The AsyncAPI Specification is a project used to describe and document message-driven APIs in a machine-readable format. It’s protocol-agnostic, so you can use it for APIs that work over any protocol (e.g., AMQP, MQTT, WebSockets, Kafka, STOMP, HTTP, Mercure, etc).
https://www.asyncapi.com/docs/reference/specification/v2.5.0
The AsyncAPI Specification defines a set of files required to describe such an API. These files can then be used to create utilities, such as documentation, integration and/or testing tools.
It is a way to allow external integrations to understand the format of Events that you are sending them, without having to share your internal models.
There is a playground if you want to get familiar with the syntax, but it is recommended for simple asyncapi.yml files, and it can crash if you have a large file.
https://studio.asyncapi.com/
We have the following internal User class:
public class User {
private String userId;
private String userName;
private Boolean isUserManager;
private LocalDate dateAdded;
private List<User> employees;
public User(String userId, String userName, Boolean isUserManager, LocalDate dateAdded, List<User> employees) {
this.userId = userId;
this.userName = userName;
this.isUserManager = isUserManager;
this.dateAdded = dateAdded;
this.employees = employees;
}
public String getUserId() {
return userId;
}
public String getUserName() {
return userName;
}
@JsonProperty("isUserManager")
public Boolean getUserManager() {
return isUserManager;
}
public LocalDate getDateAdded() {
return dateAdded;
}
public List<User> getEmployees() {
return employees;
}
}
To represent the class User, we will create an asyncapi.yml file:
asyncapi: '2.5.0'
info:
title: Example on how to generate POJOs from an asyncapi.yml
version: '1.0.0'
description: Contract for Event Consumption
servers:
dev:
url: kafka:9092
protocol: kafka-secure
description: Dev Cluster
defaultContentType: application/json
channels:
EXAMPLE.APP..EVENTS.USER:
description: The topic for all User Events to be consumed from.
publish:
summary: User events can be consumed from this topic.
message:
$ref: '#/components/messages/User'
components:
messages:
User:
name: User
title: User
summary: User
contentType: application/json
payload:
$ref: "#/components/schemas/User"
schemas:
User:
type: object
properties:
userId:
type: string
userName:
type: string
isUserManager:
type: boolean
dateAdded:
type: string
format: date
employees:
type: array
items:
$ref: "#/components/schemas/User"
operationTraits:
kafka:
bindings:
kafka:
clientId: example-asyncapi-application
The message that will be sent as a Kafka Event is of type ‘#/components/messages/User’ and in turn references the schema $ref: “#/components/schemas/User”
At the time of writing, there does not seem to be a maven plugin that takes in an asyncapi.yml file and generate Plain old Java classes from it. However there is an npm package asyncapi/generator which could be used.
This generator has a docker container option as well, which we use in the maven build instead of installing node:
https://github.com/asyncapi/generator/blob/master/docs/usage.md#cli-usage-with-docker
To run the asyncapi/generator docker image as part of the maven build, we will use docker-maven-plugin, and mount the location of the folder that contains asyncapi.yml and mount the location of /target
In order to allow docker-maven-plugin to start a docker container on the host machine, you need to enable docker daemon remote access.
On Docker Desktop, you can modify the docker settings to expose Daemon on port 2375.
On Linux, you can use /var/run/docker.sock
<plugin>
<groupId>io.fabric8</groupId>
<artifactId>docker-maven-plugin</artifactId>
<version>0.33.0</version>
<executions>
<execution>
<id>start-db-container</id>
<phase>generate-sources</phase>
<goals>
<goal>start</goal>
</goals>
<configuration>
<logStdout>true</logStdout>
<showLogs>true</showLogs>
<dockerHost>${docker.host}</dockerHost>
<images>
<image>
<name>asyncapi/generator</name>
<run>
<volumes>
<bind>
<volume>src/main/resources/asyncapi:/asyncapi</volume>
<volume>${project.build.directory}:/target</volume>
</bind>
</volumes>
<cmd>
-o /target/generated-sources/async-api-generated-code /asyncapi/asyncapi.yml
-p disableEqualsHashCode=true @asyncapi/java-spring-template --force-write
</cmd>
<wait>
<log>Check out your shiny new generated files.</log>
<time>300000</time>
</wait>
</run>
</image>
</images>
</configuration>
</execution>
<execution>
<id>stop-asyn-container</id>
<phase>pre-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<phase>generate-sources</phase>
<configuration>
<target>
<copy todir="${project.build.directory}/generated-sources/async-api">
<fileset
dir="${project.build.directory}/generated-sources/async-api-generated-code/src/main/java"
includes="**/*"/>
</copy>
<delete dir="${project.build.directory}/generated-sources/async-api-generated-code"
includeemptydirs="true"/>
<delete dir="${project.build.directory}/generated-sources/async-api/com/asyncapi/infrastructure"
includeemptydirs="true"/>
<delete dir="${project.build.directory}/generated-sources/async-api/com/asyncapi/service"
includeemptydirs="true"/>
<delete file="${project.build.directory}/generated-sources/async-api/com/asyncapi/Application.java"/>
</target>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>${project.build.directory}/generated-sources/async-api</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
Once asyncapi/generator docker image runs, it parses the asyncapi.yml file and generates Java classes including some Spring related classes that we do not need. The reason for the extra classes is due to the fact that we are using java-spring-template.
To remove the unnecessary extra classes, we use maven-antrun-plugin to clean up and move the relevant generated classes and use build-helper-maven-plugin to point to them and add them as src.
You must provide a template to asyncapi/generator so it knows the format for generation.
We tell the build to wait for Check out your shiny new generated files. log in order to determine that the generation was successful.
It should look like this:
Test:
The test verifies that if we send to Kafka our Internal Models (User), the integrated applications can use the generated AsyncAPI models to read the Kafka Events.
@Test
public void verifyAsyncAPIGeneratedModels() throws JsonProcessingException {
//If we want to work with LocalDate + LocalDateTime
objectMapper.registerModule(new JavaTimeModule());
//Internal Models
User user = new User("123", "Username", Boolean.TRUE, LocalDate.now(), null);
User user2 = new User("1234", "Username2", Boolean.TRUE, LocalDate.now(), null);
//Verify that they can be parsed into generated Async API Models
String userJSONFromInternalModel = objectMapper.writeValueAsString(user);
String user2JSONFromInternalModel = objectMapper.writeValueAsString(user2);
//Conversion should be OK!
objectMapper.readValue(userJSONFromInternalModel, com.asyncapi.model.User.class);
objectMapper.readValue(user2JSONFromInternalModel, com.asyncapi.model.User.class);
}
Profiles:
There are 2 profiles listed in the pom.xml. This is because on linux, it is usually defaulted to using the docker.sock
<profiles>
<profile>
<id>windows</id>
<activation>
<os>
<family>Windows</family>
</os>
</activation>
<properties>
<docker.host>http://localhost:2375</docker.host>
</properties>
</profile>
<profile>
<id>Linux</id>
<activation>
<os>
<family>Linux</family>
</os>
</activation>
<properties>
<docker.host>unix:///var/run/docker.sock</docker.host>
</properties>
</profile>
</profiles>
Considerations:
You might need to run your maven build from inside another docker. So it would be docker–>on–>docker.
However in this case, your <volume> mounts will not work as they should. In order for the volume mounts to work, they need to be named exactly as they would be on the host.
Example:docker run maven -v /var/run/docker.sock:/var/run/docker.sock -v /your-application/src/main/resources/asyncapi:/your-application/src/main/resources/asyncapi -v /your-application/target:/your-application/target mvn clean install
Your plugin volumes should also change:
<plugin>
<groupId>io.fabric8</groupId>
<artifactId>docker-maven-plugin</artifactId>
<version>0.33.0</version>
<executions>
<execution>
<id>start-db-container</id>
<phase>generate-sources</phase>
<goals>
<goal>start</goal>
</goals>
<configuration>
<logStdout>true</logStdout>
<showLogs>true</showLogs>
<dockerHost>${docker.host}</dockerHost>
<images>
<image>
<name>asyncapi/generator</name>
<run>
<run>
<volumes>
<bind>
<volume>/var/run/docker.sock:/var/run/docker.sock</volume <volume>/your-application/src/main/resources/asyncapi:/your-application/src/main/resources/asyncapi</volume>
<volume>/your-application/target:/your-application/target</volume>
https://github.com/fabric8io/docker-maven-plugin/issues/863
Docker-On-Docker would not work well with the docker-maven-plugin.
You might also face a similar exception to the below if your getters are not named as your syncapi.yml properties:com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "userManager" (class com.asyncapi.model.User), not marked as ignorable (5 known properties: "isUserManager", "dateAdded", "employees", "userId", "userName"])
at [Source: (String)"{"userId":"123","userName":"Username","dateAdded":[2023,1,20],"employees":null,"userManager":true}"; line: 1, column: 98] (through reference chain: com.asyncapi.model.User["userManager"])
You are seeing this because JSON is generated from the public Getters, in this case the getter is called getUserManager(). <– Auto Generated from Intellij. This was done on purpose to showcase the error.
To fix this, add @JsonProperty("isUserManager") on top of the public getter.
If you use Lombok, then put the @JsonProperty("isUserManager") on top of the private class property instead.
[INFO] Scanning for projects...
[INFO]
[INFO] -----------< com.asyncapi.example:generate-async-api-pojos >------------
[INFO] Building generate-async-api-pojos 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- docker-maven-plugin:0.33.0:start (start-db-container) @ generate-async-api-pojos ---
[INFO] DOCKER> [asyncapi/generator:latest]: Start container 3cd04b07f85c
3cd04b>
3cd04b>
3cd04b> Done! ✨
[INFO] DOCKER> Pattern 'Check out your shiny new generated files.' matched for container 3cd04b07f85c
3cd04b> Check out your shiny new generated files at /target/generated-sources/async-api-generated-code.
3cd04b>
[INFO] DOCKER> [asyncapi/generator:latest]: Waited on log out 'Check out your shiny new generated files.' 50447 ms
[INFO]
[INFO] --- maven-antrun-plugin:3.1.0:run (default) @ generate-async-api-pojos ---
[INFO] Executing tasks
[INFO] [copy] Copying 6 files to C:\Users\Downloads\generate-async-api-pojos\target\generated-sources\async-api
[INFO] [delete] Deleting directory C:\Users\Downloads\generate-async-api-pojos\target\generated-sources\async-api-generated-code
[INFO] [delete] Deleting directory C:\Users\Downloads\generate-async-api-pojos\target\generated-sources\async-api\com\asyncapi\infrastructure
[INFO] [delete] Deleting directory C:\Users\Downloads\generate-async-api-pojos\target\generated-sources\async-api\com\asyncapi\service
[INFO] [delete] Deleting: C:\Users\Downloads\generate-async-api-pojos\target\generated-sources\async-api\com\asyncapi\Application.java
[INFO] Executed tasks
[INFO]
[INFO] --- build-helper-maven-plugin:3.3.0:add-source (default) @ generate-async-api-pojos ---
[INFO] Source directory: C:\Users\Downloads\generate-async-api-pojos\target\generated-sources\async-api added.
[INFO]
[INFO] --- maven-resources-plugin:3.2.0:resources (default-resources) @ generate-async-api-pojos ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] Copying 1 resource
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.10.1:compile (default-compile) @ generate-async-api-pojos ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 3 source files to C:\Users\Downloads\generate-async-api-pojos\target\classes
[INFO]
[INFO] --- maven-resources-plugin:3.2.0:testResources (default-testResources) @ generate-async-api-pojos ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] skip non existing resourceDirectory C:\Users\Downloads\generate-async-api-pojos\src\test\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.10.1:testCompile (default-testCompile) @ generate-async-api-pojos ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to C:\Users\Downloads\generate-async-api-pojos\target\test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ generate-async-api-pojos ---
[INFO]
[INFO] --- maven-jar-plugin:3.2.2:jar (default-jar) @ generate-async-api-pojos ---
[INFO] Building jar: C:\Users\Downloads\generate-async-api-pojos\target\generate-async-api-pojos-0.0.1-SNAPSHOT.jar
[INFO]
[INFO] --- spring-boot-maven-plugin:2.7.8:repackage (repackage) @ generate-async-api-pojos ---
[INFO] Replacing main artifact with repackaged archive
[INFO]
[INFO] --- docker-maven-plugin:0.33.0:stop (stop-asyn-container) @ generate-async-api-pojos ---
[INFO] DOCKER> [asyncapi/generator:latest]: Stop and removed container 3cd04b07f85c after 0 ms
[INFO]
[INFO] --- maven-failsafe-plugin:2.22.2:integration-test (default) @ generate-async-api-pojos ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.asyncapi.example.generateasyncapipojos.GenerateAsyncApiIT
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.202 s - in com.asyncapi.example.generateasyncapipojos.GenerateAsyncApiIT
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- maven-failsafe-plugin:2.22.2:verify (default) @ generate-async-api-pojos ---
[INFO]
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ generate-async-api-pojos ---
[INFO] Installing C:\Users\Downloads\generate-async-api-pojos\target\generate-async-api-pojos-0.0.1-SNAPSHOT.jar to C:\Users\.m2\repository\com\asyncapi\example\generate-async-api-pojos\0.0.1-SNAPSHOT\generate-async-api-pojos-0.0.1-SNAPSHOT.jar
[INFO] Installing C:\Users\Downloads\generate-async-api-pojos\pom.xml to C:\Users\.m2\repository\com\asyncapi\example\generate-async-api-pojos\0.0.1-SNAPSHOT\generate-async-api-pojos-0.0.1-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 57.138 s
[INFO] Finished at: 2023-01-22T11:02:26+01:00
[INFO] ------------------------------------------------------------------------
Process finished with exit code 0
JCompetence AB 