Spring Cloud Schema Registry
When organizations have a messaging based pub/sub architecture and multiple producer and consumer microservices communicate each other, it is often necessary for all those microservices to agree on a contract that is based on a schema. When such a schema needs to evolve to accommodate new business requirements, the existing components are still required to continue to work. This project provides support for a standalone schema registry server using which aforementioned schema can be registered and used by the applications. It also contains support for avro based schema registry clients, which essentially provide message converters that communicates with the schema registry for reconciling schema during message conversion. The schema evolution support provided by this project works both with the aforementioned standalone schema registry as well as the scheam registry provided by Confluent that specifically works with Apache Kafka.
Spring Cloud Schema Registry overview
Spring Cloud Schema Registry provides support for schema evolution so that the data can be evolved over time and still work with older or newer producers and consumers and vice versa. Most serialization models, especially the ones that aim for portability across different platforms and languages, rely on a schema that describes how the data is serialized in the binary payload. In order to serialize the data and then to interpret it, both the sending and receiving sides must have access to a schema that describes the binary format. In certain cases, the schema can be inferred from the payload type on serialization or from the target type on deserialization. However, many applications benefit from having access to an explicit schema that describes the binary data format. A schema registry lets you store schema information in a textual format (typically JSON) and makes that information accessible to various applications that need it to receive and send data in binary format. A schema is referenceable as a tuple consisting of:
-
A subject that is the logical name of the schema
-
The schema version
-
The schema format, which describes the binary format of the data
Spring Cloud Schema Registry provides the following compoents
-
Standalone Schema Registry Server
By default, it is using an H2 database, but server can be used with other databases by providing appropriate datasource configuration.
-
Schema registry clients capable of message marshalling by communicating with a Schema Registry.
Currently, the client can communicate to the standalone schema registry or the Confluent Schema Registry.
Project page
You can read more about Spring Cloud Schema Registry by going to the project page
Building
Basic Compile and Test
To build the source you will need to install JDK 1.7.
Spring Cloud uses Maven for most build-related activities, and you should be able to get off the ground quite quickly by cloning the project you are interested in and typing
$ ./mvnw install
Note
|
You can also install Maven (>=3.3.3) yourself and run the mvn command
in place of ./mvnw in the examples below. If you do that you also
might need to add -P spring if your local Maven settings do not
contain repository declarations for spring pre-release artifacts.
|
Note
|
Be aware that you might need to increase the amount of memory
available to Maven by setting a MAVEN_OPTS environment variable with
a value like -Xmx512m -XX:MaxPermSize=128m . We try to cover this in
the .mvn configuration, so if you find you have to do it to make a
build succeed, please raise a ticket to get the settings added to
source control.
|
For hints on how to build the project look in .travis.yml
if there
is one. There should be a "script" and maybe "install" command. Also
look at the "services" section to see if any services need to be
running locally (e.g. mongo or rabbit). Ignore the git-related bits
that you might find in "before_install" since they’re related to setting git
credentials and you already have those.
The projects that require middleware generally include a
docker-compose.yml
, so consider using
Docker Compose to run the middeware servers
in Docker containers. See the README in the
scripts demo
repository for specific instructions about the common cases of mongo,
rabbit and redis.
Note
|
If all else fails, build with the command from .travis.yml (usually
./mvnw install ).
|
Documentation
The spring-cloud-build module has a "docs" profile, and if you switch
that on it will try to build asciidoc sources from
src/main/asciidoc
. As part of that process it will look for a
README.adoc
and process it by loading all the includes, but not
parsing or rendering it, just copying it to ${main.basedir}
(defaults to ${basedir}
, i.e. the root of the project). If there are
any changes in the README it will then show up after a Maven build as
a modified file in the correct place. Just commit it and push the change.
Working with the code
If you don’t have an IDE preference we would recommend that you use Spring Tools Suite or Eclipse when working with the code. We use the m2eclipse eclipse plugin for maven support. Other IDEs and tools should also work without issue as long as they use Maven 3.3.3 or better.
Importing into eclipse with m2eclipse
We recommend the m2eclipse eclipse plugin when working with eclipse. If you don’t already have m2eclipse installed it is available from the "eclipse marketplace".
Note
|
Older versions of m2e do not support Maven 3.3, so once the
projects are imported into Eclipse you will also need to tell
m2eclipse to use the right profile for the projects. If you
see many different errors related to the POMs in the projects, check
that you have an up to date installation. If you can’t upgrade m2e,
add the "spring" profile to your settings.xml . Alternatively you can
copy the repository settings from the "spring" profile of the parent
pom into your settings.xml .
|
Importing into eclipse without m2eclipse
If you prefer not to use m2eclipse you can generate eclipse project metadata using the following command:
$ ./mvnw eclipse:eclipse
The generated eclipse projects can be imported by selecting import existing projects
from the file
menu.
Contributing
Spring Cloud is released under the non-restrictive Apache 2.0 license, and follows a very standard Github development process, using Github tracker for issues and merging pull requests into master. If you want to contribute even something trivial please do not hesitate, but follow the guidelines below.
Sign the Contributor License Agreement
Before we accept a non-trivial patch or pull request we will need you to sign the Contributor License Agreement. Signing the contributor’s agreement does not grant anyone commit rights to the main repository, but it does mean that we can accept your contributions, and you will get an author credit if we do. Active contributors might be asked to join the core team, and given the ability to merge pull requests.
Code of Conduct
This project adheres to the Contributor Covenant code of conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to spring-code-of-conduct@pivotal.io.
Code Conventions and Housekeeping
None of these is essential for a pull request, but they will all help. They can also be added after the original pull request but before a merge.
-
Use the Spring Framework code format conventions. If you use Eclipse you can import formatter settings using the
eclipse-code-formatter.xml
file from the Spring Cloud Build project. If using IntelliJ, you can use the Eclipse Code Formatter Plugin to import the same file. -
Make sure all new
.java
files to have a simple Javadoc class comment with at least an@author
tag identifying you, and preferably at least a paragraph on what the class is for. -
Add the ASF license header comment to all new
.java
files (copy from existing files in the project) -
Add yourself as an
@author
to the .java files that you modify substantially (more than cosmetic changes). -
Add some Javadocs and, if you change the namespace, some XSD doc elements.
-
A few unit tests would help a lot as well — someone has to do it.
-
If no-one else is using your branch, please rebase it against the current master (or other target branch in the main project).
-
When writing a commit message please follow these conventions, if you are fixing an existing issue please add
Fixes gh-XXXX
at the end of the commit message (where XXXX is the issue number).
Checkstyle
Spring Cloud Build comes with a set of checkstyle rules. You can find them in the spring-cloud-build-tools
module. The most notable files under the module are:
└── src ├── checkstyle │ └── checkstyle-suppressions.xml (3) └── main └── resources ├── checkstyle-header.txt (2) └── checkstyle.xml (1)
-
Default Checkstyle rules
-
File header setup
-
Default suppression rules
Checkstyle configuration
Checkstyle rules are disabled by default. To add checkstyle to your project just define the following properties and plugins.
<properties> <maven-checkstyle-plugin.failsOnError>true</maven-checkstyle-plugin.failsOnError> (1) <maven-checkstyle-plugin.failsOnViolation>true </maven-checkstyle-plugin.failsOnViolation> (2) <maven-checkstyle-plugin.includeTestSourceDirectory>true </maven-checkstyle-plugin.includeTestSourceDirectory> (3) </properties> <build> <plugins> <plugin> (4) <groupId>io.spring.javaformat</groupId> <artifactId>spring-javaformat-maven-plugin</artifactId> </plugin> <plugin> (5) <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-checkstyle-plugin</artifactId> </plugin> </plugins> <reporting> <plugins> <plugin> (5) <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-checkstyle-plugin</artifactId> </plugin> </plugins> </reporting> </build>
-
Fails the build upon Checkstyle errors
-
Fails the build upon Checkstyle violations
-
Checkstyle analyzes also the test sources
-
Add the Spring Java Format plugin that will reformat your code to pass most of the Checkstyle formatting rules
-
Add checkstyle plugin to your build and reporting phases
If you need to suppress some rules (e.g. line length needs to be longer), then it’s enough for you to define a file under ${project.root}/src/checkstyle/checkstyle-suppressions.xml
with your suppressions. Example:
<?xml version="1.0"?> <!DOCTYPE suppressions PUBLIC "-//Puppy Crawl//DTD Suppressions 1.1//EN" "https://www.puppycrawl.com/dtds/suppressions_1_1.dtd"> <suppressions> <suppress files=".*ConfigServerApplication\.java" checks="HideUtilityClassConstructor"/> <suppress files=".*ConfigClientWatch\.java" checks="LineLengthCheck"/> </suppressions>
It’s advisable to copy the ${spring-cloud-build.rootFolder}/.editorconfig
and ${spring-cloud-build.rootFolder}/.springformat
to your project. That way, some default formatting rules will be applied. You can do so by running this script:
$ curl https://raw.githubusercontent.com/spring-cloud/spring-cloud-build/master/.editorconfig -o .editorconfig
$ touch .springformat
IDE setup
Intellij IDEA
In order to setup Intellij you should import our coding conventions, inspection profiles and set up the checkstyle plugin. The following files can be found in the Spring Cloud Build project.
└── src ├── checkstyle │ └── checkstyle-suppressions.xml (3) └── main └── resources ├── checkstyle-header.txt (2) ├── checkstyle.xml (1) └── intellij ├── Intellij_Project_Defaults.xml (4) └── Intellij_Spring_Boot_Java_Conventions.xml (5)
-
Default Checkstyle rules
-
File header setup
-
Default suppression rules
-
Project defaults for Intellij that apply most of Checkstyle rules
-
Project style conventions for Intellij that apply most of Checkstyle rules
Go to File
→ Settings
→ Editor
→ Code style
. There click on the icon next to the Scheme
section. There, click on the Import Scheme
value and pick the Intellij IDEA code style XML
option. Import the spring-cloud-build-tools/src/main/resources/intellij/Intellij_Spring_Boot_Java_Conventions.xml
file.
Go to File
→ Settings
→ Editor
→ Inspections
. There click on the icon next to the Profile
section. There, click on the Import Profile
and import the spring-cloud-build-tools/src/main/resources/intellij/Intellij_Project_Defaults.xml
file.
To have Intellij work with Checkstyle, you have to install the Checkstyle
plugin. It’s advisable to also install the Assertions2Assertj
to automatically convert the JUnit assertions
Go to File
→ Settings
→ Other settings
→ Checkstyle
. There click on the +
icon in the Configuration file
section. There, you’ll have to define where the checkstyle rules should be picked from. In the image above, we’ve picked the rules from the cloned Spring Cloud Build repository. However, you can point to the Spring Cloud Build’s GitHub repository (e.g. for the checkstyle.xml
: https://raw.githubusercontent.com/spring-cloud/spring-cloud-build/master/spring-cloud-build-tools/src/main/resources/checkstyle.xml
). We need to provide the following variables:
-
checkstyle.header.file
- please point it to the Spring Cloud Build’s,spring-cloud-build-tools/src/main/resources/checkstyle-header.txt
file either in your cloned repo or via thehttps://raw.githubusercontent.com/spring-cloud/spring-cloud-build/master/spring-cloud-build-tools/src/main/resources/checkstyle-header.txt
URL. -
checkstyle.suppressions.file
- default suppressions. Please point it to the Spring Cloud Build’s,spring-cloud-build-tools/src/checkstyle/checkstyle-suppressions.xml
file either in your cloned repo or via thehttps://raw.githubusercontent.com/spring-cloud/spring-cloud-build/master/spring-cloud-build-tools/src/checkstyle/checkstyle-suppressions.xml
URL. -
checkstyle.additional.suppressions.file
- this variable corresponds to suppressions in your local project. E.g. you’re working onspring-cloud-contract
. Then point to theproject-root/src/checkstyle/checkstyle-suppressions.xml
folder. Example forspring-cloud-contract
would be:/home/username/spring-cloud-contract/src/checkstyle/checkstyle-suppressions.xml
.
Important
|
Remember to set the Scan Scope to All sources since we apply checkstyle rules for production and test sources.
|