This repository contains different deployment templates for Driverless AI (DAI) scorers.
The structure is as follows:
common
: Code shared by multiple deployment templatesswagger
: The shared REST API definitions. Please don't duplicate.rest-{framework}-model/api
: Modules with Java classes autogenerated from the API Swagger definition.transform
: Shared transformers between mojo and API classes.
{target}-scorer
: One module per deployment target.
Note: Please look to each subproject for additional deployment information.
This is a Gradle multi-project repository. To get the resulting distribution archive, run:
$ ./gradlew distributionZip
The result of which is ./build/dai-deployment-templates-{CURRENT_VERSION}.zip
, which is in turn integrated in
the DAI build and deployment process.
Note that each of the templates is expected to inject its files in this archive in their respective gradle files. Please see and follow examples in the existing deployment templates.
The ci process for building the above artifacts also performs linting and styling checks, and will fail if there are any issues found. In order test this before pushing to the remote, these checks can be performed locally.
To test builds locally, with respect to linting and styling use the following steps:
- run
./gradlew applySpotless
- this will help clean the code base with proper linting/styling
- run
./gradlew check
- this will run the same check as in Jenkins, to check for styling/linting errors.
- will identify issues such as missing javadoc strings etc.
The deployment templates zip archives are stored in S3 at:
- Snapshots: https://s3.console.aws.amazon.com/s3/buckets/artifacts.h2o.ai/snapshots/ai/h2o/dai-deployment-templates/
- Releases: https://s3.console.aws.amazon.com/s3/buckets/artifacts.h2o.ai/releases/ai/h2o/dai-deployment-templates/
The push is handled by Jenkins from master
and release*
branches with versioning
defined by a value in gradle.properties
. The snapshots are versions with
the -SNAPSHOT
suffix in version
value defined in gradle.properties
.
The master
branch is expected to be a snapshot (otherwise Jenkins pipeline fails).
The release*
branches may be both snapshot and release versions.
Beware that Jenkins is happy to overwrite the release artifact, if there is a new commit
with the same version.
In addition to this, we maintain GitHub releases that mirror the released artifacts. For example
an artifact versioned 0.0.5
is tagged as v0.0.5
with the actual release called
Release v0.0.5
. This step is not automated yet.
- Prepare PR with two commits:
- First, "Release X.Y.Z" that removes the
-SNAPSHOT
suffix ingradle.properties
formversion
. The diff will look like:- version = 1.2.3-SNAPSHOT + version = 1.2.3
- Second, "Dev X.Y.(Z+1)" that bumps the version and adds the
-SNAPSHOT
suffix back toversion
ingradle.properties
. The diff will look like:- version = 1.2.3 + version = 1.2.4-SNAPSHOT
- Once approved, rebase the PR on top of
master
, so that both the commits exist in the history as separate commits. - Create a release branch pointing to the first commit of the two, the release one, called
release-X.Y.Z
. Once in GitHub, Jenkins will automatically pick it up and push the corresponding release packages to S3. - Create a GitHub release "Release vX.Y.Z" with meaningful description, tag
vX.Y.Z
, and branchrelease-X.Y.Z
.
- First, "Release X.Y.Z" that removes the
Currently, we have two lines of releases:
- DriverlessAI 1.7.x, bleeding edge:
These releases are meant for the latest version of DriverlessAI and contains
the most up to date version of the templates.
Git branch:
master
, release versions:1.x.y
, immutable release branch names:release-1.x.y
. - DriverlessAI 1.6.x, LTS:
These releases are meant to support the LTS branch of DriverlessAI marked as
1.6.x
. Changes in this branch must not break compatibility, so please keep the Terraform version fixed. Git branchsupport-1.6
, release versions0.0.x
, immutable release branch names:release-0.0.x
.
To upgrade the mojo2 runtime dependency version, just edit the corresponding line in the
gradle.properties
file a push a new version of the deployment templates out as described
above.
Note that in order to be able to build against the new mojo2 runtime, the mojo2 runtime implementation and the api jars have to be available in the public Maven repository: https://mvnrepository.com/artifact/ai.h2o/mojo2-runtime-api and https://mvnrepository.com/artifact/ai.h2o/mojo2-runtime-impl.
Both the steps are handled by the DAI Build: mojo2
Jenkins pipeline:
http://mr-0xc1:8080/view/H2OAI/job/mojo2/job/master/
if the doRelease
parameter is checked when clicking on Build with Parameters
on
the master
branch (again, the master should currently contain a version without
the -SNAPSHOT
suffix).
Note that there is an extra step to push the mojo2-runtime-api
to the public Maven repo.
One has to login to Sonatype using H2o public nexus credentials, manually close
the appropriate staging repository with the new artifacts (usually the first one called
aih2o.*
, but check the contents first!), and then release
it.
The Maven UI takes time to display the new version, but the artifacts themselves usually
appear in the order of seconds/minutes after the release (i.e., you will not see the new
version on https://mvnrepository.com/artifact/ai.h2o/mojo2-runtime-api, but you will be
able to build against it).