davidoliverSP2 / neo4j-documentation

About Neo4j documentation

This repository contains tools and tests for proving and generating documentation for Neo4j.

All docs sources are maintained in AsciiDoc format. The primary output of building the projects kept here are artifacts (JARs) containing docs sources, code examples, and some scripts and images. These are processed downstream with custom tools, Asciidoctor and XSLT to produce the library of documentation available at https://neo4j.com/docs/.

Building

Running mvn clean install will build and install all artifacts. Modules that test or generate documentation as part of their build will produce a -docs.jar artifact. The contents module collects these into a neo4j-manual-contents jar. Beyond this überJAR the javadocs and cypher/refcard-tests modules also produce special-purpose content JARs.

The tools and tests run against Neo4j product artifacts. These can be downloaded from Maven Central, as ordinary dependencies. It is also possible to build Neo4j locally, and either deploy to a flatfile -DaltDeploymentRepository or install in the local Maven cache. The tools and tests in this repository can then be built and run against the locally produced Neo4j artifacts. This is the workflow for building Neo4j documentation while developing Neo4j the product.

Local pipeline

The easiest way to build Neo4j and Neo4j documentation in step is to rely on the local Maven cache. Running mvn clean install -DskipTests -T1.5C in your local Neo4j clone followed by mvn clean install in your clone of this repository will achieve this. This is not very robust, however. Pretty soon Neo4j artifacts will be published to the Neo4j snapshot repository that are newer than the ones you have installed locally, and your docs build will start pulling these in instead. It is also very difficult to know which artifacts are actually being used. A module or part of a module depended on in the docs build may even be removed from the product without this surfacing and the build breaking. As long as you have some SNAPSHOT artifact for the correct version installed in the local cache this will continue to be used, until a release happens and the version is bumped.

The less easy but more robust way to develop Neo4j and dependents like documentation in step, is to emulate a CI release pipeline locally. This could involve things like setting an explicit version on the artifacts produced, and not rely on the ever-fleeting Maven SNAPSHOT. To control the dependency chain, and to avoid interfering with other builds that may share dependencies, it may be a good idea to deploy or install artifacts in custom locations. The in/ and out/ for each build in the pipeline can be configured to provide the desired insulation.

An example of achieving this would be:

Manually build neo4j/neo4j

$ cd $MY_NEO4J_CLONE
$ export ALT_DEPLOYMENT_REPOSITORY=$PWD/out/neo_artifacts
$ mvn versions:set -DnewVersion=$TARGET_VERSION
$ mvn clean deploy -DaltDeploymentRepository=releases::default::file://$ALT_DEPLOYMENT_REPOSITORY -DskipTests -T1.5C
$ mvn versions:revert                       # or `git reset --hard` if `versions:revert` is not thorough enough

Manually build neo4j/neo4j-documentation

$ cd $MY_NEO4J_DOCUMENTATION_CLONE
$ export MAVEN_REPO_LOCAL=$PWD/in/localRepository
$ export ALT_DEPLOYMENT_REPOSITORY=$PWD/out/docs_artifacts
$ ln -s $MY_NEO4J_CLONE/out/neo_artifacts $MAVEN_REPO_LOCAL
$ mvn versions:set -DnewVersion=$TARGET_VERSION
$ mvn clean deploy -DaltDeploymentRepository=releases::default::file://$ALT_DEPLOYMENT_REPOSITORY -Dmaven.repo.local=$MAVEN_REPO_LOCAL
$ mvn versions:revert

As commandline interaction this may seem a bit heavy-handed. One might then consider separating configuration (environment variables, symlinks, etc.) into a developer-common devenv and a developer-local devenv.local file to be sourced, and commands (Maven invocations) into a run-build script to be executed. An interaction cycle might then be:

Build Neo4j and Neo4j Documentation

$ cd $MY_NEO4J_CLONE
$ . ./devenv                                # which in turn sources ./devenv.local
$ ./run-build
$ cd $MY_NEO4J_DOCUMENTATION_CLONE
$ . ./devenv
$ ./run-build

This interaction might in turn easily be automated by either running one build and listen for changes to trigger others, or by scripting the steps for manual execution. Sprinkle some git commit hooks on top et voilà, you are now Snapping the CI on your own developer box.