gradle crossbuild scala plugin

Features

Multi-module projects support Supports both simple projects and multi-module projects.
In multi-module projects support mixed cases where only some of the modules needs cross compiling.
Powerful DSL Plugin DSL can be written once for all sub projects using subprojects {} block.
Specific DSL definition can be afterwards added to individual sub projects.
It supports shorthands to avoid repetitions.
Operates in both eager and lazy (wrapped in pluginManager.withPlugin {} block) apply modes.
Integrates with maven-publish plugin When used, can be leveraged to publish cross building artifacts.
Implicit/Explicit scala lib dependency declaration Supports declaring both
simple case implicit compile 3rd-party-scala-lib_2.12 type of dependencies
and also finer granular explicit crossBuildSpark24_212Compile spark-streaming-kafka-0-10_2.12 type of dependencies.
Applied easily on existing projects As the plugin maintains a strict separation between main source set configurations and crossBuildXYZ ones, a simple non cross build project can be easily and gradually transformed to a cross build one.
Testing support As mentioned above strict separation of source sets, keeps main source set test configurations intact.

Shortcomings

Cross building for test/check tasks are not supported.
No support for java-library plugin java-library api, apiComponents and runtimeComponents configurations are not supported yet.

Getting the plugin

Using the `plugins` DSL:

plugins {
    id "com.github.prokod.gradle-crossbuild-scala" version "0.12.0"
}

Using legacy `buildscript`

buildscript {
    dependencies {
        classpath("com.github.prokod:gradle-crossbuild-scala:0.12.0")
    }
}

Quick start

recommended cross building apply strategy

This is especially true for multi module projects but not just.

Wire up your build scripts in the project in such a way that you are able to successfully build it for single scala version.
Beware that this plugin does not support api configuration so avoid using it, for further details see java-library related section.

NOTE: Do not worry in this stage about publishing artifacts as cross building with publishing is supported by the plugin.
It will be some what wasted effort to do that here and then modify it to the cross build scenario.
After that add the cross building plugin without changing any of the inter and external dependencies. Follow base step - getting the plugin and then configure it, see please both plugin configuration options and basic plugin configuration

NOTE: If you have to change your dependencies because of applying the plugin and trying explicitly gradle build, something is fishy.
You see, the plugin is designed in such a way that it borrows from the state of the project's dependency tree already in place without changing it and then it adds a somewhat parallel dependency tree for each of the cross building variants.
To configure the plugin efficiently please see recommended multi module projects apply patterns.

NOTE: From version 0.12.x there is no need to have any special glob pattern to express cross build dependency for implementation/compile/runtime/... configurations - the plugin will add a correct dependency resolution according to the provided crossBuild {} plugin dsl block.
Up to version 0.11.x (inclusive) use the '?' question mark to express cross build dependency inside implementation/compile/runtime/... configurations.
Use the provided explicit crossBuildXYZImplementation/Compile/Runtime/... configuration when you need a finer granularity in expressing the cross build dependencies
Publish cross building artifacts, for that please have a look here

NOTE: cross build artifact naming is governed by archive.appendixPattern which by default is _? meaning for example, that module lib will be resolved to lib_2.11/_2.12/... according to the correlating crossBuild {} plugin dsl block
To test that everything works as expected, both gradle build (which also runs the tests) and gradle publishToMavenLocal (which goes from cross building, artifact creation and publishing) should succeed.

NOTE: Look under ~/.m2/repository/... to assert the end result is the one you have wished for.

Multi-module projects and applying cross build plugin only for some

From version 0.11.x the plugin supports multi-module projects where only some of the modules have cross build plugin applied to.
This helps with cases where some of the modules depend on legacy plugins that do not play nicely with the cross build plugin like legacy play plugin for instance :)
Thanks borissmidt for the collaboration on that.

cross building - basic plugin configuration

applying the plugin

Apply the plugin and use the provided DSL. For example:

archivesBaseName = 'lib'

apply plugin: 'com.github.prokod.gradle-crossbuild-scala'

crossBuild {
    builds {
        v211
        v212
    }
}

NOTE: Another variant which might appeal aesthetically better to some

archivesBaseName = 'lib'

apply plugin: 'com.github.prokod.gradle-crossbuild-scala'

crossBuild {
    builds {
        scala {
            scalaVersions = ['2.11', '2.12']
        }
    }
}

dependencies {
    compile ("com.google.protobuf:protobuf-java:$protobufVersion")
    compile ("joda-time:joda-time:$jodaVersion")
    // Scala 2.12 is the default cross built Scala version
    // the plugin replaces the default based on the Scala version being built
    compile ("org.scalaz:scalaz_2.12:$scalazVersion")
}

NOTE: Up to version 0.11.x (inclusive) 3rd party Scala lib dependencies are expressed using '?' question mark (implicit pattern)
dependencies {
    compile ("com.google.protobuf:protobuf-java:$protobufVersion")
    compile ("joda-time:joda-time:$jodaVersion")
    // The question mark is being replaced based on the Scala version being built
    compile ("org.scalaz:scalaz_?:$scalazVersion")
}

gradle tasks

gradle-crossbuild plugin adds the following user faced tasks to the project crossBuild211Classes, crossBuild211Jar, crossBuild212Classes, crossBuild212Jar based on the plugin DSL builds {} block

> ./gradlew tasks

------------------------------------------------------------
All tasks runnable from root project
------------------------------------------------------------

Build tasks
-----------
assemble - Assembles the outputs of this project.
build - Assembles and tests this project.
buildDependents - Assembles and tests this project and all projects that depend on it.
buildNeeded - Assembles and tests this project and all projects it depends on.
classes - Assembles main classes.
clean - Deletes the build directory.
crossBuildV211Classes - Assembles cross build v211 classes.
crossBuildV211Jar - Assembles a jar archive containing 211 classes
crossBuildV212Classes - Assembles cross build v212 classes.
crossBuildV212Jar - Assembles a jar archive containing 212 classes
jar - Assembles a jar archive containing the main classes.
testClasses - Assembles test classes.
...

gradle crossBuildV211Jar crossBuildV212Jar ...

> ./gradlew crossBuildV211Jar crossBuildV212Jar
...
Tasks to be executed: [task ':compileCrossBuildV211Java', task ':compileCrossBuildV211Scala', task ':processCrossBuildV211Resources', task ':crossBuildV211Classes', task ':crossBuildV211Jar', task ':compileCrossBuildV212Java', task ':compileCrossBuildV212Scala', task ':processCrossBuildV212Resources', task ':crossBuildV212Classes', task ':crossBuildV212Jar']
...
:crossBuildV211Jar (Thread[Connection worker,5,main]) completed. Took 0.04 secs.
...
:crossBuildV212Jar (Thread[Connection worker,5,main]) completed. Took 0.007 secs.

> ls ./build/libs
lib_2.11.jar  lib_2.12.jar

notes

When defining builds {} block, a short hand convention can be used for default values.
To be able to use that, build item should be named by the following convention, for example:
xyz211 is translated to { "build": { "scalaVersions": ["2.11"], "name": "xyz211" ... }

test/check tasks are not being cross compiled and they use the default Scala version.
If a user would like to run tests with different Scala versions, he needs to change the relevant scala-library library version and neighbouring 3rd party scala dependencies in build.gradle

cross building with publishing

leveraging gradle maven-publish plugin

Apply the plugin and add maven-publish plugin. For example:

apply plugin: 'com.github.prokod.gradle-crossbuild-scala'
apply plugin: 'maven-publish'

group = 'x.y.z'
archivesBaseName = 'lib'

crossBuild {
    builds {
        v211
    }
}

// 'maven-publish' plugin usage for publishing crossbuild artifacts
publishing {
    publications {
        // Create a publication
        crossBuildV211(MavenPublication) {
            // By default groupId equals group
            groupId = 'x.y.z'
            // By default artifactId is set to crossBuildJar task `baseName`
            artifactId = 'lib_2.11'
            // actual artifact for this publication as a Jar task from crossbuild plugin
            artifact crossBuildV211Jar
        }
    }
}
...

gradle tasks

Notice that now the following publish related user faced tasks are added to the project:

> ./gradlew tasks

------------------------------------------------------------
All tasks runnable from project :lib
------------------------------------------------------------
...
Publishing tasks
----------------
generatePomFileForCrossBuild211Publication - Generates the Maven POM file for publication 'crossBuild211'.
publish - Publishes all publications produced by this project.
publishCrossBuild211PublicationToMavenLocal - Publishes Maven publication 'crossBuild211' to the local Maven repository.
publishToMavenLocal - Publishes all Maven publications produced by this project to the local Maven cache.

gradle publishToMavenLocal

> ./gradlew publishToMavenLocal
...
Tasks to be executed: [task ':compileCrossBuild211Java', task ':compileCrossBuild211Scala', task ':processCrossBuild211Resources', task ':crossBuild211Classes', task ':crossBuild211Jar', task ':generatePomFileForCrossBuild211Publication', task ':publishCrossBuild211PublicationToMavenLocal', task ':publishToMavenLocal']
Tasks to be executed: [task ':compileCrossBuild211Java', task ':compileCrossBuild211Scala', task ':processCrossBuild211Resources', task ':crossBuild211Classes', task ':crossBuild211Jar', task ':generatePomFileForCrossBuild211Publication', task ':publishCrossBuild211PublicationToMavenLocal', task ':publishToMavenLocal']
...

Gradle configurations -> Maven Scopes

The plugin handles pom generation for the different user defined publications. The transformation from Gradle's own Configurations to Maven's Scopes is done using the following transformation matrix

Maven Scope	Gradle transformation function
`COMPILE`	`GCC - (GCC - GRC)`
`RUNTIME`	`GRC - GCC`
`PROVIDED`	`GCC - GRC`

Where GCC is Gradle's CompileClasspath dependency set; GRCis Gradle's RuntimeClasspath dependency set

overriding plugin's internal pom.withXml

The plugin as seen above handles pom generation in an opinionated way. If one wants to override this behavior, he can do that by providing his own pom.withXml handler for the relevant publications.
When the plugin detects custom pom.withXml handler, the internal handler is skipped altogether.

An example for a custom pom.withXml handler:

  pom.withXml { xml ->
      def dependenciesNode = xml.asNode().dependencies?.getAt(0)
      if (dependenciesNode == null) {
          dependenciesNode = xml.asNode().appendNode('dependencies')
      }

      project.configurations.crossBuildScala_210MavenCompileScope.allDependencies.each { dep ->
          def dependencyNode = dependenciesNode.appendNode('dependency')
          dependencyNode.appendNode('groupId', dep.group)
          dependencyNode.appendNode('artifactId', dep.name)
          dependencyNode.appendNode('version', dep.version)
          dependencyNode.appendNode('scope', 'compile')
      }

      project.configurations.crossBuildScala_210MavenRuntimeScope.allDependencies.each { dep ->
          def dependencyNode = dependenciesNode.appendNode('dependency')
          dependencyNode.appendNode('groupId', dep.group)
          dependencyNode.appendNode('artifactId', dep.name)
          dependencyNode.appendNode('version', dep.version)
          dependencyNode.appendNode('scope', 'runtime')
      }
  }

In this example, we override default behaviour, dropping provided scope dependencies from generated pom.

notes

Using pluginManager 'gradle-crossbuild' plugin leverages Gradle's pluginManager To update 'maven-publish' cross-build related publications

Behind the scenes Configurations crossBuildXYZMavenCompileScope, crossBuildXYZMavenRuntimeScope are being populated from corresponding crossBuildXYZCompileClasspath, crossBuildXYZRuntimeClasspath and afterwards being used within pom.withXml {} block.
It follows a similar line of thought as conf2ScopeMappings.addMapping() in Gradle's maven plugin.
Beware, Behind the scenes the jars and the publications are decoupled, the logical linkage between a cross built Jar and the publication is made by giving the publication item a name of the following convention crossBuildXYZ(MavenPublication) where XYZ is the build name from builds {} block following the pattern examples in table under SourceSet/s column.

For Gradle 5.x beware that publishing {} block does not support deferred configuration anymore and in that case artifact crossBuild211Jar should be wrapped in afterEvaluate {} block
Please see Gradle documentation here

cross building - configuration options

targetVersionItem.archiveAppendix, crossBuild.scalaVersionsCatalog, crossBuild211XYZ pre defined configurations

apply plugin: 'com.github.prokod.gradle-crossbuild-scala'

crossBuild {
    scalaVersionsCatalog = ['2.10': '2.10.6', '2.11': '2.11.12', '2.12':'2.12.8' ...]

    archive.appendixPattern = '_?'          // Default appendix pattern for all builds

    builds {
        v210
        v211 {
            scalaVersions = ['2.11']        // By default derived from build name in short hand build name
            archive.appendixPattern = '_?'  // By default the value is "_?"
                                            // In the default case will yield '_2.11')
                                            // If different from upper level config, it will override it.
        }
    }
}

dependencies {
    compile ("com.google.protobuf:protobuf-java:$protobufVersion") 
    compile ("joda-time:joda-time:$jodaVersion")
    compile ("org.scalaz:scalaz_2.11:$scalazVersion")                       // 'default' building flavour Scala version
    compile ('org.scala-lang:scala-library:2.11.12')
    
    compileOnly ('org.apache.spark:spark-sql_2.11:2.2.1')                   // 'default' building flavour (when calling gradle build)
    crossBuildV210CompileOnly ('org.apache.spark:spark-sql_2.10:1.6.3')     // A configuration auto generated by the plugin
    crossBuildV211CompileOnly ('org.apache.spark:spark-sql_2.11:2.2.1')     // A configuration auto generated by the plugin
}

Notes

Backward compatibility: dependency with question mark Scala tag e.g. org.scalaz:scalaz_?:$scalazVersion is being replaced based on the Scala version being built

Backward compatibility: scala-library library is needed for test/check tasks in case '?' dependencies are declared

If crossBuild.scalaVersionsCatalog is not defined, a default one will be used (might get outdated).

Per build item in builds {} block, Scala version(s) is set either by explicitly setting a build scalaVersions or implicitly through a build name.
See the different build scenarios for more details

Declaring cross building dependencies explicitly:

crossBuild {
  builds {
    v210
    v211
  }
}

dependencies {
  compileOnly ('org.apache.spark:spark-sql_2.11:2.2.1')
  crossBuildV210CompileOnly ('org.apache.spark:spark-sql_2.10:1.6.3')
  crossBuildV211CompileOnly ('org.apache.spark:spark-sql_2.11:2.2.1')
}

The plugin DSL defines in the above crossBuild {} block two cross building variants. One for Scala 2.10 and one for 2.11.
When declaring explicit cross building dependency, for instance when using Spark or Kafka 3rd party libraries, when dependency library name contains platform version, All the different variants should be declared, like shown above.

default-variant In the above example, the spark version of the dependency specified for compileOnly configuration which we refer here as default-variant one for build, test/check tasks only.
The other dependency specified for Scala versions 2.10, 2.11 respectively (crossBuild210Compile/Only, crossBuild211Compile/Only), will be used only for crossBuild210Jar, crossBuild211Jar, and other corresponding task variants (publishCrossBuild210PublicationToMavenLocal, publishCrossBuild211PublicationToMavenLocal ...)

The plugin provides pre defined configurations (sourceSets) being used by the matching pre generated Jar tasks:
crossBuild211Jar -> crossBuild211Compile, crossBuild211CompileOnly, ...

`builds {}` -> Gradle SourceSets, Configurations and Tasks

The following table shows some commonly build scenarios expressed through the plugin DSL and how they are actually resolved

build scenario	SourceSet/s	Configuration/s	Task/s
`v210`	crossBuildV210	crossBuildV210Implementation crossBuildV210Compile crossBuildV210CompileOnly crossBuildV210Runtime crossBuildV210CompileClasspath	JavaPlugin -> crossBuildV210Java ScalaPlugin -> crossBuildV210Scala crossBuildV210Jar
`v211 {` `scalaVersions = ['2.11', '2.12']` `}`	crossBuildV211_211, crossBuildV211_212	crossBuildV211_211Compile ... crossBuildV211_212Compile ...	JavaPlugin -> crossBuildV211_211Java, crossBuildV211_212Java ScalaPlugin -> crossBuildV211_211Scala, crossBuildV211_212Scala crossBuildV211_211Jar, crossBuildV211_212Jar
`v213 {` `scalaVersions = ['2.13']` `}`	crossBuildV213	crossBuildV213Compile ...	JavaPlugin -> crossBuildV213Java ScalaPlugin -> crossBuildV213Scala crossBuildV213Jar
`spark24 {` `scalaVersions = ['2.11', '2.12']` `}`	crossBuildSpark24_211, crossBuildSpark24_212	crossBuildSpark24_211Compile ... crossBuildSpark24_212Compile ...	JavaPlugin -> crossBuildSpark24_211Java, crossBuildSpark24_212Java ScalaPlugin -> crossBuildSpark24_211Scala, crossBuildSpark24_212Scala crossBuildSpark24_211Jar, crossBuildSpark24_212Jar

`implementation` configuration and `java-library` plugin

implementation java plugin based configuration is supported by the plugin and cross build variants will be added to the cross build projects.
When using implementation configuration in a multi module project together with cross build plugin applied a suggestion is to read java-library plugin doc before hand, especially for new comers from Maven, compile/runtime users.
As the cross building plugin is not supporting yet java-library plugin there is no counter part to implementation by the form of api configuration and to emulate that one should use compile configuration instead

multi-module project plugin apply patterns

To apply cross building to a multi-module project use one of the following suggested layouts:

layout 1 (a.k.a lazy apply)

In the root project build.gradle:

plugins {
    id "com.github.prokod.gradle-crossbuild-scala" version '0.12.0' apply false
}

allprojects {
    apply plugin: 'base'
    group = 'x.y.z'
    version = '1.0-SNAPSHOT'

    repositories {
        mavenCentral()
    }

    pluginManager.withPlugin('com.github.prokod.gradle-crossbuild-scala') {
        crossBuild {

            scalaVersionsCatalog = ['2.11':'2.11.12', '2.12':'2.12.8']

            builds {
                spark240_211
                spark243_212
            }
        }
    }

    pluginManager.withPlugin('maven-publish') {
        publishing {
            publications {
                crossBuildSpark240_211(MavenPublication) {
                    artifact crossBuildSpark240_211Jar
                }
                crossBuildSpark243_212(MavenPublication) {
                    artifact crossBuildSpark243_212Jar
                }
            }
        }
    }
}

In sub projects' build.gradle:

apply plugin: 'com.github.prokod.gradle-crossbuild-scala'
...

layout 2 (a.k.a eager apply)

In the root project build.gradle:

plugins {
    id "com.github.prokod.gradle-crossbuild-scala" version '0.12.0' apply false
}

allprojects {
    group = 'x.y.z'
    version = '1.0-SNAPSHOT'

    repositories {
        mavenCentral()
    }
}

subprojects {
    apply plugin: 'com.github.prokod.gradle-crossbuild-scala'
    apply plugin: 'maven-publish'

    crossBuild {

        scalaVersionsCatalog = ['2.11':'2.11.12', '2.12':'2.12.8']

        builds {
            spark233_211 {
                archive.appendixPattern = '-2-3-3_?'
            }
            spark243 {
                scalaVersions = ['2.11', '2,12']
                archive.appendixPattern = '-2-4-3_?'
            }
        }
    }

    publishing {
        publications {
            crossBuildSpark233_211(MavenPublication) {
                artifact crossBuildSpark233_211Jar
            }
            crossBuildSpark243_211(MavenPublication) {
                artifact crossBuildSpark243_211Jar
            }
            crossBuildSpark243_212(MavenPublication) {
                artifact crossBuildSpark243_212Jar
            }
        }
    }
}

Supported Gradle versions

plugin version	Tested Gradle versions
0.12.x	4.10.3, 5.6.4, 6.5
0.11.x	4.10.3, 5.6.4, 6.5
0.10.x	4.10.3, 5.6.4, 6.0.1
0.9.x	4.2, 4.10.3, 5.4.1
0.4.x	2.14, 3.0, 4.1

Contributing

This project uses gitflow process. PRs should be done against develop branch
PRs to develop should be style checked/tested locally by running ./gradlew clean check

Building

./gradlew clean build

borissmidt / gradle-crossbuild-scala

gradle crossbuild scala plugin

Features

Shortcomings

Getting the plugin

Using the `plugins` DSL:

Using legacy `buildscript`

Quick start

recommended cross building apply strategy

Multi-module projects and applying cross build plugin only for some

cross building - basic plugin configuration

applying the plugin

notes

cross building with publishing

leveraging gradle maven-publish plugin

Gradle configurations -> Maven Scopes

overriding plugin's internal pom.withXml

notes

cross building - configuration options

Notes

`builds {}` -> Gradle SourceSets, Configurations and Tasks

`implementation` configuration and `java-library` plugin

multi-module project plugin apply patterns

layout 1 (a.k.a lazy apply)

layout 2 (a.k.a eager apply)

Supported Gradle versions

Contributing

Building

About

Languages

gradle crossbuild scala plugin

Features

Shortcomings

Getting the plugin

Using the plugins DSL:

Using legacy buildscript

Quick start

recommended cross building apply strategy

Multi-module projects and applying cross build plugin only for some

cross building - basic plugin configuration

applying the plugin

notes

cross building with publishing

leveraging gradle maven-publish plugin

Gradle configurations -> Maven Scopes

overriding plugin's internal pom.withXml

notes

cross building - configuration options

Notes

builds {} -> Gradle SourceSets, Configurations and Tasks

implementation configuration and java-library plugin

multi-module project plugin apply patterns

layout 1 (a.k.a lazy apply)

layout 2 (a.k.a eager apply)

Supported Gradle versions

Contributing

Building

About

Languages

Using the `plugins` DSL:

Using legacy `buildscript`

`builds {}` -> Gradle SourceSets, Configurations and Tasks

`implementation` configuration and `java-library` plugin