Build services can only be registered if the set of applied plugins is the same on all projects

Question

Build services can only be registered if the set of applied plugins is the same on all projects

melix opened this issue 3 years ago · comments

Expected Behavior

Given a build service:

import org.gradle.api.services.BuildService
import org.gradle.api.services.BuildServiceParameters

abstract class MyService implements BuildService<BuildServiceParameters.None> {
}

a plugin which registers the build service:

def myService = gradle.sharedServices.registerIfAbsent("test", MyService) {
   maxParallelUsages.set(1)
}

and a task which uses the build service:

abstract class MyTask extends DefaultTask {
   @Internal
   abstract Property<MyService> getService()
   
   @TaskAction
   void execute() {
       println(service.get())
   }
}


tasks.register("hello", MyTask) {
   service.set(myService)
}

then the build service should be registered in any case.

Current Behavior

Build service registration fails in multi-project builds, if the projects using the build service have a different set of plugins applied:

* What went wrong:
A problem occurred configuring project ':lib'.
> Could not create task ':lib:hello'.
   > Cannot set the value of task ':lib:hello' property 'service' of type MyService using a provider of type MyService.

The error is confusing, but hints at a classloading issue: the services seem to be cached by classloader, which is by set of applied plugins. Therefore there are 2 different instances of the MyService class.

Context

Initially reported at graalvm/native-build-tools#70

Steps to Reproduce

Using the attached project attached project, cd into consumer then run: ./gradlew hello

Now edit lib/build.gradle and comment out the jmh plugin. Now both projects core and lib have the same set of plugins. Run ./gradlew hello again and see the build pass.

Your Environment

Build scan URL: https://scans.gradle.com/s/fjgx3mlcahdfu
JMH plugin commented out: https://scans.gradle.com/s/nj3oohuwdyx2q

Cédric Champeau · Answer 1 · Mon Jun 28 2021 20:57:37 GMT+0800 (China Standard Time)

Workaround is to use a Provider<Object> on the task side.

Louis Jacomet · Answer 2 · Tue Jun 29 2021 17:11:41 GMT+0800 (China Standard Time)

@gradle/configuration-cache Could you take a look at this one?

Rob Moore · Answer 3 · Fri Apr 22 2022 17:39:18 GMT+0800 (China Standard Time)

@gradle/bt-configuration-cache I encountered this earlier today and used the workaround from @melix (thank you!) which worked well. Now encountering something that looks the same, but for Extension objects:

Extension of type 'AwsDevCredentialsExtension' does not exist. Currently registered extension types: [..., AwsDevCredentialsExtension, ...]

Edit:
I think in in my case it could be because the extension is attempting to be acquired in an applied script i.e. apply(from = another-script.gradle.kts). The extension that gets created by the plugin is attached to a different class loader than the one that is used in the applied script, which is causing the issue here. It looks like this in the logs when I change to doing project.extensions.getByName("awsDevCredentials") as AwsDevCredentialsExtension.

class gebuild.aws.AwsDevCredentialsExtension cannot be cast to class gebuild.aws.AwsDevCredentialsExtension (gebuild.aws.AwsDevCredentialsExtension is in unnamed module of loader org.gradle.internal.classloader.VisitableURLClassLoader @30cb7b03; gebuild.aws.AwsDevCredentialsExtension is in unnamed module of loader org.gradle.internal.classloader.VisitableURLClassLoader @19c59d0e)

Edit 2:
Also affects NamedDomainObjectContainers.

Cédric Champeau · Answer 4 · Thu Oct 27 2022 15:31:20 GMT+0800 (China Standard Time)

Hi team! Do you plan to investigate this one? It is quite problematic as a "well designed" plugin can be facing this issue independently in a multi-project build, so testing wouldn't highlight the problem.

Michael Bailey · Answer 5 · Sat Nov 26 2022 14:32:13 GMT+0800 (China Standard Time)

@melix if you add the plugin to the root build.gradle with apply: false does it workaround the issue?

i.e. in the build.gradle for the root project (parent of core and lib):

plugins {
    id 'my.plugin' apply false
}

Benjamin Muschko · Answer 6 · Fri Jan 13 2023 00:58:19 GMT+0800 (China Standard Time)

Are you planning to provide a fix for this issue? Plugin developers now have to start putting in custom handling to notify users that they are "using it wrong". Or is the assumption that the check will be relaxed based on #23204 (comment)?

Rafael Chaves · Answer 7 · Mon Jan 16 2023 20:58:06 GMT+0800 (China Standard Time)

@bmuschko Just to be clear, the reversal of the check you mentioned only affects the failure mode - the underlying issue described in the present issue still occurs, in one case (as implemented today), at usage time, in the other, at build service registration time (which we reckon is more helpful, and which we plan to re-add in a more conservative way).

The actual issue is on our radar, but no target release at this time.

Clayton Walker · Answer 8 · Wed Jan 18 2023 00:56:35 GMT+0800 (China Standard Time)

the underlying issue described in the present issue still occurs

Is there a public discussion we can follow where progress on this issue can be tracked?

Rafael Chaves · Answer 9 · Wed Jan 18 2023 01:36:44 GMT+0800 (China Standard Time)

@Sineaggi This ticket is that place at this time, afaik.

Cédric Champeau · Answer 10 · Fri Jan 27 2023 20:20:19 GMT+0800 (China Standard Time)

Ping, we're seeing more users report this issue, it's rather annoying.

Tony Robalik · Answer 11 · Wed Mar 01 2023 03:07:34 GMT+0800 (China Standard Time)

We (Square/Block) are seeing this as well.

Archimedes Trajano · Answer 12 · Thu Mar 02 2023 03:58:02 GMT+0800 (China Standard Time)

Ping, we're seeing more users report this issue, it's rather annoying.

In my case, I think it's because of having it part of start.spring.io since I only knew about it through there and it's their first dependency option.

Cédric Champeau · Answer 13 · Tue Mar 07 2023 00:44:02 GMT+0800 (China Standard Time)

The other workaround is to move the plugins to buildSrc, but it's kind of complicated because basically it requires changing all the generated projects, and it's not strictly equivalent...

Cédric Champeau · Answer 14 · Thu May 11 2023 17:51:04 GMT+0800 (China Standard Time)

Another occurrence: micronaut-projects/micronaut-gradle-plugin#706

This is an extremely annoying bug.

David Gregory · Answer 15 · Mon Jul 17 2023 00:37:11 GMT+0800 (China Standard Time)

I ran across this as well when using https://github.com/vanniktech/gradle-maven-publish-plugin in a multi-project build:

Could not create task ':gradle-bsp-server:dropRepository'.
Cannot set the value of task ':gradle-bsp-server:dropRepository' property 'buildService' of type com.vanniktech.maven.publish.sonatype.SonatypeRepositoryBuildService using a provider of type com.vanniktech.maven.publish.sonatype.SonatypeRepositoryBuildService.

It's debilitating in this case because it breaks both IDE sync and CI publishing.

Bencze Balázs · Answer 16 · Wed Aug 09 2023 18:50:08 GMT+0800 (China Standard Time)

Same issue here Triple-T/gradle-play-publisher#1053

Are there any known workarounds?

Torsten Krah · Answer 17 · Wed Aug 09 2023 18:53:32 GMT+0800 (China Standard Time)

Same issue here Triple-T/gradle-play-publisher#1053

Are there any known workarounds?

Yeah - just read the issue, they are mentioned above.

crkurz · Answer 18 · Wed Sep 20 2023 04:34:35 GMT+0800 (China Standard Time)

In order to start using Gradle's config caching feature we need to move to a recent version of gradle-docker-plugin, but that does not work in multi-project setup gradle-docker-plugin/1123 due to GRADLE-17559.

Would anybody have an ETA? We would love to use config caching.

Clayton Walker · Answer 19 · Thu Sep 21 2023 08:55:02 GMT+0800 (China Standard Time)

In order to start using Gradle's config caching feature we need to move to a recent version of gradle-docker-plugin, but that does not work in multi-project setup gradle-docker-plugin/1123 due to GRADLE-17559.

Would anybody have an ETA? We would love to use config caching.

Does the solution mentioned in gradle-docker-plugin work for you? bmuschko/gradle-docker-plugin#1123 (comment)

crkurz · Answer 20 · Thu Sep 21 2023 14:04:07 GMT+0800 (China Standard Time)

Thank you very much for getting back on this, @Sineaggi !

The docker plugin is not directly applied in our Gradle project scripts. Due to product/build size our Gradle sub-projects are using standardized scripts. E.g. any project producing docker images uses apply from: "$rootDir/gradle/docker.gradle" . It is this docker.gradle, which applies the docker plugin via

buildscript {
    dependencies { classpath 'com.bmuschko:gradle-docker-plugin:9.3.2' }
}
...
apply plugin: com.bmuschko.gradle.docker.DockerRemoteApiPlugin

The problem for us with the above mentioned work-around is that plugins section is not allowed in scripts. It causes Only Project and Settings build scripts can contain plugins {} blocks.

So my understanding is that we only have three options

restructure overall build in all our repos (not an option, as we need standardisation)
convert docker.gradle into a binary plugin (unlikely that we will get funding for such refactoring (across dozens of repos) plus having a binary plugin makes change process harder.)
wait for this bug to be addressed (obviously our preferred option :-) )

Can you see any other options? or is there some veeeery rough ETA?

Update: it appears that just by adding the suggested plugins section into each of our project scripts works around the issue and still allows us to continue using our docker.gradle script plugin. - Not perfect, but definitely acceptable.

Update: Still stuck: putting plugins section into root project is now giving me:

Cannot query the value of task ':dataflow-service:dockerBuildImage' property 'dockerClientService' because it has no value available.

during execution phase. Tried many variations.

Archimedes Trajano · Answer 21 · Thu Sep 21 2023 21:15:09 GMT+0800 (China Standard Time)

@crkurz I am trying to work around all the Gradle Docker plugin issues by building within Docker and letting it manage the build cycle. The main issues I generally face are

caching of dependencies (too many copies and the Gradle's inability to share a common set of deps... you need the workaround with the read-only cache)
TestContainer testing because it's a container inside a container knowing about the container testing. It's possible but not straight forward. In which case I just delegate the tests outside and let Gradle handle that part.

mjustin · Answer 22 · Thu Jan 18 2024 04:26:07 GMT+0800 (China Standard Time)

@melix if you add the plugin to the root build.gradle with apply: false does it workaround the issue?

i.e. in the build.gradle for the root project (parent of core and lib):
plugins {
    id 'my.plugin' apply false
}

I can confirm that adding all my plugins to the root build.gradle with apply: false works around the issue in my own project.