apache.commons.io version collision with the one used by spring-boot-gradle-plugin

Question

apache.commons.io version collision with the one used by spring-boot-gradle-plugin

jvmlet opened this issue 5 years ago · comments

Hi

Looks like apache.commons.io you are using contradicts with version that spring boot plugin uses.
Here is my gradle.build;

buildscript {
    repositories {
        mavenCentral()
        jcenter()
    }
    dependencies {
        classpath("org.springframework.boot:spring-boot-gradle-plugin:2.1.1.RELEASE")
        classpath("name.valery1707.kaitai:kaitai-gradle-plugin:0.1.1")
    }
}

apply plugin: 'java'
apply plugin: 'org.springframework.boot'
apply plugin: 'io.spring.dependency-management'
apply plugin: 'name.valery1707.kaitai'

When I run kaitai task with buggy ksy I get this :

Caused by: java.lang.NoSuchMethodError: org.apache.commons.io.output.ByteArrayOutputStream.toString(Ljava/nio/charset/Charset;)Ljava/lang/String;
        at name.valery1707.kaitai.KaitaiGenerator.execute(KaitaiGenerator.java:224)
        at name.valery1707.kaitai.KaitaiGenerator.generate(KaitaiGenerator.java:265)
        at name.valery1707.kaitai.KaitaiGenerator$generate$4.call(Unknown Source)
        at name.valery1707.kaitai.GenerateTask.action(GenerateTask.groovy:111)

You can consider to repackage your dependencies with your unique package name.

valery1707 · Answer 1 · Mon Jan 28 2019 16:38:37 GMT+0800 (China Standard Time)

I trying to reproduce your's problem in https://gist.github.com/valery1707/65e14e9dcab89460859c339a4793db8d, but I can't to receive such message and successfully receiver original message.

I think that repacking dependencies inside my maven plugin is not good idea.

In next version I will refuse to use the problematic method.

But will be good if I can reproduce problem in tests or local project to to make sure the problem is completely gone.
Can you create minimal gradle project in which your's problem is reproduced?

valery1707 · Answer 2 · Mon Jan 28 2019 17:23:09 GMT+0800 (China Standard Time)

I can reproduce this problem if downgrade commons-io from 2.5 (released at 2016-04-22) to 2.4 (released at 2012-06-13) I don't think that org.springframework.boot:spring-boot-gradle-plugin:2.1.1.RELEASE released at 2018-11-30 really use such old version of commons-io library.

Furer Alexander · Answer 3 · Mon Jan 28 2019 18:21:15 GMT+0800 (China Standard Time)

Here are additional classpath dependencies (were defined in root project, forgot to add them)

classpath 'org.ajoberstar.grgit:grgit-gradle:3.0.0'
classpath "org.jfrog.buildinfo:build-info-extractor-gradle:4+"
classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.7'

Can you please try to reproduce with this config :

buildscript {
    repositories {
        mavenCentral()
        jcenter()
    }
    dependencies {
        classpath("org.springframework.boot:spring-boot-gradle-plugin:2.1.1.RELEASE")
        classpath("name.valery1707.kaitai:kaitai-gradle-plugin:0.1.1")
       classpath 'org.ajoberstar.grgit:grgit-gradle:3.0.0'
       classpath "org.jfrog.buildinfo:build-info-extractor-gradle:4+"
       classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.7'
    }
}

Furer Alexander · Answer 4 · Mon Jan 28 2019 18:22:42 GMT+0800 (China Standard Time)

Not sure if the order is important :

dependencies {
     classpath 'org.ajoberstar.grgit:grgit-gradle:3.0.0'
       classpath "org.jfrog.buildinfo:build-info-extractor-gradle:4+"
       classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.7'
        classpath("org.springframework.boot:spring-boot-gradle-plugin:2.1.1.RELEASE")
        classpath("name.valery1707.kaitai:kaitai-gradle-plugin:0.1.1")
  
    }

valery1707 · Answer 5 · Mon Jan 28 2019 19:34:14 GMT+0800 (China Standard Time)

No, problem is not reproduced with this dependencies.
But I already released new version on kaitai-maven-plugin which don't use method from commons-io@2.5 and soon will release new version of kaitai-gradle-plugin.

valery1707 · Answer 6 · Tue Jan 29 2019 02:03:31 GMT+0800 (China Standard Time)

Version 0.1.2 was released - please test does it works for you

Furer Alexander · Answer 7 · Tue Jan 29 2019 18:43:57 GMT+0800 (China Standard Time)

Doesn't work, with buggy ksy I'm getting
Stderr unavailable as it has been consumed by user provided stream.

valery1707 · Answer 8 · Tue Jan 29 2019 18:45:02 GMT+0800 (China Standard Time)

Can you show full error? And run gradle with --stacktrace?

valery1707 · Answer 9 · Tue Jan 29 2019 18:59:11 GMT+0800 (China Standard Time)

Because of issue kaitai-io/kaitai_struct#509, the process launcher org.buildobjects.process.Proc will never receive the message text from stderror - it is written in stdout, at least in version 0.8.
I extracting this message manually and show it in the message of upper level exception.

valery1707 · Answer 10 · Tue Jan 29 2019 19:06:10 GMT+0800 (China Standard Time)

After switch from showing raw error text to use machine-readable error log I will have to use stdout anyway (as I see in JavaMain.scala#263).

Furer Alexander · Answer 11 · Tue Jan 29 2019 19:08:04 GMT+0800 (China Standard Time)

The full stack trace :

C:\XXXX>gradlew -S kaitai
> Task :parsers:cfb-parser:kaitai FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':parsers:cfb-parser:kaitai'.
> name.valery1707.kaitai.KaitaiException: Fail to generate Java files

* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':parsers:cfb-parser:kaitai'.
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:96)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:65)
        at org.gradle.api.internal.tasks.execution.ActionEventFiringTaskExecuter.execute(ActionEventFiringTaskExecuter.java:44)
        at org.gradle.api.internal.tasks.execution.TimeoutTaskExecuter.execute(TimeoutTaskExecuter.java:53)
        at org.gradle.api.internal.tasks.execution.SnapshotAfterExecutionTaskExecuter.execute(SnapshotAfterExecutionTaskExecuter.java:38)
        at org.gradle.api.internal.tasks.execution.OutputDirectoryCreatingTaskExecuter.execute(OutputDirectoryCreatingTaskExecuter.java:51)
        at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:59)
        at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:49)
        at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:61)
        at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:101)
        at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:91)
        at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:44)
        at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:62)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:55)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54)
        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:35)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.run(EventFiringTaskExecuter.java:49)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:301)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:293)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:175)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:91)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:44)
        at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:43)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:337)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:325)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:318)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:304)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker$1.execute(DefaultPlanExecutor.java:134)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker$1.execute(DefaultPlanExecutor.java:129)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:202)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:193)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:129)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.gradle.internal.UncheckedException: name.valery1707.kaitai.KaitaiException: Fail to generate Java files
        at org.gradle.internal.UncheckedException.throwAsUncheckedException(UncheckedException.java:66)
        at org.gradle.internal.UncheckedException.throwAsUncheckedException(UncheckedException.java:40)
        at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:76)
        at org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:48)
        at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:41)
        at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:28)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:704)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:671)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.run(ExecuteActionsTaskExecuter.java:117)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:301)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:293)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:175)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:91)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:106)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:85)
        ... 38 more
Caused by: name.valery1707.kaitai.KaitaiException: Fail to generate Java files
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:83)
        at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:105)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:257)
        at name.valery1707.kaitai.GenerateTask.action(GenerateTask.groovy:119)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:73)
        ... 51 more
Caused by: name.valery1707.kaitai.KaitaiException: Fail to execute kaitai command: /meta/id: invalid meta ID: 'Connectioninfo3270', expected /^[a-z][a-z0-9_]*$/
        at name.valery1707.kaitai.KaitaiGenerator.execute(KaitaiGenerator.java:225)
        at name.valery1707.kaitai.KaitaiGenerator.generate(KaitaiGenerator.java:265)
        at name.valery1707.kaitai.KaitaiGenerator$generate$4.call(Unknown Source)
        at name.valery1707.kaitai.GenerateTask.action(GenerateTask.groovy:111)
        ... 56 more
Caused by: org.buildobjects.process.ExternalProcessFailureException: External process 'C:\XXXXXX\build\tmp\kaitai-cache\kaitai-struct-compiler-0.8\kaitai-struct-compiler-0.8\bin\kaitai-struct-compiler.bat
--target java --outdir C:\XXXX\build\generated\kaitai --java-package xxx.xxx.xxx.xxx.xxx C:\XXXXX\3270.ksy C:\XXXX
b-parser\src\main\resources\kaitai\connectioninfo3270.ksy' returned 2 after 1441ms
Stderr unavailable as it has been consumed by user provided stream.
        at org.buildobjects.process.Proc.<init>(Proc.java:124)
        at org.buildobjects.process.ProcBuilder.run(ProcBuilder.java:205)
        at name.valery1707.kaitai.KaitaiGenerator.execute(KaitaiGenerator.java:220)
        ... 59 more


* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.0/userguide/command_line_interface.html#sec:command_line_warnings

Without -S switch I get

* What went wrong:
Execution failed for task ':parsers:cfb-parser:kaitai'.
> name.valery1707.kaitai.KaitaiException: Fail to generate Java files

I would expect to see the real cause :

Fail to execute kaitai command: /meta/id: invalid meta ID: 'Connectioninfo3270', expected /^[a-z][a-z0-9_]*$/

valery1707 · Answer 12 · Tue Jan 29 2019 19:13:43 GMT+0800 (China Standard Time)

Thanks for the log.
I will try to minimize intermediate errors so that without a -S option, a clear message is displayed immediately.

valery1707 · Answer 13 · Tue Jan 29 2019 21:46:57 GMT+0800 (China Standard Time)

Please test with version 0.1.3 (already released)

Furer Alexander · Answer 14 · Tue Jan 29 2019 23:07:53 GMT+0800 (China Standard Time)

Great, thanks a lot

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':parsers:cfb-parser:kaitai'.
> name.valery1707.kaitai.KaitaiException: Fail to execute kaitai command: /meta/id: invalid meta ID: 'Connectioninfo', expected /^[a-z][a-z0-9_]*$/