Overlapping outputs prevents gradle task cacheability

Question

Overlapping outputs prevents gradle task cacheability

plastiv opened this issue 5 years ago · comments

Sergii Pechenizkyi commented 5 years ago

ComposerTask is marked as @CacheableTask and so is participating in gradle cache lifecycle but is not cooperative.

We are observing next crashes on our CI pipelines

org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':storage:testDebugComposer'.Open stacktrace
Caused by: java.lang.RuntimeException: Failed to store cache entry for task ':storage:testDebugComposer'Open stacktrace
Caused by: org.gradle.api.GradleException: Could not pack tree 'outputDir': java.io.IOException: Request to write '61599' bytes exceeds size in header of '913483' bytes for entry 'tree-outputDir/logs/emulator-5554/full.logcat'Open stacktrace
Caused by: org.gradle.api.UncheckedIOException: java.io.IOException: Request to write '61599' bytes exceeds size in header of '913483' bytes for entry 'tree-outputDir/logs/emulator-5554/full.logcat'Open stacktrace
Caused by: java.io.IOException: Request to write '61599' bytes exceeds size in header of '913483' bytes for entry 'tree-outputDir/logs/emulator-5554/full.logcat'Close stacktrace
at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.write(TarArchiveOutputStream.java:449)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2315)
at org.gradle.caching.internal.packaging.impl.TarBuildCacheEntryPacker$PackingVisitor.storeFileEntry(TarBuildCacheEntryPacker.java:457)
at org.gradle.caching.internal.packaging.impl.TarBuildCacheEntryPacker$PackingVisitor.visitFile(TarBuildCacheEntryPacker.java:389)
at org.gradle.internal.snapshot.RegularFileSnapshot.accept(RegularFileSnapshot.java:60)
at org.gradle.internal.snapshot.DirectorySnapshot.accept(DirectorySnapshot.java:58)
at org.gradle.internal.snapshot.DirectorySnapshot.accept(DirectorySnapshot.java:58)
at org.gradle.internal.snapshot.DirectorySnapshot.accept(DirectorySnapshot.java:58)
at org.gradle.internal.snapshot.CompositeFileSystemSnapshot.accept(CompositeFileSystemSnapshot.java:40)
at org.gradle.internal.fingerprint.impl.DefaultCurrentFileCollectionFingerprint.accept(DefaultCurrentFileCollectionFingerprint.java:118)
at org.gradle.caching.internal.packaging.impl.TarBuildCacheEntryPacker.packTree(TarBuildCacheEntryPacker.java:148)
at org.gradle.caching.internal.packaging.impl.TarBuildCacheEntryPacker.lambda$pack$1(TarBuildCacheEntryPacker.java:138)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution.visitOutputTrees(ExecuteActionsTaskExecuter.java:332)
at org.gradle.caching.internal.packaging.impl.TarBuildCacheEntryPacker.pack(TarBuildCacheEntryPacker.java:135)
at org.gradle.caching.internal.packaging.impl.TarBuildCacheEntryPacker.pack(TarBuildCacheEntryPacker.java:120)
at org.gradle.caching.internal.packaging.impl.GZipBuildCacheEntryPacker.pack(GZipBuildCacheEntryPacker.java:42)
at org.gradle.caching.internal.command.BuildCacheCommandFactory$StoreCommand.store(BuildCacheCommandFactory.java:173)
at org.gradle.caching.internal.controller.DefaultBuildCacheController$Pack$1.run(DefaultBuildCacheController.java:205)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:402)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:394)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:92)
at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
at org.gradle.caching.internal.controller.DefaultBuildCacheController$Pack.execute(DefaultBuildCacheController.java:201)
at org.gradle.caching.internal.controller.DefaultBuildCacheController.lambda$store$1(DefaultBuildCacheController.java:179)
at org.gradle.caching.local.internal.DefaultBuildCacheTempFileStore.withTempFile(DefaultBuildCacheTempFileStore.java:46)
at org.gradle.caching.internal.controller.DefaultBuildCacheController.store(DefaultBuildCacheController.java:178)
at org.gradle.internal.execution.steps.CacheStep.store(CacheStep.java:139)
at org.gradle.internal.execution.steps.CacheStep.lambda$executeAndStoreInCache$6(CacheStep.java:131)
at org.gradle.internal.Try$Success.ifSuccessfulOrElse(Try.java:187)
at org.gradle.internal.execution.steps.CacheStep.executeAndStoreInCache(CacheStep.java:130)
at org.gradle.internal.execution.steps.CacheStep.lambda$executeWithCache$2(CacheStep.java:107)
at org.gradle.internal.execution.steps.CacheStep.lambda$executeWithCache$3(CacheStep.java:107)
at org.gradle.internal.Try$Success.map(Try.java:162)
at org.gradle.internal.execution.steps.CacheStep.executeWithCache(CacheStep.java:76)
at org.gradle.internal.execution.steps.CacheStep.execute(CacheStep.java:66)
at org.gradle.internal.execution.steps.CacheStep.execute(CacheStep.java:41)
at org.gradle.internal.execution.steps.BroadcastChangingOutputsStep.execute(BroadcastChangingOutputsStep.java:49)
at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:44)
at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:33)
at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:38)
at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:24)
at org.gradle.internal.execution.steps.SkipUpToDateStep.executeBecause(SkipUpToDateStep.java:92)
at org.gradle.internal.execution.steps.SkipUpToDateStep.lambda$execute$0(SkipUpToDateStep.java:85)
at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:55)
at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:39)
at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:76)
at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:37)
at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:36)
at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:26)
at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:94)
at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:49)
at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:79)
at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:53)
at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:74)
at org.gradle.internal.execution.steps.SkipEmptyWorkStep.lambda$execute$2(SkipEmptyWorkStep.java:78)
at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:78)
at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:34)
at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsStartedStep.execute(MarkSnapshottingInputsStartedStep.java:39)
at org.gradle.internal.execution.steps.LoadExecutionStateStep.execute(LoadExecutionStateStep.java:40)
at org.gradle.internal.execution.steps.LoadExecutionStateStep.execute(LoadExecutionStateStep.java:28)
at org.gradle.internal.execution.impl.DefaultWorkExecutor.execute(DefaultWorkExecutor.java:33)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:174)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:166)
at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:109)
at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:62)
at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:416)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:406)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:102)
at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:41)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:374)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:361)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:354)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:340)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.lambda$run$0(DefaultPlanExecutor.java:127)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:191)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:182)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:124)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)

Running with org.gradle.caching.debug=true results in next warnings

> Task :storage:testDebugComposer
Non-cacheable because Gradle does not know how file 'build/reports/composer/debug/logs/emulator-5554/full.logcat' was created (output property 'outputDir'). Task output caching requires exclusive access to output paths to guarantee correctness. [OVERLAPPING_OUTPUTS]
Appending implementation to build cache key: com.trevjonez.composer.ComposerTask_Decorated@01ed0e65ef381133e7c3807e9a4dde77
Appending additional implementation to build cache key: com.trevjonez.composer.ComposerTask_Decorated@dda74d08a08ed49fa4e0f81bf5e6fefc
Appending input value fingerprint for 'apkInstallTimeout' to build cache key: f6bd6b3389b872033d462029172c8612
Appending input value fingerprint for 'args' to build cache key: 8222d82255460164427051d7537fa305
Appending input value fingerprint for 'debug' to build cache key: c06857e9ea338f3f3a24bb78f8fbdf6f
Appending input value fingerprint for 'debugOptions' to build cache key: 9b6eb7ad5d057645a33e4188a4ef73bb
Appending input value fingerprint for 'debugOptions.enabled' to build cache key: c06857e9ea338f3f3a24bb78f8fbdf6f
Appending input value fingerprint for 'debugOptions.port' to build cache key: d83d6605a42e65ae05e59d11497bd8b1
Appending input value fingerprint for 'debugOptions.server' to build cache key: f6d7ed39fe24031e22d54f3fe65b901c
Appending input value fingerprint for 'debugOptions.suspend' to build cache key: f6d7ed39fe24031e22d54f3fe65b901c
Appending input value fingerprint for 'defaultCharacterEncoding' to build cache key: d746f44d09fb58d2971f341d24d74c35
Appending input value fingerprint for 'devicePattern' to build cache key: f6bd6b3389b872033d462029172c8612
Appending input value fingerprint for 'devices' to build cache key: 8222d82255460164427051d7537fa305
Appending input value fingerprint for 'enableAssertions' to build cache key: c06857e9ea338f3f3a24bb78f8fbdf6f
Appending input value fingerprint for 'ignoreExitValue' to build cache key: c06857e9ea338f3f3a24bb78f8fbdf6f
Appending input value fingerprint for 'instrumentationArguments' to build cache key: 3adf042dcd5b07446bff6de870596db9
Appending input value fingerprint for 'javaVersion' to build cache key: 4a465f4e625f50133941afa67afe36be
Appending input value fingerprint for 'jvmArgs' to build cache key: 8222d82255460164427051d7537fa305
Appending input value fingerprint for 'keepOutput' to build cache key: f6bd6b3389b872033d462029172c8612
Appending input value fingerprint for 'main' to build cache key: f6bd6b3389b872033d462029172c8612
Appending input value fingerprint for 'maxHeapSize' to build cache key: f6bd6b3389b872033d462029172c8612
Appending input value fingerprint for 'minHeapSize' to build cache key: f6bd6b3389b872033d462029172c8612
Appending input value fingerprint for 'shard' to build cache key: f6bd6b3389b872033d462029172c8612
Appending input value fingerprint for 'systemProperties' to build cache key: cfcc5f871f42c7c797557fd17748dc17
Appending input value fingerprint for 'verboseOutput' to build cache key: f6bd6b3389b872033d462029172c8612
Appending input value fingerprint for 'withOrchestrator' to build cache key: f6bd6b3389b872033d462029172c8612
Appending input file fingerprints for 'apk' to build cache key: aa70ca66ca60f2ffc8ea316ae72c1d7f
Appending input file fingerprints for 'bootstrapClasspath' to build cache key: 5fd1e7396e8de4cb5c23dc6aadd7787a
Appending input file fingerprints for 'classpath' to build cache key: 5fd1e7396e8de4cb5c23dc6aadd7787a
Appending input file fingerprints for 'extraApks' to build cache key: 5fd1e7396e8de4cb5c23dc6aadd7787a
Appending input file fingerprints for 'testApk' to build cache key: aa70ca66ca60f2ffc8ea316ae72c1d7f
Appending output property name to build cache key: outputDir

Trevor Jones · Answer 1 · Fri Dec 20 2019 02:19:27 GMT+0800 (China Standard Time)

I don't believe this is actually an overlapping output based on the output path. But rather the logcat file being change outside of the time Gradle expects it to be.

What versions of adb / composer / this plugin / Gradle / Android Gradle plugin produced this? I'll try to repro on my end

Sergii Pechenizkyi · Answer 2 · Fri Dec 20 2019 04:24:33 GMT+0800 (China Standard Time)

My environment is:

Gradle 6.0.1
AGP 3.5.3
composer-gradle-plugin 0.13.1
composer 0.6.0

And I've currently applied composer plugin to 2 submodules:

rootprojectdir
├── main
│   ├── build.gradle
├── storage
│   ├── build.gradle

And try to run composer for both of them by executing ./gradlew testDebugComposer from root dir.

Composer config

apply plugin: 'com.trevjonez.composer'

composer {
    instrumentationArgument('disableAnalytics', 'true') // google reports amount of test runs by default
    if (project.hasProperty('ORCHESTRATOR')) {
        withOrchestrator true
        instrumentationArgument('clearPackageData', 'true') // orchestrator param
    }
}

dependencies {
    if (project.hasProperty('ORCHESTRATOR')) {
        androidTestUtil libraries.testOrchestrator
    }
    composer libraries.composer
}

NOTE: if I don't apply plugin: 'com.trevjonez.composer' to one of the modules the warning is gone.

Trevor Jones · Answer 3 · Fri Dec 20 2019 13:46:31 GMT+0800 (China Standard Time)

thank you for such a thorough response. i'll see if I can dig into this over the holidays. Perhaps the fix here might also fix #26 , smells maybe related.

Trevor Jones · Answer 4 · Tue Jan 21 2020 09:21:52 GMT+0800 (China Standard Time)

@plastiv I pushed rc04 which I believe should correct this issue. The thinking is that adb was left running in the background long enough to cause issues. the change today forcefully kills the spawned process during the rx disposal.

Trevor Jones · Answer 5 · Thu Jan 23 2020 01:01:45 GMT+0800 (China Standard Time)

should be fully resolved in rc05 based on feedback from others experiencing this issue periodically.

if not feel free to let me know here and we can reopen the investigation.