Copy tasks do not consider filter/eachFile/expansion properties in up-to-date checks

[Lóránt Pintér]

@lptr commented on Tue Jul 12 2016

Code review: REVIEW-6089

See JIRA issues:

• GRADLE-1646
• GRADLE-1276
• GRADLE-1298

GitHub issues:

• #861
• #1015
• #4729

---

@lptr commented on Tue Jul 12 2016

Added tests in e42e103

---

@lptr commented on Tue Jul 12 2016

Discussion on Slack with @melix about deprecating the imperative methods on CopySpec like filter(Closure): https://gradle.slack.com/archives/build-tool/p1468322257003567

---

@lptr commented on Thu Jan 12 2017

Since we don't want to cache archive or copy tasks, we are moving issues related to them to the main Gradle board. This issue is already reported on the main board (see description for links).

[Jörn Huxhorn]

@lptr Consider adding my test in #861 regarding nested (for lack of a better word) expands, i.e. the filesMatching '**/*.properties', { expand expandMap } part.

[Lóránt Pintér]

@huxi do you think we should have them besides these? e42e103

[Jörn Huxhorn]

@lptr Yes.
I was able to fix the first of the two tests in #861 but that change wouldn't have affected the second test scenario where the expand was a parameter to filesMatching.

The straight-forward idea of adding the line getInputs().properties(properties); to AbstractCopyTask.expand(Map<String, ?> properties) fixes the first but not the second test.

[Riko Eksteen]

Are there any plans to fix this issue, or a workaround I can use in the interim? How can I get my build to update the application.properties file if one of the input variables have changes?

[Lóránt Pintér]

As a workaround you can register the input variables via task.inputs.property().

[Riko Eksteen]

Thank you @lptr.

So I have:

processResources {
	filesMatching('application.properties') {
		expand(project.properties)
	}
}

If I change it to

processResources {
	inputs.properties(project.properties)
	filesMatching('application.properties') {
		expand(project.properties)
	}
}

I get this error:

Could not add entry ':processResources' to cache taskHistory.bin (C:\Development\...\.gradle\3.4\taskHistory\taskHistory.bin).
> Unable to store task input properties. Property 'classLoaderScope' with value 'org.gradle.api.internal.initialization.DefaultClassLoaderScope@2b00d37' cannot be serialized.

Any idea what is wrong? I get the same error if I use inputs.property with only the single property I want to add to the inputs (which is just a string).

I am using Gradle 3.4

[Lóránt Pintér]

project.properties contains a lot of things, some of which is not serializable, and thus we can't track changes to them. You could take the properties that you know will be used during expansion (say, propA and prop2), and add them specifically:

processResources {
	def expandProps = ["propA": propA, "prop2": prop2]
	inputs.properties(expandProps)
	filesMatching('application.properties') {
		expand(expandProps)
	}
}

[Chad Beaulac]

Using inputs.properties(props) to copy when filters change saved me! This took awhile to find. Thank you!

[Sterling Greene]

@wolfs Now that we have polymorphic inputs, can this be tackled more easily?

[Lóránt Pintér]

I think we could handle the CopySpec methods that don't take a Closure/Action (like eachFile {} does) now, yes. We could also handle the ones with the actions, but they would produce a lot of false cache misses in the typical case when the action is declared in the build script. Still, for a lot of use cases we could be doing things better.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. Given the limited bandwidth of the team, it will be automatically closed if no further activity occurs. If you're interested in how we try to keep the backlog in a healthy state, please read our blog post on how we refine our backlog. If you feel this is something you could contribute, please have a look at our Contributor Guide. Thank you for your contribution.

[stale]

This issue has been automatically closed due to inactivity. If you can reproduce this on a recent version of Gradle or if you have a good use case for this feature, please feel free to reopen the issue with steps to reproduce, a quick explanation of your use case or a high-quality pull request.

[Zak Taccardi]

we don't want to cache archive or copy tasks

Why not? My Sync tasks are generally just merging output from one or more tasks into a new folder

[Vladimir Sitnikov]

I guess the idea is that "build cache" would require Gradle to download the entry from build cache and unpack it.
It would take the same or even extra time as just performing copy/archive actions locally.

[Silvio Giebl]

While implementing a custom task type with similar inputs than a copy/archive task, I wanted to get inspired by the implementation of the Gradle types and it was unexpected that renaming is not properly implemented regarding up-to-date checks.
I think that at least a warning is missing in the documentation, with a note that the users currently need to workaround by declaring inputs themselves (for example on the working with files page).
Correct me if I was just too blind to find a note about this topic in the documentation.

[Arjen Jonathan]

This is actually an annoying issue to work around. Typically if something is not marked as an input one can just add it to the inputs of a task explicitly, but because CopySpec is also used outside of concrete tasks (for example, the distribution extension has a copyspec), this is not possible here.

In such cases you are entirely dependent on the copyspec interface to achieve sound up-to-date checking, and it is hard to find a suitable alternative to "rename" and friends.