ignore `$buildDir/generated/source`

Question

ignore `$buildDir/generated/source`

xenoterracide opened this issue 4 years ago · comments

so... due to craziness maybe have an inclusive option to ignore generated code? I have this

    disableWarningsInGeneratedCode.set(true)
    excludedPaths.set("$buildDir/generated/sources/.*")

I'm still not certain that how I'm specifying the path for excluded paths is the best way to do it... but it might be nice to have a

disableGeneratedCodeCheck()

or some such, that does this for you. Also it might be good to change excludedPaths to a collection, and then you can join them yourself, that way the above would "add" it to the collection, and not simply override it. at minimum might be good to document how to exclude this directory.

Thomas Broyer · Answer 1 · Thu Dec 10 2020 17:48:58 GMT+0800 (China Standard Time)

If you configure the property with $buildDir, then your build is not relocatable, but that should otherwise work. I would prefer ".*/build/generated/sources/.*" though.

A more generic approach would be to use something like the following so you're not hard-coding any conventional path (that will break if the directory is reconfigured to something outside the project directory, but that's very unlikely to ever happen):

excludedPaths.set(generatedSourceOutputDirectory.map { """.*/\Q${relativePath(it)}\E/*""" })

Maybe that last bit could be added to the README; not sure it's compatible with configuration cache though (and with Windows BTW), so maybe an excludeGeneratedSourceOutputDirectory() would help here…

Caleb Cushing · Answer 2 · Thu Dec 10 2020 23:59:14 GMT+0800 (China Standard Time)

$buildDir is set by gradle so why would it not be relocatable? excludeGeneratedSourceOutputDirectory() would it be possible to use a sourceset? I don't really know how to do that in that case...

Thomas Broyer · Answer 3 · Fri Dec 11 2020 01:29:06 GMT+0800 (China Standard Time)

$buildDir is an absolute path, so you're setting excludedPaths to something like /home/xenoterracide/Projects/myproject/build/generated/sources/*. This value will be used in cache keys, so moving your project around (or using a shared build cache between developers/machines) will generate cache misses.

Using Project#relativePath() on CompileOptions#generatedSourceOutputDirectory makes it possible to generate a value of .*/\Qbuild/generated/sources/annotationProcessor/java/main\E/.* (for the compileJava task using default/conventional values) which then is relocatable (as it's relative to your project's directory).

The excludeGeneratedSourceOutputDirectory() I was talking about would just be a builtin (to the plugin) equivalent to the code using relativePath and generatedSourceOutputDirectory, with possible additional tweaks for Windows compatibility (in retrospect, I think that code should be compatible with the configuration cache). I'm not going to rush adding such a helper though, as I think it's rather an edge-case (I personally prefer disableWarningsInGeneratedCode, along with -Werror so warnings in non-generated code are turned into errors), and .*/build/generated/sources/.* or .*/build/generated/sources/annotationProcessor/.* should be enough for most people otherwise.

If the relativePath+generatedSourceOutputDirectory works, I could however possibly add it to the README, along with the alternatives and their pros and cons.

Caleb Cushing · Answer 4 · Fri Dec 11 2020 03:51:39 GMT+0800 (China Standard Time)

$buildDir is an absolute path, so you're setting excludedPaths to something like

except if I move the project to another directory wouldn't that variable update?

disableWarningsInGeneratedCode

this did not work for me with the immutables library, at all. hence this path. I filed an issue with error prone.

Thomas Broyer · Answer 5 · Fri Dec 11 2020 04:38:50 GMT+0800 (China Standard Time)

$buildDir is an absolute path, so you're setting excludedPaths to something like

except if I move the project to another directory wouldn't that variable update?

Yes, and that's what lead to cache misses, because the property value depends on where the project lives.
Might not be a problem in your case though.
Only leads to task reruns when their outputs could have been restored from cache.

disableWarningsInGeneratedCode

this did not work for me with the immutables library, at all. hence this path. I filed an issue with error prone.

Didn't remember the details but might be due to checks that you promoted to error. Having them as warnings would get them ignored, and -Werror would promote the ones in your own code to errors. Doesn't fit all setups, depending one whether you'd like to keep some checks as mere warnings.

Caleb Cushing · Answer 6 · Fri Dec 11 2020 05:14:59 GMT+0800 (China Standard Time)

Not a fan of warnings, I should be releasing this code as open source in the following week or so, do you want to look at why this was the case? otherwise we can close this.

Thomas Broyer · Answer 7 · Fri Dec 11 2020 05:31:46 GMT+0800 (China Standard Time)

Not a fan of warnings

Me neither, hence -Werror; in which case you can configure ErrorProne checks as warnings (promoted to errors by JavaC in the end) such that they'll be ignored for generated code.

Caleb Cushing · Answer 8 · Fri Dec 11 2020 06:02:13 GMT+0800 (China Standard Time)

-Werror would promote the ones in your own code to errors.

ok, then maybe I don't understand, how would this not catch the generated code, vs the fact that I just have an incredibly long list of stuff I made errors.

Thomas Broyer · Answer 9 · Fri Dec 11 2020 06:25:46 GMT+0800 (China Standard Time)

When a check emit a failure, if it's configured as a warning, then ErrorProne looks if it's in generated code. If so, it'll omit the warning (disableWarningsInGeneratedCode), otherwise it emits it as usual.
Later on, at the end of the compilation, if a warning has been emitted (which can only be the case for non-generated code then) then JavaC reports a compilation failure (-Werror)

Caleb Cushing · Answer 10 · Fri Dec 11 2020 06:30:11 GMT+0800 (China Standard Time)

hmm... this command line is confusing... are experimental warnings warnings by default?

Caleb Cushing · Answer 11 · Sat Dec 12 2020 01:18:47 GMT+0800 (China Standard Time)

would you consider having disableWarningsInGeneratedCode also enabling -WError? or maybe have a property just for it and document that you need to do both.

Thomas Broyer · Answer 12 · Sat Dec 12 2020 02:30:59 GMT+0800 (China Standard Time)

The goal of this plugin is only to make it easier to configure ErrorProne, I don't want to add features or even helpers unless ones would be really needed due to usage in Gradle (I haven't seen any need for that), as anything that "automatically does X when you do Y" (particularly if Y has the same name as the Y in ErrorProne itself) could cause issues for some people so needs to be disableable, etc.
Put differently, the plugin aims at answering "how do I pass X to ErrorProne?", and not raising any question like "if I do Y with the plugin, what ErrorProne arguments does it really map to?"

So no, disableWarningsInGeneratedCode will never enable -Werror, it'll always only map to ErrorProne's -XepDisableWarningsInGeneratedCode.

I'm not opposed to an excludeGeneratedSourceOutputDirectory() but I don't think it's a good idea. As I said, for most people, excludePaths.set(".*/build/generated/sources/annotationProcessor/.*") will be enough; being smarter (such as basing the value on the $buildDir and/or generatedSourceOutputDirectory) can only be actually useful inside other plugins that want to be safe in case the project reconfigures those directories.
And as I said above, as soon as someone wants to exclude more (or less) code, they'll have to revert to using excludePaths directly anyway.

Regarding documentation, that means that the ErrorProne documentation by itself should be enough, and the plugin's documentation answers the "how do I do it with Gradle?" question. It could however add some documentation for things that are specific to Gradle, so documenting the .*/build/generated/sources/annotationProcessor/.* value could have its place there.

Caleb Cushing · Answer 13 · Sat Dec 12 2020 02:42:07 GMT+0800 (China Standard Time)

ok, I tried this, did I put the comment somewhere else?

    var foo = 2;
    var bar = 3;
    bar = 4;
    var overwrite = skel.getOverwrite();
    overwrite.booleanValue();

tasks.withType<JavaCompile>().configureEach {
  options.compilerArgs.addAll(listOf("-parameters", "-Xlint:deprecation", "-Xlint:unchecked"))

  options.errorprone {
    nullaway {
      severity.set(CheckSeverity.WARN)
      acknowledgeRestrictiveAnnotations.set(true)
      handleTestAssertionLibraries.set(true)
      checkOptionalEmptiness.set(true)
    }
    errorproneArgs.set(listOf("-Werror"))
    disableWarningsInGeneratedCode.set(true)
    excludedPaths.set("$buildDir/generated/sources/.*")
    error(
      "AmbiguousMethodReference",
       ...
    )
    warn(
      "Var" // I'd love for this to be inn the default error list, but it experimental :'(
    )

they're all warnings not errors

/Users/calebcushing/IdeaProjects/ppm/scaf/src/main/java/com/xenoterracide/scaf/PebbleTemplateProcessor.java:90: warning: [UnusedVariable] The local variable 'bar' is never read.
    var bar = 3;
        ^
    (see https://errorprone.info/bugpattern/UnusedVariable)
  Did you mean to remove this line?
/Users/calebcushing/IdeaProjects/ppm/scaf/src/main/java/com/xenoterracide/scaf/PebbleTemplateProcessor.java:89: warning: [UnusedVariable] The local variable 'foo' is never read.
    var foo = 2;
        ^
    (see https://errorprone.info/bugpattern/UnusedVariable)
  Did you mean to remove this line?
/Users/calebcushing/IdeaProjects/ppm/scaf/src/main/java/com/xenoterracide/scaf/PebbleTemplateProcessor.java:90: warning: [Var] Non-constant variable missing @Var annotation
    var bar = 3;
        ^
    (see https://errorprone.info/bugpattern/Var)
  Did you mean '@Var var bar = 3;'?
/Users/calebcushing/IdeaProjects/ppm/scaf/src/main/java/com/xenoterracide/scaf/PebbleTemplateProcessor.java:93: warning: [NullAway] dereferenced expression overwrite is @Nullable
    overwrite.booleanValue();
             ^
    (see http://t.uber.com/nullaway )

Thomas Broyer · Answer 14 · Sat Dec 12 2020 04:18:22 GMT+0800 (China Standard Time)

-Werror is a JavaC argument, so options.compilerArgs, not options.errorprone.errorproneArgs.
(note that they'll stay warnings, but the compilation will fail with an error saying that there have been warnings)

Caleb Cushing · Answer 15 · Sat Dec 12 2020 04:37:15 GMT+0800 (China Standard Time)

still get them on my generated class

tasks.withType<JavaCompile>().configureEach {
  options.compilerArgs.addAll(listOf("-parameters", "-Werror", "-Xlint:deprecation", "-Xlint:unchecked"))

  options.errorprone {
    nullaway {
      severity.set(CheckSeverity.WARN)
      acknowledgeRestrictiveAnnotations.set(true)
      handleTestAssertionLibraries.set(true)
      checkOptionalEmptiness.set(true)
    }
    disableWarningsInGeneratedCode.set(true)
    // excludedPaths.set(".*/build/generated/sources/annotationProcessor/.*")

Caleb Cushing · Answer 16 · Fri Feb 26 2021 09:58:50 GMT+0800 (China Standard Time)

yeah, coming back to this, still a problem

tasks.withType<JavaCompile>().configureEach {
  options.compilerArgs.addAll(listOf(
    "-parameters",
    "-Werror",
    "-Xlint:deprecation",
    "-Xlint:unchecked"
  ) )
  options.errorprone {
    nullaway {
      severity.set(CheckSeverity.ERROR)
      acknowledgeRestrictiveAnnotations.set(true)
      handleTestAssertionLibraries.set(true)
    }
    disableWarningsInGeneratedCode.set(true)

    error(...

/home/xeno/IdeaProjects/adhoc/proto/build/generated/sources/annotationProcessor/java/main/com/xenoterracide/uplogix/TxnLogDtoBuilder.java:45: error: [NullAway] initializer method does not guarantee @NonNull fields type (line 29), timestamp (line 30), amount (line 32) are initialized along all control-flow paths (remember to check for exceptions or early returns).
  public TxnLogDtoBuilder() {
         ^
    (see http://t.uber.com/nullaway )
2 errors

I'm kinda just glad this ticket is still open so I can see my fix.

and this doesn't compile in build.gradle.kts

  val generatedSourceOutputDirectory = options.generatedSourceOutputDirectory;
  options.compilerArgs.addAll(listOf(
    "-parameters",
    "-Werror",
    "-Xlint:deprecation",
    "-Xlint:unchecked"
  ) )
  options.errorprone {
    nullaway {
      severity.set(CheckSeverity.ERROR)
      acknowledgeRestrictiveAnnotations.set(true)
      handleTestAssertionLibraries.set(true)
    }
    disableWarningsInGeneratedCode.set(true)
    excludedPaths.set(generatedSourceOutputDirectory.map { """.*/\Q${relativePath(it)}\E/*""" })

* What went wrong:
Circular dependency between the following tasks:
:compileJava
\--- :compileJava (*)

Thomas Broyer · Answer 17 · Mon Mar 22 2021 03:09:59 GMT+0800 (China Standard Time)

I think this is a bug in Gradle, I've opened an issue: gradle/gradle#16604

Note that with Gradle < 7, it's possible to use an intermediary provider to break the circular dependency. In Gradle 6 you'll have a deprecation warning though (see linked issue, where this is described, with code snippet)

I'm going to close this as I don't think there's anything I can do here.