Giters

junit-team / junit5

✅ The 5th major version of the programmer-friendly testing framework for Java and the JVM

Home Page:https://junit.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Allow overriding JAR manifest entries for reproducibility

hboutemy opened this issue · comments

commented

in #2217, configuration has been done to ensure Reproducible Build by building 2-times on the same machine and comparing results, as checked with https://github.com/junit-team/junit5/blob/main/gradle/scripts/checkBuildReproducibility.sh

It works fine, great

There is a second level of expectation from https://reproducible-builds.org/ : a third party should be able to rebuild and get the same output

For this, we have 2 issues with the current setup:

  1. Created-By: the detailed JVM info is stored in META-INF/MANIFEST for example Created-By: 17.0.8 (Azul Systems, Inc. 17.0.8+7-LTS), which forces the rebuilder to install the exact same JVM distribution (if feasible)
  2. Build-Time and Build-Date are stored in META-INF/MANIFEST: perhaps we can extract the value from META-INF/MANIFEST and inject as SOURCE_DATE_EPOCH env variable, but it is really cumbersome

Conclusion:

  • In theory:
    the binaries published to Maven Central are reproducible, because when rebuilding in Reproducible Central https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/content/org/junit/junit5/README.md , these 2 differences are the only differences found: then in theory, the build environment can be worked out to match expectations to get the same output

  • In practice:
    this is really hard to rebuild to get the same output: it would be nice to drop these 2 aspects, or give an easy way to override detected values when rebuilding

commented

notice: for Build-Time and Build-Date, some projects use the current Git commit timestamp

commented

@hboutemy Would it help if we published the values used for these manifest attributes as part of the release notes and provided a way to override them when running a build?

commented

@hboutemy What about extracting the META-INF/MANIFEST.MF files from the artifacts stored in Maven Central and re-use them before comparing?

commented

Would it help if we published the values used for these manifest attributes as part of the release notes and provided a way to override them when running a build?

yes, this is a typical strategy when such values are generated by reference build, then need to be injected by rebuilders

What about extracting the META-INF/MANIFEST.MF files from the artifacts stored in Maven Central and re-use them before comparing?

it's what I do in general as a third party rebuilder, not even knowing about the release notes from the project

it makes the rebuilder's work harder, because there is this values discovery step, but feasible: if the values have a real interested to end users, this is a good price to pay :)

commented

@hboutemy I'm unclear about next steps here. It sounds to me as if you'd be okay with using the (info from the) original manifests when comparing and have everything you need but it's somewhat inconvenient. Is that an accurate summary?

commented

It sounds to me as if you'd be okay with using the (info from the) original manifests when comparing and have everything you need but it's somewhat inconvenient. Is that an accurate summary?

yes

commented

Alright, then I'll close this issue for the time being. Please let us know if this approach doesn't work as expected.

commented

@marcphilipp I don't understand: is there a way to inject the 2 values when rebuilding let's say JUnit 5.10.1?

I can extract them from reference build, I just need instructions on how to inject them to a build command

commented

I had understood that you were going to replace the manifests in the rebuilt jars with the original ones and compare the resulting jars with the published ones. Did I misunderstand you?

commented

@marcphilipp misunderstanding: "replace the manifests in the rebuilt jars with the original ones" is not rebuilding

the idea is to rebuild with parameters, like

./gradlew publishToMavenLocal -Djvm="17.0.8 (Azul Systems, Inc. 17.0.8+7-LTS" -Dtimestamp="2023-11-25 08:01:42.196+0000"

and accept that the values have to be extracted from reference binaries manifest to be able to create the Reproducible Build build command

commented

Thanks for the clarification! I'll see to it that future releases support this