milessabin / shapeless

Generic programming for Scala

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Ensure macro outputs are deterministic

milessabin opened this issue · comments

Comment from @retronym on playframework/play-json#263,

I've been working to make scalac itself deterministic (meaning: when sources doen't change, generated bytecode is identical from run to run, even when only a subset of sources are recompiled or when the order of source files changes).

I'd like it if the widely used macros in the ecosystem could adhere to the same principles.

Typically this involves:

  • adding .sorted to .decls / .members
  • sorting the result of knownDirectSubclasses
  • avoiding reflecting the iteration order of Hash{Map,Set}-s used in the implementation of macros in the order of generated code, by using Linked versions instead.

I think we're already in pretty good shape here, at the very least we've always sorted the output of knownDirectSubclasses, and I don't think there's anywhere where map/set iteration order would be relevant. But it would be good to make absolutely sure of this.

@retronym if you have any advice on how to test for (non-)determinism as part of the build they would be very welcome.

You could borrow parts of https://github.com/scala/scala/blob/2.13.x/test/junit/scala/tools/nsc/DeterminismTest.scala, in which I compile a set of source files in different orders/subsets and check bytecode is identical.

I wrote http://github.com/scala/jardiff to automate the process of bytecode decompilation and diffing, which is handy when to diagnose test failures.