Could not copy file, mismatching byte count
olafurpg opened this issue · comments
I encountered the following error while building a project in CI, see logs https://github.com/scalameta/metals/pull/940/checks?check_run_id=234376357
[error] java.lang.RuntimeException: Could not copy '/home/runner/work/metals/metals/mtags/target/streams/compile/buildInfo/$global/streams/sbt-buildinfo/BuildInfo.scala' to '/home/runner/work/metals/metals/mtags/target/scala-2.13/src_managed/main/sbt-buildinfo/BuildInfo.scala' (325/327 bytes copied)
[error] at scala.sys.package$.error(package.scala:30)
[error] at sbt.io.IO$.$anonfun$copyFile$4(IO.scala:841)
[error] at sbt.io.IO$.$anonfun$copyFile$4$adapted(IO.scala:829)
[error] at sbt.io.Using.apply(Using.scala:22)
[error] at sbt.io.IO$.$anonfun$copyFile$3(IO.scala:829)
[error] at sbt.io.IO$.$anonfun$copyFile$3$adapted(IO.scala:828)
[error] at sbt.io.Using.apply(Using.scala:22)
[error] at sbt.io.IO$.copyFile(IO.scala:828)
[error] at sbtbuildinfo.BuildInfo$BuildInfoTask.$anonfun$cachedCopyFile$1(BuildInfo.scala:100)
[error] at sbtbuildinfo.BuildInfo$BuildInfoTask.$anonfun$cachedCopyFile$1$adapted(BuildInfo.scala:98)
I have seen the same error happen locally, and usually it doesn't repeat after another compile.
The stack trace points to the following line
Any ideas on what might be causing the issue? 🤔
Is it running somehow running multiple builds concurrently with different Scala versions? Maybe it happened such that the copyFile was running against the same file name: /home/runner/work/metals/metals/mtags/target/streams/compile/buildInfo/$global/streams/sbt-buildinfo/BuildInfo.scala? If so, we should fix it by making sbt-buildinfo become aware of Scala version.
That could indeed be the source of the issue, we publishLocal for Scala 2.11 and 2.13 in parallel here https://github.com/scalameta/metals/blob/82acc8bff0884800d601170b2a1aefa4e38b5a9a/build.sbt#L461-L462
If so, we should fix it by making sbt-buildinfo become aware of Scala version.
How would that be implemented? I'm curious to understand 🤔
oh you're creating parallel universes there:
val (s, _) = Project
.extract(newState)
.runTask(publishLocal.in(mtags), newState)
Project
.extract(s)
.runTask(publishLocal.in(interfaces), s)The point of State is so sbt is aware of the state machine that represents your disk + settings (including Scala version). Now two instances of buildInfo tasks are using the same target directory to generate stuff. I think that's the source of the bug, not sbt-buildinfo per se.
Try sbt-projectmatrix (designed to do this), maybe?
Yeah, sbt-projectmatrix might be a good idea for our build. As an attempt to reduce the risk of using State without too drastic changes, I updated our build to run 2.11/2.13 publishLocal synchronously instead of concurrently.
Thank you for tracking down the root cause of this issue! I'm gonna close this as "wontfix" given the root case was misuse of sbt APIs.
FWIW, I just tried sbt-projectmatrix and hit on sbt/sbt-projectmatrix#14. Our build is structured like this
- mtags-interfaces (pure Java)
- mtags (CrossVersion.full: 2.11.12, 2.12.10, 2.13.1, ...), depends on mtags-interfaces
FWIW, I just tried sbt-projectmatrix and hit on sbt/sbt-projectmatrix#14. Our build is structured like this
- mtags-interfaces (pure Java)
- mtags (CrossVersion.full: 2.11.12, 2.12.10, 2.13.1, ...), depends on mtags-interfaces
Looks also like we can't cross publish for both 2.12.8 and 2.12.10, or am I totally mistaken? I can open a new issue if it matters? 🤔