`ProjectRef` source dependency does not work with multiple Scala versions

Question

`ProjectRef` source dependency does not work with multiple Scala versions

akauppi opened this issue 8 years ago · comments

Using ProjectRef to bring in sources from another repository is a useful feature. However, it only seems to compile for the default Scala version of the dependency, i.e. it does not support cross-compilation.

Steps

project A compiles for both Scala 2.11.8 and 2.12.1 (default)
project B refers to A via dependsOn(ProjectRef(...))

Problems

project B does not compile for Scala 2.11.8

Expected

project B would compile, whether it uses Scala 2.11.8 or 2.12.1

Notes

Workaround: push a separate tag of A, e.g. 0.0.7-2.11 where the main Scala version is temporarily changed to the alternative one

Platform: sbt 0.13.13 on macOS

References: StackOverflow entry

I'm sorry that I don't have the time to prepare a PR to fix this.

Project A console output (on sbt compile):

[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn] 	::          UNRESOLVED DEPENDENCIES         ::
[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn] 	:: <project-name-removed>_2.11;0.0.7: not found
[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn] 
[warn] 	Note: Unresolved dependencies path:
[warn] 		<project-name-removed>_2.11:0.0.7
[warn] 		  +- <project-name-removed>_2.11:0.0.29
sbt.ResolveException: unresolved dependency: <project-name-removed>_2.11;0.0.7: not found
	at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:313)
	at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:191)
	at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:168)
	at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:156)
	at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:156)
	at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:133)
	at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:57)
	at sbt.IvySbt$$anon$4.call(Ivy.scala:65)
	at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93)
	at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78)
	at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97)
	at xsbt.boot.Using$.withResource(Using.scala:10)
	at xsbt.boot.Using$.apply(Using.scala:9)
	at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58)
	at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48)
	at xsbt.boot.Locks$.apply0(Locks.scala:31)
	at xsbt.boot.Locks$.apply(Locks.scala:28)
	at sbt.IvySbt.withDefaultLogger(Ivy.scala:65)
	at sbt.IvySbt.withIvy(Ivy.scala:128)
	at sbt.IvySbt.withIvy(Ivy.scala:125)
	at sbt.IvySbt$Module.withModule(Ivy.scala:156)
	at sbt.IvyActions$.updateEither(IvyActions.scala:168)
	at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1442)
	at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1438)
	at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$90.apply(Defaults.scala:1473)
	at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$90.apply(Defaults.scala:1471)
	at sbt.Tracked$$anonfun$lastOutput$1.apply(Tracked.scala:37)
	at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1476)
	at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1470)
	at sbt.Tracked$$anonfun$inputChanged$1.apply(Tracked.scala:60)
	at sbt.Classpaths$.cachedUpdate(Defaults.scala:1493)
	at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1420)
	at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1372)
	at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
	at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
	at sbt.std.Transform$$anon$4.work(System.scala:63)
	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
	at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
	at sbt.Execute.work(Execute.scala:237)
	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
	at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
	at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
[error] (content-api/*:update) sbt.ResolveException: unresolved dependency: <project-name-removed>_2.11;0.0.7: not found
[error] Total time: 4 s, completed Jan 10, 2017 11:20:48 AM

Dale Wijnand · Answer 1 · Tue Jan 10 2017 19:10:45 GMT+0800 (China Standard Time)

Could you provide clear reproduction steps, please? I don't mean to be pedantic, but, for instance, it's not clear exactly what you're doing when you say "project A compiles for both Scala 2.11.8 and 2.12.1 (default)". These details matter. Thank you.

Asko Kauppi · Answer 2 · Tue Jan 10 2017 19:14:31 GMT+0800 (China Standard Time)

Sure.

$ git clone git@github.com:akauppi/sbt2901b.git
$ cd sbt2901b
$ sbt run

If it runs, there was no problem.

Dale Wijnand · Answer 3 · Thu Feb 23 2017 20:48:08 GMT+0800 (China Standard Time)

sbt +compile was suggested as the workaround.

See https://gitter.im/sbt/sbt?at=58aece2df1a33b62753bf45b

Josef Lindman Hörnlund · Answer 4 · Thu Feb 23 2017 21:43:22 GMT+0800 (China Standard Time)

So, project A in my case is akka/alpakka and they have

ivyScala := ivyScala.value.map(_.copy(overrideScalaVersion = sbtPlugin.value)) // TODO Remove once this workaround no longer needed (https://github.com/sbt/sbt/issues/2786)!

in their sbt configurations.

And if I am doing sbt +compile from my project B I get

[warn] Binary version (2.11) for dependency org.scala-lang#scala-library;2.11.8 [warn] in com.lightbend.akka#akka-stream-alpakka-s3_2.12;fbcb33f1+20170223-1431 differs from Scala binary version in project (2.12).

Might be the issues are related.

Nicolas Rouquette · Answer 5 · Mon Jun 05 2017 23:11:40 GMT+0800 (China Standard Time)

I experienced this problem when I wanted to add a source dependency on https://github.com/scala-graph/scala-graph. The workaround worked for me.

Asko Kauppi · Answer 6 · Thu Sep 21 2017 03:03:26 GMT+0800 (China Standard Time)

I am no longer using source dependencies, so from my side this issue is not urgent and could be closed. However, since others have raised comments it's probably best to keep it open?

Nicolas Rouquette · Answer 7 · Thu Sep 21 2017 05:59:46 GMT+0800 (China Standard Time)

Well me too but for different reasons... I got my fixes into scala-graph so now I can use a dependency on the published artifact instead of my fork of the source.

I think it would be useful to explain somewhere in the SBT doc the subtle difference between sbt compile and sbt +compile; if that is indeed enough to work around the problem.

Alternatively, one could also consider contributing @akauppi's example as a unit test for the current SBT development. Perhaps the problem has become moot with 1.0.

Dale Wijnand · Answer 8 · Fri Sep 22 2017 00:51:32 GMT+0800 (China Standard Time)

I understand more now than I did in February. :)

I suspect the reason is that + only considers the projects in ThisBuild. When you use ProjectRef you introduce another build.

However with the new implementation of + in sbt 1, this problem might indeed be moot.

Asko Kauppi · Answer 9 · Mon Jan 29 2018 01:38:30 GMT+0800 (China Standard Time)

Updated both the A and B projects to use sbt 1.1.0 and Scala 2.12.4, to check if it's still there.

Yes, it is.

Using sbt +update on the dependency side is a quite usable workaround, but should be mentioned somewhere (I'd add a note at the ProjectRef discussion, because that's really where the effect is).

But, without a search in sbt 1.1.0 documentation I'm not even able to find that place. Maybe that is the high priority thing to fix.