Stackage-CLI-generated sandboxes use past add-source dependencies

[Alexandr Kurilin]

I'm running into an odd behavior that I'd love to understand better. It's currently preventing me from correctly using local dependencies set up through add-source. I create an LTS 2.3 sandbox with stackage-cli:

stk sandbox init lts-2.3

I double-check that my local dependency isn't in the shared sandbox:

[.stackage]$ ag -i --hidden frontrow-entities
[.stackage]$

Now, back to my project which expects that dependency in the cabal file (note, I intentionally never ran add-source):

[fancy-api (asmt-migration)]$ cabal install --only-dep --enable-test && cabal build
Resolving dependencies...
Notice: installing into a sandbox located at
/home/alex/.stackage/sandboxes/ghc-7.8.3/lts-2.3
Configuring frontrow-entities-0.0.1.1...
Building frontrow-entities-0.0.1.1...
Installed frontrow-entities-0.0.1.1
Building fancy-api-0.0.1...
Preprocessing library fancy-api-0.0.1...

Model.hs:20:8:
    Could not find module ‘FrontRow.Breakdown’
    Use -v to see a list of the files searched for.

Model.hs:21:8:
    Could not find module ‘FrontRow.Question’
    Use -v to see a list of the files searched for.

First of all, huh? How did it find the location of the local dependency? I never ran add-source in the project that needed it. Let's double-check the shared sandbox. NOW it has that dependency listed in it:

sandboxes/ghc-7.8.3/lts-2.3/add-source-timestamps
1:[("x86_64-linux-ghc-7.8.3",[("/home/alex/code/classroom/frontrow-entities-deploy",1430180607)])]

Ok, huh, so it managed to find it somehow, and it created the add-source-timetstamps file, except it's actually using the deployment-only version (a separate folder), and not the version that I was hoping it'd use. This leads to the build error.

Now let's say I want to try to add the dependency by hand to see if that will somehow override the odd choice:

cabal sandbox add-source ../frontrow-entities

and now in the shared sandbox:

[.stackage]$ ag -i --hidden frontrow-entities
sandboxes/ghc-7.8.3/lts-2.3/add-source-timestamps
1:[("x86_64-linux-ghc-7.8.3",[("/home/alex/code/classroom/frontrow-entities",0),("/home/alex/code/classroom/frontrow-entities-deploy",1430180607)])]

Ok, the dependency I wanted showed up, but no matter what I do to the file (e.g. change the timestamp, remove the -deploy entry etc) it still keeps installing the -deploy. Doesn't matter if I unregister the package with stk sandbox unregister frontrow-entities and run cabal install, still keeps using the -deploy version.

I have to switch back to using regular local sandboxes in order to have the add-source dependency work as intended for the time being. I can't get it to pick up the right add-source folder that I want it to. Any advice on this? Would love to be able to switch to the stackage sandboxes full-time.

[Michael Snoyman]

This looks like (surprise) buggy cabal sandbox behavior: it should be able to change the location of add-source directories. As to how it's magically finding the location in a new project: I'm guessing that's the behavior of cabal sandboxes: putting the information in the shared sandbox instead of the local config file, but I'm not certain.

[Dan Burton]

I haven't looked into making add-source play nicely with shared stackage
sandboxes. If you need add-source then I must regretfully say that this
is a use case which stackage sandbox does not (yet?) support. I'm not sure
what we can do on the stackage-cli end of things to address this but we'll
look into it.

My workflow for cases where I want to use a local or patched dependency for
all my other projects is to simply install the dependency directly to the
shared sandbox. So far I've only had to deal with fairly simple cases,
which this workflow handles well.

For example, there was a small bug in optparse-simple-0.0.1. (This is a
dependency of stackage-cli, which I was working on at the time.) I made a
patch and installed the patched version into my shared sandbox. Then I was
able to continue development of stackage-cli as usual. Once
optparse-simple-0.0.2 was released, I unregistered my patched 0.0.1 and
installed that dep from hackage.

If an add-source dependency is later modified, it is reinstalled

automatically.

This is the feature that my workflow does not support. If a local
dependency/patch is modified, then I just manually reinstall that
dependency to the shared sandbox, and manually reinstall anything depending
on it as well.

Again, we'll see what we can do to make things work for your use case. This
might mean prodding cabal sandbox folks for a bugfix.

-- Dan Burton

On Mon, Apr 27, 2015 at 8:48 PM, Michael Snoyman [e-mail]
wrote:

This looks like (surprise) buggy cabal sandbox behavior: it should be able
to change the location of add-source directories. As to how it's magically
finding the location in a new project: I'm guessing that's the behavior of
cabal sandboxes: putting the information in the shared sandbox instead of
the local config file, but I'm not certain.

—
Reply to this email directly or view it on GitHub
#36 (comment).

[Alexandr Kurilin]

Thanks for looking into this! Out of curiosity, is my workflow uncommon? I don't have much to compare it to. I basically have one or more repos (each a standalone haskell library / local cabal package) that other repos depend on, and I almost always work on all of them in parallel. It's thus very convenient to have the main project automatically rebuild the dependency when anything about it changes.

It works very well with yesod devel, which will pick up changes in add-source dependencies and rebuild both the altered deps and the yesod-based app as well.

[Michael Snoyman]

I don't think it's terribly uncommon. I typically don't use sandboxes very much, and definitely not the add-source feature, since sandboxes have been so notoriously buggy. At FP Complete, we have a completely different build tool that has more robust sandboxing (and yes, we're currently cleaning it up so that we can open source it).