DISCUSSION: Mark GHCJS support as deprecated?
snoyberg opened this issue · comments
From what I can tell:
- The GHCJS codepaths are not currently being actively maintained in Stack
- Most of the GHCJS community recommends Nix for setting up the toolchain
- None of the regular Stack contributors are using the GHCJS support
Given this, I'd recommend in the next major release of Stack to mark GHCJS support as deprecated and begin printing warnings when it's used. We can later worry about when, if ever, we actually remove the support.
Another possibility could be to only remove support for installing GHCJS itself, but retain the ability to build with it.
Another possibility could be to only remove support for installing GHCJS itself, but retain the ability to build with it.
I'm in favor of this approach. IIRC the code for using ghcjs to build is quite small and so not a big maintenance burden, but the code for installing ghcjs is complicated and error prone. Considering that stack setup for ghcjs can take a long time, it seems sensible to encourage users to find a different approach to installation, perhaps involving prebuilt binaries.
When I last tried the GHCJS support in the fall, it was very frustrating to get working. Documentation was incomplete and out of date, there didn't seem to be a good way to get it to work with recent GHC versions or Stackage snapshots, and it all just felt very hacky because you needed these special GHCJS distributions with specific versions of base packages that were difficult to generate. To be fair, a big part of this is the fact that there wasn't any release process for GHCJS versions, it just seemed to be in perpetual development. I'd be in favour of deprecating unless some real effort is put into making this more usable.
How's a warning that'll usually get triggered at the start?
$ /home/callen/work/stack/.stack-work/install/x86_64-linux/lts-11.6/8.2.2/bin/stack build
Building a GHCJS project. Note that GHCJS support in Stack is DEPRECATED and it will be removed in a future release of Stack.
miso-0.21.1.0: unregistering (flags changed from ["-fexamples","--ghcjs"] to ["--ghcjs"])
miso-0.21.1.0: configure (lib)
Configuring miso-0.21.1.0...
miso-0.21.1.0: build (lib)
Preprocessing library miso-0.21.1.0...
[... snip ...]
[34 of 34] Compiling Miso ( ghcjs-src/Miso.hs, .stack-work/dist/x86_64-linux/Cabal-1.24.0.0_ghcjs/build/Miso.js_o )
miso-0.21.1.0: copy/register
Installing library in
/home/callen/work/miso/.stack-work/install/x86_64-linux/lts-6.20/ghcjs-0.2.0.9006020_ghc-7.10.3/lib/x86_64-linux-ghcjs-0.2.0.9006020-ghc7_10_3/miso-0.21.1.0-2K9hamfIcJSIhYT9IvbE75
Registering miso-0.21.1.0...
I'll put up a PR, you can tell me if you want it to spit a warning at the end or not.
For what it's worth, I use stack and ghcjs together for a couple of client-server projects. I believe @matchwood is working on creating some ghcjs tarballs for more recent resolvers at https://github.com/matchwood/ghcjs-stack-dist. I understand the arguments above, but I'd be sad to see ghcjs support go.
Only just seen this, thanks for the heads up @gleachkr !
@snoyberg I think your three points are quite closely related, and really come down to the last one: "None of the regular Stack contributors are using the GHCJS support".
The move to nix seems (to me) at least partially because of the patchy support for ghcjs offered by stack. But deprecating support, rather than improving it, is a rather sad backward step. Haskell is a hard enough sell by itself, ghcjs doubly so, but then adding in a requirement to master nix just to get a working compilation environment seems to be consigning ghcjs to the margins of usability. Which would be a great shame, given the potential promise of end-to-end type safety, code reuse, etc etc in client server applications.
As @gleachkr mentioned, I have put together tarballs for a couple of resolvers, and while it is a somewhat tedious and slow process it is not inherently very difficult.
On a related note, when experimenting with more recent ghcjs versions (ghc 8.2) I struggled to get stack to even build anything with a separately installed compiler, so some changes will probably have to be made to the build code as well as the setup code.
In the next few weeks I was planning to spend some time getting a working ghcjs 8.2 tarball together, and as part of that I will almost certainly have to make some changes to stack, so it might be worth holding off on this issue to give me chance to look at the current code. In principle I would be interested in committing to developing and supporting ghcjs support in stack, but at present I can't make any promises.
while it is a somewhat tedious and slow process it is not inherently very difficult.
It's good to hear this isn't very difficult. It's unfortunate, though, that it is slow and tedious. @matchwood what would be needed to change that?
I migrated code away from ghcjs because of tooling issues, like installing the compiler (this was back when arch and nix weren't playing nicely). I was having some joy with @matchwood's tools, actually. But it was too haphazard, and a key project requirement was ease of building the frontend. Unless much has changed (and "I struggled to get stack to even build anything with a separately installed compiler" suggests not) ghcjs is unsupported in stack, and I would argue stack should tell users that.
Moving forward: I've just come across haskell/process#128, which includes a very long comment about the business need for ghcjs (from two months ago). The author of ghcjs then followed up, essentially saying that the project is not ready for patches to boot libraries to be pushed upstream. I haven't found any indications of when it may reach this point. Is there a roadmap for ghcjs?
just to throw an annoying vote in; i'd love to stack to support ghcjs somehow.
echoing others, i really can't be bothered getting nix to work; it's a bridge a bit too far, and i was really excited to see this from matchwood https://github.com/matchwood/ghcjs-stack-dist ; so i'd love to see ghcjs become much more easy to work with. seems like stack is a nice way to go here. but i can imagine it may require a heap of effort on stack's part.
a particularly amusing recent somewhat-related work here is this: https://github.com/nomeata/ghcjs2gh-pages by @nomeata; note that it doesn't use stack at all! the travis.yml is comically complicated.
note that it doesn't use stack at all! the travis.yml is comically complicated
I'd say that's mostly in order to work around resource constraints on Travis
The primary reason for deprecating the support is because GHCJS in practice hasn't been supported. If someone is willing to step up and volunteer to be the maintainer for its support, I don't mind un-deprecating it (after all, we haven't released the deprecation yet).
I think it would make sense to put some dedicated subsystem maintainers into the README.md. This would include things like GHCJS, Docker, and Nix support. How's this: whoever wants to volunteer for that role, please send a PR to undo the recent deprecation and add your name as a subsystem maintainer to README.md.
CC @bitemyapp
@dbaynard Sadly I think the slowness can't really be helped without improving ghcjs itself - iterating over the ghcjs build and boot process is just inherently slow! The tedium could be helped with some automation - https://github.com/tolysz/prepare-ghcjs was the tool used in the past for this, but I abandoned it after running into some issues. It is also worth saying that the ghcjs build and setup process changed significantly with the 8.2 branch, and hopefully that will make things easier going forward (it might, of course, have made things more difficult - I haven't played around with it enough yet to be sure).
@snoyberg When is the next release planned? I would be willing to volunteer for maintaining ghcjs support, but I'm not sure how long it will take to get a working PR together that addresses the current problems. In light of this, I think marking ghcjs support as 'experimental', or similar, would be the best route to go down, with the docs changed to reflect this. That way we don't have the issue that new users are given false expectations, while those of us who are using it can continue to do so. I would be happy to update the docs (in fact I volunteered to do this some time ago (https://groups.google.com/forum/#!topic/haskell-stack/CMvbzhb2zdw) but received no response).
We have a tentative release planned in about 2 weeks.
I think the term "experimental" is perfect in this case. If you can put together a PR to update the docs and change the terminology used from "deprecated" to "experimental," plus add yourself as the subsystem maintainer in README.md, that should be plenty for the upcoming release, and we can worry about getting it into a better state post-1.8. Thank you!
See PR. I left the warning as a logWarn though maybe logInfo would be more appropriate?
Unless anybody objects, I'll close this when the PR is merged (just waiting on a review).
Throwing my hat into the ring. Please don't remove GHCJS support! Miso's Windows build depends on it. Also, stack is great at keeping things simple and easy for new users. Many newcomers don't want to adopt nix, and setting up GHCJS manually is quite a pain. Stack really streamlines the installation. I'd rather it be marked as experimental as opposed to deprecated.
I think it would make sense to put some dedicated subsystem maintainers into the README.md. This would include things like GHCJS, Docker, and Nix support. How's this: whoever wants to volunteer for that role, please send a PR to undo the recent deprecation and add your name as a subsystem maintainer to README.md.
I think there's an actual CODEOWNERs file format that some teams use to request review from people affected by a PR among other things. Could look into that if maintenance responsibility is to be formalized, might as well be in a format that's more widely used.
Here's what I found:
-
https://github.com/softprops/codeowners Rust library
-
https://github.com/softprops/git-codeowners git extension that understands code owners
-
https://github.com/meetup/ownerus generate code owners from history
-
https://github.com/NixOS/nixpkgs/blob/master/.github/CODEOWNERS Nix's use of this pattern
GitHub's documentation: https://help.github.com/articles/about-codeowners/
In addition to marking GHCJS installation experimental, it'd be good for Stack to support GHCJS with Nix integration, as GHCJS is much more reliably packaged and installed with Nix. Stack does reliably build with GHCJS in my experience, once it's installed. So allowing users to have it installed more reliably via Nix would make the situation much better. Currently it's just actually impossible to use Stack + GHCJS on NixOS because you get this:
Note: Enabling Nix integration, as it is required under NixOS
Only GHC is supported by stack --nix
We're now about 9 months later, and working on the Stack 2.0 release. and I'm not sure if anything has changed. Can anyone demonstrate that the GHCJS support in Stack either:
- Is currently working, or
- There's work to fix it
Reviewing the issue tracker makes it seem like we're still in the same situation I described when opening this issue.
CC @matchwood
It might depend on what you mean by "working", but it's definitely possible to build GHCJS projects using the current version of stack. I'm still actively using stack for this purpose, and I gather on IRC that I'm not the only one. It may be that Miso has moved on, but I'm not sure. It would be good to hear from them.
It's still necessary to use older resolvers, since as far as I know, @matchwood's work at ghcjs-stack-dist is the most up to to date version of ghcjs with appropriate patches (not sure what these are). It looks like @alexanderkjeldaas also has something working for 8.4.1, although I haven't tried it yet. Discussion at #4030. Anyway, this suggests that the ghcjs-specific code in stack is fine, we just need more up-to-date packaging for ghcjs.
Apologies for not contributing more - I haven't had as much time for this as I hoped. But I am still planning to support more recent snapshots in ghcjs-stack-dist, and to do some work on improving the integration with stack.
Is there anything about the current state of GHCJS in Stack that is blocking other code pathways or features? Or are you just concerned about the weight of legacy code?
Is there anything about the current state of GHCJS in Stack that is blocking other code pathways or features? Or are you just concerned about the weight of legacy code?
It's really about the weight of legacy code. We've been attempting to clean up the code base and simplify, removing features which have been sources of bugs and user frustration. GHCJS has been one of those features.
It'd be nice to keep GHCJS support in stack at least until Asterius / WebGHC are viable options. A lot of new users to miso look to stack for building. Also, there are miso users on Windows that use stack. But, if the weight of GHCJS legacy code is burdensome enough, I'm not feeling any kind of way about it being removed. It's your call @snoyberg, I have no way of gauging the development burden of keeping GHCJS support in stack.
I agree with @dmjio that stack, despite bugs and user frustration, is the easiest way to get started with GHCJS. It does work in many cases, and removing it would effectively (for projects that have been using it) prevent users from ever upgrading stack. If there were an easy alternative that integrates nicely with a stack project then it wouldn't be much of an issue, but as far as I am aware there is not.
That's all well and good. But if we don't have someone maintaining the code properly, it's setting up false expectations. This resurfaced not too long ago when a coworker attempted to use Stack for a GHCJS project and found the process daunting.
We're trying to maintain a high level of usability in Stack, and I honestly don't believe we're hitting that mark with GHCJS. And I don't see that situation changing based on the discussions and actions since this issue was originally opened 10 months ago.
@snoyberg, Out of curiosity, what specifically did the coworker find daunting ? In the miso repo there are stack files that provision GHCJS and it has worked well for users that have gone down the stack path. The alternative of procuring GHCJS manually or with nix (because of its steep learning curve) is far more daunting I'd say.
Just checked in with him. We had a project that was on Nix, and had been asked to help migrate it over to Stack. It was very difficult to find a version of GHCJS that was compatible with both the existing project and could be built with Stack.
Perhaps the compromise position would be as suggested above - to keep the ability to build with GHCJS but to deprecate installation? It should then be possible for me (or others) to put together an independent tool that handles the installation process.
OK, let's make that the next step for Stack 2.0. See #4744.
My understanding is that stack has removed the ability to setup GHCJS, but retained the ability to build with GHCJS.
However, can someone clarify how I can use stack to build with a ghcjs that I have installed manually in my PATH?
The documentation for custom ghcjs says to use stack --compiler ghcjs-8.6.0.1_ghc-8.6.5 build.
But that appears to call setup since it errors with I don't know how to install GHC on your system configuration, please install manually which is an error message from Setup.hs.
ghcjs-8.6.0.1_ghc-8.6.5 is ghcjs that I have built manually from source, and is installed in my PATH as ghcjs
How do I tell stack to use ghcjs from my PATH but not setup?
Running with verbose gave more clues. I think I have to copy ghcjs into local-install-root
stack --verbose --compiler ghcjs-8.6.0.1_ghc-8.6.5 build
2019-10-08 05:50:21.117546: [debug] Checking for project config at: /Users/louis/repo/js/javascript-extras/stack.yaml
2019-10-08 05:50:21.118072: [debug] Loading project config file stack.yaml
2019-10-08 05:50:21.124604: [debug] SELECT COUNT(*) FROM "last_performed" WHERE ("action"=?) AND ("timestamp">=?); [PersistInt64 1,PersistUTCTime 2019-10-06 18:50:21.124476 UTC]
2019-10-08 05:50:21.125475: [debug] Using package location completions from a lock file
2019-10-08 05:50:21.287672: [debug] Asking for a supported GHC version
2019-10-08 05:50:21.287945: [debug] Installed tools:
- ghc-8.6.5
2019-10-08 05:50:21.288072: [debug] Found already installed GHC builds:
2019-10-08 05:50:21.618596: [error] I don't know how to install GHC on your system configuration, please install manually```
Copying the ghcjs bin, lib directory into ~/.stack/programs/x86_64-osx to mirror the existing~/.stack/programs/x86_64-osx/ghc-8.6.5 is not working for me (not being detected).
Can someone clarify how stack determines if a program is installed?
I can't find any definition of listInstalled used here
https://github.com/commercialhaskell/stack/blob/master/src/Stack/Setup.hs#L664
installed <- listInstalled localPrograms
Sorry for the continual spam. I found listInstalled here.
I'll comment again when I'm truly stuck or when I have a solution.
Relevant spam though.
Does --system-ghc work?
Thanks for the suggestion
Running stack --verbose --system-ghc --compiler ghcjs-8.6.0.1_ghc-8.6.5 build gets me further, but it seems to try (and fail) to install dependencies/package.cache in the current directory. Maybe there is something wrong with the way I compiled ghcjs.
Yes, I think ghcjs-pkg is not working properly ghcjs-pkg with dump --expand-pkgroot is returning blank.
@ElvishJerricco, I didn't thank you enough, --system-ghc was what I needed to get stack to use the ghcjs in my PATH.
I suspect this works seamlessly for ghcjs 7.x.
My other problems is a separate issue from having bad stack.yaml file, and from using ghcjs 8.6.
Unfortunately, even though stack can find ghcjs on my PATH, it looks like ghcjs 8.6 hangs when using stack, but I don't think that is necessarily stack's fault.
I think it is due to ghcjs -build-runner vis node.js of Setup.hs scripts is slower in ghcjs 8.6.
cabal build works, but that might be because cabal has tricks that avoid going via node.js
Looks like I have to use cabal if I want to experiment with ghcjs 8.6
Tried what @louispan did with no luck.
I built and installed ghcjs in PATH and it works.
For sure, this is my installation of ghcjs in macos. (click to show)
brew install cabal-install bash autoconf automake ghc@8.6 gnu-tar cabal-install
# add /usr/local/opt/ghc@8.6/bin to PATH
git clone --branch ghc-8.6 https://github.com/ghcjs/ghcjs.git
cd ghcjs
git submodule update --init --recursive
./utils/makePackages.sh
stack intsall
ghcjs-bootStack.yaml is simple. I created project by stack new foo and added compiler configuration.
resolver: lts-14.22
compiler: ghcjs-8.6.0.1_ghc-8.6.5
packages:
- .When I tried stack build -v, it refuses ghcjs because it can't find runghcjs.
$ stack build -v
Version 2.1.3, Git revision 0fa51b9925decd937e4a993ad90cb686f88fa282 (7739 commits) x86_64 hpack-0.31.2
2020-02-03 17:29:12.804027: [debug] Checking for project config at: /Users/my_username/Downloads/ghcjs_sample_proj/stack.yaml
2020-02-03 17:29:12.804529: [debug] Loading project config file stack.yaml
2020-02-03 17:29:12.809396: [debug] SELECT COUNT(*) FROM "last_performed" WHERE ("action"=?) AND ("timestamp">=?); [PersistInt64 1,PersistUTCTime 2020-02-02 08:29:12.809348 UTC]
2020-02-03 17:29:12.810176: [debug] Using package location completions from a lock file
2020-02-03 17:29:12.999858: [debug] Running hpack on /Users/my_username/Downloads/ghcjs_sample_proj/package.yaml
2020-02-03 17:29:13.004192: [debug] hpack output unchanged in /Users/my_username/Downloads/ghcjs_sample_proj/as.cabal
2020-02-03 17:29:13.007931: [debug] Asking for a supported GHC version
2020-02-03 17:29:13.008029: [debug] Getting system compiler version
2020-02-03 17:29:13.009671: [debug] SELECT "id","actual_version","arch","ghc_path","ghc_size","ghc_modified","ghc_pkg_path","runghc_path","haddock_path","cabal_version","global_db","global_db_cache_size","global_db_cache_modified","info","global_dump" FROM "compiler_cache" WHERE "ghc_path"=?; [PersistText "/Users/my_username/.local/bin/ghcjs"]
2020-02-03 17:29:13.010225: [debug] Looking for executable(s): ["/Users/my_username/.local/bin/ghcjs-pkg"]
2020-02-03 17:29:13.010334: [debug] Looking for executable(s): ["/Users/my_username/.local/bin/runghcjs"]
2020-02-03 17:29:13.010417: [debug] Not using compiler at "/Users/my_username/.local/bin/ghcjs": Found an invalid compiler at "/Users/my_username/.local/bin/ghcjs": Control.Exception.Safe.throwString called with:
Could not find any of: ["/Users/my_username/.local/bin/runghcjs"]
Called from:
throwString (src/Stack/Setup.hs:721:25 in stack-2.1.3-AhLY8rQIRDSIaiiXlxnn1Y:Stack.Setup)
2020-02-03 17:29:13.010980: [debug] Installed tools:
- ghc-8.6.5
- ghc-8.8.2
2020-02-03 17:29:13.011219: [debug] Found already installed GHC builds:
2020-02-03 17:29:13.011849: [error] No compiler found, expected minor version match with ghcjs-8.6.0.1_ghc-8.6.5 (x86_64) (based on resolver setting in /Users/my_username/Downloads/ghcjs_sample_proj/stack.yaml).
To install the correct GHC into /Users/my_username/.stack/programs/x86_64-osx/, try running "stack setup" or use the "--install-ghc" flag. To use your system GHC installation, run "stack config set system-ghc --global true", or use the "--system-ghc" flag.system-ghc: true is in ~/.stack/config.yaml.
stack --verbose --system-ghc --compiler ghcjs-8.6.0.1_ghc-8.6.5 build creates same error. Did I missed some options so stack treats ghcjs as ghc? Or is this a ghcjs-upstream error?
@openingnow I gave up on stack support. The newer version of ghcjs works with cabal and nix. Have you tried installing from my instructions on reddit? https://www.reddit.com/r/haskell/comments/dg4dvs/install_ghc_in_25mins_or_ghcjs_in_one_sleep_with
@louispan I read that article but it seems like nix is broken in mac. There is a workaround, but I'd like to wait until somewhat official support comes.
Then, does built-from-source ghcjs works for cabal?
@openingnow I remembered that I was able to successfully install it without nix by following the cabal instructions on https://github.com/ghcjs/ghcjs (remember to also to the do the "Booting GHCJS" steps).
However, it was not straightforward (there were some missing steps or errors I had to fix manually), but I can't remember what they were...