How to [new-]build with Cabal 3?
Bodigrim opened this issue · comments
I cannot build time with cabal build, which defaults to new-build in recent Cabal:
git clone https://github.com/haskell/time.git
cd time
autoreconf -i
cabal build
Resolving dependencies...
Build profile: -w ghc-8.10.2 -O1
In order, the following will be built (use -v for more details):
- time-1.11.1.1 (lib:time) (first run)
Configuring time-1.11.1.1...
<...>
configure: creating ./config.status
config.status: creating lib/include/HsTimeConfig.h
Preprocessing library for time-1.11.1.1..
Building library for time-1.11.1.1..
...
[46 of 46] Compiling Data.Time.Format.ISO8601 ( lib/Data/Time/Format/ISO8601.hs, /private/tmp/time/dist-newstyle/build/x86_64-osx/ghc-8.10.2/time-1.11.1.1/build/Data/Time/Format/ISO8601.o, /private/tmp/time/dist-newstyle/build/x86_64-osx/ghc-8.10.2/time-1.11.1.1/build/Data/Time/Format/ISO8601.dyn_o )
cabal: can't find include file HsTimeConfig.h
$ cabal --version
cabal-install version 3.4.0.0
compiled using version 3.4.0.0 of the Cabal library
$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.10.2
However, cabal v1-build still works fine. The issues seems to be similar to haskell/cabal#5223, but that one is closed, so I'm a bit confused.
This must be a defect in cabal, right?
Can you build earlier versions of time in this manner? haskell/cabal#5223 is for time-1.9.1.
In any case, you should be getting time from Hackage, not from git.
Can you build earlier versions of time in this manner?
In any case, you should be getting time from Hackage, not from git.
I checked all versions starting from 1.9 (both from Git tags and from Hackage), cabal new-build consistently fails with the same error.
This must be a defect in cabal, right?
To be honest, I'm not well acquainted with time.cabal to judge. I decided that in any case it is worth bringing to your attention: even if this is a cabal defect indeed, could there possibly exist a reasonable workaround to apply on time side? I hit this issue on CI, and resorting to stack or cabal v1-build is not as straightforward as in local environment.
If cabal fails with time-1.9.1, isn't this exactly the same as haskell/cabal#5223?
That’s why I’m confused: haskell/cabal#5223 seems to be closed as resolved with regards to time. CC @phadej
time should use autogen-includes field.
diff --git a/time.cabal b/time.cabal
index ceb4b14..c977653 100644
--- a/time.cabal
+++ b/time.cabal
@@ -1,7 +1,8 @@
+cabal-version: 3.0
name: time
version: 1.11.1.1
stability: stable
-license: BSD3
+license: BSD-3-Clause
license-file: LICENSE
author: Ashley Yakeley
maintainer: <ashley@semantic.org>
@@ -11,7 +12,6 @@ synopsis: A time library
description: Time, clocks and calendars
category: Time
build-type: Configure
-cabal-version: >=1.10
tested-with:
GHC == 8.0.2,
GHC == 8.2.2,
@@ -76,7 +76,7 @@ library
Data.Time.Format.ISO8601,
Data.Time
other-modules:
- Data.Format
+ Data.Format,
Data.Time.Calendar.Types,
Data.Time.Calendar.Private,
Data.Time.Calendar.Days,
@@ -96,7 +96,7 @@ library
Data.Time.Clock.Internal.UTCDiff,
Data.Time.LocalTime.Internal.TimeZone,
Data.Time.LocalTime.Internal.TimeOfDay,
- Data.Time.LocalTime.Internal.CalendarDiffTime
+ Data.Time.LocalTime.Internal.CalendarDiffTime,
Data.Time.LocalTime.Internal.LocalTime,
Data.Time.LocalTime.Internal.ZonedTime,
Data.Time.Format.Parse,
@@ -110,6 +110,8 @@ library
install-includes:
HsTime.h
else
+ autogen-includes:
+ HsTimeConfig.h
install-includes:
HsTime.h
HsTimeConfig.h(With newer cabal-version parser is also pickier about commas, if you choose to use ones in exposed-modules, they should be between every element).
It seems like the cabal format has become stricter in recent versions, and the time package no longer complies, is that correct?
@phadej thanks! Would not autogen-includes: HsTimeConfig.hs be enough? Otherwise, shall one add autogen-includes: HsTime.hs to if os(windows) branch as well?
@Bodigrim yes, just HsTimeConfig.h.
EDIT: Update the above diff.
@AshleyYakeley Not exactly. cabal v2-build is stricter, and isolates build environments better. Format is the same, and newer versions have added fields to let explicitly declare previously implicit things. (Cabal just used to look for files, and try to install them).
Also as part of v2-build cabal registers the package in (project) local package database. That is the step where things fail, as HsTimeConfig.h is not found.
In fact, without the patch above cabal sdist fails to make a source distribution to begin with!
% cabal sdist
cabal: can't find include file HsTimeConfig.h
autogen-includes: (and other autogen- fields) tell cabal sdist to not look for these, but trust that they will be generated by configure (or some other means). Also makes install look for files in folders where autogenerated files are put.
For the record: I haven't made the patch earlier (e.g. when the Cabal issue was closed), because Hackage didn't support uploading cabal-version: 3.0 packages then. I think it's now possible, (yet, I haven't tried).
I think it's now possible, (yet, I haven't tried).
It is possible. Even base-4.14.1.0 requires cabal-version: 3.0, and there are many more examples.
https://hackage-search.serokell.io/?q=cabal-version%3A%5Cs*3.0
OK, I can fix this (or you can make a PR). I'll make a 1.11.1.2 release.
I don't think release is necessary. It affects only local builds, so you can wait until the release is needed otherwise.
I'll make a PR right away
OK, fix checked in. I'm going to make a 1.11.1.2 version anyway. Thanks everyone.
time is also rebuilt, when building a project involving a patched boot library (bytestring in my case). A release would be much appreciated.
time is also rebuilt, when building a project involving a patched boot library (bytestring in my case).
How so? That seems like a bug in cabal-install logic. Are you using cabal-install-3.4? (Which changed how source-repository-package are handled?)
Why would it be a bug? time depends on bytestring via Win32.
Ah. Windows. Who uses that for development :)
(FWIW, release won't help with rebuilding then. You'll rebuild newer time still).
Well, I don't mind rebuilding as long as it succeeds.
There are problems building with older GHC versions. This might take awhile to figure out.
There are problems building with older GHC versions. This might take awhile to figure out.
What you mean by that?
It means don't expect a release today...
I don't understand. CI is green, also locally I have
9.0.0.20201227 8.10.3 8.8.4 8.6.5 8.4.4 8.2.2 8.0.2 7.10.3 7.8.4 7.6.3 7.4.2 7.2.2 7.0.4
time OK OK OK OK OK OK OK NO-IP NO-IP NO-IP NO-IP NO-IP NO-IP
(This is on Linux)
@AshleyYakeley I suggest that you consider @Bodigrim's offer about CI, https://github.com/haskell/bytestring/runs/1619829279?check_suite_focus=true looks quite nice (and checks Windows too, e.g.).
(@Bodigrim https://github.com/haskell/actions/tree/main/setup should AFAIU support all GHCs supported by hvr-ppa, it tries various sources incl ppa, ghcup and preinstalled ones, so you won't need to split into old/new GHC).
The problem is it's no longer possible to install time-1.11.1.2 with an older GHC using stack. This is because version 3 of the Cabal library needs to be installed first, but that itself depends on the time library.
Previously, this wasn't a problem, because the stock Cabal that came with GHC (back to 8.0.2) was always capable of reading the time.cabal file. This isn't the case in the current master branch.
Might have to revert the PR. Reopening.
Why stack have such problems. cabal-install doesn't have.
I'm reluctant to drop support for stack, in any case.
Stack wants a consistent set of tools, libraries, and file formats, and an order for building. That's the point of it: everything is stable, consistent, and reproducible.
I think support for stack with older GHCs means that the time.cabal file is restricted to a format that can be read by the corresponding versions of the Cabal library.
You are not supporting recent cabal-install then. Your choice.
cabal-install used to work, and now it doesn't. Why not fix it so it works like it used to?
fair. Please revert my patch.
The other possibility is to drop support for older GHC versions. I'll try to find out what expectations are.
Ah, interesting.
I do not want to sound partial, but I don't expect there are many Stack users, who build time-1.11.1.2 against GHC 8.6. I mean, not that many people are unhappy with wired-in-GHC version of time, and the absense of solver makes this endeavour more difficult: you are on your own with extra-deps on top of a year-old lts-14.
Besides this, Stack users with GHC 8.6 can still happily build time-1.11.1.1 or any other version. But currently there is no version of time at all which works with cabal [new-]build.
It would be helpful to know what is causing the problem. The trace clearly says creating lib/include/HsTimeConfig.h, and yet later on GHC cannot find this file. Why is that?
@AshleyYakeley See #145 (comment) above
I see, lib/include/HsTimeConfig.h is being created, but in a special folder, and cabal install version 3 does not by default pass that folder to GHC.
It seems the best solution would be to fix cabal-install, so that if it comes across a version 2 cabal file, it interprets it the same way as the old cabal-install did.
Leaving isolation of build environments and cabal build aside for a moment (at least there is v1-build still), let's look at cabal sdist, which is probably more helpful to highlight the issue.
time.cabal declares that lib/include/HsTimeConfig.h is a part of the package, but in fact it is nowhere to be found. I don't think it is fair to expect from Cabal just to guess "ah, probably this file will appear after some magic manipulations and everything will be just fine, let's silently create sdist without it". The autogen-includes field is exactly a principled way to communicate this information to Cabal and any other build system. (It is unfortunate that this field appeared so late, and old versions of Cabal had no alternative to guesses and prayers).
Note that there is no v1-sdist, which leaves cabal users without an alternative at all.
Upd.: Apparently v1-sdist existed, but was decomissioned in cabal-3.4.
That's true, and that's certainly a motivation to upgrade time.cabal to version 3 format. Currently cabal sdist doesn't work unless you run configure first.
Note that there is no
v1-sdist, which leavescabalusers without an alternative at all.
Who runs cabal sdist besides me when I'm preparing a new release?
Currently
cabal sdistdoesn't work unless you runconfigurefirst.
Yes, and then sdist distributes your locally generated HsTimeConfig.h. For instance, when I unpack time-1.11.1.1.tar.gz, I receive HsTimeConfig.h with outdated entries like #define PACKAGE_VERSION "1.10", because that was probably the last time you ran configure locally. Luckily, cabal seems to insist on re-running configure again, regenerating headers.
Who runs
cabal sdistbesides me when I'm preparing a new release?
Most probably nobody :) But sdist could be used more often than that, it is often a part of CI workflow, like git clone; cabal sdist; cabal get; cabal test to ensure that sdist contains everything for reproducible build. So someone maintaining a private fork of time could theoretically be affected.
Luckily,
cabalseems to insist on re-runningconfigureagain, regenerating headers.
Sadly that doesn't help, I forgot to run autoconf... (#148)
OK, I've asked for community feedback here on the possibility of dropping support for old GHC versions.
I do not quite understand why inability to use Stack with GHC <= 8.6 necessitates dropping support of old GHCs altogether. Is it because Cabal has not been able to build tests before #147?
If people are using old GHCs with new time, I'd prefer to support stack. In that case, I will revert time.cabal to the old version.
I'll accept your decision as a maintainer on this matter, so just a few data points:
-
People using old GHCs with new
timecan always resort totime-compat, this is its very raison d'etre. -
It is a question for @phadej, but since the issue deals with the isolation of build environments, I doubt Cabal will provide a workaround. We can postpone
cabal-version: 3.0a bit later, but not indefinitely.
This is still a regression in cabal-install, is it not? Older versions of cabal-install work with released versions of time, while cabal-install version 3 does not.
All cabal-install work with released versions of time. This issue doesn't manifest itself when package is installed as a dependency from Hackage.
The issue is that time cannot be used as a local package, i.e. part of a project.
In other words, it's impossible for cabal users to contribute to the development of time.
I see. I use stack to build the time library locally, partly because it's easier to manage multiple GHC versions.
Generally, I would like to move time.cabal to version 3 if I can. If there is no-one who is using older GHCs with newer time, then I can drop support for older GHC versions. This would not only solve this problem, but also let me remove some of the CPP conditional code.
@Bodigrim, why are you building your own time library from the repo? Maybe there's an alternative solution for what you're trying to do?
I run bytestring test suite. Building local bytestring from the repo triggers rebuilds of all boot libraries, depending on bytestring, and on Windows it includes time. So even while I grab time directly from Hackage and not from the repo, it is still a "local build" and fails as described.
There are certain workarounds available indeed. E. g., I can recommend bytestring contributors to resort to cabal v1-build when using Windows, and setup CI accordingly. But this approach does not scale well: if I wish to test any other package or application on Windows against a yet unreleased bytestring (as a part of release preparation, for example), I'm in trouble because of v1-build limitations (probably one can conjure a sandbox or something, but...)
For as long as time supports GHC 8.6 and earlier, time.cabal must use the old format. As far as I can tell, the time.cabal file in releases is correct within that format.
It seems to me that version 3 of cabal-install is not properly backwards-compatible with the old .cabal format, and that this is a defect in cabal-install.
It's true that cabal new-build is not fully backward-compatible with cabal v1-build. Otherwise there would have been no reason for separate commands and no transition period to adapt to new workflows. This is by design.
I use stack to build the
timelibrary locally, partly because it's easier to manage multiple GHC versions.
I used to think the same, but it appeared that ghcup is absolutely up to the task.
Has cabal new-build always failed with the time library? Or is this something new with version 3 of cabal-install?
This is not something new with Cabal 3.X series, because new-build fails in Cabal 2.4 as well.
Not sure I can help you then. At some point I will drop support for GHC 8.6, and at that point I will migrate time.cabal to version 3.
Given recent changes in master, could we possibly get back to this?
If support for older GHCs is dropped please remember to update lower bounds.
Done.