https://static.rust-lang.org/dist/index.html no longer works
infinity0 opened this issue · comments
This prevents Debian infrastructure from automatically downloading old versions of the source code using our generic scraper tool uscan, used for archival and maintenance purposes.
Is there an alternative?
The github releases page of "rustc" is not suitable since it only contains stuff from the top-level git repository and e.g. not vendor crates.
We indeed no longer update that page (primarily due to budget/time of index costs); this has been the case for I think up to 4-6 release cycles now.
What is the specific use case/requirement? In theory, all minor stable releases are at "well known" URLs (i.e., stable release dates are every 6 weeks). Beyond that, we don't currently have any good way of scanning all of the data.
The use case is that sometimes I'm too slow in packaging the latest rustc for Debian so I need to download older versions. Unfortunately the uscan tool can't work with a "well known" URL directly, it needs some sort of index page with explicit hyperlinks to the targets. I could ask them to implement the ability to directly use a URL pattern but that would take some more time, and in the meantime I thought there was a good chance you guys had merely replaced/moved dist/index.html than removing it completely.
Unfortunately not. FWIW, we do have http://static.rust-lang.org/dist/channel-rust-1.13.0.toml and so on for stable releases -- nightly release manifest are found in dist/YYYY-MM-DD/channel-rust-nightly.toml (e.g., https://static.rust-lang.org/dist/2019-04-05/channel-rust-nightly.toml). Maybe that's enough for your purposes? Presumably the nightly index wouldn't be too useful anyway since almost every date has one...
@pietroalbini and I separately discovered that S3/AWS has a feature, S3 Inventory, that may be helpful to restore functionality here (perhaps fully, perhaps even more usefully). I think it's likely that if you wanted this to happen sooner rather than later we would be happy to integrate something into our release process. I've not personally investigated too much but a list of steps or something along those lines would help us move faster here I think.
@Mark-Simulacrum The toml file works for my use-case so we can close this. Hopefully no plans to drop it any time soon?
No, the toml files are necessary for rustup and more broadly, so I would not expect us to drop them. I'll close this then.