eth-cscs / stackinator

Home Page:https://eth-cscs.github.io/stackinator/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Easy pushing of partial builds to the build cache

bcumming opened this issue · comments

Stackinator pushes all built packages to the build cache once all packages in individual spack environment has been successfully built.

If an error occurs towards the end of building an environment, the already built packages are not pushed to the cache. For large environments this can waste a lot of developer time.

Proposed solution: provide a script named something like force-push-cache that will push all packages that have been installed in the build tree in ${build_path}/store to the build cache.
Make this opt-in, because fixing the underlying issue might require rebuilding a different set of dependencies, and we want to avoid pushing unnecessary packages to the build cache.

Thoughts...
Option 1 - provide a script that the user runs as part of the post install process (is that feasible/sensible to put it in the post install script if we opt-in?)
Option 2 - same as above, but also add a flag in the cache config yaml files that allows the user to just specify that all builds are pushed and then have it done automagically by stackinator

The post install script is only run if the stack has been built successfully, i.e. when all the targets required to build store.squashfs have been completed successfully. So that approach wouldn't address our specific concern.

I could add a makefile target make cache-push?