Vendor dirs and package duplication

[Etienne Marais]

Hello, dear opam-monorepo maintainers,
I'm currently working on creating a lock file with opam-monorepo for irmin to be able to reproduce the build.
The past version of irmin has a vendors directory which contains tezos-context-hash. Moreover, as irmin relies on index, which also vendors tezos-context-hash, two different versions of tezos-context-hash are available for dune at build time. It creates the following error;

Error: Too many opam files for package "tezos-context-hash-irmin":
- vendors/tezos-context-hash/tezos-context-hash-irmin.opam
- duniverse/index/vendors/tezos-context-hash/tezos-context-hash-irmin.opam

Is there a way to specify in the locked file to not install one of the vendor dir?

[Etienne Millon]

Hi,

Nested vendoring is difficult to fix at the level of opam-monorepo because it can not know which copy of the library to use.

The most expressive way to solve this would be to extend dune so that several copies of a library can be present in the same workspace, with rules about which parts use which one (the "scoping" proposal).

Otherwise: assuming that a single version of the nested library can be used by both parts of the codebase, we could solve that by removing the unwanted bits of the dependency. That could be either done by opam-monorepo itself (by allowing to pull only a subset of a tarball using a wildcard, what I'd call the "post-checkout filter") or with the help of dune (it knows which files belong to which package - that would be "dune stripping").

But opam-monorepo is not in a position to determine what to do in an automatic way.

I can suggest some workarounds. They assume that you don't need two different versions of tezos-context-hash.

• you can use a version of index that does not vendor tezos-context-hash. Either strip the tarball to remove the offending components (they might not even be used by irmin) or change it so that it exposes its tezos-context-hash dependency to opam.
• as a alternative to this solution, you can modify the vendoring so that the tezos-context-hash library is made private.
• you can manually tweak the output of opam monorepo pull. This requires checking out the contents of duniverse/ in your git repo, but you can do anything you want between opam monorepo pull and dune build - including removing directories that prevent the workspace from building. You can package that cleanup phase as a Makefile target for example.

[Etienne Marais]

Thanks for your answer and your suggestions 😄
I'll go for the first one you suggested!