Feature Request: Test for incompatible versions used in the inputs tree of affected packages

Question

Feature Request: Test for incompatible versions used in the inputs tree of affected packages

doronbehar opened this issue 4 years ago · comments

In NixOS/nixpkgs#99956, we learned how disastrous are the consequences when incompatible versions of qt, are used in transitive inputs of a package. We learned that the hard way, by experiencing the issues on runtime, and many users experienced the same issue in various packages.

@jorsn wrote a script / nix function that checks that:

https://paste.sr.ht/~jorsn/307bd7a99855c7d6f34923ebe2f41a9f2d0f6f8a

Called like this (using nixUnstable):

nix eval --impure --expr 'with import ./. {}; import ./dep-ver-consistent.nix lib false 1 "qtbase" carla'

Maybe ofborg should perform this check upon every attribute in the changed paths?

Johannes Rosenberger · Answer 1 · Fri Oct 09 2020 08:59:24 GMT+0800 (China Standard Time)

If ofborg builds a package it should rather check the store paths in its closure (e.g. using nix path-info -r. But I think, by default it only evaluates them, doesn't it?

I updated the script: https://paste.sr.ht/~jorsn/71a68250987766c94f82651442a8d9dc6704b32f.
It is now almost resistant to larger then necessary search depths, because it is now really lazy, if and only if the package is broken.

The function searches a restricted dependency tree of limited depth for packages with a given package name (as given by getName). If such packages are found, the versions are compared to the first one.

This is not quite fast. I tried to register already checked packages and prevent double-checking them. This didn't decrease execution times; in some cases it even increased them. If anyone has suggestions how this can be optimized, I'm happy to apply changes.

But since ofborg builds the package anyway, it is maybe better to only check the output of

Here is the output of two sample calls with time measured on my notebook:

1. with broken `carla`:

$ for i in 1 2 4 8; do time nix eval --impure --expr "with import ./. {}; import ./dep-ver-consistent-lc.nix lib false $i \"qtbase\" carla"; done
trace: x.pkg.version == 5.15.0 != 5.14.2 == dep0.pkg.version
  where
    dep0.parent.outPath == /nix/store/xmgiwrnglzmkyyz2ccg54jjsswjhayzd-carla-2.1.1
    dep0.pkg.outPath == /nix/store/vm9m827nfcnlf97y53bn65zaadw8pasp-qtbase-5.14.2-dev
    x.parent.outPath == /nix/store/7414sivcqpizf81pjrj99qbib9wx50kk-python3.8-PyQt5-5.15.1-dev
    x.pkg.outPath == /nix/store/njywy8s62w93ymr822sc9cl03nv72q7r-qtbase-5.15.0-dev

false
nix eval --impure --expr   1.32s user 0.21s system 83% cpu 1.833 total
trace: x.pkg.version == 5.15.0 != 5.14.2 == dep0.pkg.version
  where
    dep0.parent.outPath == /nix/store/xmgiwrnglzmkyyz2ccg54jjsswjhayzd-carla-2.1.1
    dep0.pkg.outPath == /nix/store/vm9m827nfcnlf97y53bn65zaadw8pasp-qtbase-5.14.2-dev
    x.parent.outPath == /nix/store/7414sivcqpizf81pjrj99qbib9wx50kk-python3.8-PyQt5-5.15.1-dev
    x.pkg.outPath == /nix/store/njywy8s62w93ymr822sc9cl03nv72q7r-qtbase-5.15.0-dev

false
nix eval --impure --expr   1.29s user 0.22s system 83% cpu 1.803 total
trace: x.pkg.version == 5.15.0 != 5.14.2 == dep0.pkg.version
  where
    dep0.parent.outPath == /nix/store/xmgiwrnglzmkyyz2ccg54jjsswjhayzd-carla-2.1.1
    dep0.pkg.outPath == /nix/store/vm9m827nfcnlf97y53bn65zaadw8pasp-qtbase-5.14.2-dev
    x.parent.outPath == /nix/store/7414sivcqpizf81pjrj99qbib9wx50kk-python3.8-PyQt5-5.15.1-dev
    x.pkg.outPath == /nix/store/njywy8s62w93ymr822sc9cl03nv72q7r-qtbase-5.15.0-dev

false
nix eval --impure --expr   1.26s user 0.18s system 84% cpu 1.699 total
trace: x.pkg.version == 5.15.0 != 5.14.2 == dep0.pkg.version
  where
    dep0.parent.outPath == /nix/store/xmgiwrnglzmkyyz2ccg54jjsswjhayzd-carla-2.1.1
    dep0.pkg.outPath == /nix/store/vm9m827nfcnlf97y53bn65zaadw8pasp-qtbase-5.14.2-dev
    x.parent.outPath == /nix/store/7414sivcqpizf81pjrj99qbib9wx50kk-python3.8-PyQt5-5.15.1-dev
    x.pkg.outPath == /nix/store/njywy8s62w93ymr822sc9cl03nv72q7r-qtbase-5.15.0-dev

false
nix eval --impure --expr   1.55s user 0.23s system 85% cpu 2.073 total

2. with fixed `carla`

$ for i in 1 2 4 8; do time nix eval --impure --expr "with import ./. {}; import ./dep-ver-consistent-lc.nix lib false $i \"qtbase\" carla"; done
true
nix eval --impure --expr   0.73s user 0.16s system 90% cpu 0.984 total
true
nix eval --impure --expr   0.78s user 0.16s system 93% cpu 1.009 total
true
nix eval --impure --expr   0.97s user 0.18s system 95% cpu 1.217 total
true
nix eval --impure --expr   6.89s user 0.32s system 104% cpu 6.882 total

Johannes Rosenberger · Answer 2 · Fri Oct 09 2020 18:10:18 GMT+0800 (China Standard Time)

Now, by using beadth-first search it (https://paste.sr.ht/~jorsn/49940df504ab8860715c38b6e697ac6b555ea8a4) really achieves constant time if there is an inconsistency, and a saturation if there is none:

1. with broken `carla`:

for i in 1 8 64 128; do echo "depth == $i:" >&2; time nix eval --impure --expr "with import ./. {}; import ./dep-ver-consistent-bf.nix lib false $i \"qtbase\" carla"; done
depth == 1:
trace: x.pkg.version == 5.15.0 != 5.14.2 == dep0.pkg.version
  where
    dep0.parent.outPath == /nix/store/xmgiwrnglzmkyyz2ccg54jjsswjhayzd-carla-2.1.1
    dep0.pkg.outPath    == /nix/store/vm9m827nfcnlf97y53bn65zaadw8pasp-qtbase-5.14.2-dev
    x.parent.outPath    == /nix/store/y6kf7b0bf91if7i11pk84rxszvbggnqk-file-5.39
    x.pkg.outPath       == /nix/store/njywy8s62w93ymr822sc9cl03nv72q7r-qtbase-5.15.0-dev

false
nix eval --impure --expr   1.25s user 0.21s system 83% cpu 1.760 total
depth == 8:
trace: x.pkg.version == 5.15.0 != 5.14.2 == dep0.pkg.version
  where
    dep0.parent.outPath == /nix/store/xmgiwrnglzmkyyz2ccg54jjsswjhayzd-carla-2.1.1
    dep0.pkg.outPath    == /nix/store/vm9m827nfcnlf97y53bn65zaadw8pasp-qtbase-5.14.2-dev
    x.parent.outPath    == /nix/store/y6kf7b0bf91if7i11pk84rxszvbggnqk-file-5.39
    x.pkg.outPath       == /nix/store/njywy8s62w93ymr822sc9cl03nv72q7r-qtbase-5.15.0-dev

false
nix eval --impure --expr   1.24s user 0.20s system 83% cpu 1.726 total
depth == 64:
trace: x.pkg.version == 5.15.0 != 5.14.2 == dep0.pkg.version
  where
    dep0.parent.outPath == /nix/store/xmgiwrnglzmkyyz2ccg54jjsswjhayzd-carla-2.1.1
    dep0.pkg.outPath    == /nix/store/vm9m827nfcnlf97y53bn65zaadw8pasp-qtbase-5.14.2-dev
    x.parent.outPath    == /nix/store/y6kf7b0bf91if7i11pk84rxszvbggnqk-file-5.39
    x.pkg.outPath       == /nix/store/njywy8s62w93ymr822sc9cl03nv72q7r-qtbase-5.15.0-dev

false
nix eval --impure --expr   1.26s user 0.19s system 83% cpu 1.735 total
depth == 128:
trace: x.pkg.version == 5.15.0 != 5.14.2 == dep0.pkg.version
  where
    dep0.parent.outPath == /nix/store/xmgiwrnglzmkyyz2ccg54jjsswjhayzd-carla-2.1.1
    dep0.pkg.outPath    == /nix/store/vm9m827nfcnlf97y53bn65zaadw8pasp-qtbase-5.14.2-dev
    x.parent.outPath    == /nix/store/y6kf7b0bf91if7i11pk84rxszvbggnqk-file-5.39
    x.pkg.outPath       == /nix/store/njywy8s62w93ymr822sc9cl03nv72q7r-qtbase-5.15.0-dev

false
nix eval --impure --expr   1.26s user 0.22s system 82% cpu 1.776 total

2. with fixed `carla`:

for i in 1 2 4 8 16 32 64 128; do echo "depth == $i:" >&2; time nix eval --impure --expr "with import ./. {}; import ./dep-ver-consistent-bf.nix lib false $i \"qtbase\" carla"; done
depth == 1:
true
nix eval --impure --expr   0.42s user 0.08s system 91% cpu 0.553 total
depth == 2:
true
nix eval --impure --expr   0.42s user 0.08s system 94% cpu 0.529 total
depth == 4:
true
nix eval --impure --expr   0.50s user 0.10s system 94% cpu 0.641 total
depth == 8:
true
nix eval --impure --expr   1.68s user 0.16s system 97% cpu 1.880 total
depth == 16:
true
nix eval --impure --expr   5.07s user 0.16s system 103% cpu 5.033 total
depth == 32:
true
nix eval --impure --expr   5.28s user 0.17s system 103% cpu 5.271 total
depth == 64:
true
nix eval --impure --expr   5.23s user 0.16s system 103% cpu 5.197 total
depth == 128:
true
nix eval --impure --expr   5.27s user 0.15s system 103% cpu 5.244 total

Johannes Rosenberger · Answer 3 · Tue Oct 13 2020 23:35:22 GMT+0800 (China Standard Time)

Maybe this should print the whole parent stacks. This would probably really help debugging.

Doron Behar · Answer 4 · Tue Oct 13 2020 23:48:01 GMT+0800 (China Standard Time)

I don't know if there's more to print, just perhaps the message could be improved, I figured out what's wrong with kdoctools only after I read the source code for a few minutes, and adding builtins.trace myself to figure out what's going on.

Johannes Rosenberger · Answer 5 · Tue Oct 13 2020 23:58:14 GMT+0800 (China Standard Time)

If the whole parent stack is printed, you see the whole chain of dependencies pulling in the conflicting packages. Maybe I can even display the parent tree, i.e. the list of deps until the last common parent and then for each package its branch separately.

Edit: Yes, the error message can probably be improved.

Johannes Rosenberger · Answer 6 · Tue Oct 13 2020 23:59:59 GMT+0800 (China Standard Time)

I figured out what's wrong with kdoctools only after I read the source code for a few minutes, and adding builtins.trace myself to figure out what's going on.

The source of this function or the source of kdoctools?

Doron Behar · Answer 7 · Wed Oct 14 2020 00:04:34 GMT+0800 (China Standard Time)

The source of this function.