Testing functions with "passthru" tests.
roberth opened this issue · comments
Functions producing derivations can have their own tests
attribute, just like a derivation can have a tests
attribute using mkDerivation
's passthru.tests
. While a plain lambda can't have attributes, a callable attrset with __functor
, as produced by callPackage
for example, can.
While some functions are already tested indirectly by having calls in the package tree (mkDerivation
has quite some coverage this way...), some functions have surprisingly little. Oddly, the fetchers are among those, because they often don't even have to run, as their outputs are already available.
With NixOS/nixpkgs#136022 we can use a trick to run fetchers whenever their implementations change, which is great for testing. However, ofborg can't run them yet. It uses nix-build
(or a similar command) to run the tests, but it tries to invoke the __functor
function instead of going directly for the attribute. nix build
does not suffer from this problem, but alternatively, we can introduce a packageTests
attribute in Nixpkgs as an alias for the expected behavior. I've elided it from that PR, but it could be reintroduced for ofborg specifically. See NixOS/nixpkgs#136022 (comment)
So it seems that all ofborg needs to do to support running these tests is to either
a. switch to nix build
, or
b. switch from *.tests
(eqv. *.passthru.tests
if it insists?) to packageTests.*
AFAIK, we currently put NixOS tests and (other) package tests of functions under pkgs.tests
, such as tests.trivial-builders.<name>
.
It would be much more intuitive to move tests specific to a function under <function>.passthru.tests
.
Update:
As callables are plain attribute sets that won't update themselves with their passthru
attributes (the way packages returned by stdenv.mkDerivation
do), <function>.tests
is more sensible.