randombit / botan

Cryptography Toolkit

Home Page:https://botan.randombit.net

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Certstore system tests fail if certificates are not installed.

FAlbertDev opened this issue · comments

After building Botan on Windows, the Botan certstore system tests (src/tests/test_certstor_system.cpp) failed on my system. The root certificate (D-TRUST Root Class 3 CA 2 EV 2009) was somehow not installed on my system, which caused this test to fail. The test implicitly asserts that some well-known CA certificates are installed. There are various options for how to handle this issue:

  1. We can ignore it and hope my system was the only one where the respective certificate was not installed. Maybe we could at least add a comment to the test that it fails if the certificate is not found on the system.
  2. We skip the test if the certificate is not found instead of failing it.
  3. Same as 2, but we add a flag to botan-test to avoid that we skip this test in CI.

In my opinion, it's not very urgent since no one has complained so far. It could be frustrating for a user when installing Botan, though.

The dependence on "certain" well-known root certificates in the test has always been somewhat an Achilles heel, unfortunately.

I'm in favor of (3), in a sense that we "somehow" flag the tests that assume the trust in certain root certs as "may-fail" in the test setup. AFAIK Botan's test system doesn't have this functionality, yet.

I'm envisioning something like:

auto test_that_assumes_existence_of_some_root() {
   Test::Result result("Test OS certstor, MAY_FAIL);
   /* ... test all the things ... */
   return result
}

... if the test fails, it should print a warning by default but not set the return code of ./botan-test.exe. Except the tests were invoked with ./botan-test.exe --strict (or similar).