How would I write tests with the 0.11 versions?

[Tim Satke]

I would like to be able to execute my kernel tests with QEMU with a simple cargo test, like in the versions before 0.11.
From what I see though, with the bindeps, when I call cargo test, the kernel is not even built in "test mode", and only the root crate (which I will call "runner" from now on) is being tested.

Is there an easy way to boot up QEMU and run the kernel tests, or do I have to manually build the kernel with cargo test --no-run --package kernel --target x86_64-unknown-none?
When running this, I get an error

   Compiling kernel v0.1.0 (<path>)
error[E0463]: can't find crate for `test`

without any additional context, so I'm not sure whether this is the way to go.

The way this bootloader runs tests is also not applicable I think, because as far as I see, it runs kernels, not tests inside kernels.
Correct me if I'm wrong somewhere please

[Philipp Oppermann]

Thanks for raising this issue! You're right that the cargo test approach used with bootloader v0.9 does no longer work. Unfortunately, I haven't implemented a good alternative yet. Given that the eRFC for the current custom test framework feature (rust-lang/rust#50297) was closed, I think it would be better to create our own solution instead. My current ideas are:

• Create a proc macro to collect tests, similar to the official #[test] attribute. We could base this on the inventory crate.
• Add support for passing arguments to kernel, e.g. via serial input or via some special QEMU feature.
• When a --test argument is given, the kernel could run the collected tests instead of doing a normal start. To signal success or failure, we could use the special QEMU exit device as before.
• To get support for cargo test again, the root crate (with the artifact dependency on kernel) could define a normal #[test] function that starts the kernel with the --test argument in QEMU and check the exit code.

[Tim Satke]

Create a proc macro to collect tests, similar to the official #[test] attribute. We could base this on the inventory crate.

Sounds intuitive, however, when doing so, I get stopped with error: #[ctor] is not supported on the current target (https://github.com/mmastrac/rust-ctor/blob/master/ctor/src/lib.rs#L173-L174).

I'll continue on this for a bit, maybe I can get something to work.

[Burkhard Mittelbach]

I didn't yet find the time to actually try this, but the inventory crate points to a similar package that might work.
https://github.com/dtolnay/linkme
I looked into it shortly and linkme does not seem to have the same target restrictions inventory does.

[Burkhard Mittelbach]

I'm currently trying to implement tests using linkeme and it seems to be possible.

However there is a rather large issue I ran into. Just cost me the better part of today to debug this. I had to add

[target.x86_64-unknown-none]
rustflags = [
    "-C", "link-arg=-z",
    "-C", "link-arg=nostart-stop-gc"
]

to my .cargo/config. Otherwise linkme will cause some rather cryptic linker errors.

[Jarl Evanson]

I ran into the same error that you did and solved it the same way.

I just got my linkme testing implementation working. I even got automatic paths/naming through dynamic trait objects, though it requires a wrapper type. The interface is still clunky though, so if you can think of a better way, that would be nice.

[Burkhard Mittelbach]

I also use a wrapper type, but you can generate that using a proc macro.

mod test {
    #[kernel_test]
    fn foo() {
    }
}

becomes something like

mod test {
    #[distributed_slice(testing::KERNEL_TESTS)]
    static __KERNEL_TEST_foo: testing::KernelTestDescription = testing::KernelTestDescription {
        name: "foo",
        fn_name: "foo",
        expected_exit: TestExitState::Succeed,
        test_fn: foo,
        test_location: testing::SourceLocation {
            module: "test",
            file: file!(),
            line: line!()
            column: column!(),
        },
    }
    fn foo() {
    }
}

feel free to use (or improve) my proc macro https://github.com/Wasabi375/WasabiOS/blob/main/testing/derive/src/lib.rs

[Jarl Evanson]

I just modified your proc macro to have support for including the function in different lists, and the functions to have varying input types. That way I can have a slice for modules, and those modules can call their own tests and pass in different inputs.

#[kernel_test(list = TEST_MODULE, name = "BOOT INFO", kind = u64)]
fn boot_info_tests(num: u64) -> Result<(), KernelTestError>

Thanks for sharing

[Tim Satke]

Works for me as well. This is more of an integration test solution though. I'm starting the tests via #[test] methods from my boot crate, and there's not really a way to run the tests otherwise with cargo test, at least not that I can think of.
That works as well as the tests with bootloader <0.11, but is still not a very nice solution.

However, I think the answer of this ticket has been answered, and it can be closed.

@phil-opp maybe it's worth thinking about including this approach in edition 3 for testing, since the only other way to get unit tests for stuff in the kernel, is to extract that functionality into a separate no_std crate. Let me know what you think, I'd be happy to help with a post.