`bootimage test` failing with `ok` at the end of output

[Phoebe Bell]

I want to use serial output for pretty much everything, but runs into some problems with what bootimage test expects. For example, this test adapted from your blog fails when it probably shouldn't:

RUN: test-exception-breakpoint
FAIL: Invalid Output:
    EXCEPTION: BREAKPOINT
    ExceptionStackFrame {
        instruction_pointer: VirtAddr(0x205ae1),
        code_segment: 8,
        cpu_flags: 0x16,
        stack_pointer: VirtAddr(0x57ac001fffa0),
        stack_segment: 0
    }
    ok

Unless there is a good reason for this behavior which I'm unaware of, this is what I suggest:

diff --git a/src/test.rs b/src/test.rs
index 6d31af4..39b94a8 100644
--- a/src/test.rs
+++ b/src/test.rs
@@ -92,15 +92,15 @@ pub(crate) fn test(args: Args) -> Result<(), Error> {
                     let output = fs::read_to_string(&output_file).with_context(|e| {
                         format_err!("Failed to read test output file {}: {}", output_file, e)
                     })?;
-                    if output.starts_with("ok\n") {
-                        test_result = TestResult::Ok;
-                        println!("OK: {}", target.name);
-                    } else if output.starts_with("failed\n") {
+                    if output.contains("failed\n") {
                         test_result = TestResult::Failed;
                         writeln!(io::stderr(), "FAIL:")?;
                         for line in output[7..].lines() {
                             writeln!(io::stderr(), "    {}", line)?;
                         }
+                    } else if output.contains("ok\n") {
+                        test_result = TestResult::Ok;
+                        println!("OK: {}", target.name);
                     } else {
                         test_result = TestResult::Invalid;
                         writeln!(io::stderr(), "FAIL: Invalid Output:")?;

With this patch, the test passes.

(It might be better to replace, e.g., if output.contains("failed\n") with something like if Regex::new(r"^failed$").unwrap().is_match(output), but I'll let you decide if you end up keeping this change.)

[Philipp Oppermann]

Thanks for the suggestion!

I think there is an even better solution: The qemu exit device supports setting an exit code, which we could use for indicating success/failure. We already do this in an integration test for the bootloader.

To implement this, we would need to check for the exit code in QEMU:

• If it's 0, do everything the same way as before for backwards compatibility.
• An exit code of 2 indicates success -> mark the test as succeeded without looking at the serial output
• An exit code of 3 indicates failure -> mark the test as failed and print every serial output after a special string such as failed\n or test failed:.

What do you think about this approach?

[Phoebe Bell]

I like that solution a lot! Would you like me to write the patch?

[Philipp Oppermann]

@foeb That would be great!

[Philipp Oppermann]

Implemented in #33