Optionally allow output of assert commands

[rugk]

assertTrue/assertFalse and similar currently always suppress the whole output. That is bad if you really want to dig into things and see why a test failed.

So it would be nice if you could allow the output as usual.

The only (not so nice) workaround I found is writing all STDOUT/STDERR with tee or so into a file and then reading that file afterwards to display its content.

[James Telfer]

Assuming your assertTrue is checking the output of the command (as opposed to the return code), you could capture the output into a variable, output its contents and then assert? e.g.

output=$(commandUnderTest)
echo "${output}"
assertTrue "Expected Output" "${output}"

[rugk]

But no, assertTrue is checking the return code (I hope, at least). I thought that's what it is supposed to do…

[James Telfer]

And that's why I shouldn't comment late on a Friday...

So to correct my example for the actual behaviour:

output=$(commandUnderTest)
result=$?
echo "${output}"
assertTrue $result

Which is a bit clunky. If you only wanted the output in the failing cases:

output=$(commandUnderTest)
assertTrue "${output}" $?

[rugk]

Yeah that's a good workaround. It would still be better, if it is included in shunit2 by default.

[Kate Ward]

I believe that the example @jameswtelfer gave is the best.

None of the functions of shunit2 are specifically designed to run your functions for the user. Rather, the test code runs the function, and shunit2 framework is used to validate expected behavior.

When unit testing other languages, one always captures the output of function calls into variables, and then tests or utilizes that output in some way. The unit test framework isn't expected to do that heavy lifting for the user though.

If I were doing the same in Go, I too would provide the output (in this case the error message) to the test function (in this case t.Errorf()).

Given the following source code (https://play.golang.org/p/GYIr9fGaKQ):

package main

import (
	"fmt"
	"math"
	"math/rand"
	"time"
)

func randError() error {
	// Generate an error if the random value is greater than 1073741823.
	if rand.Int() > math.MaxInt32/2 {
		return fmt.Errorf("big number")
	}
	return nil
}

func main() {
	if err := randError(); err != nil {
		fmt.Printf("error: %s", err)
	}
}

func init() {
	rand.Seed(time.Now().UnixNano())
}

I would test it something like this. (Note, this is only meant to demonstrate how I'd pass the output. For example, I'd need to mock the output of rand.Int() to have full test coverage.)

package main

import "testing"

func testRandError(t *testing.T) {
  if err := randError(); err != nil {
    t.Errorf("unexpected error; %s", err)
  }
}