Update Runner API to allow stepping through tests
moorereason opened this issue · comments
I'm trying to incorporate this test suite into a Go testing framework without using the toml-test binary interface.
I'd like to step through the tests individually using the Go testing.Run API. Currently, Runner.Run() will run all tests and then let you step through the results (see BurntSushi/toml for example).
With the current API, if a test causes a panic, no clues are offered as to which test caused the panic. It would be helpful and, I think, better fit in testing.Run if the Runner API was extended to offer a way to step through the tests one at a time.
I think you can already do this; if instead of calling Runner.Run() you loop over Runner.List(), more or less similar to what Runner.Run() does. The implementation in the TOML library is a bit simplistic and probably not the best example; I had a lot of different stuff I was doing at the time with both toml and toml-test 😅
The "directly use from Go" is a bit of a rare use case so I don't want to add too much complexity for it, but I don't mind adding some helper methods or tweaking the API a bit to make this easier if that proves useful. I think exporting Runner.findTests() might be enough?
I'll be unlikely to work on this in the coming weeks myself, but try it and see where you run in to problems; I'll happily review and merge PRs or make simple changes.
Closing. The existing API is fine.