Document why some TOML tests are skipped
EpicWink opened this issue · comments
Some tests are declared as being skipped. These should be documented as to why in the test code
I can offer insight here: it's because that TOML test suite was written to target TOML 0.4.0 and has not been updated since; there's a number of tests that would be false-negatives against a newer TOML parser (e.g. mixed-type arrays).
There's a complementary set of tests updated to support 1.0.0-rc1 here: https://github.com/iarna/toml-spec-tests. They're designed to plug in to the same test harness as the BurntSushi ones so they can be used side-by-side. Might be worth adding these too, @bobfang1992.
Side note: I haven't actually tested toml++ against any of these tests directly, since the integrations required in C++ are a bit tedious. I opted to add matching cases to the internal catch2-based test suite instead. There might be one or two compliance cases I've missed; I'd be keenly interested in finding out if you discover anything fun.
Perhaps adding the iarna/toml-spec-tests suite is the way to go, especially as we only need the TOML and JSON files
Edit: specifically adding, not replacing with
I think using both and opting out of some of the 0.4.0-specific tests (as in the current behaviour) is the right approach, since iarna's tests don't cover some of the basics and are intended to be used in a complementary fashion alongside BurntSushi's.
That being said, the TOML group are working on bringing these together in one official place at https://github.com/toml-lang/compliance, though that work is ongoing (more info here).