[discussion] Ideas for specs/tests
neochrome opened this issue · comments
Please have a look at https://github.com/neochrome/bucklescript-tea/tree/specs for a simple take on how specs/tests possible could be implemented.
For now it is expected to first be compiled using the bucklescript compiler, then executed using node (see package.json). It's possible it could work to execute more like a script using the -bs-main switch of the compiler, but I couldn't get it to work right away.
Would something like this be interesting?
Hmm, I'll probably want it being tested from the OCaml side as I do intend for native-compilation rendering to be built in sometime, and it looks like this would work for that as well yes?
I'm not actually terribly familiar with the whole 'describe' method as I tend to use more traditional tests especially via fuzzers and case generation. Is there a way to use at least case generation inside these as well? I've used janestreet's Quickcheck library in the past for that but I hear there are others as well?
Hmm, I'll probably want it being tested from the OCaml side as I do intend for native-compilation rendering to be built in sometime, and it looks like this would work for that as well yes?
Yes, nothing is stopping it from working on the OCaml side (that's how I've used it before).
Though in it's current state it doesn't give terrible good error/failure messages and as such is of limited use. That's something a ppx could help mitigate, and in the case of testing the bucklescript-tea codebase I think that wouldn't be a problem since it would only be a dev dependency. I also saw https://github.com/andrenth/ospec which looked kinda nice.
I'm not actually terribly familiar with the whole 'describe' method as I tend to use more traditional tests especially via fuzzers and case generation. Is there a way to use at least case generation inside these as well?
Currently - no - nothing is done in that direction, although it might be possible to add. But then it's probably better to look into any existing tooling/framework for testing this codebase. And on that note, it might be useful to keep as separate concerns how to test this codebase and applications built using it. I'm thinking maybe something like https://github.com/gdotdesign/elm-spec could be useful for the latter?
and in the case of testing the bucklescript-tea codebase I think that wouldn't be a problem since it would only be a dev dependency.
Likewise, if any of it leaked into user code then something's wrong anyway. ^.^;
Currently - no - nothing is done in that direction, although it might be possible to add. But then it's probably better to look into any existing tooling/framework for testing this codebase.
You really should look into the quickcheck method. It is a significantly better testing methodology than simple inline tests (and you can always fall back into a simple inline test anyway, though I always try to structure things to not need that).
And on that note, it might be useful to keep as separate concerns how to test this codebase and applications built using it.
Hmm?
I'm thinking maybe something like gdotdesign/elm-spec could be useful for the latter?
That looks like it is more for testing client code in the browser though?
That looks like it is more for testing client code in the browser though?
Exactly, or maybe also from the cli, but specifically for clients, yes. And that's what I meant by keeping it separate from testing bucklescript-tea :-)
Cool. :-)
But yeah, definitely look at how quickcheck works (common in haskell/ocaml/erlang/etc... languages for testing). :-)
I will definitely look into that kind of testing - I feel like I've missed that, maybe due to coming more from JavaScript and and other languages with inferior type systems.... :-)
In essence with the property check style testing, instead of testing specific calls to specific returns, you test streams of calls with randomly generated inputs and verify that the end result is coherent and expected (like an old example is testing an elevator system, you test that when 3 is pressed it goes to level 3, etc...). :-)
Some simple reading on the concepts (uses erlang as example code):
https://ferd.ca/property-based-testing-basics.html
https://propertesting.com/