Add tests that applies all node types to all children.
mbj opened this issue · comments
As ruby is an expression based language (so almost everything is an expression), nodes that wrap other nodes should be tested against all combinations of child node types.
This will catch cases like #66.
This test case will generate lots of examples that should get tested against: parse(unparse(generated)) == generated.
I think this would be a perfect problem for something like Property Based Testing.
Alternatively (or combined with PBT) you could create a markov chain using the RubySpec source as the corpus and use that to generate random nodes which you can pass through parser/unparser to make sure they can be round-tripped.
Not only would this test unparser, but I think this would exercise the language itself. I've often wondered why language designers don't use something like this since it seems like it'd be a good technique for uncovering parser problems.
Yes, basically this ticket is about providing the AST generator for a property based test.
Also all property based tests I've ran into also have a "reduce failing test case step", here mutants mutation engine could be mounted to reduce a generated failing test case.
If anyone is curious about the approach I mentioned, here's an article that explains it in a bit more detail and the results they obtained by applying it to a Makefile parser: http://electric-cloud.com/blog/2009/09/using-markov-chains-to-generate-test-input/
@dkubb I think that the AST generator I plan for the first iteration will be fully deterministic. It'll simply place all node types into all children "slots". The amount of combinations will be huge but exercise the unparsing engine much better.
After that we can look into markov chains, here I think its best to do both a token and an AST node based one. The first will likely crash rubies parser ;)
I'll close this as #148 adds all tests from parser to the unparser test corpus which is close enough to this issue and as stated below there is no way to know ahead of time if a given tree of AST nodes can actually be produced by parser.