Automatic Test Generation and Parameterization

[Vivek Joshy]

Is your feature request related to a problem? Please describe.
When a model is rewritten or improved, due to changes internally, the expected API outputs will change significantly. Re-entering the correct values into the test suite to verify determinism is wasted effort on the developer's part long term.

Describe the solution you'd like
Use Hypothesis to generate tests and pytest parameterization.

Tasks:

• Decouple benchmarks into its own top-level package for loading different kinds of data.
• Create a command line utility to run benchmarks and regenerate tests.
• Import benchmarks package to load data for testing purposes.

[Vivek Joshy]

Using hypothesis offline to help generate tests is a good idea. It will not be added to the test suite though.