Provide experimental foundation for assisted experimentation

[Wesley Baugh]

For any given machine learning task, we often perform experiments by setting the training and test data, along with trying out various combinations of feature extraction strategies and tuning variable values. The usual way this is done is to:

1. Write a script to perform the experiment.
2. Run the experiment and get the performance.
3. Make changes either in a new copy of the script or by commenting out the old code in-place
4. Get the new performance and repeat the process.

It would be useful to have a programmatic way of automating or assisting this process that is modular so that it can be used across different tasks. Some of the modular components should be:

• Perform any required initial setup to get the model ready.
• Parse the training dataset and yield training instances.
• Parse the testing dataset and store a copy in memory.
• Train the model given a training instance.
• Make a prediction for a test instance using the current model.

This would be a way to make the implementation of gh-7 and gh-8 easier.