shamilcm / nea

Neural Essay Assessor: An Automatic Essay Scoring System Based on Deep Neural Networks

Neural Essay Assessor

An automatic essay scoring system based on convolutional and recurrent neural networks, including GRU and LSTM.

Set Up

• Install Keras
• Prepare data
• Run train_nea.py

Data

We have used 5-fold cross validation on ASAP dataset to evaluate our system. This dataset (training_set_rel3.tsv) can be downloaded from here. After downloading the dataset, create training, development and test data according to the essay IDs in the data directory. You should keep the TSV header in all the generated files.

Options

You can see the list of available options by running:

python train_nea.py -h

Example

The following command trains a model for prompt 1 in the ASAP dataset, using the training and development data from fold 0 and evaluates it.

THEANO_FLAGS="device=gpu0,floatX=float32" train_nea.py
	-tr fold_0/train.tsv
	-tu fold_0/dev.tsv
	-ts fold_0/test.tsv
	-p 1	# Prompt ID
	--emb embeddings.w2v.txt
	-o output_dir

License

Neural Essay Assessor is licensed under the GNU General Public License Version 3. Separate commercial licensing is also available. For more information contact:

• Kaveh Taghipour ([e-mail])
• Hwee Tou Ng ([e-mail])

Publication

Kaveh Taghipour and Hwee Tou Ng. 2016. A neural approach to automated essay scoring. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.