Essay Scoring experiments on the following datasets:
| # | Data Set | language | # essays | # prompts | scores |
|---|---|---|---|---|---|
| 1. | Falko | German | 248 | 4 | B2-C2 |
| 2. | TestDaF | German | 3,275 | 12 | 0-5 |
| 3. | ASAP | English | 12,978 | 8 | various |
| 4. | ICNALE | English | 5,419 | 2 | 1-4 |
| 5. | MEWS | English | 2,720 | 4 | 0-5 |
| 6. | SWELL | Swedish | |||
| 7. | COPLE2 | Portuguese | 966 | 131 | 1-5 |
The experiments are run using DKPro (https://dkpro.github.io/) based Escrito framework (https://github.com/ltl-ude/escrito).
Prerequisites
The project uses Java 1.8. You need to set a DKPRO_HOME variable as described here https://zoidberg.ukp.informatik.tu-darmstadt.de/jenkins/job/DKPro%20TC%20Documentation%20(GitHub)/org.dkpro.tc%24dkpro-tc-doc/doclinks/1/#QuickStart
References:
- Zesch, Torsten, and Horbach, Andrea. “ESCRITO - An NLP-Enhanced Educational Scoring Toolkit.” Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), 2018.
- Eckart de Castilho, R. and Gurevych, I. (2014). A broad-coverage collection of portable NLP components for building shareable analysis pipelines. In Proceedings of the Workshop on Open Infrastructures and Analysis Frameworks for HLT (OIAF4HLT) at COLING 2014, p 1-11, Dublin, Ireland.