AmbiAttach
Project for interpreting RNN LM syntactic knowledge using ambiguous relative clause attachment in English and Spanish.
Dependencies
Requires the following python packages (available through pip):
- pytorch v1.0.0
Quick Usage
Due to file size constraints on github all models can be found here. When you download the models put them in models/ under subdirectories en_models and es_models respectively.
To get by-word compelxity results
python run_test.py [es|en] [0-5]
The first arugment corresponds to the language and the second to the model you wish to evaluate. The complexity metrics are derived from the repository neural-complexity. For ease of use I have included the necessary files here, but see that repo for more details.
Extra Details
As mentioned in the Quick Usage, the models used in the paper (English, Spanish, and synthetic) can be found on our Zenodo repo. Additionally, the raw results from each of the models is given (if you want to save compute time, especially for the larger stimuli).
References
Forrest Davis and Marten van Schijndel. "Recurrent Neural Networks Always learn English-Like Relative Clause Attachment." In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020). 2020.