This code illstrates some unexplained details of the paper
Textbook Question Answering under Instructor Guidance with Memory Networks
Authors: Juzheng Li, Hang Su, Jun Zhu, Siyu Wang and Bo Zhang
Created by Juzheng Li at Tsinghua University
Introduction
- The original paper introduces a hierarchical model to deal with the Textbook Question Answering problem
- Due to space limitation, the Contradiction Enitity-Relationship Graph (CERG) is not presented in detail in the paper
- This code illstrates what CERG is and how it is built
- This code only works on the true or false questions
Performance
The % accuracy results:
Task | Random Ave | Random Best | Trick | MemN + VQA | BiDAF + DPG | IGMN (This code) | IGMN (full model) |
---|---|---|---|---|---|---|---|
Text TF | 49.00 | 50.10 | 51.90 | 50.50 | 50.40 | 55.31 | 57.41 |
License
This code is released under the MIT License (refer to the LICENSE file for details)
Citing IGMN
If you find IGMN useful in your research, please consider citing:
@inproceedings{liCVPR18igmn,
Author = {Juzheng Li and Hang Su and Jun Zhu and Siyu Wang and Bo Zhang},
Title = {Textbook Question Answering under Instructor Guidance with Memory Networks},
Booktitle = {Conference on Computer Vision and Pattern Recognition ({CVPR})},
Year = {2018}
}
Requirements
1. Python 2.7
2. No other requirements
Install and Run
git clone https://github.com/freerailway/igmn.git
cd igmn
python tfproc.py
Usage
I want to edit the rules
This code only have 25 rules. You can modify them as much as you need.
-
Files
./dicts/entdis.txt ... Rules for Entity Distinction type ./dicts/caulsality.txt ... Rules for Caulsality type ./dicts/structure.txt ... Rules for Structure type
-
Rule Formation
All the rules have the same formation:Subtree|TypicalWords|Output
Subtree: the tree structure and dependency tags to match the full dependency tree of a sentence
formation: 1stEdge 2ndEdge LinkWhichNode ForwardOrBackword 3rdEdge LinkWhichNode ForwardOrBackword ...
TypicalWords: telling whether the typical node in the subtree is or is not a certain word
formation: WhichNode WordList WhichNode WordList WhichNode WordList ... WordList: Word^Word^Word...^!Word^!Word^!Word... (! means the node is not the word, no ! means the node is one of the words)
Output: the extracted relationship including two numbers
for Entity Distinction: NodeHeadword NodeModifier for Caulsality: NodeCause NodeResult for Structure: NodeMember NodeEntirety
Example:
Rule: nsubj dobj 0 f|0 have^contain^include|2 1 Tree: node_1 <-----nsubj------ node_0 -----dobj-----> node_2 Word: node_0 must be "have" or "contain" or "include" Output: node_2 inside node_1
I want to test a different dataset
-
You need to generate dependency trees from the sentences in your dataset.
-
The Stanford CoreNLP Toolkit (Manning et al., ACL 2014) is suggested to make sure the dependency tags are consistent.
-
You need to save the dependency trees in json files:
./dependencies/lesson_tf.json ./dependencies/question_tf.json
following the formation:
for Lessons: {LessonID:{SentenceIndex:Tree}} for Questions: {QuestionID:[Tree, Tree, Tree, ...]} Tree: [{NodeId:Word}, [[Head, Tail, Tag], [Head, Tail, Tag], ...]] (Notice that the indices start at 1)
I want to work on the multiple choice questions
Not suggested. But if you really need, you will face a number of engineering work to deal with the multiple choice formation. At least you need to:
- Combine the question stems and the options
- Make the results in a continuous space
I want to work on the diagram questions
- Extract the words, lines and arrows (and even the stick figures) from the diagram. Record their positions.
- Design your own rules based on the positions. You may refer to those introduced in the paper.
- Build the CERG based on your rules.
- Dealing with expressions which are not suited to the CERG but indeed useful, such as "in this picture", "not in this picture", "how many xxx appear", ect.