JianshuZhang / TAP

Track, Attend and Parse for Online Handwritten Mathematical Expression Recognition

TAP

This repository contains the source code for TAP introduced in the following papers:

• v1: A gru-based encoder-decoder approach with attention for online handwritten mathematical expression recognition
  
• v2: Track, attend and parse (TAP): An end-to-end framework for online handwritten mathematical expression recognition
  

Here, v1 employs the coverage based spatial attention model, v2 employs the guided hybrid attention model.

Requirements

• Install cuda-8.0 cudnn-v7
• Install Theano.0.10.0 with libgpuarray

Citation

If you find TAP useful in your research, please consider citing:

@inproceedings{zhang2017icdar,
  title={A GRU-based Encoder-Decoder Approach with Attention for Online Handwritten Mathematical Expression Recognition},
  author={Jianshu Zhang and Jun Du and Lirong Dai},
  booktitle={International Conference on Document Analysis and Recognition},
  volume={1},
  pages={902--907},
  year={2017}
}


@article{zhang2019track,
  title={Track, Attend and Parse (TAP): An End-to-end Framework for Online Handwritten Mathematical Expression Recognition},
  author={Zhang, Jianshu and Du, Jun and Dai, Lirong},
  journal={IEEE Transactions on Multimedia},
  volume={21},
  number={1},
  pages={221--233},
  year={2019}
}

Description

• Train TAP without using weightnoise and save the best model in terms of WER

  $ bash train.sh
  

• Anneal the best model by using weightnoise and save the new best model

  $ bash train_weightnoise.sh
  

• Reload the new best model and generate the testing latex strings

  $ bash test.sh
  

Contact

xysszjs at mail.ustc.edu.cn
West campus of University of Science and Technology of China
Any discussions, suggestions and questions are welcome!