xinxiaozhe12345 / Pytorch-Handwritten-Mathematical-Expression-Recognition

This program uses Attention and Coverage to realize HMER and this program is based on Pytorch.

Handwritten-Mathematical-Expression-Recognition (Pytorch)

2019/8/13 README.md has been sorted out and you can see the previous version in version_before.md.

This program uses Attention and Coverage to realize HMER (HandWritten Mathematical Expression Recognition) and written by Hongyu Wang refer to Dr. Jianshu Zhang. Any discussion and questions are welcome to contact me ([e-mail]).

Requirements

Python 3.6
Pytorch == 1.0 

Training and Testing

1. Install Requirements and pretrained Densenet weights can be download here) .
2. Decompression files in off_image_train and off_image_test, and this will be your training data and testing data.
3. python 'gen_pkl.py'. This python file will compress your training pictures or testing pictures into a '.pkl' file. Moreover, you should write the correct location of your data files.
4. python 'Train.py' for training.
5. python 'Densenet_testway.py' for testing.
6. Open the source code of HMER V2.0. You can see detials in HMER_v2.0.

Experiment

• This model is testing in CROHME 2016 dataset. All of my experiments are running in two TITAN XP GPUs. The batch_size is 6, the max len is 48 and the max Image size is 100000.

• The best result of this model is:

• WER loss: 17.160%
  ExpRate: 38.595%

• The HMER V2.0

• Visualization of results

• Visualization of Attention

Input image

step by step