Tensorflow-Signature-Recognition

A siamese network implementation for signature recognition. A Resnet50 based siamese network is trained with triplet loss with the choice of online semi-hard negative mining or hard negative mining. [Blog for triplet loss] [Blog for simaese] [Trained Model]

The network is trained on open-source signature dataset. Please note that this code is not explicitly written for signature detection and can be used for any siamese task such as Face-Recognition (Face alignment logic should be implemented), writer recognition using handwritten text, etc. See Notes Section.

Dependencies

Python 2.7 || 3.5 - 3.8
Tensorflow == 1.12.0 (Should work on lower versions with minor changes)

The code is tested on :- Ubuntu 14.04 LTS with CPU architecture x86_64 + Nvidia Titan X 1070 + cuda9.0.

Getting Started

Training

First download the dataset and extract it to appropriate folder (divide dataset in two subsets train and val). The code is not data format dependent, thus can be used with any custom data.

To run training, edit and run train.sh:

#!/usr/bin/env bash
CUDA_VISIBLE_DEVICES=0 python main.py \
    --output_dir /handwritten-data/experiment_sign_semi_reg \
    --summary_dir /handwritten-data/experiment_sign_semi_reg/log/ \
    --mode train \
    --train_dir /handwritten-data/signatures/full_org \
    --val_dir /handwritten-data/signatures/val \
    --val_dataset_name kaggle_signature \
    --learning_rate 0.0001 \
    --loss semi-hard \
    --decay_step 50000 \
    --decay_rate 0.1 \
    --stair True \
    --beta 0.9 \
    --loss_margin 0.5 \
    --max_iter 200000

For all available options, check main.py.

To observe the training losses, learning rate, graph and images, use tensorboard:

tensorboard --logdir [summary_dir]

Inference

I have implemented a spring boot Java application with react frontend to serve the model for inference, which is available here. To use your model with Java app. Use graph_serialize_utils to convert (and visualize) tf model to (frozen) pb file. and follow instruction on above repo.

Notes

The pretrained model is not explicitly trained for signature fraud detection.
This code can be trained for any other siamese task by just providing appropriate train and val folder. No code change required.
Other signature dataset
Writer recognition dataset
- IAM
face dataset (For face reco training, face alignment logic would be required)
- CASIA-WebFace
- VGGFACE

S0soo / signature-recognition