Author: Zixuan Zhang, Chengxuan Cai
In Hollywood crimes movies we often see detectives solving crimes with the help from one of their computer geeks who can reveal hidden information from blurred, low-quality images. This project is an effort to achieve the same task, but on one specific type of image - license plates. License plate enhancement is a detailed application of a broader field called Single Image Super Resolution (SISR).
The project is inspired by several state-of-the-art SRSR models such as:
- Photo-realistic single image super resolution using a Generative Adversarial Network
- Residual Dense Network for Image Super Resolution
- ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
- Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network
The dataset used in this project is called the Chinese City Parking Dataset, a large-scale collection of plate images in various conditions.
Read my post on Medium for further understanding
Preprocessing
- Dask >= 2.11.0
- PIL >= 6.2.2
Training & Evaluation
- tensorflow >= 2.1.0
- numpy >= 1.18.1
- matplotlib >= 3.1.3
Before training the model it is important to preprocess the raw dataset using the preprocess.py script
Our plate enhancer model is trained in an adversarial fashion(GAN), meaning the generator is trained to create realistic
reconstruction of images that can fool the discriminator, which is a binary classifier. Why GANs? Well, according to several
papers, GAN network tend to create more realistic image reconstruction comparing to model solely trained in the supervised
fashion. For instance, models that minimize Mean Square Error tend to have over-smoothing artifacts.
Therefore, there are two models - the generator(reconstructor) and the discriminator(classifier).
The generator is trained to minimize a novel hybrid loss function, namely the perceptual loss defined in the SRGAN paper
I'd like to thank Olaoluwa Adigun for his amazing suggestions during the span of this project!
This project won the Best Deep Learning Design Award in USC EE599-Deep Learning. Here's the link
to our amazing rojects done by my classmates!
Also, this project stands on the shoulder of many other SISR projects: