IPNUISTlegal / funie-gan

Fast underwater image enhancement using GANs. Implementation of FUnIE-GAN, UGAN, Cycle-GAN, and more.

Resources

• Implementations of FUnIE-GAN for underwater image enhancement
• Simplified implementations of UGAN and its variants (original repo)
• Cycle-GAN and other relevant modules
• Modules for quantifying image quality base on UIQM, SSIM, and PSNR
• Implementation: TensorFlow >= 1.11.0, Keras >= 2.2, and Python 2.7

Perceptual enhancement	Color and sharpness	Hue and contrast

Enhanced underwater imagery	Improved detection and pose estimation

FUnIE-GAN Pointers

• Paper: https://arxiv.org/pdf/1903.09766.pdf
• Datasets: http://irvlab.cs.umn.edu/resources/euvp-dataset
• Bibliography entry for citation:

   article{islam2019fast,
       title={Fast Underwater Image Enhancement for Improved Visual Perception},
       author={Islam, Md Jahidul and Xia, Youya and Sattar, Junaed},
       journal={arXiv preprint arXiv:1903.09766},
       year={2019}
   }
  

Usage

• Download the data, setup data-paths in the training-scripts
• Use paired training for FUnIE-GAN/UGAN, and unpaired training for FUnIE-GAN-up/Cycle-GAN
  • Checkpoints: checkpoints/model-name/dataset-name
  • Samples: data/samples/model-name/dataset-name
• Use the test-scripts for evaluating different models
  • A few test images: data/test/A (ground-truth: GTr_A), data/test/random (unpaired)
  • Output: data/output
• Use the measure.py for quantitative analysis based on UIQM, SSIM, and PSNR
• A few saved models are provided in saved_models/ (base model: gen1/)

Constraints and Challenges

• Trade-offs between performance and running time. Requirements:
  • Running time >= 10 FPS on Jetson-TX2
  • Model size <= 17MB (no quantization)
• Issues with unpaired training (as discussed in the paper)
  • Inconsistent coloring, inaccurate modeling of sunlight
  • Often poor hue rectification (dominant blue/green hue)
  • Hard to achieve training stability

Underwater Image Enhancement: Recent Research and Resources

2019

Paper	Theme	Code	Data
FUnIE-GAN	Fast cGAN-based model, loss function and dataset formulation	GitHub	EUVP dataset
Multiscale Dense-GAN	Residual multiscale dense block as generator
Fusion-GAN	FGAN-based model, loss function formulation		U45
UDAE	U-Net denoising autoencoder
VDSR	ResNet-based model, loss function formulation
JWCDN	Joint wavelength compensation and dehazing	GitHub
AWMD-Cycle-GAN	Adaptive weighting for multi-discriminator training
WAug Encoder-Decoder	Encoder-decoder module with wavelet pooling and unpooling	GitHub

2017-18

Paper	Theme	Code	Data
UGAN	Several GAN-based models, dataset formulation	GitHub	Uw-imagenet
Underwater-GAN	Loss function formulation, cGAN-based model
LAB-MSR	Multi-scale Retinex-based framework
Water-GAN	Data generation from in-air image and depth pairings	GitHub	MHL, Field data
UIE-Net	CNN-based model for color correction and haze removal

Non-deep Models

• Local Color Mapping Combined with Color Transfer (code)
• Real-time Model-based Image Color Correction for Underwater Robots
• All-In-One Underwater Image Enhancement using Domain-Adversarial Learning (code)
• Difference Backtracking Deblurring Method for Underwater Images
• Guided Trigonometric Bilateral Filter and Fast Automatic Color correction

Reviews, Metrics, and Benchmarks

• Real-world Underwater Enhancement: Challenges, Benchmarks, and Solutions
• Human-Visual-System-Inspired Underwater Image Quality Measures
• An Underwater Image Enhancement Benchmark Dataset and Beyond
• An Experimental-based Review of Image Enhancement and Image Restoration Methods
• Diving Deeper into Underwater Image Enhancement: A Survey

Acknowledgements

• https://github.com/floodsung/Deep-Learning-Papers-Reading-Roadmap
• https://github.com/PacktPublishing/Advanced-Deep-Learning-with-Keras
• https://github.com/cameronfabbri/Underwater-Color-Correction
• https://github.com/eriklindernoren/Keras-GAN
• https://github.com/phillipi/pix2pix
• https://github.com/wandb/superres
• https://github.com/aiff22/DPED
• https://github.com/roatienza/Deep-Learning-Experiments
• https://github.com/CMU-Perceptual-Computing-Lab/openpose