| 60cm Input | 30cm SR Output |
|---|---|
 |
 |
RFSR is an adaptation and major twist on other random-forest super-resolution techniques such as SRF by Schulter et al. Our method uses a random forest regressor with a few simple standard parameters. All parameters were finely tweaked using empirical testing to maximize PSNR scores while maintaining minimal training time (4 hours or less per level of enhancement using 200 million pixels on a 64GB RAM CPU). The technique is trained only using the luminance component from a YCbCr converted image.
HR images are degraded to create LR and HR image pairs. The degraded LR image is then shifted by a set dimension in each direction versus the HR image. Each unique shift is stored and then compressed into a 3-dimensional array with a stack of uniquely shifted LR images. All images are then padded with zeros using a two-pixel width to minimize edge effects. The original up-sampled LR image is then subtracted from the 3-dimensional LR stack, and from the HR image for a residual training schema. This normalizes the LR stack and HR image pair and also removes homogenous areas, emphasizing important edge effects. After training and inference the interpolated LR image is then added back to the output image from the model to create the super-resolved output.
After the LR image stack and corresponding HR images are created they are randomly sampled at a 10% rate for training. Based on empirical observation, this sampling removes a significant amount of redundant data and enhances training speed with little to no recognizable affect on performance. RFSR can only produce one level of enhancement at a time and is not built for concurrent training across multiple levels of enhancement. Average training time per level of enhancement on ~200 million pixel examples (10% sampled) is 3.6 hours. Average inference speed on a 544x544 pixel image is 0.7 seconds.
Initial blog on Super-Resolution: Blog
arXiv paper: Shermeyer & Van Etten, 2018
Python 2.7+, GDAL, scikit-learn, scipy, numpy, opencv, and Jupyter Notebooks are the baseline requirements. All are listed in Requirements.txt and are easily installable via pip or conda.
pip install -r Requirements.txt
Simply launch a jupyter notebook instance and open the notebook. Follow the instructions cell by cell and initialize all functions. RFSR features the ability to train new models, test performance on various datasets, and run inference to enhance coarser imagery based on a set user scale. Sample data from the xView dataset is provided.
Feel free to modify the code, work with new data, and try to improve your performance.
Check out our 8-bit conversion code: https://github.com/jshermeyer/SR_Utils
SpaceNet Utilities is a recommended toolkit for working with geospatial data and deep learning
Very Deep Super Resolution (VDSR) 4 Geospatial (VDSR4GEO) is a partner repository and also used in the the arXiv paper listed above.
| 120cm Input | 30cm SR Output |
|---|---|
 |
 |
| 30cm Input | 15cm SR Output |
|---|---|
 |
 |
| 60cm Input | 30cm SR Output |
|---|---|
 |
 |
| 30cm Input | 15cm SR Output |
|---|---|
 |
 |