- ClearRealityV1 (4X) by Kim2091
- SPANkendata (4X) by terrainer
- 2xHFA2kSpan (2x) by Phhofm
https://openmodeldb.info/models/2x-sudo-shuffle-cugan-9-584-969
Python Binding for upscale-ncnn-py with PyBind11
Span go brrr for upscaling images.
System | Status | CPU (32bit) | CPU (64bit) | GPU (32bit) | GPU (64bit) |
---|---|---|---|---|---|
Linux (Clang) | — | ✅ | — | ✅ | |
Linux (GCC) | — | ✅ | — | ✅ | |
Windows | — | ✅ | — | ✅ | |
MacOS | — | ✅ | — | ✅ | |
MacOS (ARM) | — | ✅ | — | ✅ |
Python >= 3.6 (>= 3.9 in MacOS arm)
To use this package, simply install it via pip:
pip install upscale-ncnn-py
Then, import the upscale class from the package:
from upscale_ncnn_py import UPSCALE
To initialize the model:
upscale = UPSCALE(gpuid: int = 0, tta_mode: bool = False, tilesize: int = 0, model: int = 0)
# model can be 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19; 0 for default
#span
0: {"param": "spanx2_ch48.param", "bin": "spanx2_ch48.bin", "scale": 2, "folder": "models/SPAN"},
1: {"param": "spanx2_ch52.param", "bin": "spanx2_ch52.bin", "scale": 2, "folder": "models/SPAN"},
2: {"param": "spanx4_ch48.param", "bin": "spanx4_ch48.bin", "scale": 4, "folder": "models/SPAN"},
3: {"param": "spanx4_ch52.param", "bin": "spanx4_ch52.bin", "scale": 4, "folder": "models/SPAN"},
#custom span
4: {"param": "2x_ModernSpanimationV1.param", "bin": "2x_ModernSpanimationV1.bin", "scale": 2, "folder": "models/SPAN"},
5: {"param": "4xSPANkendata.param", "bin": "4xSPANkendata.bin", "scale": 4, "folder": "models/SPAN"},
6: {"param": "ClearReality4x.param", "bin": "ClearReality4x.bin", "scale": 4, "folder": "models/SPAN"},
#esrgan
7: {"param": "realesr-animevideov3-x2.param", "bin": "realesr-animevideov3-x2.bin", "scale": 2, "folder": "models/ESRGAN"},
8: {"param": "realesr-animevideov3-x3.param", "bin": "realesr-animevideov3-x3.bin", "scale": 3, "folder": "models/ESRGAN"},
9: {"param": "realesr-animevideov3-x4.param", "bin": "realesr-animevideov3-x4.bin", "scale": 4, "folder": "models/ESRGAN"},
10: {"param": "realesrgan-x4plus-x4.param", "bin": "realesrgan-x4plus.bin", "scale": 4, "folder": "models/ESRGAN"},
11: {"param": "realesrgan-x4plus-anime.param", "bin": "realesrgan-x4plus-anime.bin", "scale": 4, "folder": "models/ESRGAN"},
#cugan-se models
12: {"param": "up2x-conservative.param", "bin": "up2x-conservative.bin", "scale": 2, "folder": "models/CUGAN/models-se"},
13: {"param": "up2x-no-denoise.param", "bin": "up2x-no-denoise.bin", "scale": 2, "folder": "models/CUGAN/models-se"},
14: {"param": "up2x-denoise1x.param", "bin": "up2x-denoise1x.bin", "scale": 2, "folder": "models/CUGAN/models-se"},
15: {"param": "up2x-denoise2x.param", "bin": "up2x-denoise2x.bin", "scale": 2, "folder": "models/CUGAN/models-se"},
16: {"param": "up2x-denoise3x.param", "bin": "up2x-denoise3x.bin", "scale": 2, "folder": "models/CUGAN/models-se"},
17: {"param": "up3x-conservative.param", "bin": "up3x-conservative.bin", "scale": 3, "folder": "models/CUGAN/models-se"},
18: {"param": "up3x-no-denoise.param", "bin": "up3x-no-denoise.bin", "scale": 3, "folder": "models/CUGAN/models-se"},
19: {"param": "up3x-denoise3x.param", "bin": "up3x-denoise3x.bin", "scale": 3, "folder": "models/CUGAN/models-se"},
20: {"param": "up4x-conservative.param", "bin": "up4x-conservative.bin", "scale": 4, "folder": "models/CUGAN/models-se"},
21: {"param": "up4x-no-denoise.param", "bin": "up4x-no-denoise.bin", "scale": 4, "folder": "models/CUGAN/models-se"},
22: {"param": "up4x-denoise3x.param", "bin": "up3x-denoise3x.bin", "scale": 4, "folder": "models/CUGAN/models-se"},
#cugan-pro models
23: {"param": "up2x-denoise3x.param", "bin": "up2x-denoise3x.bin", "scale": 2, "folder": "models/CUGAN/models-pro"},
24: {"param": "up2x-conservative.param", "bin": "up2x-conservative.bin", "scale": 2, "folder": "models/CUGAN/models-pro"},
25: {"param": "up2x-no-denoise.param", "bin": "up2x-no-denoise.bin", "scale": 2, "folder": "models/CUGAN/models-pro"},
26: {"param": "up3x-denoise3x", "bin": "denoise3x-up3x", "scale": 3, "folder": "models/CUGAN/models-pro"},
27: {"param": "up3x-conservative", "bin": "up3x-conservative.bin", "scale": 3, "folder": "models/CUGAN/models-pro"},
28: {"param": "up3x-no-denoise.param", "bin": "up3x-no-denoise.bin", "scale": 3, "folder": "models/CUGAN/models-pro"},
#shufflecugan
29: {"param": "sudo_shuffle_cugan-x2.param", "bin": "sudo_shuffle_cugan-x2.bin", "scale": 2, "folder": "models/SHUFFLECUGAN"},
Here, gpuid specifies the GPU device to use, tta_mode enables test-time augmentation, tilesize specifies the tile size for processing (0 or >= 32), and model specifies the num of the pre-trained model to use.
Once the model is initialized, you can use the upscale method to super-resolve your images:
from PIL import Image
upscale = upscale(gpuid=0)
with Image.open("input.jpg") as image:
image = upscale.process_pil(image)
image.save("output.jpg", quality=95)
import cv2
upscale = upscale(gpuid=0)
image = cv2.imdecode(np.fromfile("input.jpg", dtype=np.uint8), cv2.IMREAD_COLOR)
image = upscale.process_cv2(image)
cv2.imencode(".jpg", image)[1].tofile("output_cv2.jpg")
import subprocess as sp
# your ffmpeg parameters
command_out = [FFMPEG_BIN, ........]
command_in = [FFMPEG_BIN, ........]
pipe_out = sp.Popen(command_out, stdout=sp.PIPE, bufsize=10 ** 8)
pipe_in = sp.Popen(command_in, stdin=sp.PIPE)
realesrgan = Realesrgan(gpuid=0)
while True:
raw_image = pipe_out.stdout.read(src_width * src_height * 3)
if not raw_image:
break
raw_image = realesrgan.process_bytes(raw_image, src_width, src_height, 3)
pipe_in.stdin.write(raw_image)
The project just only been tested in Ubuntu 18+ and Debian 9+ environments on Linux, so if the project does not work on your system, please try building it.
The following references were used in the development of this project:
xinntao/Real-ESRGAN-ncnn-vulkan - This project was the main inspiration for our work. It provided the core implementation of the Real-ESRGAN algorithm using the ncnn and Vulkan libraries.
Real-ESRGAN - Real-ESRGAN is an AI super resolution model, aims at developing Practical Algorithms for General Image/Video Restoration.
media2x/realsr-ncnn-vulkan-python - This project was used as a reference for implementing the wrapper. Special thanks to the original author for sharing the code.
ncnn - ncnn is a high-performance neural network inference framework developed by Tencent AI Lab.
This project is licensed under the BSD 3-Clause - see the LICENSE file for details.