jquesnelle / txt2imghd

A port of GOBIG for Stable Diffusion

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Integrate FidelityFX 1.0 as an optional passe

LifeIsStrange opened this issue · comments

What is FSR 1.0?
FSR is the most popular classical upscaler on earth, and has received a tremendous success in video games and emulators.
For example see this difference in Zelda Breath of the wild: https://youtu.be/2OiujuKOwl4?t=327
Strengths:
It works on any GPU,
is computationally extremely cheap/real time
It is not AI based and is I believe complementary as it can be freely mixed with an AI upscaler for free

OK looks like a nice improvement for free but why not DLSS or XeSS or FSR 2.0?
Those newer algorithms are temporal based upscalers, they require motion vectors. I am not aware of them being fit for static images.
However FSR 1.0 is a classical spatial upscaler, except it's very good at its job.

usage:

FSR 1 is a spatial upscaler based on the Lanczo's algorithm requiring an anti aliased lower resolution image
The library is open sauce on github (one should set the quality profile to highest)
https://github.com/GPUOpen-Effects/FidelityFX-FSR

Opportunities:

  1. the obvious one is to use FSR as a postprocessing step after the final image generated by txt2imghd
  2. the less obvious and intriguing one is to use FSR as an intermediary step, AKA use it on the initial 700px stable diffusion output, to give better, more sharpened information to the txt2imgupscaler.
    While it does increase information (e.g. by making clearer what is a corner, a line, a round angle in an image, how well an AI upscaler will respond to it is an open question and algorithm/normalizations dependent.
  3. one can do 2) and then as a bonus reapply FSR on the final image

https://www.tomshardware.com/news/use-amd-fsr-to-upscale-your-own-images

Note that 1) and 2) would be interesting for other SOTA spatial upscalers but I believe FSR is one of the bests/generic (much superior to e.g. bicubic)