[bug]: Meta Data Lost With Built-In Upscaler
brainbent opened this issue · comments
Is there an existing issue for this problem?
- I have searched the existing issues
Operating system
Windows
GPU vendor
Nvidia (CUDA)
GPU model
RTX 3060
GPU VRAM
12GB
Version number
4.2.0A3
Browser
Chrome
Python dependencies
No response
What happened
When using the built-in upscaler from the toolbar, the output image loses it's meta data.
What you expected to happen
Back in v2.3 (at least), meta data was maintained when an upscale was performed.
How to reproduce the problem
Click the upscale icon from the toolbar, and choose any option.
Additional context
I have no idea when the function broke, as I've jumped from 2.3.5.post2 straight to 4..2.0a3, when an upgrade to 3.0.0 broke my install and I gave up on the project until now...
Discord username
serenity.brainbent
Hi, this has been the behaviour for quite some time now. Internally, the app works very, very differently than v2.3.5, and some operations do not retain metadata.
I'd be happy to review a PR to improve on this. Here are my thoughts on implementation.
It is probably fairly tricky to implement a generalized backend solution. The main problem is that a node may do anything with a given image and have any number of outputs, so it's probably not feasible to automatically copy metadata for all image operations. We'd need to have special handling to retrieve and save out metadata in every node.
A client solution for this one specific case is fairly straightforward. When building the ad-hoc upscaling graph, you could fetch the metadata for the image using RTK Query's imperative API and add it to the graph. Here's an example of using this API in the app (you'd be using the metadata endpoint instead of the DTO endpoint).
ESRGANInvocation accepts an arbitrary dict for metadata - you can just put the retrieved metadata into the node:
You could optionally extend the retrieved image metadata, adding some upscaling related metadata to it. Keep in mind, though, that we do not support generating an image and upscaling it in one go, so you won't be able to "recall" the upscale.
Thanks for the reply! Sadly, I'm too dumb to do anything at the level of custom API integration, but I appreciate the suggestion anyway..!
I've found that the latest builds of Photoshop and Topaz Photo both retain the meta, so I have a way of accomplishing the upscale in software; it's just a shame it's not something I can do from the Invoke UI now...
Didn't mean to suggest you specifically should implement the functionality - just some guidance for any contributor who might want to address the problem.
I'm curious, how do PS/Topaz compare to the upscaling we do? Ours isn't anything special, we just use the standard ESRGAN models widely used in open source apps. I would have guessed PS and Topaz give better results.
Well, Topaz has a lot of control, which you'd expect for the price! I'm running it in trial mode for now - but I'd rather not fork over 200 bucks If I don't need to!
The 2024 version of Photoshop adds a bunch of generative stuff, but it's not close to a proper SD product...