[Feature Request] Intergrate the new PatchFusion Depth Estimation Model into this project.
J-Cott opened this issue · comments
The Detail they have achieved looks very impressive, would it be possible to use this model in Automatic1111? :
https://github.com/zhyever/PatchFusion
Is it better than BOOST?
Very Impressive! The hugging faces repo suggests it requires a large amount of VRAM, they suggest 24g. But the results look very good. This might be a more challenging integration.
24g wow, that would be a beefy GPU! We went from something that could reasonably run on a CPU (smol Midas), then to average GPUs (BOOST or Zoedepth) and then into server hardware. Transformers might be a hot topic, but depth approximation is also growing rapidly and this beast is hungry for more flops! 😄
Ok, then who knows, maybe one day. If somebody can create a MR for this it would be a pleasure to merge. An integration would require refactoring is_boost boolean type of thing into patching enum kind of thing, with options [no, BOOST, PatchFusion] or something like this. The same for the UI.
It looks like it uses midas, zoedepth, stable diffusion, and controlnet without offload vram though soo it should be possible to significantly reduce the requirement.
Wait, it uses all these things at the same time? Interesting...