Use fp32 instead of fp16
andreasjansson opened this issue · comments
Discord user cakeofzerg#3653 investigated the effect of using fp16 instead of fp32 and found that fp32 produces better results overall.
We should change the Replicate model to use fp32.
This has snuck into #24. We should compare average run time of both to make sure there isn't a significant difference.
Regardless of run time difference (if you find there is any), it would be good to have the option to use fp32 (opt-in setting with warning?)
I believe it hits memory limits sooner for higher resolutions as well.
@andreasjansson Is this resolved now?
This seems to be live, though we are hitting memory limits lower, especially when trying to do a set of 4. Example I hit a memory error for 512x1024 or 512x896 if you try to do 4 at once. Since it works with 1 at a time i think that could be optimized in code, maybe doing one by one instead of in parallel or something like that?
Good idea. 👍 #28