faster inference

Question

faster inference

rom1504 opened this issue 2 years ago · comments

using caching @borzunov implemented 10x faster generation at https://github.com/learning-at-home/dalle-pytorch/pull/3/files

I think this could be useful

Phil Wang commented 2 years ago

🙏 🙏

Phil Wang · Answer 1 · Sun Jan 09 2022 09:29:06 GMT+0800 (China Standard Time)

oh nice! yeah, I can get this done for both dalle and nuwa in one go when I get a free stretch of time

Alexander Borzunov · Answer 2 · Tue Jan 11 2022 00:02:18 GMT+0800 (China Standard Time)

Hey! I'll make a PR to this repo with a finished version of this code today or tomorrow :)

Alexander Borzunov · Answer 3 · Tue Jan 11 2022 03:38:49 GMT+0800 (China Standard Time)

I think the code is ready, see #409. This git branch is based on the branch from #408 (however, there's no direct dependency, so one can use cached inference without merging code for sharing weights).

Phil Wang · Answer 4 · Tue Jan 11 2022 21:11:33 GMT+0800 (China Standard Time)

@borzunov merged! thank you for this amazing contribution!