Inference VRAM requirements?
illtellyoulater opened this issue · comments
Could you share requirements for inference in terms of VRAM?
For fp16 24GB for fp32 34GB would be sufficient.
Interesting! Do you think there is room for optimization in order to bring that number further down below?
Anyway, keep up with your great work!