Lighter text encoder

Question

igor-davidyuk opened this issue a year ago · comments

Hey guys, thank you for the awesome repo!
Is there any chance that you will support a lighter tex encoder?

phalexo · Answer 1 · Wed May 10 2023 00:05:22 GMT+0800 (China Standard Time)

If by lighter you mean requiring less VRAM, then you can try to set the dtype to float16. T5 is then about 11.6GiB.

Igor Davidyuk · Answer 2 · Wed May 10 2023 18:09:30 GMT+0800 (China Standard Time)

If by lighter you mean requiring less VRAM, then you can try to set the dtype to float16. T5 is then about 11.6GiB.

Thank you for your advice, but my CU does not support fp16 operations. Anyway, it looks like T5 XXL is too big and a smaller embedder would do