Does codegen-16B-mono support "load_in_8bit" in huggingface?

Question

Does codegen-16B-mono support "load_in_8bit" in huggingface?

Leolty opened this issue a year ago · comments

Tianyang Liu commented a year ago

As titled. I feel that I cannot load it with int8 on a single GPU (24G)

Tianyang Liu · Answer 1 · Mon May 08 2023 09:06:01 GMT+0800 (China Standard Time)

nvm, I have solved this. It works perfectly fine now. Thanks

peppa · Answer 2 · Tue Jul 25 2023 22:54:36 GMT+0800 (China Standard Time)

Do you know how to fine-tune the codegen model with your own code dataset?

Tianyang Liu · Answer 3 · Wed Jul 26 2023 16:08:59 GMT+0800 (China Standard Time)

@peppa-nwpu Hi, maybe you can refer this finetune.py, this is what I did to finetune. You can customize the CodeGenDataset for your own code data.