What is the hardware requirement for fine tuning codegen 2B and higher models?
Kushalamummigatti opened this issue · comments
I have tried to finetune 2B on 40GB GPU which i faced memory out of error. Any suggestion to finetune 2B and above models?
salesforce / CodeGen
CodeGen is a family of open-source model for program synthesis. Trained on TPU-v4. Competitive with OpenAI Codex.