salesforce / CodeGen

CodeGen is an open-source model for program synthesis. Trained on TPU-v4. Competitive with OpenAI Codex.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How do I train this on custom data?

hg0428 opened this issue · comments

I am completely new to AI, I would like to know how I can train it to recognize a new language.
I have no idea what to do. I can find no docs online about this. I have attempted to train it using the Trainer from transformers, but I keep coming up with errors. Can I have a code example for this?

You can find a minimal example of fine-tuning the models with DeepSpeed here:

https://github.com/salesforce/jaxformer#a100-fine-tune

You can find a minimal example of fine-tuning the models with DeepSpeed here:

https://github.com/salesforce/jaxformer#a100-fine-tune

How can I do it with my own custom data? I have a dict of expected inputs to expected outputs. I am trying to teach it a new language.
And also, should the dict be input:input+generation or input:generation.
I'd think it would be the former.
BTW, I have no GPU.

You can find a minimal example of fine-tuning the models with DeepSpeed here:

https://github.com/salesforce/jaxformer#a100-fine-tune

@enijkamp I need help here, I can't find any online resources.