Lichang-Chen / AlpaGasus

A better Alpaca Model Trained with Less Data (only 9k instructions of the original set)

AlpaGasus: Training a Better Alpaca Model with Fewer Data

Lichang Chen*, Shiyang Li*, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, Hongxia Jin

*Denotes equal contribution

Project page | Paper

Our Model "AlpaGasus"is pronounced as "/ˈælpəˈɡeɪsəs/", or "/ˈælpəˈɡəsəs/". The logo is generated by Midjourney

News

• [2023.7] We release our paper. If you have any questions about our project, please send email to [e-mail]
• [2023.9] Thanks to @GPT4animal for reimplementing the results in our paper. Please check this fantastic repo: https://github.com/gpt4life/alpagasus.
• [2023.9] Thanks to @gauss5930 and @YooYunS who implemented the QLoRA version of Alpagasus-7B and 13B, which could be run on the customer-level GPUs. please refer to their repo: Alpagasus2-QLoRA They also show that tuning LLaMA-2 could achieve better performance.

Citation

If you find our paper useful, please consider citing:

@article{chen2023alpagasus,
  title={AlpaGasus: Training a Better Alpaca with Fewer Data},
  author={Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, Hongxia Jin},
  journal={arXiv preprint arXiv:2307.08701},
  year={2023}
}