bjm88620 / OpenLLaMA-Chinese

OpenLLaMA-Chinese, a permissively licensed open source instruction-following models based on OpenLLaMA

OpenLLaMA-Chinese

OpenLLaMA-Chinese is a 100% free Chinese large language model, and can be utilized for both non-commercial and commercial purposes.

OpenLLaMA-Chinese is built on OpenLLaMA, which is a permissively licensed open-source reproduction of Meta AI's LLaMA 7B and 13B models, trained on the RedPajama dataset. OpenLLaMA also includes a smaller 3B variant of the LLaMA model. We have conducted fine-tuning on Chinese and English instructions using the OpenLLaMA base models and have made our weights publicly available.

Chinese Instructions Fine-tuning Weights:

• OpenLLaMA 3B
• OpenLLaMA 7B
• [OpenLLaMA 13B](coming soon!)

English Instructions Fine-tuning Weights:

• [OpenLLaMA 3B](coming soon!)
• [OpenLLaMA 7B](coming soon!)
• [OpenLLaMA 13B](coming soon!)

Chinese+English Instructions Fine-tuning Weights:

• OpenLLaMA 3B
• OpenLLaMA 7B
• [OpenLLaMA 13B](coming soon!)

Data

For Chinese fine-tuning, we utilized the alpaca_data_zh_51k.json from the Chinese-LLaMA-Alpaca project.

For English fine-tuning, we employed the alpaca_data.json from the StanfordAlpaca project.

For fine-tuning with both English and Chinese instructions, we used data from both sources.

Usage

We provide inference code based on Jittor and PyTorch

Jittor Inference

coming soon.

Torch Inference

We modified the generate code from LLaMA-X.

To use the PyTorch inference code, follow these steps:

1. Download the weights and update the base_model path in inference/gen_torch.py.
2. Run the following command:

python inference/gen_torch.py

Pretraining and Finetuning

FittenTech offers LLMs pretraining and fine-tuning services. For more details, please visit https://llm.fittentech.com/.

Acknowledgments

We would like to express our gratitude to the developers of the following open-source projects, as our project builds upon their work:

• LLaMA
• OpenLLaMA
• Jittor
• JittorLLMs
• LLaMA-X
• Chinese-LLaMA-Alpaca
• StanfordAlpaca
• transformers

License

We adopt the Apache License, following OpenLLaMA's license.