nikileshsa/h2ogpt

h2oGPT - The world's best open source GPT

Our goal is to make the world's best open source GPT!

Try h2oGPT now

Live hosted instances:

Screen.Recording.2023-04-18.at.4.10.58.PM.mov

Current state

Open-source repository with fully permissive, commercially usable code, data and models
Code for preparing large open-source datasets as instruction datasets for fine-tuning of large language models (LLMs), including prompt engineering
Code for fine-tuning large language models (currently up to 20B parameters) on commodity hardware and enterprise GPU servers (single or multi node)
Code to run a chatbot on a GPU server, with shareable end-point with Python client API
Code to evaluate and compare the performance of fine-tuned LLMs

All open-source datasets and models are posted on 🤗 H2O.ai's Hugging Face page.

Also check out H2O LLM Studio for our no-code LLM fine-tuning framework!

Roadmap items

Integration of code and resulting LLMs with downstream applications and low/no-code platforms
Complement h2oGPT chatbot with search and other APIs
High-performance distributed training of larger models on trillion tokens
Improve code completion, reasoning, mathematics, factual correctness, hallucinations and avoid repetitions

Chat with h2oGPT

git clone https://github.com/h2oai/h2ogpt.git
cd h2ogpt
pip install -r requirements.txt
python generate.py --base_model=h2oai/h2ogpt-oig-oasst1-512-6.9b

and then use browser at http://0.0.0.0:7860 or the public live URL printed by the server.

For help installing a Python 3.10 environment or CUDA toolkit, see the installation instructions

You can also use Docker for inference.

Larger models require more GPU memory

Depending on available GPU memory, you can load differently sized models. For multiple GPUs, automatic sharding can be enabled with --infer_devices=False, but that is disabled by default since cuda:x cuda:y mismatches can occur.

For GPUs with at least 24GB of memory, we recommend:

python generate.py --base_model=h2oai/h2ogpt-oasst1-512-12b

For GPUs with at least 48GB of memory, we recommend:

python generate.py --base_model=h2oai/h2ogpt-oasst1-512-20b

The numbers 256 and 512 in the model names indicate the cutoff lengths (in tokens) used for fine-tuning. Shorter values generally result in faster training and more focus on the last part of the provided input text (consisting of prompt and answer).

More information about the models can be found on H2O.ai's Hugging Face page.

Why H2O.ai?

Our Makers at H2O.ai have built several world-class Machine Learning, Deep Learning and AI platforms:

#1 open-source machine learning platform for the enterprise H2O-3
The world's best AutoML (Automatic Machine Learning) with H2O Driverless AI
No-Code Deep Learning with H2O Hydrogen Torch
Document Processing with Deep Learning in Document AI

We also built platforms for deployment and monitoring, and for data wrangling and governance:

H2O MLOps to deploy and monitor models at scale
H2O Feature Store in collaboration with AT&T
Open-source Low-Code AI App Development Frameworks Wave and Nitro
Open-source Python datatable (the engine for H2O Driverless AI feature engineering)

Many of our customers are creating models and deploying them enterprise-wide and at scale in the H2O AI Cloud:

Multi-Cloud or on Premises
Managed Cloud (SaaS)
Hybrid Cloud
AI Appstore

We are proud to have over 25 (of the world's 280) Kaggle Grandmasters call H2O home, including three Kaggle Grandmasters who have made it to world #1.

Development

Follow the installation instructions to create a development environment for training and generation.
Follow the fine-tuning instructions to fine-tune any LLM models on your data.
Follow the Docker instructions to create a container for deployment.

Help

FAQs

Acknowledgements

Some training code was based upon March 24 version of Alpaca-LoRA.
Used high-quality created data by OpenAssistant.
Used base models by EleutherAI.
Used OIG data created by LAION.

Disclaimer

Please read this disclaimer carefully before using the large language model provided in this repository. Your use of the model signifies your agreement to the following terms and conditions.

Biases and Offensiveness: The large language model is trained on a diverse range of internet text data, which may contain biased, racist, offensive, or otherwise inappropriate content. By using this model, you acknowledge and accept that the generated content may sometimes exhibit biases or produce content that is offensive or inappropriate. The developers of this repository do not endorse, support, or promote any such content or viewpoints.
Limitations: The large language model is an AI-based tool and not a human. It may produce incorrect, nonsensical, or irrelevant responses. It is the user's responsibility to critically evaluate the generated content and use it at their discretion.
Use at Your Own Risk: Users of this large language model must assume full responsibility for any consequences that may arise from their use of the tool. The developers and contributors of this repository shall not be held liable for any damages, losses, or harm resulting from the use or misuse of the provided model.
Ethical Considerations: Users are encouraged to use the large language model responsibly and ethically. By using this model, you agree not to use it for purposes that promote hate speech, discrimination, harassment, or any form of illegal or harmful activities.
Reporting Issues: If you encounter any biased, offensive, or otherwise inappropriate content generated by the large language model, please report it to the repository maintainers through the provided channels. Your feedback will help improve the model and mitigate potential issues.
Changes to this Disclaimer: The developers of this repository reserve the right to modify or update this disclaimer at any time without prior notice. It is the user's responsibility to periodically review the disclaimer to stay informed about any changes.

By using the large language model provided in this repository, you agree to accept and comply with the terms and conditions outlined in this disclaimer. If you do not agree with any part of this disclaimer, you should refrain from using the model and any content generated by it.

nikileshsa / h2ogpt