minhub is a collection of minimal implementations of deep learning models inspired by minGPT. Each model is designed to be self-contained in a single file with no external dependencies, making them easy to copy and integrate into your own projects.
The primary goal of minhub is to provide clean and readable code, prioritizing simplicity and understanding over speed optimization. These models are particularly suitable for educational purposes, experimentation, and as a starting point for more complex implementations.
Additionally, minhub supports loading weights from pre-trained models available in the Hugging Face model hub.
minhub currently implements:
- GPT-NeoX-20B: An Open-Source Autoregressive Language Model: 🤗gpt_neox, 📄gptneox.R
- SantaCoder: don't reach for the stars!: 🤗gpt_bigcode, 📄gptbigcode.R
- Language Models are Unsupervised Multitask Learners: 🤗gpt2, 📄gpt2.R
- LLaMA: Open and Efficient Foundation Language Models: 🤗llama, 📄llama.R
- Llama 2: Open Foundation and Fine-Tuned Chat Models: 🤗llama 2, 📄llama.R
You can install the development version of minhub like so:
remotes::install_github("mlverse/minhub")Note:
- dev versions of torch, tok and hfhub are required
- tok requires a Rust installation. See installation instructions in its repository.