mahamatnoumai / open-llms

A list of open LLMs available for commercial use

Open LLMs

These LLMs are all licensed for commercial use (e.g., Apache 2.0). Contributions and corrections welcome!

• T5

  • Checkpoints: T5 & Flan-T5, Flan-T5-xxl (HF)
  • Paper/blog: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
  • Size: 60M - 11B
  • Licence: Apache 2.0

• UL2

  • Checkpoints: UL2 & Flan-UL2, Flan-UL2 (HF)
  • Paper/blog: UL2 20B: An Open Source Unified Language Learner
  • Size: 20B
  • Licence: Apache 2.0

• Cerebras-GPT

  • Checkpoints: Cerebras-GPT
  • Paper/blog: Cerebras-GPT: A Family of Open, Compute-efficient, Large Language Models, Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster
  • Size: 111M - 13B
  • Licence: Apache 2.0

• Pythia

  • Checkpoints: pythia 70M - 12B
  • Paper/blog: Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
  • Size: 70M - 12B
  • License: Apache 2.0

• Dolly

  • Checkpoints: dolly-v2-12b
  • Paper/blog: Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM
  • Size: 3B, 7B, 12B
  • Licence: MIT

• RWKV

  • Checkpoints: RWKV, ChatRWKV
  • Paper/blog: The RWKV Language Model (and my LM tricks)
  • Size: 100M - 14B
  • Licence: Apache 2.0

• GPT-J-6B

  • Checkpoints: GPT-J-6B, GPT4All-J
  • Paper/blog: GPT-J-6B: 6B JAX-Based Transformer
  • Size: 6B
  • Licence: Apache 2.0

• StableLM

  • Checkpoints: StableLM
  • Paper/blog: Stability AI Launches the First of its StableLM Suite of Language Models
  • Size: 3B - 65B
  • Licence: CC BY-SA-4.0 license

• StarCoder

  • Checkpoints: starcoder
  • Paper/blog: StarCoder: A State-of-the-Art LLM for Code, StarCoder: May the source be with you!
  • Size: 15B
  • Licence: BigCode OpenRAIL-M v1

• MPT-7B

  • Checkpoints: MPT-7B base, instruct, etc
  • Paper/blog: Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs
  • Size: 7B
  • Licence: Apache 2.0 for base and storywriter

---

Want to contribute? Just add to the above with the following

• Name of model
  • Checkpoints:
  • Paper/blog:
  • Size:
  • Licence:

---

Improvements

• Add context size?
• Add (links to) eval benchmarks?
• Update to use table format