SolerHo / large-model-zoo

large model Zoo collect various of large-scale model, include CV and NLP, multiModel Etc.

🏰 large model Zoo

• Introduction
• NLP models💬
• CV models👀
• Multimodels
• TODO Lists 🚩
• Reference

Introduction

This project collects various of large-scale models as follows:

• NLP
• CV

Links to resource: Github, Paper, Hugging Face Etc.

All models are not sorted by any items, may be sorted by date or parameter size, etc.

NLP models💬

Model Name	Release Date	Developer/Institute	Size of Parameter	Github	Hugging Face	modelscope(魔搭)	Framework	Paper	Closed/Open source
Transformer	2017.06	Google	N / A	[Link]	[Link]	[Link] (Alibaba DAMO)	----	[Link]	Open
GPT 1.0	2018.06	OpenAI	117M	[Link]	[Link]	----	PyTorch	[Link]	Open
Bert	2018.10	Google	110M/340M	[Link]	[Link]	[Link] (Alibaba DAMO)	TF	[Link]	Open
GPT-2	2019.02	OpenAI	124M/1158M	[Link]	[Link]	----	PyTorch	[Link]	Open
XLNet	2019.06	CMU & Google	110M/240M	[Link]	[Link]	----	TF	[Link]	Open
T5	2019.10	Google	60M/220M/770M	[Link]	[Link]	----	TF / JAX	[Link]	Open
mT5	2020.10	Google	13B	[Link]	[Link]	----	TF	[Link]	Open
GPT-3	2020.05	OpenAI	175B	[Link]	----	----	PyTorch	[Link]	Closed
Pangu-Alpha	2020.07	Huawei & Peng Cheng Lab	2.6B	[Link]	[Link]	[Link]	mindspore	[Link]	Open
CPM-2	2021.06	Tsinghua University & BAAI(北京智源AI研究院)	11B/198B	[Link]	----	----	PyTorch	[Link]	Open
T0	2021.03	Hugging Face	11B	[Link]	[Link]	----	PyTorch	[Link]	Open
PLUG	2021.04	Alibaba DAMO	27B	[Link]	----	[Link]	PyTorch	----	Open
Bloom	2021.08	Bloom	176B	----	[Link]	[Link] (langboat Tech)	PyTorch	[Link]	Closed
Codex (based on GPT3)	2021.07	OpenAI	----	----	----	----	----	[Link]	Closed
LaMDA	2022.01	Google	2B	[Link]	----	----	----	[Link]	Open
OPT	2022.01	FaceBook(Meta)	125M ~ 175M	[Link]	[Link]	----	PyTorch	[Link]	Closed
MT-NLG	2022.01	Microsoft	530B	----	----	----	PyTorch	[Link]	Closed
FLAN-T5v1.1	2021.09	Google	245B	[Link]	[Link]	----	TF	[Link]	Open
LLaMA	2023.02	FaceBook(Meta)	7B ~ 65B	[Link]	[Link]	[Link (Fengshenbang)]	PyTorch	[Link]	Open
WebGPT	2021.12	OpenAI	175B	----	----	----	----	[Link]	Closed
PaLM	2022.04	Google	540B	[Link]	----	----	PyTorch	[Link]	Open
Gopher	2021.12	DeepMind	280B	----	----	----	----	[Link]	Closed
PALM	2020.04	Alibaba DAMO	257M/483M	[Link]	----	[Link] (Alibaba DAMO)	PyTorch	[Link]	Open
GPT-NeoX	2022.04	EleutherAI	20B	[Link]	[Link]	----	PyTorch	[Link]	Open
AlphaCode	2021.01	DeepMind	----	----	----	----	----	[Link]	Closed
InstructGPT	2022.01	OpenAI	1.3B	----	----	----	----	[Link]	Closed
CodeGen	2022.01	SaleForce Research	350M/1B/3B/7B/16B	[Link]	[Link]	----	PyTorch	[Link]	Open

CV models👀

Model Name	Release Date	Developer/Firms	Size of Parameter	Domain	Github	Hugging Face	Supported Framework	Paper	Closed / Open source	FLOPS	Top-1 Error	Top-5 Error
ResNet

Multimodels

TODO Lists 🚩

• NLP models
• CV models
• Hybrid models
• Other

Reference

• LLM：https://arxiv.org/pdf/2303.18223.pdf
• https://www.datalearner.com/ai-models/pretrained-models?&aiArea=2&openSource=-1&publisher=-1