continual-pre-training

There are 1 repository under continual-pre-training topic.

pjlab-sys4nlp / llama-moe
⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (EMNLP 2024)
llama llm mixture-of-experts moe continual-pre-training expert-partition
Language:Python 987
GAIR-NLP / ProX
Offical Repo for "Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale"
data-centric-ai data-quality llm neural-symbolic pre-training continual continual-pre-training llama mistral
Language:Python 234
RUC-GSAI / Llama-3-SynE
Llama-3-SynE: A Significantly Enhanced Version of Llama-3 with Advanced Scientific Reasoning and Chinese Language Capabilities | 继续预训练提升 Llama-3 的科学推理和中文能力
chinese continual-pre-training large-language-model reasoning science
Language:Python 32
SliMM-X / CoMP-MM
Official repository of "CoMP: Continual Multimodal Pre-training for Vision Foundation Models"
continual-pre-training large-multimodal-models vision-foundation-model
Language:Python 20
adithya-s-k / Indic-llm
A open-source framework designed to adapt pre-trained Language Models (LLMs), such as Llama, Mistral, and Mixtral, to a wide array of domains and languages.
continual-pre-training dpo finetuning finetuning-llms llm lora
Language:Python 14

pjlab-sys4nlp / llama-moe