nzjin

LoRAMoE

LoRAMoE: Revolutionizing Mixture of Experts for Maintaining World Knowledge in Language Model Alignment

DeepSeek-MoE

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

OpenMoE

A family of open-sourced Mixture-of-Experts (MoE) Large Language Models

AdaMix

This is the implementation of the paper AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning (https://arxiv.org/abs/2205.12410).

EMoE

Official PyTorch Implementation of EMoE: Unlocking Emergent Modularity in Large Language Models [main conference @ NAACL2024]

parameter-efficient-moe

Switch-NeRF

Codes for Switch-NeRF (ICLR 2023)

awesome_moe

The collections of MOE (Mixture Of Expert) papers, code and tools, etc.

ODTQA

The data and processing code of the paper "Enhancing Open-Domain Table Question Answering via Syntax- and Structure-aware Dense Retrieval"

Chinese-Mixtral-8x7B

中文Mixtral-8x7B（Chinese-Mixtral-8x7B）

mixture-of-experts

A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models

flash-attention

Fast and memory-efficient exact attention

llama-moe

⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training

llama-moe-v1

LLaMA-Factory

A WebUI for Efficient Fine-Tuning of 100+ LLMs (ACL 2024)

LLMSurvey

The official GitHub page for the survey paper "A Survey of Large Language Models".

Awesome-Chinese-LLM

整理开源的中文大语言模型，以规模较小、可私有化部署、训练成本较低的模型为主，包括底座模型，垂直领域微调及应用，数据集与教程等。

LLM-Agent-Paper-List

The paper list of the 86-page paper "The Rise and Potential of Large Language Model Based Agents: A Survey" by Zhiheng Xi et al.

codedemo

Few-shot-NL2SQL-with-prompting

Open_WikiTable

Open-WikiTable :Dataset for Open Domain Question Answering with Complex Reasoning over Table

www2018-table

sigir2019-table2vec

GTR

[SIGIR 2021] Retrieving Complex Tables with Multi-Granular Graph Representation Learning.

Web-Table-Retrieval-Benchmark

trec_eval

Evaluation software used in the Text Retrieval Conference

table-retrieval

TestSuiteEval

"Semantic Evaluation for Text-to-SQL with Distilled Test Suite", EMNLP2020

RESDSQL

The Pytorch implementation of RESDSQL (AAAI 2023).

rasat

The official implementation of the paper "RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL"(EMNLP 2022)