pretrained-language-model

There are 5 repositories under pretrained-language-model topic.

wenge-research / YAYI2
YAYI 2 是中科闻歌研发的新一代开源大语言模型，采用了超过 2 万亿 Tokens 的高质量、多语言语料进行预训练。(Repo for YaYi 2 Chinese LLMs)
artificial-intelligence chat chinese gpt natural-language-generation pretrained-language-model yayi
Language:Python 3598
microsoft / torchscale
Foundation Architecture for (M)LLMs
computer-vision machine-learning multimodal natural-language-processing pretrained-language-model speech-processing transformer translation
Language:Python 2967
Separius / awesome-sentence-embedding
A curated list of pretrained sentence and word embedding models
awesome awesome-list bert contextualized-representation cross-lingual embedding-models language-model natural-language nlp pretrained-embedding pretrained-language-model pretrained-models sentence-embeddings sentence-representations subword-models unsupervised-learning word-embeddings wordembedding
Language:Python 2200
THUDM / P-tuning-v2
An optimized deep prompt tuning strategy comparable to fine-tuning across scales and tasks
natural-language-processing p-tuning parameter-efficient-learning pretrained-language-model prompt-tuning
Language:Python 1925
xcfcode / Summarization-Papers
Summarization Papers
chatgpt natural-language-processing nlp pretrained-language-model summarization text-generation
Language:TeX 976
thunlp / OpenDelta
A plug-and-play library for parameter-efficient-tuning (Delta Tuning)
nlp nlp-library pretrained-language-model parameter-efficient-learning deep-learning
Language:Python 955
AndrewZhe / lawyer-llama
中文法律LLaMA (LLaMA for Chinese legel domain)
alpaca large-language-models legal-ai llama llm nlp plm pretrained-language-model pretrained-models
Language:Python 798
allenai / dont-stop-pretraining
Code associated with the Don't Stop Pretraining ACL 2020 paper
natural-language-processing pretrained-language-model
Language:Python 522
OpenBMB / CPM-Live
Live Training for Open-source Big Models
deep-learning multi-task-learning natural-language-generation natural-language-understanding nlp parameter-efficient-learning pretrained-language-model natural-language-processing
Language:Python 510
gaoisbest / NLP-Projects
word2vec, sentence2vec, machine reading comprehension, dialog system, text classification, pretrained language model (i.e., XLNet, BERT, ELMo, GPT), sequence labeling, information retrieval, information extraction (i.e., entity, relation and event extraction), knowledge graph, text generation, network embedding
word2vec sentence2vec machine-reading-comprehension dialogue-systems text-classification pretrained-language-model sequence-labeling information-retrieval information-extraction knowledge-graph text-generation network-embedding
Language:OpenEdge ABL 506
RenzeLou / awesome-instruction-learning
Papers and Datasets on Instruction Tuning and Following. ✨✨✨
pretrained-language-model instruction-learning paper-list awesome-list datasets in-context-learning large-language-models prompt survey instruction instruction-tuning
Language:Python 426
Hzfinfdu / Diffusion-BERT
ACL'2023: DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models
bert conditional-generation diffusion-models pretrained-language-model text-generation unconditional-generation
Language:Python 278
LYH-YF / MWPToolkit
MWPToolkit is an open-source framework for math word problem(MWP) solvers.
deep-learning pytorch pretrained-language-model sequence-to-sequence sequence-to-tree graph-to-tree math-word-problem
Language:Python 159
ZhengZixiang / ATPapers
Worth-reading papers and related resources on attention mechanism, Transformer and pretrained language model (PLM) such as BERT. 值得一读的注意力机制、Transformer和预训练语言模型论文与相关资源集合
attention-mechanism awesome bert pretrained-language-model transformer
129
yueyu1030 / AttrPrompt
[NeurIPS 2023] This is the code for the paper `Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias`.
attributed-text data-centric-ai large-language-models natural-language-processing pretrained-language-model text-classification training-data-generation zero-shot-learning
Language:Python 124
microsoft / COCO-LM
[NeurIPS 2021] COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining
language-model pretrained-language-model natural-language-processing natural-language-understanding pretraining representation-learning deep-learning transformers contrastive-learning
Language:Python 120
hyintell / awesome-refreshing-llms
EMNLP'23 survey: a curation of awesome papers and resources on refreshing large language models (LLMs) without expensive retraining.
awesome-list continual-learning knowledge-editing large-language-models llm llms natural-language-processing nlp paper pretrained-language-model refreshing retrieval-augmented-generation review survey update-llm
117
git-disl / BERT4ETH
BERT4ETH: A Pre-trained Transformer for Ethereum Fraud Detection (WWW23)
bert transformer deanonymization blockchain ethereum fraud-detection www2023 phishing-detection pretrained-language-model
Language:Python 95
thunlp / Prompt-Transferability
On Transferability of Prompt Tuning for Natural Language Processing
nlp parameter-efficient-learning parameter-efficient-tuning pretrained-language-model pretrained-language-models pretrained-models prompt prompt-tuning pytorch transfer-learning
Language:Python 92
SJTU-IPADS / Bamboo
Bamboo-7B Large Language Model
large-language-models llm powerinfer pretrained-language-model pretrained-models sparse-llm
86
yumeng5 / TopClus
[WWW 2022] Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations
language-model pretrained-language-model topic-modeling topic-discovery clustering
Language:Python 84
RUCAIBox / UniCRS
[KDD22] Official PyTorch implementation for "Towards Unified Conversational Recommender Systems via Knowledge-Enhanced Prompt Learning".
conversation conversational-ai conversational-bots dialog dialogue dialogue-systems pretrained-language-model pretrained-models pretraining prompt prompt-tuning prompts recommendation recommender-system
Language:Python 71
GanjinZero / CODER
CODER: Knowledge infused cross-lingual medical term embedding for term normalization. [JBI, ACL-BioNLP 2022]
nlp medical pretrained-language-model umls multi-language embeddings
Language:Python 69
yumeng5 / SuperGen
[NeurIPS 2022] Generating Training Data with Language Models: Towards Zero-Shot Language Understanding
zero-shot-learning natural-language-processing natural-language-understanding pretrained-language-model text-generation text-classification
Language:Python 59
FranxYao / PoincareProbe
Implementation of ICLR 21 paper: Probing BERT in Hyperbolic Spaces
bert bert-embeddings bert-model bertology hyperbolic hyperbolic-embeddings hyperbolic-geometry pretrained-language-model probing probing-tasks
Language:Jupyter Notebook 54
SKplanet / Dialog-KoELECTRA
ELECTRA기반 한국어 대화체 언어모델
korean-language pretrained-language-model electra-models dialog
Language:Python 54
EagleW / Scientific-Inspiration-Machines-Optimized-for-Novelty
Official implementation of the ACL 2024: Scientific Inspiration Machines Optimized for Novelty
gpt4 hypothesis-generation llm pretrained-language-model pytorch retrieval-augmented-generation text-generation acl2024
Language:Python 52
GanjinZero / BioBART
BioBART: Pretraining and Evaluation of A Biomedical Generative Language Model [ACL-BioNLP 2022]
biomedical generative pretrained-language-model
Language:Python 50
ChangwenXu98 / TransPolymer
Implementation of "TransPolymer: a Transformer-based language model for polymer property predictions" in PyTorch
deep-learning polymer pretrained-language-model pytorch self-supervised-learning transformer
Language:Python 49
OpenMatch / COCO-DR
[EMNLP 2022] This is the code repo for our EMNLP‘22 paper "COCO-DR: Combating Distribution Shifts in Zero-Shot Dense Retrieval with Contrastive and Distributionally Robust Learning".
bert contrastive-learning dense-retrieval distributionally distributionally-robust information-retrieval nlp pretrained-language-model transformer zero-shot zero-shot-retrieval
Language:Python 46
heraclex12 / NLP2SPARQL
Translate Natural Language Processing to SPARQL Query and vice versa
sparql question-answering knowledge-base pretrained-language-model machine-translation bert2bert spbert
Language:Python 45
SuperBruceJia / Awesome-LLM-Self-Consistency
Awesome LLM Self-Consistency: a curated list of Self-consistency in Large Language Models
chain-of-thought chatgpt compositional-consistency factual-consistency gpt-3 gpt-4 hypothetical-consistency llms llms-reasoning logical-consistency pretrained-language-model reasoning self-consistency self-consistency-benchmark self-consistency-learning self-consistent-generation semantics semantics-consistency semantics-preserving
43
zjukg / DUET
[Paper][AAAI 2023] DUET: Cross-modal Semantic Grounding for Contrastive Zero-shot Learning
pretrained-language-model pytorch transformer zero-shot-learning cross-modal grounding semantic knowledge-transfer visual-grounding
Language:Python 41
txsun1997 / Metric-Fairness
EMNLP'2022: BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generation
bertscore deep-learning fairness-ai fairness-ml metric-bias natural-language-processing pretrained-language-model text-generation
Language:Jupyter Notebook 37
yzhan238 / CGExpan
The source code used for paper "Empower Entity Set Expansion via Language Model Probing", published in ACL 2020.
bert entity-set-expansion masked-language-models pretrained-language-model set-expansion
Language:Python 33
cheneydon / efficient-bert
This repository contains the code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation".
nlp natural-language-processing knowledge-distillation model-compression natural-language-understanding automl neural-architecture-search pretrained-language-model
Language:Python 32