pretrain

There are 1 repository under pretrain topic.

brightmart / nlp_chinese_corpus
大规模中文自然语言处理语料 Large Scale Chinese Corpus for NLP
chinese-dataset chinese-corpus pretrain word2vec nlp bert language-model wiki news question-answering chinese corpus chinese-nlp dataset text-classification
9776
SparK
keyu-tian / SparK
[ICLR'23 Spotlight🔥] The first successful BERT/MAE-style pretraining on any convolutional network; Pytorch impl. of "Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling"
bert convnet convolutional-neural-networks masked-image-modeling pre-trained-model self-supervised-learning sparse-convolution ssl cnn iclr iclr2023 deep-learning object-detection pytorch instance-segmentation mask-rcnn mae masked-autoencoder pretrain pretraining
Language:Python 1355
CLUEbenchmark / CLUECorpus2020
Large-scale Pre-training Corpus for Chinese 100G 中文预训练语料
chinese chinese-corpus datasets pretrain corpus nlp bert roberta albert
982
yangjianxin1 / Firefly-LLaMA2-Chinese
Firefly中文LLaMA-2大模型，支持增量预训练Baichuan2、Llama2、Llama、Falcon、Qwen、Baichuan、InternLM、Bloom等大模型
firefly llama llama-2 llama2 llm baichuan baichuan-13b bloom chatglm falcon internlm lora pretrain qlora qwen xverse baichaun2
Language:Python 413
microsoft / UniVL
An official implementation for " UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation"
multimodality video-text pretraining youcookii retrieval-task msrvtt caption pretrain video caption-task video-language multimodal-sentiment-analysis localization segmentation coin joint alignment video-text-retrieval
Language:Python 350
xcfcode / What-I-Have-Read
Paper Lists, Notes and Slides, Focus on NLP. For summarization, please refer to https://github.com/xcfcode/Summarization-Papers
aaai acl conversation data-augmentation emnlp gan generation gnn graph-neural-networks knowledge-distillation meta-learning naacl nlp non-autoregressive notes presentation presentations pretrain slides summarization
165
THUNLP-AIPoet / BERT-CCPoem
BERT-CCPoem is an BERT-based pre-trained model particularly for Chinese classical poetry
bert poetry pretrain
Language:Python 153
thunlp / RE-Context-or-Names
Bert-based models(BERT, MTB, CP) for relation extraction.
relation-extraction pytorch bert contrastive-learning pretrain
Language:Python 101
huzongxiang / MatDGL
MatDGL is a neural network package that allows researchers to train custom models for crystal modeling tasks. It aims to accelerate the research and application of material science.
machine-learning deep-learning neural-networks graph transformer massagepassing tensorflow materials pretrain
Language:Python 52
CoinCheung / MFM
code for paper "Masked Frequency Modeling for Self-Supervised Visual Pre-Training" (https://arxiv.org/pdf/2206.07706.pdf)
fft frequency mfm pretrain self-supervised-learning ssl
Language:Python 24
SalesforceAIResearch / pretrain-time-series-cloudops
Official code repository for the paper "Pushing the Limits of Pre-training for Time Series Forecasting in the CloudOps Domain"
cloudops forecasting pretrain time-series
Language:Python 23
nancheng58 / SSL4SR
[CCIR 2023] Self-supervised learning for Sequential Recommender Systems
baseline pretrain recommendation recommender-system self-supervised-learning sequential-recommendation
Language:Python 20
bayartsogt-ya / albert-mongolian
ALBERT trained on Mongolian text corpus
albert pretrained-model pretrain language-model mongolian transformers masked-autoencoder
Language:Jupyter Notebook 18
KennethanCeyer / diy-generative-ai-lm
Make your Generative AI LM model from the scratch (Including pretraining / SFT with LoRA)
colab genai generativeai llm lm lora nlp pretrain sft torch transformer
Language:Python 16
yongzhuo / MacroGPT-Pretrain
macrogpt大模型全量预训练(1b3,32层), 多卡deepspeed/单卡adafactor
deepspeed gpt llm macro micro pretrain 1b3
Language:Python 13
mrzjy / hoyo_public_wiki_parser
Parsing Hoyoverse game text corpus from public wikipedia
conversation corpus dialogue game genshin-impact honkai-star-rail hoyoverse llm mihoyo nlp pretrain wiki
Language:Python 9
janelu9 / EasyLLM
Running Large Language Model easily.
deepspeed megatron-lm fine-tuning pretrain llama npu qwen qwen-vl deepseek
Language:Python 8
pskliff / vtb-data-fusion
This repository provides code solution for Data Fusion Contest task 1
distilbert rubert nlp bert fine-tuning pretrain classification receipts retail huggingface
Language:Jupyter Notebook 8
arrrrrmin / albert-guide
Understanding "A Lite BERT". An Transformer approach for learning self-supervised Language Models.
albert-guide albert-models guide language-modeling nlp pretrain pretraining
Language:Python 7
tianhao-ai / Detecting-Machine-Generated-Text-COMP90051-2023S1-Project-1
This project is about to detecting the text generated by different LLM given prompt. The instance is labeled by Human and Machine, and this project utilised both traditional machine learning method and deep learning method to classify the instance.
attention-gru bidirectional-gru comp90051 domain-adaptation fine-tuning lgbmclassifier pretrain pretraining pytorch
Language:Jupyter Notebook 4
afogarty85 / applied_nlp_demos
pytorch bert natural-language-processing accelerate chatbot deepspeed nlp transformers lora t5-model pretrain
Language:Python 2
stoneyang / cv-arxiv-daily
🎓Automatically Update CV Papers Daily using Github Actions (Update Every 24th hours)
pretrain pretrained pretraining
Language:Python 1

pretrain

brightmart / nlp_chinese_corpus

keyu-tian / SparK

CLUEbenchmark / CLUECorpus2020

yangjianxin1 / Firefly-LLaMA2-Chinese

microsoft / UniVL

xcfcode / What-I-Have-Read

THUNLP-AIPoet / BERT-CCPoem

thunlp / RE-Context-or-Names

huzongxiang / MatDGL

CoinCheung / MFM

SalesforceAIResearch / pretrain-time-series-cloudops

nancheng58 / SSL4SR

bayartsogt-ya / albert-mongolian

KennethanCeyer / diy-generative-ai-lm

yongzhuo / MacroGPT-Pretrain

mrzjy / hoyo_public_wiki_parser

janelu9 / EasyLLM

pskliff / vtb-data-fusion

arrrrrmin / albert-guide

tianhao-ai / Detecting-Machine-Generated-Text-COMP90051-2023S1-Project-1

afogarty85 / applied_nlp_demos

stoneyang / cv-arxiv-daily