hyunwoongko

kss

KSS: Korean String processing Suite

pecab

Pecab: Pure python Korean morpheme analyzer based on Mecab

kiwipiepy

Python API for Kiwi

transformer

Transformer: PyTorch Implementation of "Attention Is All You Need"

hyunwoongko

miscellaneous

OpenRLHF

An Easy-to-use, Scalable and High-performance RLHF Framework (Support 70B+ full tuning & LoRA & Mixtral & KTO)

Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

mamba

stripedhyena

fast_retention

Speed up Parallel Retention about 2x times

openchat

OpenChat: Easy to use opensource chatting framework via neural networks

stop-sequencer

Implementation of stop sequencer for Huggingface Transformers

text-generation-inference

Large Language Model Text Generation Inference

kocrawl

Collection of useful Korean crawlers

kochat

Opensource Korean chatbot framework

python-mecab-kor

Yet another python binding for mecab-ko

beyond-lm

Beyond LM: How can language model go forward in the future?

Open-Assistant

OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

gpt2-tokenizer-java

Java implementation of GPT2 tokenizer.

trl

Train transformer language models with reinforcement learning.

summarizers

Package for controllable summarization

pydatrie

Pure python implementation of DARTS (Double ARray Trie System)

nlp-datasets

Curation note of NLP datasets

DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

code-pipeline

Test repository for AWS code-pipeline deployment.

ooo-backprop

kobart-transformers

Kobart model on Huggingface transformers

simpleDATrie

Simple Double Array Trie implementation

python-nori

Pynori - Lucene Nori, Korean Mopological Analyzer, in Python

dialobot

Opensource chatbot framework