i4never's repositories
bert
TensorFlow code and pre-trained models for BERT
bert4keras
keras implement of transformers for humans
bert4keras-CMLM
CMLM预训练的非官方实现
blog.archive
GitHub pages, you can visit my blog:
deep_trader
This project uses reinforcement learning on stock market and agent tries to learn trading. The goal is to check if the agent can learn to read tape. The project is dedicated to hero in life great Jesse Livermore.
DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
DeepSpeedExamples
Example models using DeepSpeed
flash-attention
Fast and memory-efficient exact attention
gpt-2
Code for the paper "Language Models are Unsupervised Multitask Learners"
keras-drop-connect
Drop-connect wrapper
Megatron-DeepSpeed
Ongoing research training transformer language models at scale, including: BERT & GPT-2
OpenTLD
Official source code for TLD
reinforce_py
reinforcement learning in Python
reinforcement-learning
Implementation of Reinforcement Learning Algorithms. Python, OpenAI Gym, Tensorflow. Exercises and Solutions to accompany Sutton's Book and David Silver's course.
ring-flash-attention
Ring attention implementation with flash attention
text-generation-inference
Large Language Model Text Generation Inference
TigerBot
TigerBot: A multi-language multi-task LLM
toolkit
常用工具
TransformerEngine
A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilization in both training and inference.
transformers
🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
trl
Train transformer language models with reinforcement learning.