CarperAI / trlx

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)

CarperAI/trlx Issues

Issue of tensors share memory
Updated 22 days ago1
TRLX Environment customization
Updated a month ago
Unable to load and run inference on finetuned Alpaca model
Closed 9 months ago6
Unable to load the trained model to do the inference
Closed 9 months ago9
when i use trlx ppotrainer train a model llama 13b model, but saved huggingface mode ,but when it inference , it has some strange keys ,and the inference result did not show ,it also have no error , it seems the result disapper
Updated 3 months ago1
Runtime error when running examples (ilql_sentiments_t5.py)
Updated 3 months ago2
[New Feature Request] Add KTO
Updated 4 months ago
RLHF text summarization diverges
Updated 4 months ago
Integration of Self-Play Fine-Tuning (SPIN) Method for Enhancing Large Language Models
Updated 4 months ago
MPT is not working
Updated 5 months ago
Attention mask when calculating log ratio for PPO
Updated 6 months ago
Multi-GPU training errors with peft
Updated 6 months ago
Issue since most recent transformers update
Updated 6 months ago
multigpu support for summarization ppo example
Updated 7 months ago3
resume_from_checkpoint doesn't work
Closed 7 months ago1
Support parallel reward_fn in PPO training
Closed 7 months ago
`position_ids` error in accelerate PPO trainer
Closed 7 months ago3
TypeError: reward_fn() got an unexpected keyword argument 'tokenizer'
Closed 7 months ago1
Question about saving peft checkpoint
Updated 7 months ago2
Problem with LLama training with LoRA
Updated 7 months ago3
How to generate reward-labeled dataset
Updated 8 months ago
How to train LLaMA2 on the summarize_rlhf example?
Updated 8 months ago
Increasing max new tokens for generation arguments lead to errors
Updated 8 months ago3
Training stuck generating rollouts
Closed 9 months ago6
strange design
Closed 9 months ago1
Sanity check: SFT Model should be frozen (PPO)
Closed 9 months ago2
Reward model negative numbers meaning
Closed 9 months ago2
Model does not load in the expected dtype
Closed 9 months ago5
RuntimeError: module must have its parameters and buffers on device
Closed 9 months ago4
Add support for Falcon 7B/40B
Updated 9 months ago1
Memory occupy with multi GPUs Training
Updated 9 months ago1
Caught signal 7 (Bus error: nonexistent physical address)
Closed 10 months ago5
ILQL training batch2 tensor dimensions error
Updated 10 months ago2
Direct Policy Optimization
Updated 10 months ago4
Add support for LLaMA2
Closed 10 months ago1
Implements beam search of AutoModelForCausalLMWithILQLHeads
Closed 10 months ago1
Implement Asynchronous PPO
Updated 10 months ago
SFT wrong for Anthropic-HH, leading to poor model quality?
Closed 10 months ago2
ppo using GLM2-6b as a backbone?
Updated 10 months ago1
support base model + multi adapter for actor, critic, ref and reward model
Updated 10 months ago
8-bit inference
Updated a year ago
sanity check: PPO `log_ratio` should be zero when training is disabled
Closed a year ago1
Error loading T5 model checkpoint for inference after training with ILQL
Updated a year ago
Add support for safetensors
Updated a year ago
About the weight of word embedding being nan
Updated a year ago1
Use tiny models for the tests
Updated a year ago
Type hint doesn't match documentation for prompts arg of PromptPipeline.__init__
Closed a year ago1
Support for CodeGenForCausalLM
Updated a year ago3
Support logging for non-scalar metrics
Updated a year ago2
My loss graph seems weird. (PPO training)
Closed a year ago8