Giters

CarperAI / trlx

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

8-bit inference

glerzing opened this issue · comments

commented

🐛 Describe the bug

When using 8-bit inference, there is a hook function that is created by the accelerate library and that replaces the forward method. When using inspect.getfullargspec to set self.forward_kwargs with the arguments of forward, we actually inspect the arguments of the hook instead of the arguments of the real forward function called by the hook.

Which trlX version are you using?

0.7.0

Additional system and package information

No response