kyegomez / BitNet

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch

Home Page:https://discord.gg/qUtxnK2NMf

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

forward method in Class BitLinear

guoqixin1 opened this issue · comments

hello, thanks for your Implementation.
I was a bit confused while reading the bitnet/bitlinear.py forward()
as the paper shown:
企业微信截图_09dcb834-cb93-4cd5-8d02-84d33f63a955
i think the forward method should be:
image
did i misunderstand the process?

@guoqixin1 Hey thanks for pointing this out, we changed it yesterday so now there is a new verison let me know if you find any new issues