kyegomez / BitNet

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch

https://discord.gg/qUtxnK2NMf

Consider techniques from official training paper

EwoutH opened this issue 3 months ago · comments

Ewout ter Hoeven commented 3 months ago

Microsoft released a new paper, which contains details and tips on training a ternary LLM. Might be useful!

https://github.com/microsoft/unilm/blob/master/bitnet/The-Era-of-1-bit-LLMs__Training_Tips_Code_FAQ.pdf

Kye Gomez commented 3 months ago

@EwoutH Yes, i have integrated the new codes!

github-actions commented a month ago

Stale issue message