EleutherAI / pythia

The hub for EleutherAI's work on interpretability and learning dynamics

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Weights tying

link-er opened this issue · comments

Hello,

I tried to understand from the config if the weights tying (i.e., sharing weights between embedding and un-embedding layers) was used when training, but was confused by the name of the parameter (no-weights-tying=True) - does it mean that no weights tying was used?

Weight tying was not used.