joey00072 / TinyLora

Low-Rank Adaptation of Large Language Models clean implementation

TinyLora

Easy hackable for Low-Rank Adaptations,

LoRA: Low-Rank Adaptation of Large Language Models https://arxiv.org/abs/2106.09685

TODO

• Update Readme
• Add way to save only lora weights
• Add merge layer
• Add unmerge layer