epfml / landmark-attention

Landmark Attention: Random-Access Infinite Context Length for Transformers

https://arxiv.org/abs/2305.16300

epfml/landmark-attention Issues

Question about training stability
Updated 7 months ago
[Q]: Quantize Per-Trained model Using QLoRa or LoRa , PFET Technique
Updated 9 months ago
Will this work with Qlora and 4bit inference?
Updated 10 months ago36
when using auto_insert_landmarks, it appears to be view problems
Updated 10 months ago1
Assertion `srcIndex < srcSelectDimSize` failed while running test.
Closed a year ago2
Peft module
Closed a year ago
How much VRAM do you need to run Inference?
Updated a year ago
Is true attention (full-context) calculation needed during training?
Closed a year ago3
Question about Training vs LLaMA Fine Tuning
Closed a year ago2
Unlimited context length and no prompt token limit
Closed a year ago
hello! i reached out on twitter before the release and received the link here when you guys dropped this
Closed a year ago
Readme clarification
Closed a year ago1
model_name_or_path defaults
Updated a year ago