allenai / OLMo

Modeling, training, eval, and inference code for OLMo

https://allenai.org/olmo

allenai/OLMo Issues

Problem with HF loading from model checkpoint
Updated 12 days ago4
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Updated 13 days ago1
Gradient Checkpointing
Updated 14 days ago2
Problems with multi-epoch training
Updated 16 days ago
OLMo-1B's results seem very bad on olmo-eval
Updated 17 days ago
PyTorch 2.3 Support for for OLMo
Updated 21 days ago
Inference using llama.cpp
Updated 25 days ago
Support DDP
Updated a month ago
the loss spike
Updated a month ago4
OLMo 7B finetuning w/ CPU offloading does not work
Updated a month ago4
Clarification about weight tying mechanism in OLMo 1B: shared modules vs. shared weights
Closed a month ago6
Shape mismatch error when resizing token embeddings in OLMo modeling code
Closed a month ago2
I'm interested in OLMo-twin, but I found no more information except its name.
Closed a month ago3
why customized optimizers?
Updated a month ago5
About Pre-Training Corpus
Closed 3 months ago1
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory
Updated a month ago1
Question about Pre-training Olmo 7B
Closed a month ago2
nan loss encountered
Closed a month ago2
Training on a single GPU
Closed a month ago2
Does not support flash attention 2.0 on transformers.AutoModelForCausalLM.from_pretrained
Closed a month ago1
Why does training not stop after max_duration steps?
Closed a month ago2
scripts/prepare_tulu_data.py ERROR
Closed a month ago1
Why is OLMo not integrated into Transformers?
Closed a month ago2
OLMoThreadError
Updated a month ago2
Break at 1 epoch "Training epoch complete", can't pretraining beyond 1 epoch ?
Updated a month ago3
No module named 'torch.distributed.device_mesh'
Closed a month ago6
Flash attention 2.0 support
Closed a month ago3
How can I download the checkpoint
Closed 2 months ago1
Flash Attention for AMD GPUs
Closed a month ago3
Checkpoints for instruction-tuned model
Closed a month ago1
SFT and DPO Finetuning Code Available?
Closed a month ago3
Issue training on multiple nodes
Updated a month ago2
Duplicate tokenizer entry in `config.json` `auto_map` section
Updated a month ago1
Something weird with Instruct Model
Updated 2 months ago1
IndexError in OLMo-7B pre-training dataset
Updated 2 months ago
how to train model base on v1_6-sample dataset on local trainset
Updated 2 months ago1
1B generates whitespace after a specific amount of fine-tuning
Closed 2 months ago1
TypeError: forward() got an unexpected keyword argument 'cache_position'
Closed 2 months ago4
Question about optim.pt
Updated 2 months ago
Training stability without loss scaling
Closed 2 months ago1
nan loss encountered
Updated 2 months ago2
Question about the tokens/per second/GPU
Updated 2 months ago
prepare_memmap_dataset.py seems to use wrong eos_token_id for the tokenizer
Closed 2 months ago2
FSDPMixedPrecision setting, Logit norm growth, z-loss.
Updated 2 months ago
could some one help why the tokens begin with G.
Closed 2 months ago2
olmo-torch2-base does not exist in docker hub
Updated 2 months ago2
Full OLMo checkpoints?
Closed 3 months ago2
How can I finetune Olmo-7B using this repo?
Closed 3 months ago2
cannot import name 'OLMoConfig' from partially initialized module 'hf_olmo' (most likely due to a circular import)
Closed 3 months ago2
Did OLMo train its own tokenizer?
Closed 3 months ago1