Lightning-AI / litgpt

Pretrain, finetune, deploy 20+ LLMs on your own data. Uses state-of-the-art techniques: flash attention, FSDP, 4-bit, LoRA, and more.

Home Page:https://lightning.ai

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Explore support for urls as configs

carmocca opened this issue · comments

Right now the tutorials assume that the config hub is locally available.

But if litgpt is used as a package, this is not true.

Can we make litgpt pretrain --config https://github.com/Lightning-AI/litgpt/blob/wip/config_hub/pretrain/tinyllama.yaml work?

We could potentially include the config_hub in the distribution. But I think I'd be even better to link to the URL like you suggest. This way, you can also use "new" config files that may not be on PyPI yet and it also shows people where to find the configs in case they have suggestions or want to contribute