facebookresearch / MetaCLIP

ICLR2024 Spotlight: curation/training code, metadata, distribution and pre-trained models for MetaCLIP; CVPR 2024: MoDE: CLIP Data Experts via Clustering

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Potential Feature Request: Adding checkpoints to `pretrain.py`

Andy-LZH opened this issue · comments

Congrats on getting this insightful paper out, and I will definitely use it as a backbone in my research, really nice work.

However I do have one minor feedback on how to load MetaCLIP checkpoints at this stage. Right now's process(hopefully I did not do something wrong) was a bit tedious, as I cannot strictly follow below line

model, _, preprocess = open_clip.create_model_and_transforms('ViT-B-32-quickgelu', pretrained='metaclip/b32_400m.pt')

to download weights and load models. I'm wondering whether the team is considering to add few simple lines of code to include the model checkpoints for a little bit better user experience. Thanks.

thx, it's in our TODO list to download online w/o artificial local path. We are limit on man power so feel free to PR a solution if you have one.

@howardhsu Thanks for the reply and very happy to help, it's an exciting project and please do let me know what I could do in the future to help bring item down from TODO list.