Zasder3 / train-CLIP

A PyTorch Lightning solution to training OpenAI's CLIP from scratch.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

customwrapped load checkpoint test

tonyhuang33 opened this issue · comments

I have some problems about how to load checkpoing with Customwrapped. Could give some code example to me? I would be very grateful to you

I trained with CustomCLIPWrapper, and successfully loaded checkpoints. Here my code for this

    img_encoder = resnet50(pretrained=False)
    img_encoder.fc = torch.nn.Linear(2048, 768)

    txt_encoder = AutoModel.from_pretrained("vinai/phobert-base")

    model = CustomCLIPWrapper(img_encoder, txt_encoder, 64, avg_word_embs=True)

    checkpoint = torch.load(ckpt_path)
    model.load_state_dict(checkpoint["state_dict"])