Vchitect / Latte

When I'm loading the pretrained ucf101 checkpoint, the error appears:
RuntimeError: Error(s) in loading state_dict for Latte:
Unexpected key(s) in state_dict: "y_embedder.embedding_table.weight".
All the other checkpoints can generate videos successfully, can you check this? Thank you

When I'm loading the pretrained ucf101 checkpoint, the error appears: RuntimeError: Error(s) in loading state_dict for Latte: Unexpected key(s) in state_dict: "y_embedder.embedding_table.weight". All the other checkpoints can generate videos successfully, can you check this? Thank you

Hi, can I confirm whether the extras parameter is 2 when you use the ucf101 pre-train model? See:

Latte/configs/ucf101/ucf101_sample.yaml

Line 14 in 9ededbe

extras: 2 # [1, 2] 1 unconditional generation, 2 class-conditional generation

Do you have pretrained models for unconditional generation?

Do you have pretrained models for unconditional generation?

All pre-trained models are unconditional generation except ucf101 and t2v.

Thanks. How can I control the class being sampled, is it "sample_names" in the ucf101_sample.yaml? But I didn't found where you reference this value in your code

Thanks. How can I control the class being sampled, is it "sample_names" in the ucf101_sample.yaml? But I didn't found where you reference this value in your code

You can change the class label in here,

Latte/sample/sample.py

Line 90 in 9ededbe

y = torch.randint(0, args.num_classes, (1,), device=device)

What's you training loss after converge? I want to reproduce but met some problem.

What's you training loss after converge? I want to reproduce but met some problem.

Training loss decreases quickly and oscillates around a value.

Extra key in ucf101.pt