The generated videos are shaking and unstable.

[nankepan]

I trained a stdit model with webvid. The generated videos are shaking and unstable. Can you give me some reasons or suggestions？Is there some training tricks?

sample_0.2.mp4

sample_1.2.mp4

sample_2.2.mp4

[icesunL]

same problem

[silentycj]

same problem

[Zheng Zangwei (Alex Zheng)]

Although we did not try Webvid dataset, I suppose it is because your number of training steps is too small. Could you share how many number of steps have you run?

[Zheng Zangwei (Alex Zheng)]

It seems you use a very large batch size (32x72). I think although the passed number of data is the same, you still need enough iterations to get a good result. If you continue the training, you will see the generated video to have better quality.