cloneofsimo / minRF

Minimal implementation of scalable rectified flow transformers, based on SD3's approach

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Problem in loss function

mephisto28 opened this issue · comments

batchwise_mse = ((z1 - x - vtheta) ** 2).mean(dim=list(range(1, len(x.shape))))

I guess the loss should be batchwise_mse = ((z1 - x - vtheta * texp) ** 2).mean(dim=list(range(1, len(x.shape))))... is it?

The current code says the predict target in training should be the distance other than velocity. If not changing the training target the inference code could be modified by dividing the predicted value by timestep to gain the velocity.

Sorry what?

Sorry what?

If mse = (model_pred - x) ** 2, it should be model_pred = z1 - vtheta * t instead of model_pred = z1 - vtheta ... is it?

nah