Factor training losses not contracting

[BlockWaving]

Hi, this is an impressive project, and thanks for sharing w/ community!

I've been trying to learn the model with my test data. The test data has about 70 samples, each with about 2300 timesteps.

However, in final stage, the Recovery Loss in Rolling Validation is getting bigger and bigger each round, and early stopped at 0.308, which caused the much worse wape and wape_global metrics than the baseline:
{'wape': 0.39331427, 'mape': 0.36864823, 'smape': 0.4937316, 'mae': 5.487852, 'rmse': 8.775283, 'nrmse': 0.47228432, 'wape_global': 0.582235, 'mape_global': 0.56812644, 'smape_global': 0.84549224, 'mae_global': 8.123833, 'rmse_global': 11.685119, 'nrmse_global': 0.47228432, 'baseline_wape': 0.11834013, 'baseline_mape': 0.11296055, 'baseline_smape': 0.11496856}

Could you provide some insights how can I improve the training and get better resulsts?

Thanks!!!

===========================================================

Last round of Recovery Loss stats:
GLO: rolling_validation(): Current window wape: 0.5014139
GLO: recover_future_X(): Recovery Loss(0/100000): 1.002367615699768
GLO: recover_future_X(): Recovery Loss(1000/100000): 0.628299355506897
GLO: recover_future_X(): Recovery Loss(2000/100000): 0.4282535910606384
GLO: recover_future_X(): Recovery Loss(3000/100000): 0.3461550176143646
GLO: recover_future_X(): Recovery Loss(4000/100000): 0.3201618790626526
GLO: recover_future_X(): Recovery Loss(5000/100000): 0.310817688703537
GLO: recover_future_X(): Recovery Loss(6000/100000): 0.3080223500728607
GLO: recover_future_X(): Recovery Loss(7000/100000): 0.3077664375305176
GLO: rolling_validation(): Current window wape: 0.45383096

In addition, Factorization Loss F, Factorization Loss X, Validation Loss ended at (0.214, 0.205, 0.294) level, early stopped, while Temporal Loss hovered around 0.017 level.

Training of Xseq and Yseq, (training loss, validation loss) progress down to (0.074, 0.052) and (0.054, 0.021) respectively.