Giters

JulesBelveze / time-series-autoencoder

PyTorch Dual-Attention LSTM-Autoencoder For Multivariate Time Series

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Question about model evaluation

myalos opened this issue · comments

commented

Thanks for sharing the code, i learned a lot from it.
I see that in eval.py evaluation is performed on the target scaled with StandardScaler
I think that evaluation will decrease the actual mse.
I'm new to time series forecasting and don't know if it's reasonable to evaluate it on standardized data.

commented

Hey @myalos thanks for your interest!
Well my idea behind using the scaled value to compute the MSE is that if you are trying to predict a target which takes values in the range [0, 0.1] you would get a way smaller MSE than if you are trying to predict a target which ranges between 0 and 1000, right? So by scaling the values you make sure that you end up with comparable values.
What do you think?

commented

Thanks for reply. I argree with what you said. I think that mae could be better metric to compare the models and rmse in original scale could make people better understand the power of the model, since the value between [0, 1] is not very intuitive.