Giters

wzhwzhwzh0921 / S-D-Mamba

Code for "Is Mamba Effective for Time Series Forecasting?"

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Directional Accuracy metric

ericleonardo opened this issue · comments

commented

Hi... Zihan Wang! Please, can you add the Mean Directional Accuracy to the Exchange dataset result?
I would like to know if the Mamba can predict if the Exchange rate (Forex) price will go Up or Down with more than 50% directional accuracy.
Thank you!

commented

Hi, Eric, when the input length is 96 and the forecast length is 96, the Mean Directional Accuracy (MDA) on different variates of the Exchange dataset are [0.484, 0.442,0.505, 0.505, 0.421, 0.474, 0.495].

commented

Really it is hard to get more than 50% directional accuracy on Forex.
It is the most useful prediction, but ignored by most of papers and AI researchers.
Repeating the last known value may get better RMSE than many AI, but RMSE alone is useless. Direction is more important.
I will try to apply your S-D-Mamba to another type of Forex sample because regular time series has lot of noise.
Maybe sample based on big movements only, or when occur Directional Change/reversion above 1% threshold.

Thank you!