How to apply the dataset to different models?

[sugar-hub]

Hi, there!

I would like to appreciate it for your opening source code. I'm very interested in your work! I noticed that you have experimented so many other models with prediction steps 3, 6, 12 in METR-LA and PEMS-BAY. How did you apply those two datasets to other models with such prediction steps? I tried some work but failed.

Best wishes!

[wengwenchao]

Thank you for your interest in this work. Regarding your question, the main modification involves changing the test part of other models to achieve this. Some models only provide the average result of the overall 12-step prediction. If we need to output the prediction result for each step, we need to use a for loop to output the single-step prediction results.

In relation to my train code, the relevant part is as follows:

for q in range(args.Q):
    mae, rmse, mape = utils.metric(testPred[:, q], testY[:, q])
    MAE.append(mae)
    RMSE.append(rmse)
    MAPE.append(mape)
    utils.log_string(log, 'step: %02d         %.2f\t\t%.2f\t\t%.2f%%' %
                     (q +1, mae, rmse, mape * 100))

You can use this part of the code as a reference for debugging, then try to integrate it into the test section of other models to achieve single-step prediction output. Alternatively, you can also try porting other models into a framework that already has single-step prediction functionality for making predictions. This is also a viable approach.

I hope this can help you, and you are welcome to discuss it with me.

[sugar-hub]

Dear Weng,

I'm very surprised and excited that you replied me so quickly! I have read some of other models mentioned in your paper and I found that some of them are just tested on PEMS04, PEMS08. Then I would like to know that how you experimented them on METR-LA and PEMS-BAY.

Thank you for your help! I am very grateful to you!

[wengwenchao]

Regarding the models tested on PEMS04 and PEMS08, such as AGCRN, when I tested its performance on LA and BAY, I used the AGCRN framework itself. Specifically, I added the LA and BAY datasets to the AGCRN framework, enabling data reading and splitting the data according to the 7:1:2 ratio convention used in LA and BAY (of course, you can also directly use the pre-split train/val/test sets from Graph WaveNet). Then, I modified the model's input and output parameters (since the split data from Graph WaveNet includes a time axis dimension in addition to data x, adjustments need to be made accordingly). This way, the testing can proceed smoothly. Additionally, note that LA and BAY usually use mask_mae to calculate accuracy, which masks out data where the true value is 0. AGCRN does not mask by default, so this needs to be adjusted.

Generally speaking, each model's framework can implement the functions required for training and testing. However, we need to modify the data reading and construction according to our needs, as well as the results we want to obtain in the final test phase.

I hope this helps you.

[sugar-hub]

Thank you very much! I appreciate it. Let me have a try these days.

[wengwenchao]

You are weclome.