Results are too sensitive to hyperparameters
crishna0401 opened this issue · comments
Hi, I just found out that adjusted learning rate, batch size significantly (increase/decrease) changes the results.
Although all of the compared methods are using same style of training, I feel the hyperparameters shouldn't be the deciding factor.
Hyperparameters like learning rate and batch size are not deciding factors, they are influencing factors. All the deep learning models (Transformer, MLP, CNN) are influenced by the choice of learning rate and batch size. The study of the relationship between batch size and learning rate, and their impact on generalization has been a particular field. If you are interested, we refer to [1] [2] [3] and their related research for further study.
Our model chooses almost the same learning rate (0.0003) and batch size (16 or 32) for all the datasets. We think our model, along with the choice of hyperparameters, is robust across different circumstances.
I understand the influence of hyperparameters. But my concern is for some datasets model is converged within 1-4 epochs itself. what is your comment on that.
For example I ran weather data lookback=96 and pred_len=96, the model converged in first epoch itself.
Args in experiment:
Namespace(task_name='long_term_forecast', is_training=1, model_id='weather_96_96', model='SOFTS', data='custom', root_path='/DATA/krishna/mvltf/dataset/', data_path='weather.csv', features='M', target='OT', freq='h', checkpoints='./checkpoints/', seq_len=96, label_len=48, pred_len=96, seasonal_patterns='Monthly', enc_in=21, dec_in=21, c_out=21, d_model=512, d_core=128, e_layers=3, d_layers=1, d_ff=512, moving_avg=25, factor=1, distil=True, dropout=0.0, embed='timeF', activation='gelu', output_attention=False, attention_type='full', use_norm=True, num_workers=4, itr=1, train_epochs=10, batch_size=16, patience=10, learning_rate=0.0003, des='Exp', loss='MSE', lradj='cosine', use_amp=False, use_gpu=True, gpu=0, use_multi_gpu=False, devices='0,1,2,3', save_model=False)
Use GPU: cuda:0
start training : long_term_forecast_weather_96_96_SOFTS_custom_ftM_sl96_ll48_pl96_dm512_el3_dl1_df512_fc1_ebtimeF_dtTrue_Exp>>>>>>>>>>>>>>>>>>>>>>>>>>
train 36696
val 5175
test 10444
iters: 100, epoch: 1 | loss: 0.4036454
speed: 0.0235s/iter; left time: 537.3747s
iters: 200, epoch: 1 | loss: 0.3538151
speed: 0.0142s/iter; left time: 323.6224s
iters: 300, epoch: 1 | loss: 0.3341010
speed: 0.0138s/iter; left time: 312.5898s
iters: 400, epoch: 1 | loss: 0.3313734
speed: 0.0138s/iter; left time: 312.0042s
iters: 500, epoch: 1 | loss: 0.3725422
speed: 0.0149s/iter; left time: 333.6391s
iters: 600, epoch: 1 | loss: 2.2886326
speed: 0.0152s/iter; left time: 339.5225s
iters: 700, epoch: 1 | loss: 0.4027846
speed: 0.0166s/iter; left time: 368.6571s
iters: 800, epoch: 1 | loss: 0.3414143
speed: 0.0175s/iter; left time: 386.4240s
iters: 900, epoch: 1 | loss: 0.4444550
speed: 0.0171s/iter; left time: 376.6025s
iters: 1000, epoch: 1 | loss: 0.3779259
speed: 0.0175s/iter; left time: 383.3128s
iters: 1100, epoch: 1 | loss: 0.3262408
speed: 0.0174s/iter; left time: 379.6096s
iters: 1200, epoch: 1 | loss: 0.2622951
speed: 0.0175s/iter; left time: 379.7748s
iters: 1300, epoch: 1 | loss: 0.5229304
speed: 0.0178s/iter; left time: 384.5851s
iters: 1400, epoch: 1 | loss: 0.5858208
speed: 0.0192s/iter; left time: 414.5758s
iters: 1500, epoch: 1 | loss: 0.5290294
speed: 0.0204s/iter; left time: 437.1839s
iters: 1600, epoch: 1 | loss: 0.3211349
speed: 0.0204s/iter; left time: 434.5036s
iters: 1700, epoch: 1 | loss: 0.3488162
speed: 0.0203s/iter; left time: 431.8828s
iters: 1800, epoch: 1 | loss: 0.3670591
speed: 0.0203s/iter; left time: 430.0920s
iters: 1900, epoch: 1 | loss: 0.2346632
speed: 0.0204s/iter; left time: 428.7342s
iters: 2000, epoch: 1 | loss: 0.6296965
speed: 0.0204s/iter; left time: 426.3540s
iters: 2100, epoch: 1 | loss: 0.3551054
speed: 0.0203s/iter; left time: 423.3616s
iters: 2200, epoch: 1 | loss: 0.2917550
speed: 0.0203s/iter; left time: 421.3748s
Epoch: 1 cost time: 41.27806258201599
Epoch: 1, Steps: 2294 | Train Loss: 0.4738744 Vali Loss: 0.4414336 Test Loss: 0.1900593
Validation loss decreased (inf --> 0.441434). Saving model ...
Updating learning rate to 0.00029265847744427303
iters: 100, epoch: 2 | loss: 0.6401179
speed: 0.0706s/iter; left time: 1449.7168s
iters: 200, epoch: 2 | loss: 0.3250378
speed: 0.0135s/iter; left time: 275.2172s
iters: 300, epoch: 2 | loss: 0.2748268
speed: 0.0135s/iter; left time: 274.1024s
iters: 400, epoch: 2 | loss: 0.6726312
speed: 0.0135s/iter; left time: 272.6539s
iters: 500, epoch: 2 | loss: 0.2553286
speed: 0.0136s/iter; left time: 273.6538s
iters: 600, epoch: 2 | loss: 0.4415316
speed: 0.0136s/iter; left time: 272.8549s
iters: 700, epoch: 2 | loss: 0.4522270
speed: 0.0137s/iter; left time: 272.3512s
iters: 800, epoch: 2 | loss: 0.4513119
speed: 0.0131s/iter; left time: 259.6996s
iters: 900, epoch: 2 | loss: 0.4280919
speed: 0.0138s/iter; left time: 271.9492s
iters: 1000, epoch: 2 | loss: 0.2037832
speed: 0.0138s/iter; left time: 270.6034s
iters: 1100, epoch: 2 | loss: 0.3414104
speed: 0.0138s/iter; left time: 269.3662s
iters: 1200, epoch: 2 | loss: 0.3679063
speed: 0.0138s/iter; left time: 268.3948s
iters: 1300, epoch: 2 | loss: 2.2532523
speed: 0.0138s/iter; left time: 266.9653s
iters: 1400, epoch: 2 | loss: 0.3440191
speed: 0.0160s/iter; left time: 307.6901s
iters: 1500, epoch: 2 | loss: 0.3955808
speed: 0.0154s/iter; left time: 295.5054s
iters: 1600, epoch: 2 | loss: 0.4907824
speed: 0.0138s/iter; left time: 263.0443s
iters: 1700, epoch: 2 | loss: 2.2307138
speed: 0.0140s/iter; left time: 264.6075s
iters: 1800, epoch: 2 | loss: 0.3165238
speed: 0.0161s/iter; left time: 303.6546s
iters: 1900, epoch: 2 | loss: 0.3220946
speed: 0.0161s/iter; left time: 302.2597s
iters: 2000, epoch: 2 | loss: 0.2449237
speed: 0.0170s/iter; left time: 317.1356s
iters: 2100, epoch: 2 | loss: 0.3095628
speed: 0.0170s/iter; left time: 315.8705s
iters: 2200, epoch: 2 | loss: 0.2945173
speed: 0.0170s/iter; left time: 313.6160s
Epoch: 2 cost time: 33.299219608306885
Epoch: 2, Steps: 2294 | Train Loss: 0.5480080 Vali Loss: 0.4419284 Test Loss: 0.1754540
EarlyStopping counter: 1 out of 10
Updating learning rate to 0.0002713525491562421
iters: 100, epoch: 3 | loss: 0.2709894
speed: 0.0637s/iter; left time: 1163.6169s
iters: 200, epoch: 3 | loss: 0.2794941
speed: 0.0165s/iter; left time: 300.3870s
iters: 300, epoch: 3 | loss: 0.3516423
speed: 0.0165s/iter; left time: 298.4946s
iters: 400, epoch: 3 | loss: 0.3174662
speed: 0.0155s/iter; left time: 278.3513s
iters: 500, epoch: 3 | loss: 0.3157647
speed: 0.0163s/iter; left time: 290.3424s
iters: 600, epoch: 3 | loss: 0.2292653
speed: 0.0165s/iter; left time: 292.6444s
iters: 700, epoch: 3 | loss: 0.2290015
speed: 0.0165s/iter; left time: 291.4030s
iters: 800, epoch: 3 | loss: 0.2470866
speed: 0.0165s/iter; left time: 290.1491s
iters: 900, epoch: 3 | loss: 0.2827386
speed: 0.0165s/iter; left time: 288.1084s
iters: 1000, epoch: 3 | loss: 0.3288465
speed: 0.0165s/iter; left time: 286.5521s
iters: 1100, epoch: 3 | loss: 0.4127340
speed: 0.0174s/iter; left time: 300.6309s
iters: 1200, epoch: 3 | loss: 0.2658502
speed: 0.0179s/iter; left time: 307.8953s
iters: 1300, epoch: 3 | loss: 0.2911725
speed: 0.0179s/iter; left time: 305.8910s
iters: 1400, epoch: 3 | loss: 0.2256956
speed: 0.0180s/iter; left time: 304.4094s
iters: 1500, epoch: 3 | loss: 0.3485063
speed: 0.0179s/iter; left time: 301.6632s
iters: 1600, epoch: 3 | loss: 0.3210509
speed: 0.0149s/iter; left time: 249.0080s
iters: 1700, epoch: 3 | loss: 0.3115217
speed: 0.0136s/iter; left time: 226.7412s
iters: 1800, epoch: 3 | loss: 0.4391301
speed: 0.0134s/iter; left time: 221.6049s
iters: 1900, epoch: 3 | loss: 0.3646879
speed: 0.0136s/iter; left time: 222.9945s
iters: 2000, epoch: 3 | loss: 0.2265420
speed: 0.0136s/iter; left time: 221.8785s
iters: 2100, epoch: 3 | loss: 0.2624652
speed: 0.0164s/iter; left time: 267.0513s
iters: 2200, epoch: 3 | loss: 0.8389145
speed: 0.0165s/iter; left time: 266.9960s
Epoch: 3 cost time: 36.725884199142456
Epoch: 3, Steps: 2294 | Train Loss: 0.3254803 Vali Loss: 0.4416376 Test Loss: 0.1746636
EarlyStopping counter: 2 out of 10
Updating learning rate to 0.00023816778784387094
iters: 100, epoch: 4 | loss: 0.2669576
speed: 0.0619s/iter; left time: 987.0867s
iters: 200, epoch: 4 | loss: 0.3026834
speed: 0.0144s/iter; left time: 228.4021s
iters: 300, epoch: 4 | loss: 0.2467037
speed: 0.0166s/iter; left time: 262.0724s
iters: 400, epoch: 4 | loss: 0.2076253
speed: 0.0166s/iter; left time: 260.3927s
iters: 500, epoch: 4 | loss: 0.2368142
speed: 0.0166s/iter; left time: 258.9158s
iters: 600, epoch: 4 | loss: 0.1926093
speed: 0.0167s/iter; left time: 257.4968s
iters: 700, epoch: 4 | loss: 0.2499007
speed: 0.0166s/iter; left time: 254.2027s
iters: 800, epoch: 4 | loss: 3.8899975
speed: 0.0175s/iter; left time: 267.6255s
iters: 900, epoch: 4 | loss: 0.4037665
speed: 0.0185s/iter; left time: 280.3921s
iters: 1000, epoch: 4 | loss: 0.2400860
speed: 0.0192s/iter; left time: 289.2192s
iters: 1100, epoch: 4 | loss: 0.2650346
speed: 0.0193s/iter; left time: 288.0967s
iters: 1200, epoch: 4 | loss: 0.2793321
speed: 0.0200s/iter; left time: 297.8566s
iters: 1300, epoch: 4 | loss: 0.2470463
speed: 0.0161s/iter; left time: 237.5367s
iters: 1400, epoch: 4 | loss: 0.2521619
speed: 0.0139s/iter; left time: 203.9402s
iters: 1500, epoch: 4 | loss: 0.2568092
speed: 0.0148s/iter; left time: 215.6603s
iters: 1600, epoch: 4 | loss: 0.2051758
speed: 0.0167s/iter; left time: 242.0249s
iters: 1700, epoch: 4 | loss: 0.2405885
speed: 0.0176s/iter; left time: 252.6166s
iters: 1800, epoch: 4 | loss: 0.2696667
speed: 0.0188s/iter; left time: 267.8078s
iters: 1900, epoch: 4 | loss: 0.1836121
speed: 0.0195s/iter; left time: 276.2933s
iters: 2000, epoch: 4 | loss: 0.1976945
speed: 0.0205s/iter; left time: 287.6593s
iters: 2100, epoch: 4 | loss: 0.3522930
speed: 0.0204s/iter; left time: 285.0292s
iters: 2200, epoch: 4 | loss: 0.2470250
speed: 0.0203s/iter; left time: 281.6042s
Epoch: 4 cost time: 40.027498960494995
Epoch: 4, Steps: 2294 | Train Loss: 0.4197084 Vali Loss: 0.4251894 Test Loss: 0.1686089
Validation loss decreased (0.441434 --> 0.425189). Saving model ...
Updating learning rate to 0.0001963525491562421
iters: 100, epoch: 5 | loss: 0.3512476
speed: 0.0692s/iter; left time: 945.5842s
iters: 200, epoch: 5 | loss: 0.2025531
speed: 0.0140s/iter; left time: 189.6152s
iters: 300, epoch: 5 | loss: 0.3151135
speed: 0.0138s/iter; left time: 186.3076s
iters: 400, epoch: 5 | loss: 0.1649124
speed: 0.0137s/iter; left time: 182.9132s
iters: 500, epoch: 5 | loss: 0.1982203
speed: 0.0134s/iter; left time: 178.0392s
iters: 600, epoch: 5 | loss: 0.3465936
speed: 0.0150s/iter; left time: 197.7103s
iters: 700, epoch: 5 | loss: 0.2815661
speed: 0.0151s/iter; left time: 197.4270s
iters: 800, epoch: 5 | loss: 0.3126697
speed: 0.0137s/iter; left time: 177.7352s
iters: 900, epoch: 5 | loss: 0.2094218
speed: 0.0137s/iter; left time: 176.3205s
iters: 1000, epoch: 5 | loss: 2.2910328
speed: 0.0138s/iter; left time: 175.7294s
iters: 1100, epoch: 5 | loss: 0.1714488
speed: 0.0138s/iter; left time: 174.9900s
iters: 1200, epoch: 5 | loss: 0.5156409
speed: 0.0138s/iter; left time: 173.4002s
iters: 1300, epoch: 5 | loss: 0.1571699
speed: 0.0138s/iter; left time: 171.9752s
iters: 1400, epoch: 5 | loss: 0.3079891
speed: 0.0138s/iter; left time: 170.7172s
iters: 1500, epoch: 5 | loss: 0.2646823
speed: 0.0137s/iter; left time: 168.3070s
iters: 1600, epoch: 5 | loss: 0.3137054
speed: 0.0137s/iter; left time: 166.5632s
iters: 1700, epoch: 5 | loss: 0.2070766
speed: 0.0138s/iter; left time: 166.1373s
iters: 1800, epoch: 5 | loss: 0.2626841
speed: 0.0162s/iter; left time: 193.9877s
iters: 1900, epoch: 5 | loss: 0.2151637
speed: 0.0169s/iter; left time: 200.2149s
iters: 2000, epoch: 5 | loss: 0.2760842
speed: 0.0173s/iter; left time: 203.8911s
iters: 2100, epoch: 5 | loss: 0.2975026
speed: 0.0190s/iter; left time: 221.6073s
iters: 2200, epoch: 5 | loss: 0.2145984
speed: 0.0190s/iter; left time: 219.5113s
Epoch: 5 cost time: 34.31195425987244
Epoch: 5, Steps: 2294 | Train Loss: 0.3580489 Vali Loss: 0.4476494 Test Loss: 0.1756839
EarlyStopping counter: 1 out of 10
Updating learning rate to 0.00015
iters: 100, epoch: 6 | loss: 0.2923902
speed: 0.0681s/iter; left time: 773.8002s
iters: 200, epoch: 6 | loss: 0.1475460
speed: 0.0136s/iter; left time: 153.8053s
iters: 300, epoch: 6 | loss: 0.2080690
speed: 0.0135s/iter; left time: 151.2166s
iters: 400, epoch: 6 | loss: 0.3924242
speed: 0.0136s/iter; left time: 151.0847s
iters: 500, epoch: 6 | loss: 0.3908802
speed: 0.0137s/iter; left time: 150.8331s
iters: 600, epoch: 6 | loss: 0.1254497
speed: 0.0098s/iter; left time: 106.4976s
iters: 700, epoch: 6 | loss: 0.2812734
speed: 0.0097s/iter; left time: 104.0888s
iters: 800, epoch: 6 | loss: 0.3186392
speed: 0.0097s/iter; left time: 103.0860s
iters: 900, epoch: 6 | loss: 0.1809346
speed: 0.0094s/iter; left time: 99.6321s
iters: 1000, epoch: 6 | loss: 0.1881777
speed: 0.0138s/iter; left time: 144.2360s
iters: 1100, epoch: 6 | loss: 0.1834134
speed: 0.0137s/iter; left time: 141.6584s
iters: 1200, epoch: 6 | loss: 0.2215981
speed: 0.0130s/iter; left time: 133.5223s
iters: 1300, epoch: 6 | loss: 0.1686825
speed: 0.0144s/iter; left time: 146.0337s
iters: 1400, epoch: 6 | loss: 0.2317298
speed: 0.0166s/iter; left time: 166.8591s
iters: 1500, epoch: 6 | loss: 0.2422193
speed: 0.0165s/iter; left time: 164.9172s
iters: 1600, epoch: 6 | loss: 2.1373572
speed: 0.0167s/iter; left time: 164.6783s
iters: 1700, epoch: 6 | loss: 0.1828119
speed: 0.0174s/iter; left time: 169.8618s
iters: 1800, epoch: 6 | loss: 0.2676963
speed: 0.0197s/iter; left time: 190.4514s
iters: 1900, epoch: 6 | loss: 0.4651260
speed: 0.0197s/iter; left time: 188.9532s
iters: 2000, epoch: 6 | loss: 0.1579365
speed: 0.0157s/iter; left time: 148.9484s
iters: 2100, epoch: 6 | loss: 0.1624753
speed: 0.0136s/iter; left time: 127.4397s
iters: 2200, epoch: 6 | loss: 0.2036208
speed: 0.0160s/iter; left time: 148.4296s
Epoch: 6 cost time: 32.924118757247925
Epoch: 6, Steps: 2294 | Train Loss: 0.3250205 Vali Loss: 0.4720847 Test Loss: 0.1833609
EarlyStopping counter: 2 out of 10
Updating learning rate to 0.0001036474508437579
iters: 100, epoch: 7 | loss: 0.2263691
speed: 0.0598s/iter; left time: 542.5842s
iters: 200, epoch: 7 | loss: 0.1706614
speed: 0.0147s/iter; left time: 132.3575s
iters: 300, epoch: 7 | loss: 0.4450584
speed: 0.0150s/iter; left time: 133.5679s
iters: 400, epoch: 7 | loss: 0.1596253
speed: 0.0149s/iter; left time: 130.5566s
iters: 500, epoch: 7 | loss: 0.1552347
speed: 0.0149s/iter; left time: 129.5256s
iters: 600, epoch: 7 | loss: 0.1781175
speed: 0.0143s/iter; left time: 122.5565s
iters: 700, epoch: 7 | loss: 0.2057821
speed: 0.0150s/iter; left time: 127.2036s
iters: 800, epoch: 7 | loss: 0.1817286
speed: 0.0154s/iter; left time: 128.6435s
iters: 900, epoch: 7 | loss: 0.1354985
speed: 0.0162s/iter; left time: 134.4697s
iters: 1000, epoch: 7 | loss: 0.3534313
speed: 0.0163s/iter; left time: 132.8846s
iters: 1100, epoch: 7 | loss: 2.1183684
speed: 0.0163s/iter; left time: 131.8062s
iters: 1200, epoch: 7 | loss: 2.1941707
speed: 0.0184s/iter; left time: 146.6815s
iters: 1300, epoch: 7 | loss: 0.1697403
speed: 0.0196s/iter; left time: 154.2662s
iters: 1400, epoch: 7 | loss: 0.2164193
speed: 0.0196s/iter; left time: 152.4956s
iters: 1500, epoch: 7 | loss: 0.1487056
speed: 0.0192s/iter; left time: 147.7702s
iters: 1600, epoch: 7 | loss: 0.4190980
speed: 0.0188s/iter; left time: 142.2020s
iters: 1700, epoch: 7 | loss: 0.2058418
speed: 0.0188s/iter; left time: 140.8787s
iters: 1800, epoch: 7 | loss: 0.1226447
speed: 0.0194s/iter; left time: 142.8435s
iters: 1900, epoch: 7 | loss: 0.2356446
speed: 0.0205s/iter; left time: 148.8279s
iters: 2000, epoch: 7 | loss: 0.2367059
speed: 0.0206s/iter; left time: 147.8381s
iters: 2100, epoch: 7 | loss: 0.1676833
speed: 0.0206s/iter; left time: 145.6715s
iters: 2200, epoch: 7 | loss: 0.1725331
speed: 0.0199s/iter; left time: 139.0891s
Epoch: 7 cost time: 39.72669458389282
Epoch: 7, Steps: 2294 | Train Loss: 0.3917756 Vali Loss: 0.4726515 Test Loss: 0.1887089
EarlyStopping counter: 3 out of 10
Updating learning rate to 6.183221215612904e-05
iters: 100, epoch: 8 | loss: 0.1433359
speed: 0.0677s/iter; left time: 459.1125s
iters: 200, epoch: 8 | loss: 0.2055279
speed: 0.0139s/iter; left time: 93.1049s
iters: 300, epoch: 8 | loss: 0.2131681
speed: 0.0163s/iter; left time: 107.2426s
iters: 400, epoch: 8 | loss: 0.2196651
speed: 0.0163s/iter; left time: 105.5213s
iters: 500, epoch: 8 | loss: 0.3819869
speed: 0.0163s/iter; left time: 103.9812s
iters: 600, epoch: 8 | loss: 0.1870438
speed: 0.0163s/iter; left time: 102.2659s
iters: 700, epoch: 8 | loss: 0.0745629
speed: 0.0138s/iter; left time: 85.6146s
iters: 800, epoch: 8 | loss: 0.2042980
speed: 0.0138s/iter; left time: 84.0793s
iters: 900, epoch: 8 | loss: 0.1475536
speed: 0.0139s/iter; left time: 83.4538s
iters: 1000, epoch: 8 | loss: 0.4048573
speed: 0.0140s/iter; left time: 82.2775s
iters: 1100, epoch: 8 | loss: 0.2045363
speed: 0.0140s/iter; left time: 80.7272s
iters: 1200, epoch: 8 | loss: 0.2384610
speed: 0.0140s/iter; left time: 79.3232s
iters: 1300, epoch: 8 | loss: 0.1296934
speed: 0.0161s/iter; left time: 89.8580s
iters: 1400, epoch: 8 | loss: 0.2461748
speed: 0.0177s/iter; left time: 96.8697s
iters: 1500, epoch: 8 | loss: 0.1433063
speed: 0.0177s/iter; left time: 95.1735s
iters: 1600, epoch: 8 | loss: 2.0703864
speed: 0.0177s/iter; left time: 93.7113s
iters: 1700, epoch: 8 | loss: 0.3135197
speed: 0.0195s/iter; left time: 100.8796s
iters: 1800, epoch: 8 | loss: 0.2275418
speed: 0.0197s/iter; left time: 100.1432s
iters: 1900, epoch: 8 | loss: 0.2258315
speed: 0.0207s/iter; left time: 102.9563s
iters: 2000, epoch: 8 | loss: 0.5402485
speed: 0.0206s/iter; left time: 100.6643s
iters: 2100, epoch: 8 | loss: 0.1666155
speed: 0.0206s/iter; left time: 98.3946s
iters: 2200, epoch: 8 | loss: 0.3003441
speed: 0.0204s/iter; left time: 95.3980s
Epoch: 8 cost time: 38.66363549232483
Epoch: 8, Steps: 2294 | Train Loss: 0.3176663 Vali Loss: 0.4808291 Test Loss: 0.1920194
EarlyStopping counter: 4 out of 10
Updating learning rate to 2.8647450843757897e-05
iters: 100, epoch: 9 | loss: 0.5826761
speed: 0.0683s/iter; left time: 306.4070s
iters: 200, epoch: 9 | loss: 0.1596118
speed: 0.0138s/iter; left time: 60.7506s
iters: 300, epoch: 9 | loss: 0.2060688
speed: 0.0138s/iter; left time: 59.1969s
iters: 400, epoch: 9 | loss: 0.2674173
speed: 0.0138s/iter; left time: 57.8440s
iters: 500, epoch: 9 | loss: 0.1885380
speed: 0.0129s/iter; left time: 52.7814s
iters: 600, epoch: 9 | loss: 0.1056275
speed: 0.0125s/iter; left time: 50.0567s
iters: 700, epoch: 9 | loss: 0.0917342
speed: 0.0126s/iter; left time: 49.1035s
iters: 800, epoch: 9 | loss: 2.0841510
speed: 0.0126s/iter; left time: 47.7344s
iters: 900, epoch: 9 | loss: 0.3349448
speed: 0.0126s/iter; left time: 46.4777s
iters: 1000, epoch: 9 | loss: 0.1489390
speed: 0.0128s/iter; left time: 46.1043s
iters: 1100, epoch: 9 | loss: 0.1504216
speed: 0.0128s/iter; left time: 44.7529s
iters: 1200, epoch: 9 | loss: 0.1942857
speed: 0.0127s/iter; left time: 43.0960s
iters: 1300, epoch: 9 | loss: 0.0933912
speed: 0.0127s/iter; left time: 41.7903s
iters: 1400, epoch: 9 | loss: 0.1358986
speed: 0.0127s/iter; left time: 40.3545s
iters: 1500, epoch: 9 | loss: 0.3633728
speed: 0.0120s/iter; left time: 37.0319s
iters: 1600, epoch: 9 | loss: 0.1545264
speed: 0.0130s/iter; left time: 38.9192s
iters: 1700, epoch: 9 | loss: 0.1448196
speed: 0.0135s/iter; left time: 39.0197s
iters: 1800, epoch: 9 | loss: 0.2245946
speed: 0.0137s/iter; left time: 38.2689s
iters: 1900, epoch: 9 | loss: 0.1365054
speed: 0.0137s/iter; left time: 36.7369s
iters: 2000, epoch: 9 | loss: 0.4193476
speed: 0.0137s/iter; left time: 35.5790s
iters: 2100, epoch: 9 | loss: 0.2448702
speed: 0.0138s/iter; left time: 34.2597s
iters: 2200, epoch: 9 | loss: 0.1035335
speed: 0.0138s/iter; left time: 33.0561s
Epoch: 9 cost time: 30.47970199584961
Epoch: 9, Steps: 2294 | Train Loss: 0.2970580 Vali Loss: 0.5224582 Test Loss: 0.1955109
EarlyStopping counter: 5 out of 10
Updating learning rate to 7.34152255572697e-06
iters: 100, epoch: 10 | loss: 0.0942249
speed: 0.0648s/iter; left time: 142.1427s
iters: 200, epoch: 10 | loss: 0.1276101
speed: 0.0140s/iter; left time: 29.3143s
iters: 300, epoch: 10 | loss: 0.1668937
speed: 0.0154s/iter; left time: 30.7609s
iters: 400, epoch: 10 | loss: 0.2278446
speed: 0.0172s/iter; left time: 32.6500s
iters: 500, epoch: 10 | loss: 0.4736909
speed: 0.0175s/iter; left time: 31.4656s
iters: 600, epoch: 10 | loss: 0.1455495
speed: 0.0143s/iter; left time: 24.2843s
iters: 700, epoch: 10 | loss: 0.5455921
speed: 0.0141s/iter; left time: 22.5467s
iters: 800, epoch: 10 | loss: 0.0883168
speed: 0.0141s/iter; left time: 21.0407s
iters: 900, epoch: 10 | loss: 0.2184256
speed: 0.0145s/iter; left time: 20.2428s
iters: 1000, epoch: 10 | loss: 0.1521411
speed: 0.0149s/iter; left time: 19.3338s
iters: 1100, epoch: 10 | loss: 0.2715932
speed: 0.0138s/iter; left time: 16.4973s
iters: 1200, epoch: 10 | loss: 0.2269915
speed: 0.0138s/iter; left time: 15.1206s
iters: 1300, epoch: 10 | loss: 0.1844854
speed: 0.0138s/iter; left time: 13.7492s
iters: 1400, epoch: 10 | loss: 0.4068148
speed: 0.0138s/iter; left time: 12.3498s
iters: 1500, epoch: 10 | loss: 0.1411258
speed: 0.0138s/iter; left time: 10.9729s
iters: 1600, epoch: 10 | loss: 1.9868090
speed: 0.0138s/iter; left time: 9.5946s
iters: 1700, epoch: 10 | loss: 0.1957721
speed: 0.0138s/iter; left time: 8.1814s
iters: 1800, epoch: 10 | loss: 0.1239839
speed: 0.0137s/iter; left time: 6.8046s
iters: 1900, epoch: 10 | loss: 0.1818440
speed: 0.0138s/iter; left time: 5.4356s
iters: 2000, epoch: 10 | loss: 0.2910723
speed: 0.0138s/iter; left time: 4.0713s
iters: 2100, epoch: 10 | loss: 0.1157951
speed: 0.0138s/iter; left time: 2.6874s
iters: 2200, epoch: 10 | loss: 0.2038301
speed: 0.0138s/iter; left time: 1.3093s
Epoch: 10 cost time: 32.87806439399719
Epoch: 10, Steps: 2294 | Train Loss: 0.2986548 Vali Loss: 0.5184059 Test Loss: 0.1979387
EarlyStopping counter: 6 out of 10
Updating learning rate to 0.0
testing : long_term_forecast_weather_96_96_SOFTS_custom_ftM_sl96_ll48_pl96_dm512_el3_dl1_df512_fc1_ebtimeF_dtTrue_Exp<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
test 10444
mse:0.1686088588118713, mae:0.20914341695678046
Yes, some datasets are hard to predict due to their inherent non-stationarity. Our method may exhibit overfitting and achieve lower test loss in the early epochs. It is a common phenomenon for some time series data. Our approach aims to provide a better general solution on average. For special cases, like weather, we recommend making targeted adjustments, e.g., weight decay, robust loss, or certain preprocessing.