Error in optim/adamw.py

Question

Error in optim/adamw.py

caodoanh2001 opened this issue 3 years ago · comments

🐛 Bug

Hi,

I think in optim/adamw.py has a small mistake with alignment at line 110.

    F.adamw(params_with_grad,
                        grads,
                        exp_avgs,
                        exp_avg_sqs,
                        max_exp_avg_sqs,
                        state_steps,
                        amsgrad,
                        beta1,
                        beta2,
                        group['lr'],
                        group['weight_decay'],
                        group['eps'])

At line 110, I think it should be increased by 1 tab.

I met this bug when using mmdetection toolbox.

cc @vincentqb

Natalia Gimelshein · Answer 1 · Tue Apr 13 2021 08:42:34 GMT+0800 (China Standard Time)

What is the bug that you are encountering?

Doanh B C · Answer 2 · Tue Apr 13 2021 17:59:54 GMT+0800 (China Standard Time)

What is the bug that you are encountering?

Hi, when I start training model by mmdetection, I get this error:

UnboundLocalError: local variable 'beta1' referenced before assignment

When I decrease 1 tab at line 110 in file adamw.py, it seems that can solve this problem.
Maybe I think the variable beta1, beta2 are defined outside your loop so that it occurs the mentioned issues.

Natalia Gimelshein · Answer 3 · Wed Apr 14 2021 00:21:38 GMT+0800 (China Standard Time)

This was fixed in #52944, what pytorch version are you using?

FlameSky · Answer 4 · Thu May 06 2021 20:55:14 GMT+0800 (China Standard Time)

I also encountered this problem, i'm using pytorch 1.8.1 (py3.9_cuda11.1_cudnn8.0.5_0).

Stas Bekman · Answer 5 · Tue May 18 2021 02:41:05 GMT+0800 (China Standard Time)

I confirm that pytorch-1.8.1 doesn't have this fix included. And getting the same problem.

Yan Ren · Answer 6 · Fri May 28 2021 15:02:56 GMT+0800 (China Standard Time)

I confirm that pytorch-1.8.1 doesn't have this fix included. And getting the same problem.

Me too confirm this

JunHyungKang · Answer 7 · Fri May 28 2021 15:30:55 GMT+0800 (China Standard Time)

It seems that
beta1, beta2 = group['betas']
have to be moved to line 76??

f8238d7#diff-46de6ea1d9fce81c27638ecd7f137c781fd64d02acea698c432a8ddb916ea51f

iramazanli · Answer 8 · Fri Jun 18 2021 01:55:15 GMT+0800 (China Standard Time)

This was fixed in #52944, what pytorch version are you using?

This issue has been indeed solved with #52944 and it's in the latest master.

T.T. Tang · Answer 9 · Mon Jan 10 2022 00:29:49 GMT+0800 (China Standard Time)

hmmm, it seems that 1.8.2 LTS doesn't have this fix included.

zhongzee · Answer 10 · Thu Jun 15 2023 21:48:08 GMT+0800 (China Standard Time)

It seems that beta1, beta2 = group['betas'] have to be moved to line 76??

f8238d7#diff-46de6ea1d9fce81c27638ecd7f137c781fd64d02acea698c432a8ddb916ea51f
it does work