Lookahead has no attribute 'state'

[Hyungjin Chung]

When trying to save optimizer_state_dict,
I get an error saying >>Lookahead has no attribute 'state'
I thought this was due to not initializing parent class inside the class Lookahead
and so I added the line

super(Lookahead).__init__()

However, this still does not solve the issue. Other than saving, there seems no issue. Do you have any idea how to solve this issue?

[Kyeongpil Kang]

I got same problem...

[Kyeongpil Kang]

@HJ-harry Do you use horovod?

It happens when I use horovod.

I solved the problem by changing the position of warpping.

ex)
base_optimizer = Adam()
base_optimizer = hvd.DistributedOptimizer(base_optimizer, named_parameters=...)
hvd.broadcast_parameters(model.state_dict(), root_rank=0)
hvd.broadcast_optimizer_state(base_optimizer, root_rank=0)
optimizer = Lookahead(base_optimizer, k=5, alpha=0.5)

[Kyeongpil Kang]

I got a same error when saving the Lookahead optimizer.
Lookahead optimizer should have the state_dict function.

[Oscar Smith]

I just ran into this too. Any fixes?

[Kaushik Amar Das]

@oscardssmith optimizer.optimizer.state_dict() seems to solve the issue for me. But i still think, there should be a state for the entire optimizer as whole.