Error when using SGDW in a complex project
eypros opened this issue · comments
I was trying to use the SGDW with a project but it seems to be causing an error
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot assign a device for operation resample_p6/conv2d/kernel/Initializer/random_uniform/sub: Could not satisfy explicit device specification '' because the node {{colocation_node resample_p6/conv2d/kernel/Initializer/random_uniform/sub}} was colocated with a group of nodes that required incompatible device '/job:localhost/replica:0/task:0/device:GPU:0'. All available devices [/job:localhost/replica:0/task:0/device:CPU:0, /job:localhost/replica:0/task:0/device:XLA_CPU:0, /job:localhost/replica:0/task:0/device:XLA_GPU:0, /job:localhost/replica:0/task:0/device:GPU:0].
The error seems to be caused only when the SGDW optimizer and not the AdamW one (I haven't tried the NAdamW one).
The project I tried to apply the SGDW is EfficentDet which is quite complex project. Nevertheless, this shouldn't happen. I am not sure which is the cause of the problem. Also, when used in a small network as the one provided in the example.py there doesn't seem to be any problem.
Thanks for the report. Custom architectures may utilize resource allocation behavior that cannot be otherwise tested with API defaults. I could take a look if provided with a minimal reproducible example.
OK sorry, false alarm. I had mistyped the optimizer. (Although an accumulative optimizer I am implementing it produces the message).