len is not well defined for symbolic tensors AND using a `tf.Tensor` as a Python `bool` is not allowed in Graph execution

Question

len is not well defined for symbolic tensors AND using a `tf.Tensor` as a Python `bool` is not allowed in Graph execution

EduardDurech opened this issue 4 years ago · comments

There is an error in the error-handling of the ddpg.py and dqn.py agents in keras-rl/rl/agents while using Tensorflow 2.0, Keras 2.3.1, Keras-rl 0.4.2
dqn.py line 108
ddpg.py line 29, 31
It comes about by calling len(model.output)

Error:

Traceback (most recent call last):
  File "foo.py", line x, in <module>
    agent = DDPGAgent(...)
  **File "foo\ddpg.py", line 29, in __init__
    if hasattr(actor.output, '__len__') and len(actor.output) > 1:**
  File "foo\ops.py", line 741, in __len__
    "shape information.".format(self.name))
TypeError: len is not well defined for symbolic Tensors. (dense_5/BiasAdd:0) Please call `x.shape` rather than `len(x)` for shape information.

Possible solution:
What I've been using as a fix is summing the shape of the tensor:
sum(x is not None for x in model.output.shape)
Implementation example:
if hasattr(model.output, '__len__') and sum(x is not None for x in model.output.shape) > 1:

There is then another error in ddpg.py at line 130:
if i == self.critic_action_input:
Error:
tensorflow.python.framework.errors_impl.OperatorNotAllowedInGraphError: using a tf.Tensoras a Pythonbool is not allowed in Graph execution. Use Eager execution or decorate this function with @tf.function.
Including

import tensorflow as tf
tf.compat.v1.enable_eager_execution()

Does not seem to help, I have also tried creating a session and using tf.equal(i, self.critic_action_input).eval(session=sess) but I'm having issues, as of now I've tried

import tensorflow as tf

with tf.compat.v1.Session(graph=self.critic_action_input.graph) as sess:
            for i in self.critic.
                if tf.equal(i, self.critic_action_input).eval(session=sess): #if i == self.critic_action_input:
                    combined_inputs.append([])
                else:
                    combined_inputs.append(i)
                    critic_inputs.append(i)

But I cannot get it to work

Thank you

EduardDurech · Answer 1 · Sun Dec 22 2019 16:04:51 GMT+0800 (China Standard Time)

Note: the if hasattr(model.output, '__len__') and sum(x is not None for x in model.output.shape) > 1: is pretty redundant since these models do not have an attribute len

For those that would like a "solution", downgrade Tensorflow[1.13.1] and/or Keras[2.2.4], see apple/coremltools#462 (comment)

Ciprian-Octavian Truică · Answer 2 · Mon Mar 09 2020 22:20:56 GMT+0800 (China Standard Time)

Hi,

I have run some test the code and I got the following error when initializing the class DQNAgent

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/python-3.7.6/lib/python3.7/site-packages/rl/agents/dqn.py", line 108, in __init__
  if hasattr(model.output, '__len__') and len(model.output) > 1:
File "/opt/python-3.7.6/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 733, in __len__
  "shape information.".format(self.name))
TypeError: len is not well defined for symbolic Tensors. (activation_4/Identity:0) Please call `x.shape` rather than `len(x)` for shape information.

I have fixed the error by changing the specified line of code (i.e., 108) with:

if hasattr(model.output, '__shape__') and len(model.output.shape) > 2:

I hope this is a correct fix.

All the best,
Ciprian

P.S. I use

Keras==2.3.1

with the backend

tensorflow==2.1.0
tensorflow-estimator==2.1.0

yingshaoxo · Answer 3 · Mon Mar 16 2020 07:47:03 GMT+0800 (China Standard Time)

I think all of you may have mistaken what this library was capable of doing. It's a Keras module!

It can only work by using import keras instead of from tensorflow import keras.

apatsekin · Answer 4 · Wed Apr 29 2020 04:35:04 GMT+0800 (China Standard Time)

@yingshaoxo the first mentioned (len()) error shows up with import keras as well

Elliot V Pourmand · Answer 5 · Mon May 18 2020 11:42:06 GMT+0800 (China Standard Time)

#348 (comment) Thanks @cipriantruica , this resolved the issue for me.

keras_rl==0.4.2
Keras==2.2.4
numpy==1.16.2
tensorflow==1.15.2
gym==0.12.1
rl==3.0

Yanhui Chen · Answer 6 · Thu May 21 2020 15:39:51 GMT+0800 (China Standard Time)

if hasattr(model.output, '__shape__') and len(model.output) > 1: is ok.

elnazsn1988 · Answer 7 · Mon Jun 01 2020 09:20:37 GMT+0800 (China Standard Time)

Why has the change not been added to the package to make it compatible with tf>2.0?

Wizard · Answer 8 · Fri Jun 26 2020 01:10:21 GMT+0800 (China Standard Time)

please upgrade the code to make it compatible with Tensorlfow 2.0+.
Thank you !

Daan Klijn · Answer 9 · Sat Jul 04 2020 16:24:19 GMT+0800 (China Standard Time)

This fork does support TF2: https://github.com/wau/keras-rl2

EduardDurech · Answer 10 · Sat Jul 04 2020 19:02:12 GMT+0800 (China Standard Time)

This fork does support TF2: https://github.com/wau/keras-rl2

Yes, but it seems the author recommends other libraries

No fix yet, I recommend https://github.com/danaugrs/huskarl for newer algorithms and a cleaner implementation. That repo has a functional ddpg working with Tensorflow 2.

Originally posted by @wau in inarikami/keras-rl2#2 (comment)

There are a few libraries which are compatible with TF2 and have multiple native agents
RLLib: https://github.com/ray-project/ray/tree/master/rllib
Tensorforce:
https://github.com/tensorforce/tensorforce
Coach:
https://github.com/NervanaSystems/coach

Hanker · Answer 11 · Fri Jul 17 2020 15:39:47 GMT+0800 (China Standard Time)

Hi,

I have run some test the code and I got the following error when initializing the class DQNAgent
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/python-3.7.6/lib/python3.7/site-packages/rl/agents/dqn.py", line 108, in __init__
  if hasattr(model.output, '__len__') and len(model.output) > 1:
File "/opt/python-3.7.6/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 733, in __len__
  "shape information.".format(self.name))
TypeError: len is not well defined for symbolic Tensors. (activation_4/Identity:0) Please call `x.shape` rather than `len(x)` for shape information.
I have fixed the error by changing the specified line of code (i.e., 108) with:

if hasattr(model.output, '__shape__') and len(model.output.shape) > 2:

I hope this is a correct fix.

All the best,
Ciprian

P.S. I use
Keras==2.3.1
with the backend
tensorflow==2.1.0
tensorflow-estimator==2.1.0

it works for me, i using tensorflow 1.15, and keras 2.3

Carl Boettiger · Answer 12 · Thu Sep 10 2020 04:32:38 GMT+0800 (China Standard Time)

I see

TypeError: len is not well defined for symbolic Tensors. (activation_4/Identity:0) Please call `x.shape` rather than `len(x)` for shape information.

When using Tensorflow 1.15.0. Downgrading to Tensorflow 1.14.2 resolves this, but Tensorflow 1.14.2 is subject to two high-security vulnerabilities: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/security/advisory/tfsa-2019-002.md

sylvain-bougnoux · Answer 13 · Fri Sep 11 2020 01:17:47 GMT+0800 (China Standard Time)

All these fixes pass but are semantically wrong. Both in dqn & ddpg the tests are used to insure that the number of action is 1.
The syntax if hasattr(model.output, '__len__') is used to insure that the global method len can be called on this particular object (actor.output) so mixing len & shape is a total nonsense. Globally your test pass and it works because your actor has only one action ; but basically commenting everything out just work as well. The issue is that Tensors from TF2.? do have the __len__ attribute (at least some of them) but does not really know how to compute it on some Tensor, hence the warning. In my particular example my actor.output.shape is [none,1] so the right test in this case is if hasattr(model.output, 'shape') and model.output.shape[1]!=1: ....
I am not familiar with TF enough to insure that all TF*.* versions will accept this test. A simple design would be something like

ok=False
try: ok = len(model.output)==1
except: pass
try: ok = model.output.shape[0]==None and model.output.shape[1]==1
except: pass
...
if not ok: raise ...

Now as it is awkward to write for each tensor, create a function robust to any tensor e.g.:

def robustLen(tensor):
    try: return len(tensor)
    except: pass
    try: 
        l=1
        for i in tensor.shape: l*=i if i else 1
        return l
    except: pass
    raise ...

stale · Answer 14 · Sun Dec 13 2020 09:27:15 GMT+0800 (China Standard Time)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Shravan Suthar · Answer 15 · Thu Feb 11 2021 20:04:21 GMT+0800 (China Standard Time)

C:\Python\Python37\lib\site-packages\keras_rl-0.4.2-py3.7.egg\rl\agents\dqn.py in init(self, model, policy, test_policy, enable_double_dqn, enable_dueling_network, dueling_type, *args, **kwargs)
106
107 # Validate (important) input.
--> 108 if hasattr(model.output, 'len') and len(model.output) > 1:
109 raise ValueError('Model "{}" has more than one output. DQN expects a model that has a single output.'.format(model))
110 if model.output._keras_shape != (None, self.nb_actions):

C:\Python\Python37\lib\site-packages\tensorflow\python\keras\engine\keras_tensor.py in len(self)
238
239 def len(self):
--> 240 raise TypeError('Keras symbolic inputs/outputs do not '
241 'implement __len__. You may be '
242 'trying to pass Keras symbolic inputs/outputs '

TypeError: Keras symbolic inputs/outputs do not implement __len__. You may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model. This error will also get raised if you try asserting a symbolic input/output directly.

please help me .....

Satpal Singh Rathore · Answer 16 · Mon Jul 18 2022 16:42:38 GMT+0800 (China Standard Time)

Issue still persist.

len is not well defined for symbolic tensors *AND* using a `tf.Tensor` as a Python `bool` is not allowed in Graph execution

len is not well defined for symbolic tensors AND using a `tf.Tensor` as a Python `bool` is not allowed in Graph execution