[Bug] AttributeError: module 'tensorflow.python.util.tf_inspect' has no attribute 'Parameter'

Question

[Bug] AttributeError: module 'tensorflow.python.util.tf_inspect' has no attribute 'Parameter'

ashok-arora opened this issue 5 months ago · comments

🐛 Bug

To Reproduce

import gymnasium as gym

from sbx import DDPG, DQN, PPO, SAC, TD3, TQC, DroQ

env = gym.make("Pendulum-v1")

model = TQC("MlpPolicy", env, verbose=1)
model.learn(total_timesteps=10_000, progress_bar=True)

vec_env = model.get_env()
obs = vec_env.reset()
for i in range(1000):
    action, _states = model.predict(obs, deterministic=True)
    obs, reward, done, info = vec_env.step(action)
    vec_env.render()

vec_env.close()

error:

Traceback (most recent call last):
  File "check_sbx.py", line 3, in <module>
    from sbx import DDPG, DQN, PPO, SAC, TD3, TQC, DroQ
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/sbx/__init__.py", line 5, in <module>
    from sbx.droq import DroQ
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/sbx/droq/__init__.py", line 1, in <module>
    from sbx.droq.droq import DroQ
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/sbx/droq/droq.py", line 7, in <module>
    from sbx.tqc.policies import TQCPolicy
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/sbx/tqc/__init__.py", line 1, in <module>
    from sbx.tqc.tqc import TQC
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/sbx/tqc/tqc.py", line 19, in <module>
    from sbx.tqc.policies import TQCPolicy
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/sbx/tqc/policies.py", line 13, in <module>
    from sbx.common.distributions import TanhTransformedDistribution
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/sbx/common/distributions.py", line 7, in <module>
    tfd = tfp.distributions
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/tensorflow_probability/python/internal/lazy_loader.py", line 53, in __getattr__
    module = self._load()
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/tensorflow_probability/python/internal/lazy_loader.py", line 40, in _load
    module = importlib.import_module(self.__name__)
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/tensorflow_probability/substrates/jax/__init__.py", line 41, in <module>
    from tensorflow_probability.substrates.jax import bijectors
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/tensorflow_probability/substrates/jax/bijectors/__init__.py", line 19, in <module>
    from tensorflow_probability.substrates.jax.bijectors.absolute_value import AbsoluteValue
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/tensorflow_probability/substrates/jax/bijectors/absolute_value.py", line 17, in <module>
    from tensorflow_probability.python.internal.backend.jax.compat import v2 as tf
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/tensorflow_probability/python/internal/backend/jax/__init__.py", line 19, in <module>
    from tensorflow_probability.python.internal.backend.jax import compat
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/tensorflow_probability/python/internal/backend/jax/compat.py", line 18, in <module>
    from tensorflow_probability.python.internal.backend.jax import v2
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/tensorflow_probability/python/internal/backend/jax/v2.py", line 27, in <module>
    from tensorflow_probability.python.internal.backend.jax import linalg
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/tensorflow_probability/python/internal/backend/jax/linalg.py", line 28, in <module>
    from tensorflow_probability.python.internal.backend.jax.gen import adjoint_registrations as _adjoint_registrations
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/tensorflow_probability/python/internal/backend/jax/gen/adjoint_registrations.py", line 37, in <module>
    from tensorflow_probability.python.internal.backend.jax.gen import linear_operator
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/tensorflow_probability/python/internal/backend/jax/gen/linear_operator.py", line 58, in <module>
    from tensorflow_probability.python.internal.backend.jax.gen import linear_operator_algebra
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/tensorflow_probability/python/internal/backend/jax/gen/linear_operator_algebra.py", line 40, in <module>
    from tensorflow_probability.python.internal.backend.jax import tf_inspect
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/tensorflow_probability/python/internal/backend/jax/tf_inspect.py", line 26, in <module>
    Parameter = inspect.Parameter
AttributeError: module 'tensorflow.python.util.tf_inspect' has no attribute 'Parameter'

Expected behavior

Should have started training.

### System Info

- OS: Linux-6.5.0-17-generic-x86_64-with-glibc2.10 # 17~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Jan 16 14:32:32 UTC 2
- Python: 3.8.5
- Stable-Baselines3: 2.3.0a2
- PyTorch: 2.2.0+cu121
- GPU Enabled: True
- Numpy: 1.23.0
- Cloudpickle: 3.0.0
- Gymnasium: 0.29.1
- OpenAI Gym: 0.18.3

Additional context

Add any other context about the problem here.

Checklist

I have checked that there is no similar issue in the repo (required)
I have read the documentation (required)
I have provided a minimal working example to reproduce the bug (required)

Antonin RAFFIN · Answer 1 · Wed Feb 14 2024 19:59:02 GMT+0800 (China Standard Time)

Hello,
what is your tensorflow probability version?
and tensorflow version?

SBX was successfully tested with tensorflow-probability==0.20.1, tensorflow-probability==0.23.0 and without tensorflow installed.

Ashok Arora · Answer 2 · Wed Feb 14 2024 21:57:39 GMT+0800 (China Standard Time)

I have tensorflow-probability==0.21.1 installed.

Antonin RAFFIN · Answer 3 · Wed Feb 14 2024 22:08:59 GMT+0800 (China Standard Time)

I have tensorflow-probability==0.21.1 installed.

and tensorflow?
if you have tensorflow installed, try uninstalling it.

You can also upgrade tf probability.

Antonin RAFFIN · Answer 4 · Wed Feb 14 2024 22:16:52 GMT+0800 (China Standard Time)

Anyway, it sounds more like a tf probablity issue than a sbx one, I guess if you import tf proba and try to use it, you would have the same error?

In that case, I would open an issue their repo.

Ashok Arora · Answer 5 · Thu Feb 15 2024 11:50:01 GMT+0800 (China Standard Time)

tensorflow version is 2.13.1.

Ashok Arora · Answer 6 · Thu Feb 15 2024 11:53:56 GMT+0800 (China Standard Time)

I uninstalled tensorflow and now got a different error:

Traceback (most recent call last):
  File "check_sbx.py", line 2, in <module>
    import stable_baselines3 as sb3
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/stable_baselines3/__init__.py", line 3, in <module>
    from stable_baselines3.a2c import A2C
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/stable_baselines3/a2c/__init__.py", line 1, in <module>
    from stable_baselines3.a2c.a2c import A2C
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/stable_baselines3/a2c/a2c.py", line 7, in <module>
    from stable_baselines3.common.buffers import RolloutBuffer
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/stable_baselines3/common/buffers.py", line 16, in <module>
    from stable_baselines3.common.utils import get_device
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/stable_baselines3/common/utils.py", line 20, in <module>
    from torch.utils.tensorboard import SummaryWriter
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/torch/utils/tensorboard/__init__.py", line 12, in <module>
    from .writer import FileWriter, SummaryWriter  # noqa: F401
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/torch/utils/tensorboard/writer.py", line 18, in <module>
    from ._embedding import get_embedding_info, make_mat, make_sprite, make_tsv, write_pbtxt
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/torch/utils/tensorboard/_embedding.py", line 9, in <module>
    _HAS_GFILE_JOIN = hasattr(tf.io.gfile, "join")
  File "/home/user/miniconda3/envs/decision-transformer-gym/lib/python3.8/site-packages/tensorboard/lazy.py", line 65, in __getattr__
    return getattr(load_once(self), attr_name)
AttributeError: module 'tensorflow' has no attribute 'io'

Antonin RAFFIN · Answer 7 · Thu Feb 15 2024 13:33:40 GMT+0800 (China Standard Time)

it seems that tf is still partially installed, i would recommend start from an empty virtual env, that should solve your issue.

Ashok Arora · Answer 8 · Thu Feb 15 2024 13:54:14 GMT+0800 (China Standard Time)

I am using tensorboard for logging values so that might be the cause of partial tf, anything else I can do aside from empty venv?

Antonin RAFFIN · Answer 9 · Sun Mar 31 2024 03:51:13 GMT+0800 (China Standard Time)

tensorboard doesn't need tf normally.

incompatibility might be fixed by #28

commit: deabd7f