[BUG] Potential bug in the function " detect_action_space(actions: NDArray) -> ActionSpace" in d3rlpy/dataset/utils.py

[wenxuhaskell]

Hello,

I am using d3rlpy for a RL model with continuous action space.

Though the library works greatly for most of the time, I should have identified a possible bug in the function detect_action_space(actions: NDArray) -> ActionSpace, (line 360, /d3rlpy/dataset/utils.py)

def detect_action_space(actions: NDArray) -> ActionSpace:
if np.all(np.array(actions, dtype=np.int32) == actions):
return ActionSpace.DISCRETE
else:
return ActionSpace.CONTINUOUS

If the "actions" is created with type as np.float64 but has values as [[20.0], [30.0], [40.0], [50.0]], "detect_action_space" will return ActionSpace.DISCRETE, since np.array(actions, dtype=np.int32) == actions) always leads to True, while ActionSpace.CONTINUOUS is the right type of the "actions".

I attach a piece of code fragment for reproducing the error:

import numpy as np

actions = [[20.0], [30.0], [40.0], [50.0]]

r = np.all(np.array(actions, dtype=np.int32) == actions)

print(type(actions[0][0]))
print(r)

Output from the code fragment above:
<class 'float'>
True

[Takuma Seno]

@wenxuhaskell Thanks for the issue. Yes, that's possibly happening if you use datasets like that. The workaround is to explicitly specify action_space as described here #338 . This feature is already available in the latest release.

[wenxuhaskell]

@takuseno thank you for the very quick guidance! I will use the new feature to specify action_space.