[BUG] Regarding the division of dataset for five-fold cross-validation

[Xiuding Cai]

Hello,
Thank you for your amazing work. I am seeking clarification on how to divide the new version of d3rlpy dataset for five-fold cross-validation. When I execute "train_episodes, test_episodes = train_test_split(dataset, test_size=cfg.test_size, random_state=cfg.seed)", it raises an error: "TypeError: Singleton array array(<d3rlpy.dataset.replay_buffer.ReplayBuffer object at 0x0000028C17AE7040>, dtype=object) cannot be considered a valid collection."
I would greatly appreciate any guidance on resolving this issue. Thank you for your time and assistance.

[Takuma Seno]

@XiudingCai Hi, thanks for the issue. In the latest version, you need to do something like this:

import d3rlpy
from sklearn.model_selection import train_test_split

dataset, _ = d3rlpy.datasets.get_cartpole()
train_episodes, test_episodes = train_test_split(dataset.episodes, test_size=0.2)
train_dataset = d3rlpy.dataset.create_infinite_replay_buffer(episodes=train_episodes)

dqn = d3rlpy.algos.DQNConfig().create()
dqn.fit(train_dataset, n_steps=100000, evaluators={"td_error": d3rlpy.metrics.TDErrorEvaluator(test_episodes)})

[Xiuding Cai]

@XiudingCai Hi, thanks for the issue. In the latest version, you need to do something like this:

import d3rlpy
from sklearn.model_selection import train_test_split

dataset, _ = d3rlpy.datasets.get_cartpole()
train_episodes, test_episodes = train_test_split(dataset.episodes, test_size=0.2)
train_dataset = d3rlpy.dataset.create_infinite_replay_buffer(episodes=train_episodes)

dqn = d3rlpy.algos.DQNConfig().create()
dqn.fit(train_dataset, n_steps=100000, evaluators={"td_error": d3rlpy.metrics.TDErrorEvaluator(test_episodes)})

Thank you for your prompt reply! I will try it later. Thanks a lot!