H5py dataset
kaijingxjtu opened this issue · comments
Kaijing commented
Thanks for your great job!
By the way, does training processes run much more faster with h5py dataset than normal format ?
Can you release the scripts for changing format ?
Kevin commented
Hi @kaijingxjtu Thanks for your interest. during training, we need to load the vid / text features frequently, which may cause some IO issues (e.g., slow), h5py is a good way you can first load all features in cache, and then you can avoid any IO loading after.
I have submitted a script under data/create_h5py.py
Please take a look.