How could we train on our own dataset?
CoinCheung opened this issue · comments
Did you load all training and eval data to memory before trying the cnn model? What if there are much more data in the dataset that cannot be load to memory at one time?
You can try with flow_from_directory to load the dataset. I haven't tried that. maximum I loaded 60-70k image numpy file of 160x160.
Loading jpg/png files takes a lot of time instead of npy file. And converting whole lot of frames to npy format was not feasible, that's why I created a train_data.npy of 50000, 160, 160, 3 dimension. My gpu server had enough ram to load it.
You can also do one thing. Create a csv file of list of all frames and then create your batch generator class and pass that list. I did it with in train_C3D.py. You can also look at the following link for more info: Link. That's a more plausible solution.