The program did not finish running an epoch, a path error occurred.
emily19941102 opened this issue · comments
The program did not finish running an epoch, a path error occurred.I checked the dataset and there was no problem.I made the data set smaller, and it will give an error when executing 49/50, and this problem will also occur when executing 79/80.Thanks！
@GuangmingZhu
@emily19941102 What is the problem exactly? It seems that the training of this epoch has been finished, but problems occur during the process of testing or saving models.
The problem began when the first epoch didn't end during training.Basically, there was a path error, but I checked the path.The same thing happens when I make the data set smaller or different modes.
@GuangmingZhu
@GuangmingZhu
Epoch 1/10
49/50 [============================>.] - ETA: 3s - loss: 6.4515 - acc: 0.0000e+00 Exception in thread threading_and_return:
Traceback (most recent call last):
File "E:\anaconda\envs\tensorflow\lib\threading.py", line 914, in _bootstrap_inner
self.run()
File "E:\anaconda\envs\tensorflow\lib\threading.py", line 862, in run
self._target(*self._args, **self._kwargs)
File "./networks\datagen.py", line 24, in apply_fn
results[i] = fn(data, **kwargs)
File "./networks\inputs.py", line 35, in prepare_iso_rgb_data
assert os.path.exists(video_path)
AssertionError
Exception in thread threading_and_return:
Traceback (most recent call last):
File "E:\anaconda\envs\tensorflow\lib\threading.py", line 914, in _bootstrap_inner
self.run()
File "E:\anaconda\envs\tensorflow\lib\threading.py", line 862, in run
self._target(*self._args, **self._kwargs)
File "./networks\datagen.py", line 24, in apply_fn
results[i] = fn(data, **kwargs)
File "./networks\inputs.py", line 35, in prepare_iso_rgb_data
assert os.path.exists(video_path)
AssertionError
Traceback (most recent call last):
File "", line 1, in
runfile('F:/IsoGD/AttentionConvLSTM-master/training_res3d_clstm_mobilenet.py', wdir='F:/IsoGD/AttentionConvLSTM-master')
File "E:\anaconda\envs\tensorflow\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
execfile(filename, namespace)
File "E:\anaconda\envs\tensorflow\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "F:/IsoGD/AttentionConvLSTM-master/training_res3d_clstm_mobilenet.py", line 120, in
initial_epoch=init_epoch,
File "E:\anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\keras\engine\training.py", line 1779, in fit_generator
initial_epoch=initial_epoch)
File "E:\anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\keras\engine\training_generator.py", line 225, in fit_generator
max_queue_size=max_queue_size)
File "E:\anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\keras\engine\training_generator.py", line 336, in evaluate_generator
outs = model.test_on_batch(x, y, sample_weight=sample_weight)
File "E:\anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\keras\engine\training.py", line 1596, in test_on_batch
x, y, sample_weight=sample_weight)
File "E:\anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\keras\engine\training.py", line 878, in _standardize_user_data
exception_prefix='input')
File "E:\anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\keras\engine\training_utils.py", line 182, in standardize_input_data
'with shape ' + str(data_shape))
ValueError: Error when checking input: expected input_3 to have 5 dimensions, but got array with shape (2, 1)
@emily19941102 You may double-check the path of the test subset.
@emily19941102@GuangmingZhu
I also have the similar problem,Can you tell me what shoud I do?
The Error is:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "./networks/datagen.py", line 24, in apply_fn
results[i] = fn(data, **kwargs)
File "./networks/inputs.py", line 62, in prepare_iso_rgb_data
assert os.path.exists(image_file)
AssertionError
1240/2242 [===============>..............] - ETA: 3761s - loss: 5.9959 - acc: 0.0211Traceback (most recent call last):
File "training_res3d_clstm_mobilenet.py", line 166, in
initial_epoch=init_epoch,
File "/home/thf/tensorflow/local/lib/python2.7/site-packages/tensorflow/contrib/keras/python/keras/engine/training.py", line 1880, in fit_generator
x, y, sample_weight=sample_weight, class_weight=class_weight)
File "/home/thf/tensorflow/local/lib/python2.7/site-packages/tensorflow/contrib/keras/python/keras/engine/training.py", line 1624, in train_on_batch
check_batch_axis=True)
File "/home/thf/tensorflow/local/lib/python2.7/site-packages/tensorflow/contrib/keras/python/keras/engine/training.py", line 1300, in _standardize_user_data
exception_prefix='model input')
File "/home/thf/tensorflow/local/lib/python2.7/site-packages/tensorflow/contrib/keras/python/keras/engine/training.py", line 138, in _standardize_input_data
' dimensions, but got array with shape ' + str(array.shape))
ValueError: Error when checking model input: expected input_1 to have 5 dimensions, but got array with shape (16, 1)
@peach2wanzi You can print the shape of the input before feeding it into the Input Layer. I guess that something may be wrong when you read images for the input of this iteration, so that this iteration does not has the right input. Before training, you can double check the existance of all images using a shell script or a python program.