Failed in training hdf5 dataset
elysion122 opened this issue · comments
state {
phase: TRAIN
level: 0
stage: ""
}
layer {
name: "data"
type: "HDF5Data"
top: "data"
top: "label"
include {
phase: TRAIN
}
hdf5_data_param {
source: "examples/3DSR/train.txt"
batch_size: 30
}
}
layer {
name: "conv1a"
type: "NdConvolution"
bottom: "data"
top: "conv1a"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 64
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
pad_shape {
dim: 1
dim: 1
dim: 1
}
kernel_shape {
dim: 3
dim: 3
dim: 3
}
stride_shape {
dim: 1
dim: 1
dim: 1
}
}
}
layer {
name: "relu1a"
type: "ReLU"
bottom: "conv1a"
top: "conv1a"
}
layer {
name: "conv2a"
type: "NdConvolution"
bottom: "conv1a"
top: "conv2a"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 32
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 1
}
pad_shape {
dim: 0
dim: 1
dim: 1
}
kernel_shape {
dim: 3
dim: 3
dim: 3
}
stride_shape {
dim: 1
dim: 1
dim: 1
}
}
}
layer {
name: "relu2a"
type: "ReLU"
bottom: "conv2a"
top: "conv2a"
}
layer {
name: "conv3a"
type: "NdConvolution"
bottom: "conv2a"
top: "conv3a"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 1
}
pad_shape {
dim: 0
dim: 1
dim: 1
}
kernel_shape {
dim: 3
dim: 3
dim: 3
}
stride_shape {
dim: 1
dim: 1
dim: 1
}
}
}
layer {
name: "relu3a"
type: "ReLU"
bottom: "conv3a"
top: "conv3a"
}
layer {
name: "loss"
type: "EuclideanLoss"
bottom: "conv3a"
bottom: "label"
top: "loss"
}
I0420 21:40:06.688657 5774 layer_factory.hpp:77] Creating layer data
I0420 21:40:06.688671 5774 net.cpp:100] Creating Layer data
I0420 21:40:06.688676 5774 net.cpp:408] data -> data
I0420 21:40:06.688694 5774 net.cpp:408] data -> label
I0420 21:40:06.688702 5774 hdf5_data_layer.cpp:79] Loading list of HDF5 filenames from: examples/3DSR/train.txt
I0420 21:40:06.688720 5774 hdf5_data_layer.cpp:93] Number of HDF5 files: 2
I0420 21:40:06.689285 5774 hdf5.cpp:32] Datatype class: H5T_FLOAT
I0420 21:40:06.723724 5774 net.cpp:150] Setting up data
I0420 21:40:06.723754 5774 net.cpp:157] Top shape: 30 1 5 33 33 (163350)
I0420 21:40:06.723759 5774 net.cpp:157] Top shape: 30 1 1 33 33 (32670)
I0420 21:40:06.723762 5774 net.cpp:165] Memory required for data: 784080
I0420 21:40:06.723768 5774 layer_factory.hpp:77] Creating layer conv1a
I0420 21:40:06.723815 5774 net.cpp:100] Creating Layer conv1a
I0420 21:40:06.723824 5774 net.cpp:434] conv1a <- data
I0420 21:40:06.723834 5774 net.cpp:408] conv1a -> conv1a
I0420 21:40:06.835284 5774 net.cpp:150] Setting up conv1a
I0420 21:40:06.835314 5774 net.cpp:157] Top shape: 30 64 5 33 33 (10454400)
I0420 21:40:06.835317 5774 net.cpp:165] Memory required for data: 42601680
I0420 21:40:06.835332 5774 layer_factory.hpp:77] Creating layer relu1a
I0420 21:40:06.835381 5774 net.cpp:100] Creating Layer relu1a
I0420 21:40:06.835386 5774 net.cpp:434] relu1a <- conv1a
I0420 21:40:06.835392 5774 net.cpp:395] relu1a -> conv1a (in-place)
I0420 21:40:06.835623 5774 net.cpp:150] Setting up relu1a
I0420 21:40:06.835633 5774 net.cpp:157] Top shape: 30 64 5 33 33 (10454400)
I0420 21:40:06.835635 5774 net.cpp:165] Memory required for data: 84419280
I0420 21:40:06.835638 5774 layer_factory.hpp:77] Creating layer conv2a
I0420 21:40:06.835649 5774 net.cpp:100] Creating Layer conv2a
I0420 21:40:06.835651 5774 net.cpp:434] conv2a <- conv1a
I0420 21:40:06.835666 5774 net.cpp:408] conv2a -> conv2a
I0420 21:40:06.837594 5774 net.cpp:150] Setting up conv2a
I0420 21:40:06.837605 5774 net.cpp:157] Top shape: 30 32 3 33 33 (3136320)
I0420 21:40:06.837608 5774 net.cpp:165] Memory required for data: 96964560
I0420 21:40:06.837615 5774 layer_factory.hpp:77] Creating layer relu2a
I0420 21:40:06.837621 5774 net.cpp:100] Creating Layer relu2a
I0420 21:40:06.837625 5774 net.cpp:434] relu2a <- conv2a
I0420 21:40:06.837638 5774 net.cpp:395] relu2a -> conv2a (in-place)
I0420 21:40:06.837836 5774 net.cpp:150] Setting up relu2a
I0420 21:40:06.837846 5774 net.cpp:157] Top shape: 30 32 3 33 33 (3136320)
I0420 21:40:06.837848 5774 net.cpp:165] Memory required for data: 109509840
I0420 21:40:06.837852 5774 layer_factory.hpp:77] Creating layer conv3a
I0420 21:40:06.837858 5774 net.cpp:100] Creating Layer conv3a
I0420 21:40:06.837862 5774 net.cpp:434] conv3a <- conv2a
I0420 21:40:06.837877 5774 net.cpp:408] conv3a -> conv3a
I0420 21:40:06.838711 5774 net.cpp:150] Setting up conv3a
I0420 21:40:06.838722 5774 net.cpp:157] Top shape: 30 1 33 33 (32670)
I0420 21:40:06.838726 5774 net.cpp:165] Memory required for data: 109640520
I0420 21:40:06.838732 5774 layer_factory.hpp:77] Creating layer relu3a
I0420 21:40:06.838739 5774 net.cpp:100] Creating Layer relu3a
I0420 21:40:06.838752 5774 net.cpp:434] relu3a <- conv3a
I0420 21:40:06.838757 5774 net.cpp:395] relu3a -> conv3a (in-place)
I0420 21:40:06.838881 5774 net.cpp:150] Setting up relu3a
I0420 21:40:06.838888 5774 net.cpp:157] Top shape: 30 1 33 33 (32670)
I0420 21:40:06.838891 5774 net.cpp:165] Memory required for data: 109771200
I0420 21:40:06.838893 5774 layer_factory.hpp:77] Creating layer loss
I0420 21:40:06.838898 5774 net.cpp:100] Creating Layer loss
I0420 21:40:06.838901 5774 net.cpp:434] loss <- conv3a
I0420 21:40:06.838904 5774 net.cpp:434] loss <- label
I0420 21:40:06.838920 5774 net.cpp:408] loss -> loss
I0420 21:40:06.838950 5774 net.cpp:150] Setting up loss
I0420 21:40:06.838964 5774 net.cpp:157] Top shape: (1)
I0420 21:40:06.838966 5774 net.cpp:160] with loss weight 1
I0420 21:40:06.838979 5774 net.cpp:165] Memory required for data: 109771204
I0420 21:40:06.838981 5774 net.cpp:226] loss needs backward computation.
I0420 21:40:06.838984 5774 net.cpp:226] relu3a needs backward computation.
I0420 21:40:06.838996 5774 net.cpp:226] conv3a needs backward computation.
I0420 21:40:06.838999 5774 net.cpp:226] relu2a needs backward computation.
I0420 21:40:06.839001 5774 net.cpp:226] conv2a needs backward computation.
I0420 21:40:06.839004 5774 net.cpp:226] relu1a needs backward computation.
I0420 21:40:06.839006 5774 net.cpp:226] conv1a needs backward computation.
I0420 21:40:06.839010 5774 net.cpp:228] data does not need backward computation.
I0420 21:40:06.839011 5774 net.cpp:270] This network produces output loss
I0420 21:40:06.839018 5774 net.cpp:283] Network initialization done.
I0420 21:40:06.839059 5774 solver.cpp:60] Solver scaffolding done.
I0420 21:40:06.839226 5774 caffe.cpp:251] Starting Optimization
I0420 21:40:06.839231 5774 solver.cpp:279] Solving 3DSR
I0420 21:40:06.839232 5774 solver.cpp:280] Learning Rate Policy: step
F0420 21:40:06.852252 5774 cudnn_ndconv_layer.cu:43] Check failed: status == CUDNN_STATUS_SUCCESS (3 vs. 0) CUDNN_STATUS_BAD_PARAM
I run UCF101, it worked well.
When it comes to my net, it failed. I used hdf5 data as input. my data 5X33X33(l X h X w) and my label is 1X33X33. I don't know why it fail. What does the error mean?
Thank you.
Thank you for your time. The refactor branch worked well
Closing.