Weights format mismatches with Darkflow / Darknet
sjain-stanford opened this issue · comments
Issue
I was trying to (numerically) compare the DW2TF converted TF weights with Darkflow converted TF weights. I found that the weights were reshaped incorrectly in DW2TF, which results in different output tensors from convolutional layers from the two graphs. This renders the pre-trained graph pretty much useless.
Details
Convolutional weights read from Darknet .weights are of shape:
[n, c, ksize, ksize]
[reference]
They are then transposed to match with TensorFlow:
[ksize, ksize, c, n]
[reference]
However, in DW2TF the resulting weights from reshape+transpose are of shape:
[ksize, ksize, n, c]
[reference]
Why was this not caught due to shape mismatch
The fact that any tensor fed to tf.initializers.constant() is stored as raw bytes (serialized in memory) means that any shape mismatch is silently ignored as long as the overall size matches. To catch such dangerous shape mismatches, the argument verify_shape=True should be provided to the initializer.
For example, if expected weights.shape is [3, 3, 32, 64], but the weights const initializer is fed a tensor of shape [3, 3, 64, 32], TensorFlow doesn't complain. UNLESS verify_shape=True is provided, in which case it fails with the error:
File "/scratch/anaconda3/envs/venv/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 1487, in get_variable
aggregation=aggregation)
File "/scratch/anaconda3/envs/venv/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 1237, in get_variable
aggregation=aggregation)
File "/scratch/anaconda3/envs/venv/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 540, in get_variable
aggregation=aggregation)
File "/scratch/anaconda3/envs/venv/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 492, in _true_getter
aggregation=aggregation)
File "/scratch/anaconda3/envs/venv/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 922, in _get_single_variable
aggregation=aggregation)
File "/scratch/anaconda3/envs/venv/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 183, in __call__
return cls._variable_v1_call(*args, **kwargs)
File "/scratch/anaconda3/envs/venv/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 146, in _variable_v1_call
aggregation=aggregation)
File "/scratch/anaconda3/envs/venv/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 125, in <lambda>
previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
File "/scratch/anaconda3/envs/venv/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 2444, in default_variable_creator
expected_shape=expected_shape, import_scope=import_scope)
File "/scratch/anaconda3/envs/venv/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 187, in __call__
return super(VariableMetaclass, cls).__call__(*args, **kwargs)
File "/scratch/anaconda3/envs/venv/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 1329, in __init__
constraint=constraint)
File "/scratch/anaconda3/envs/venv/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 1437, in _init_from_args
initial_value(), name="initial_value", dtype=dtype)
File "/scratch/anaconda3/envs/venv/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 896, in <lambda>
shape.as_list(), dtype=dtype, partition_info=partition_info)
File "/scratch/anaconda3/envs/venv/lib/python3.6/site-packages/tensorflow/python/ops/init_ops.py", line 220, in __call__
self.value, dtype=dtype, shape=shape, verify_shape=verify_shape)
File "/scratch/anaconda3/envs/venv/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 208, in constant
value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "/scratch/anaconda3/envs/venv/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 492, in make_tensor_proto
(tuple(shape), nparray.shape))
TypeError: Expected Tensor's shape: (3, 3, 32, 64), got (3, 3, 64, 32).
Fix
Update this line to
weights = weights.reshape(filters, C, size, size).transpose([2, 3, 1, 0])
and add verify_shape=True to all const initializers.
PR to follow.