delira.training.PyTorchNetworkTrainer missing argument
An-Shank opened this issue · comments
Hi,
I'm getting an error while running train_single_shapenet.py. It says that a required positional argument (key_mapping) is missing in the function call. Please find the output below:
$ python train_single_shapenet.py -c ~/shapenet/shapenet/example_configs/helen.config -v
WARNING: Logging before flag parsing goes to stderr.
W0726 12:15:12.281908 139805229729536 deprecation_wrapper.py:119] From /home/anand/env_16.04_py3/lib/python3.5/site-packages/delira/models/abstract_network.py:251: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.
/home/anand/env_16.04_py3/lib/python3.5/site-packages/shapenet/utils/load_config_file.py:50: YAMLLoadWarning: calling yaml.load_all() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
for doc in docs:
Number of Parameters: 6248029
Traceback (most recent call last):
File "train_single_shapenet.py", line 141, in
train_shapenet()
File "train_single_shapenet.py", line 71, in train_shapenet
**config_dict["training"])
TypeError: init() missing 1 required positional argument: 'key_mapping'
You are using a delira-version that is to new, you are probably using delira 0.4.0 or the master branch (which will be 0.4.1), but you need delira 0.3.1 to make it run successfully. I have not yet found the time to update it, but I'll try my best to do this ASAP
Thank you for your prompt response. However, reverting back to delira 0.3.1 still throws me the following error:
$ python train_single_shapenet.py -c ~/shapenet/shapenet/example_configs/helen.config -v
Traceback (most recent call last):
File "train_single_shapenet.py", line 141, in
train_shapenet()
File "train_single_shapenet.py", line 17, in train_shapenet
from shapedata.single_shape import SingleShapeDataset
File "/home/anand/env_16.04_py3/lib/python3.5/site-packages/shapedata/init.py", line 9, in
from .single_shape import SingleShapeDataProcessing, SingleShapeDataset,
File "/home/anand/env_16.04_py3/lib/python3.5/site-packages/shapedata/single_shape/init.py", line 9, in
from .dataset import ShapeDataset as SingleShapeDataset
File "/home/anand/env_16.04_py3/lib/python3.5/site-packages/shapedata/single_shape/dataset.py", line 3, in
from delira.data_loading import AbstractDataset
File "/home/anand/env_16.04_py3/lib/python3.5/site-packages/delira/data_loading/init.py", line 2, in
from .data_loader import BaseDataLoader
File "/home/anand/env_16.04_py3/lib/python3.5/site-packages/delira/data_loading/data_loader.py", line 5, in
from .sampler import AbstractSampler, SequentialSampler
File "/home/anand/env_16.04_py3/lib/python3.5/site-packages/delira/data_loading/sampler/init.py", line 4, in
from .random_sampler import RandomSampler, PrevalenceRandomSampler,
File "/home/anand/env_16.04_py3/lib/python3.5/site-packages/delira/data_loading/sampler/random_sampler.py", line 1, in
from random import shuffle, choices
ImportError: cannot import name 'choices'
The shapedata version is 0.1.0 and delira is 0.3.1
This is due to your python version. delira 0.3.1 and below was targeted for Python 3.6 and above. The problem ist, that Python 3.5 does not containt the choices function. Sorry for the inconvenience. To support Python 3.5, we changed this starting with delira 0.3.2. I'll see, if I manage to update this tomorrow.
I was able to train the model using Python 3.6 and delira 0.3.1. However I'm getting the following error in export_to_jit.py
$ python export_to_jit.py -c ~/shapenet/shapenet/example_configs/helen.config -w ~/shapenet/shapenet/shapenet/Results/helen/checkpoint_epoch_99.pth -o ~/shapenet/shapenet/model.ptj
/home/anand/env_16.04_py3.6/lib/python3.6/site-packages/torch/utils/cpp_extension.py:4: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
/home/anand/env_16.04_py3.6/lib/python3.6/site-packages/shapenet/utils/load_config_file.py:50: YAMLLoadWarning: calling yaml.load_all() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
for doc in docs:
Traceback (most recent call last):
File "export_to_jit.py", line 44, in create_jit_net_from_config_and_weight
map_location="cpu")["state_dict"]["model"]
KeyError: 'state_dict'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "export_to_jit.py", line 76, in
main()
File "export_to_jit.py", line 70, in main
os.path.abspath(args.weight_file))
File "export_to_jit.py", line 48, in create_jit_net_from_config_and_weight
torch.load(weight_file, map_location="cpu")
File "/home/anand/env_16.04_py3.6/lib/python3.6/site-packages/torch/nn/modules/module.py", line 777, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for ShapeNetwork:
Missing key(s) in state_dict: "_out_layer._indices_shape_params", "_out_layer._indices_homogen_params", "_out_layer._shape_layer._layer._shape_mean", "_out_layer._shape_layer._layer._shape_components", "_out_layer._homogen_trafo._indices_scale_params", "_out_layer._homogen_trafo._indices_rotation_params", "_out_layer._homogen_trafo._indices_translation_params", "_out_layer._homogen_trafo._layer._trafo_matrix", "_model.model.conv_1._conv.bias", "_model.model.conv_1._conv.weight", "_model.model.conv_2._conv.bias", "_model.model.conv_2._conv.weight", "_model.model.down_conv_1._conv.bias", "_model.model.down_conv_1._conv.weight", "_model.model.conv_3._conv.bias", "_model.model.conv_3._conv.weight", "_model.model.conv_4._conv.bias", "_model.model.conv_4._conv.weight", "_model.model.down_conv_2._conv.bias", "_model.model.down_conv_2._conv.weight", "_model.model.conv_5._conv.bias", "_model.model.conv_5._conv.weight", "_model.model.conv_6._conv.bias", "_model.model.conv_6._conv.weight", "_model.model.down_conv_3._conv.bias", "_model.model.down_conv_3._conv.weight", "_model.model.conv_7._conv.bias", "_model.model.conv_7._conv.weight", "_model.model.conv_8._conv.bias", "_model.model.conv_8._conv.weight", "_model.model.down_conv_4._conv.bias", "_model.model.down_conv_4._conv.weight", "_model.model.conv_9._conv.bias", "_model.model.conv_9._conv.weight", "_model.model.conv_10._conv.bias", "_model.model.conv_10._conv.weight", "_model.model.conv_11._conv.bias", "_model.model.conv_11._conv.weight", "_model.model.conv_12._conv.bias", "_model.model.conv_12._conv.weight", "_model.model.final_conv.weight", "_model.model.final_conv.bias".
Unexpected key(s) in state_dict: "optimizer", "model", "epoch".
This should easily be solved by just removing the ["state_dict"] in line 44 of that file.
I will update the repo this afternoon
So I updated the repo just now, and the tests are passing.
However, on my current machine I can't test the training so there might be some minor issues to resolve before releasing. If you want to, you can try it by installing from this repo via
pip install git+https://github.com/justusschock/shapenet
It's working fine. Thank you for your help.