performance at different checkpoints

[Xin Yi]

Thanks for sharing the code of this great work. I have a question while using the code regarding the performance at different checkpoints (files in the shared models folder).

I loaded each model and used it to annotate a car in the supplied Road image. It seems that results from mle, rl and evaluator all look very bad. Some predicted vertices are not even on the edges. The result from ggnn, on the other hand, works really well. I'm wondering if that's an expected behaviour or I did something wrong.

Below are snapshots to explain what I meant.

1. result from mle
   

2. result from rl
   

3. result from evaluator
   

4. result from ggnn_epoch5
   

PS: ggnn_epoch8_step21000.pth cannot be loaded. It seems the shared checkpoint is not compatible with the defined model.
RuntimeError: Error(s) in loading state_dict for PolyRNNpp:
While copying the parameter named "ggnn.encoder.conv_final.0.weight", whose dimensions in the model are torch.Size([256, 256, 3, 3]) and whose dimensions in the checkpoint are torch.Size([128, 256, 3, 3]).

[Amlan Kar]

Hi,

I think what's happening is that the you have use GGNN turned on even while using checkpoints that do not have the GGNN. The GGNN would have random weights, therefore giving you random results (centered around the actual border since the GGNN only predicts deltas)

The GGNN checkpoint with epoch8 is from our model without a shared encoder (ResNet, check reproduction results in the README).

The model built in tool.py currently builds all parts of the model by default. However, the model building function (builder for the class), has options for skipping various parts, and these can be edited in tool.json.

Thanks for the great question! I hope this fixes it.

[Xin Yi]

Thanks for the clarification. Editing tool.json does the trick.

[WeiminWang]

Hi @amlankar

I am having the same issue here:

My tool.json is (following your ggnn.json):

{
"mode": "tool",
"max_poly_len": 71,
"temperature": 0.3,
"use_evaluator": true,
"use_bn_lstm": true,
"use_ggnn": true,
"ggnn_n_steps":5,
"ggnn_state_dim":1024,
"ggnn_grid_size": 112,
"ggnn_output_dim":15,
"use_separate_encoder": false,
"return_attention": false,
"dataset": {
"train_val" : {
"batch_size": 1,
"max_poly_len": 71,
"grid_side": 28,
"img_side": 224,
"flip": false,
"random_start": false,
"random_context": [0.15,0.15]
}
}
}
and I can't load ../models/ggnn_epoch8_step21000.pth, and got same error as xinario .

However, when I load ../models/ggnn_epoch5_step14000.pth instead, my results is bad:

I tried different combinations of weights and json settings, but could never got a good result.

Could you please help with me with the correct json settings and weights?

Thanks,