fidler-lab / polyrnn-pp-pytorch

PyTorch training/tool code for Polygon-RNN++ (CVPR 2018)

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

performance at different checkpoints

xinario opened this issue · comments

Thanks for sharing the code of this great work. I have a question while using the code regarding the performance at different checkpoints (files in the shared models folder).

I loaded each model and used it to annotate a car in the supplied Road image. It seems that results from mle, rl and evaluator all look very bad. Some predicted vertices are not even on the edges. The result from ggnn, on the other hand, works really well. I'm wondering if that's an expected behaviour or I did something wrong.

Below are snapshots to explain what I meant.

  1. result from mle
    mle

  2. result from rl
    rl

  3. result from evaluator
    evaluator

  4. result from ggnn_epoch5
    ggnn_epoch5

PS: ggnn_epoch8_step21000.pth cannot be loaded. It seems the shared checkpoint is not compatible with the defined model.
RuntimeError: Error(s) in loading state_dict for PolyRNNpp:
While copying the parameter named "ggnn.encoder.conv_final.0.weight", whose dimensions in the model are torch.Size([256, 256, 3, 3]) and whose dimensions in the checkpoint are torch.Size([128, 256, 3, 3]).

Hi,

I think what's happening is that the you have use GGNN turned on even while using checkpoints that do not have the GGNN. The GGNN would have random weights, therefore giving you random results (centered around the actual border since the GGNN only predicts deltas)

The GGNN checkpoint with epoch8 is from our model without a shared encoder (ResNet, check reproduction results in the README).

The model built in tool.py currently builds all parts of the model by default. However, the model building function (builder for the class), has options for skipping various parts, and these can be edited in tool.json.

Thanks for the great question! I hope this fixes it.

Thanks for the clarification. Editing tool.json does the trick.

Hi @amlankar

I am having the same issue here:

My tool.json is (following your ggnn.json):

{
"mode": "tool",
"max_poly_len": 71,
"temperature": 0.3,
"use_evaluator": true,
"use_bn_lstm": true,
"use_ggnn": true,
"ggnn_n_steps":5,
"ggnn_state_dim":1024,
"ggnn_grid_size": 112,
"ggnn_output_dim":15,
"use_separate_encoder": false,
"return_attention": false,
"dataset": {
"train_val" : {
"batch_size": 1,
"max_poly_len": 71,
"grid_side": 28,
"img_side": 224,
"flip": false,
"random_start": false,
"random_context": [0.15,0.15]
}
}
}
and I can't load ../models/ggnn_epoch8_step21000.pth, and got same error as xinario .

However, when I load ../models/ggnn_epoch5_step14000.pth instead, my results is bad:

Screen Shot 2019-12-20 at 10 07 30 AM

I tried different combinations of weights and json settings, but could never got a good result.

Could you please help with me with the correct json settings and weights?

Thanks,

ezoic increase your site revenue