very low accuracy rates for ADE20K dataset??

Question

very low accuracy rates for ADE20K dataset??

xinyuh opened this issue 7 years ago · comments

Here is the results for two images, ADE_val_00001977.png and ADE_val_00000041.png. Any suggestions?

c:\ademxapp>python issegm/voc.py --data-root data\ade20k --output output --phase val --weight models\ade20k_rna-a1_cls150_s8_ep-0001.params --split val --test-scales 504 --test-flipping --test-steps 2 --gpus 0
2017-01-13 10:47:14,700 Host start with arguments Namespace(backward_do_mirror=False, base_lr=None, cache_images=None, check_start=1, check_step=4, data_root='data\ade20k', dataset=None, debug=False, from_epoch=1, from_model='models\ade20k_rna-a1_cls150_s8_ep', gpus='0', kvstore='device', log_file='ade20k_rna-a1_cls150_s8_ep-0001.log', model='ade20k_rna-a1_cls150_s8', output='output', phase='val', prefetch_threads=1, prefetcher='thread', save_predictions=False, save_results=True, split='val', stop_epoch=None, test_flipping=True, test_scales='504', test_steps=2, to_epoch=None, weights='models\ade20k_rna-a1_cls150_s8_ep-0001.params')
2017-01-13 10:47:14,716 Host and model specs {'classes': 150, 'net_type': 'rna', 'net_name': 'a1', 'feat_stride': 8, 'dataset': 'ade20k'}
Level 0
[(64L, 3L, 224L, 224L)]
Level 1
[(64L, 64L, 224L, 224L)]
Level 2
First block on level 2, stride: 2, dilate: 1
[(64L, 128L, 112L, 112L)]
Level 3
First block on level 3, stride: 2, dilate: 1
[(64L, 256L, 56L, 56L)]
Level 4
First block on level 4, stride: 2, dilate: 1
[(64L, 512L, 28L, 28L)]
Level 5
First block on level 5, stride: 2, dilate: 1
[(64L, 1024L, 28L, 28L)]
Level 6
First block on level 6, stride: 2, dilate: 2
[(64L, 2048L, 28L, 28L)]
Level 7
[(64L, 4096L, 28L, 28L)]
2017-01-13 10:47:51,694 Host Done 1/2 with speed: 0.04/s
2017-01-13 10:47:51,694 Host pixel acc: 0.00%, mean acc: 0.00%, mean iou: 0.00%
2017-01-13 10:47:51,694 Host
[ 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00]
2017-01-13 10:47:51,694 Host
[ 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00]
2017-01-13 10:48:04,592 Host Done 2/2 with speed: 0.05/s
2017-01-13 10:48:04,592 Host pixel acc: 25.81%, mean acc: 0.67%, mean iou: 0.17%
2017-01-13 10:48:04,592 Host
[100.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00]
2017-01-13 10:48:04,592 Host
[25.81 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00]
2017-01-13 10:48:04,608 Host Done in 40.04 s.

Zifeng Wu · Answer 1 · Sat Jan 14 2017 01:19:28 GMT+0800 (China Standard Time)

@xinyuh No idea. Have you made any change to the code?

xinyuh · Answer 2 · Sat Jan 14 2017 01:23:02 GMT+0800 (China Standard Time)

no any changes. just downloaded the model and source codes and run the command

xinyuh · Answer 3 · Sat Jan 14 2017 01:26:05 GMT+0800 (China Standard Time)

one possible reason is that the model parameters are not loaded correctly. Any idea to check loaded parameters?

xinyuh · Answer 4 · Sat Jan 14 2017 01:28:00 GMT+0800 (China Standard Time)

I just checked the image before running the mod.forward(...). The input image has no problem.

Zifeng Wu · Answer 5 · Sat Jan 14 2017 01:31:29 GMT+0800 (China Standard Time)

@xinyuh Check the md5sum of your downloaded model.
1faf29850bfa194678f0b8e1cbbffa98

xinyuh · Answer 6 · Sat Jan 14 2017 01:39:23 GMT+0800 (China Standard Time)

The two checksum values are matched

Zifeng Wu · Answer 7 · Sat Jan 14 2017 02:18:16 GMT+0800 (China Standard Time)

Are you using the MIT Scene Parsing dataset for ILSVRC 2016 (not the original ADE20K dataset), which is released on this site http://sceneparsing.csail.mit.edu?

…

On 01/14/2017 04:09 AM, xinyuh wrote: The two checksum values are matched — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#9 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADm2M9992BKGgL5vGOU9KsKvGJZn6xCrks5rR7bLgaJpZM4LjB82>.

-- Dr Zifeng Wu Australian Centre for Visual Technologies School of Computer Science University of Adelaide, Adelaide, SA 5005, Australia

xinyuh · Answer 8 · Sat Jan 14 2017 03:43:35 GMT+0800 (China Standard Time)

yes, that is the link I used. "Data: train/val (922MB)"

xinyuh · Answer 9 · Sat Jan 14 2017 03:53:13 GMT+0800 (China Standard Time)

here is another test of two images (ILSVRC2012_val_00000005.JPEG and ILSVRC2012_val/ILSVRC2012_val_00000006.JPEG) based on the models for ILSVRC 2012 classification. It seems results are good. I guess the models or codes for ADE20K may have problems.

c:\ademxapp>python iclass/ilsvrc.py --data-root data\ilsvrc12 --output output --batch-images 2 --phase val --weight models\ilsvrc-cls_rna-a1_cls1000_ep-0001.params --split val --test-scales 320 --gpus 0 --no-choose-interp-method
2017-01-13 14:47:58,598 Host Run with arguments: Namespace(backward_do_mirror=False, base_lr=None, batch_images=2, choose_interpolation_method=False, crop_size=None, data_root='data\ilsvrc12', dataset=None, debug=False, from_epoch=0, gpus='0', kvstore='device', log_file='ilsvrc-cls_rna-a1_cls1000_ep-0001.log', lr_steps=None, lr_type=None, model='ilsvrc-cls_rna-a1_cls1000', output='output', phase='val', pool_top_infer_style=None, prefetch_threads=1, prefetcher='thread', split='val', stop_epoch=None, test_3crops=False, test_flipping=False, test_scales='320', to_epoch=None, weights='models\ilsvrc-cls_rna-a1_cls1000_ep-0001.params')
2017-01-13 14:47:58,601 Host and model specs: {'batch_images': 2, 'classes': 1000, 'net_type': 'rna', 'net_name': 'a1', 'feat_stride': 32, 'crop_size': 320, 'lr_params': {'base': 0.1, 'type': 'fixed', 'args': None}, 'dataset': 'ilsvrc-cls'}
Level 0
[(64L, 3L, 224L, 224L)]
Level 1
[(64L, 64L, 224L, 224L)]
Level 2
First block on level 2, stride: 2, dilate: 1
[(64L, 128L, 112L, 112L)]
Level 3
First block on level 3, stride: 2, dilate: 1
[(64L, 256L, 56L, 56L)]
Level 4
First block on level 4, stride: 2, dilate: 1
[(64L, 512L, 28L, 28L)]
Level 5
First block on level 5, stride: 2, dilate: 1
[(64L, 1024L, 14L, 14L)]
Level 6
First block on level 6, stride: 2, dilate: 1
[(64L, 2048L, 7L, 7L)]
Level 7
[(64L, 4096L, 7L, 7L)]
Waited for 0.0 seconds
(2L, 1000L)
2017-01-13 14:48:08,240 Host Crop size 320, done 1/1 at speed: 0.50/s, Top1 0.0000%
2017-01-13 14:48:08,240 Host Done crop size 320 in 4.0040s.
2017-01-13 14:48:08,267 Host Done testing crop_size [320]
2017-01-13 14:48:08,269 Host Top1 0.0000%, Top5 0.0000%

zuoxin.xiahou · Answer 10 · Sun Jan 22 2017 09:59:28 GMT+0800 (China Standard Time)

Not get IoU as the paper too,

2017-01-12 17:13:48,914 Host Done 1211/2000 with speed: 0.14/s
2017-01-12 17:13:48,914 Host pixel acc: 80.91%, mean acc: 53.63%, mean iou: 43.45%
2017-01-12 17:13:48,916 Host 
[89.13 90.79 96.97 92.09 88.10 89.96 89.40 93.84 72.54 85.78 64.34 71.98
 91.42 50.39 51.66 70.88 68.93 59.27 79.49 65.94 93.08 66.44 81.88 78.39
 60.07 53.94 90.06 55.35 51.99 31.98 52.17 77.18 51.92 69.19 68.75 57.84
 70.34 71.09 44.50 58.00 20.52 27.25 53.66 43.65 66.91 30.10 68.89 83.49
 48.52 83.06 67.36 60.36 12.76 38.98 90.17 30.39 95.54 66.77 76.11 33.85
 26.80 49.28 48.50 36.97 77.09 92.37 46.67 60.33  8.84 45.29 74.47 80.28
 63.33 32.64 80.91 55.31 79.33 14.37 26.42 27.53 94.89 61.33 53.52 17.31
 41.93 69.03 12.02 38.61 22.29 78.53 94.78 12.77 52.62 29.82  0.57  8.40
 20.07 53.04 31.97 64.17 27.54 11.41 29.07 81.82 23.37 86.00 17.25 65.89
 40.50 48.90 53.53 74.11 25.51 87.45  0.00  7.25 70.62 96.90 48.92 63.57
 47.89 12.33 42.48 18.90 75.75 39.63 53.14 71.38  3.64 61.14 79.14  1.77
 57.55 70.77 45.15 57.78 49.66 10.34 58.71 60.86 40.26 24.45 53.56 37.48
 58.44  4.41 43.71 12.41 45.24 17.73]
2017-01-12 17:13:48,917 Host 
[73.70 79.89 93.31 78.06 71.39 79.92 81.00 85.78 55.50 68.71 53.58 56.45
 76.89 37.12 38.64 52.83 55.32 46.69 67.58 51.38 85.02 53.58 68.48 62.52
 41.53 37.03 57.45 49.98 46.38 24.13 36.83 61.75 40.82 48.98 44.46 36.61
 58.48 67.81 33.10 44.63 16.67 21.94 45.86 32.82 47.39 21.89 43.00 68.36
 40.06 72.37 61.35 27.45  9.61 32.86 86.89 24.35 90.48 56.08 66.53 26.32
 15.90 45.37 26.92 34.48 61.23 83.81 35.40 41.80  7.98 38.56 55.58 73.04
 44.04 21.81 64.43 45.69 72.61 11.48 25.87 23.55 84.84 50.06 44.91 13.84
 31.64 57.13  9.55 30.65 22.03 69.81 78.32  6.46 39.47 21.36  0.44  6.35
 19.13 46.45 25.49 57.96 23.92  9.36 22.81 37.47 22.56 79.41 15.45 64.55
 28.27 27.19 40.78 65.96 23.29 57.44  0.00  6.63 64.38 89.96 39.44 44.53
 37.75 10.13 37.90 15.67 62.78 34.54 51.01 52.00  3.58 56.03 64.71  1.65
 46.29 64.30 36.80 36.43 35.90  5.42 42.95 50.06 33.49 17.40 37.17 27.28
 44.42  3.27 39.77 11.52 35.53 16.81]
2017-01-12 17:13:55,935 Host Done 1212/2000 with speed: 0.14/s
2017-01-12 17:13:55,935 Host pixel acc: 80.90%, mean acc: 53.62%, mean iou: 43.45%
2017-01-12 17:13:55,937 Host 
[89.12 90.79 96.97 92.10 88.10 89.98 89.40 93.84 72.48 85.78 64.34 71.98
 91.32 50.39 51.63 70.88 68.93 59.27 79.49 65.94 93.08 66.44 81.88 78.39
 60.07 53.94 90.06 55.35 51.99 31.98 52.17 77.18 51.92 69.19 68.75 57.84
 70.34 71.09 44.50 58.00 20.52 27.25 53.66 43.65 66.91 30.10 68.89 83.49
 48.52 83.06 67.36 59.32 12.76 38.98 90.17 30.39 95.54 66.77 76.11 33.85
 26.80 49.28 48.50 36.97 77.09 92.37 46.67 60.33  8.84 45.29 74.47 80.28
 63.33 32.64 80.91 55.31 79.33 14.37 26.42 27.53 94.89 61.33 53.52 17.31
 41.93 69.03 12.02 38.61 22.29 78.53 94.78 12.77 52.62 29.82  0.57  8.40
 20.07 53.04 31.97 64.17 27.54 11.41 29.07 81.82 23.37 86.00 17.25 65.89
 40.50 48.90 53.53 74.11 25.51 87.45  0.00  7.25 70.62 96.90 48.92 63.29
 47.89 12.33 42.48 18.90 75.75 39.63 53.14 71.38  3.64 61.14 79.14  1.77
 57.55 70.77 45.15 57.78 49.66 10.34 58.71 60.86 40.26 24.45 53.56 37.48
 58.44  4.41 43.71 12.41 45.24 17.73]
2017-01-12 17:13:55,939 Host 
[73.69 79.89 93.31 78.07 71.39 79.88 81.00 85.78 55.46 68.71 53.58 56.45
 76.81 37.12 38.63 52.83 55.32 46.69 67.58 51.38 85.02 53.58 68.48 62.52
 41.53 37.03 57.45 49.98 46.38 24.13 36.83 61.75 40.82 48.98 44.46 36.61
 58.48 67.81 33.10 44.63 16.67 21.94 45.86 32.82 47.39 21.89 43.00 68.36
 40.06 72.37 61.35 27.82  9.61 32.86 86.89 24.35 90.48 56.08 66.53 26.32
 15.90 45.37 26.92 34.48 61.23 83.81 35.40 41.80  7.98 38.56 55.58 73.04
 44.04 21.81 64.43 45.69 72.61 11.48 25.87 23.55 84.84 50.06 44.91 13.84
 31.64 57.13  9.55 30.65 22.03 69.81 78.32  6.46 39.47 21.36  0.44  6.35
 19.13 46.45 25.49 57.96 23.92  9.36 22.81 37.47 22.56 79.41 15.45 64.55
 28.27 27.19 40.78 65.96 23.29 57.44  0.00  6.63 64.38 89.96 39.44 44.40
 37.75 10.13 37.90 15.67 62.78 34.54 51.01 52.00  3.58 56.03 64.71  1.65
 46.29 64.30 36.80 36.43 35.90  5.42 42.95 50.06 33.49 17.40 37.17 27.28
 44.42  3.27 39.77 11.52 35.53 16.81]

IoU changes from low(2.0%) to high(about 43.%). The segmented image result not so smooth as paper.

And 0.14/s means 0.14 images per seconds. if thus, it seems the model may too slow for application.

Zifeng Wu · Answer 11 · Sun Jan 22 2017 10:23:13 GMT+0800 (China Standard Time)

Have you ever tested on the whole val set? In that case, there should be a line with `Done 2000/2000' (not `Done 1211/2000') in the log file. See the line below, which shows the mean IoU on the whole val set.

…

On 01/22/2017 12:29 PM, xiahouzuoxin wrote: Not get IoU as the paper too, |2017-01-12 17:13:48,914 Host Done 1211/2000 with speed: 0.14/s 2017-01-12 17:13:48,914 Host pixel acc: 80.91%, mean acc: 53.63%, mean iou: 43.45% 2017-01-12 17:13:48,916 Host [89.13 90.79 96.97 92.09 88.10 89.96 89.40 93.84 72.54 85.78 64.34 71.98 91.42 50.39 51.66 70.88 68.93 59.27 79.49 65.94 93.08 66.44 81.88 78.39 60.07 53.94 90.06 55.35 51.99 31.98 52.17 77.18 51.92 69.19 68.75 57.84 70.34 71.09 44.50 58.00 20.52 27.25 53.66 43.65 66.91 30.10 68.89 83.49 48.52 83.06 67.36 60.36 12.76 38.98 90.17 30.39 95.54 66.77 76.11 33.85 26.80 49.28 48.50 36.97 77.09 92.37 46.67 60.33 8.84 45.29 74.47 80.28 63.33 32.64 80.91 55.31 79.33 14.37 26.42 27.53 94.89 61.33 53.52 17.31 41.93 69.03 12.02 38.61 22.29 78.53 94.78 12.77 52.62 29.82 0.57 8.40 20.07 53.04 31.97 64.17 27.54 11.41 29.07 81.82 23.37 86.00 17.25 65.89 40.50 48.90 53.53 74.11 25.51 87.45 0.00 7.25 70.62 96.90 48.92 63.57 47.89 12.33 42.48 18.90 75.75 39.63 53.14 71.38 3.64 61.14 79.14 1.77 57.55 70.77 45.15 57.78 49.66 10.34 58.71 60.86 40.26 24.45 53.56 37.48 58.44 4.41 43.71 12.41 45.24 17.73] 2017-01-12 17:13:48,917 Host [73.70 79.89 93.31 78.06 71.39 79.92 81.00 85.78 55.50 68.71 53.58 56.45 76.89 37.12 38.64 52.83 55.32 46.69 67.58 51.38 85.02 53.58 68.48 62.52 41.53 37.03 57.45 49.98 46.38 24.13 36.83 61.75 40.82 48.98 44.46 36.61 58.48 67.81 33.10 44.63 16.67 21.94 45.86 32.82 47.39 21.89 43.00 68.36 40.06 72.37 61.35 27.45 9.61 32.86 86.89 24.35 90.48 56.08 66.53 26.32 15.90 45.37 26.92 34.48 61.23 83.81 35.40 41.80 7.98 38.56 55.58 73.04 44.04 21.81 64.43 45.69 72.61 11.48 25.87 23.55 84.84 50.06 44.91 13.84 31.64 57.13 9.55 30.65 22.03 69.81 78.32 6.46 39.47 21.36 0.44 6.35 19.13 46.45 25.49 57.96 23.92 9.36 22.81 37.47 22.56 79.41 15.45 64.55 28.27 27.19 40.78 65.96 23.29 57.44 0.00 6.63 64.38 89.96 39.44 44.53 37.75 10.13 37.90 15.67 62.78 34.54 51.01 52.00 3.58 56.03 64.71 1.65 46.29 64.30 36.80 36.43 35.90 5.42 42.95 50.06 33.49 17.40 37.17 27.28 44.42 3.27 39.77 11.52 35.53 16.81] 2017-01-12 17:13:55,935 Host Done 1212/2000 with speed: 0.14/s 2017-01-12 17:13:55,935 Host pixel acc: 80.90%, mean acc: 53.62%, mean iou: 43.45% 2017-01-12 17:13:55,937 Host [89.12 90.79 96.97 92.10 88.10 89.98 89.40 93.84 72.48 85.78 64.34 71.98 91.32 50.39 51.63 70.88 68.93 59.27 79.49 65.94 93.08 66.44 81.88 78.39 60.07 53.94 90.06 55.35 51.99 31.98 52.17 77.18 51.92 69.19 68.75 57.84 70.34 71.09 44.50 58.00 20.52 27.25 53.66 43.65 66.91 30.10 68.89 83.49 48.52 83.06 67.36 59.32 12.76 38.98 90.17 30.39 95.54 66.77 76.11 33.85 26.80 49.28 48.50 36.97 77.09 92.37 46.67 60.33 8.84 45.29 74.47 80.28 63.33 32.64 80.91 55.31 79.33 14.37 26.42 27.53 94.89 61.33 53.52 17.31 41.93 69.03 12.02 38.61 22.29 78.53 94.78 12.77 52.62 29.82 0.57 8.40 20.07 53.04 31.97 64.17 27.54 11.41 29.07 81.82 23.37 86.00 17.25 65.89 40.50 48.90 53.53 74.11 25.51 87.45 0.00 7.25 70.62 96.90 48.92 63.29 47.89 12.33 42.48 18.90 75.75 39.63 53.14 71.38 3.64 61.14 79.14 1.77 57.55 70.77 45.15 57.78 49.66 10.34 58.71 60.86 40.26 24.45 53.56 37.48 58.44 4.41 43.71 12.41 45.24 17.73] 2017-01-12 17:13:55,939 Host [73.69 79.89 93.31 78.07 71.39 79.88 81.00 85.78 55.46 68.71 53.58 56.45 76.81 37.12 38.63 52.83 55.32 46.69 67.58 51.38 85.02 53.58 68.48 62.52 41.53 37.03 57.45 49.98 46.38 24.13 36.83 61.75 40.82 48.98 44.46 36.61 58.48 67.81 33.10 44.63 16.67 21.94 45.86 32.82 47.39 21.89 43.00 68.36 40.06 72.37 61.35 27.82 9.61 32.86 86.89 24.35 90.48 56.08 66.53 26.32 15.90 45.37 26.92 34.48 61.23 83.81 35.40 41.80 7.98 38.56 55.58 73.04 44.04 21.81 64.43 45.69 72.61 11.48 25.87 23.55 84.84 50.06 44.91 13.84 31.64 57.13 9.55 30.65 22.03 69.81 78.32 6.46 39.47 21.36 0.44 6.35 19.13 46.45 25.49 57.96 23.92 9.36 22.81 37.47 22.56 79.41 15.45 64.55 28.27 27.19 40.78 65.96 23.29 57.44 0.00 6.63 64.38 89.96 39.44 44.40 37.75 10.13 37.90 15.67 62.78 34.54 51.01 52.00 3.58 56.03 64.71 1.65 46.29 64.30 36.80 36.43 35.90 5.42 42.95 50.06 33.49 17.40 37.17 27.28 44.42 3.27 39.77 11.52 35.53 16.81] | IoU changes from low(2.0%) to high(about 43.%). The segmented image result not so smooth as paper. And 0.14/s means 0.14 images per seconds. if thus, it seems the model may too slow for application. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#9 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADm2MybtHVC4RBNRcA3baU7mEDHZpEyhks5rUrgAgaJpZM4LjB82>.

-- Dr Zifeng Wu Australian Centre for Visual Technologies School of Computer Science University of Adelaide, Adelaide, SA 5005, Australia

zuoxin.xiahou · Answer 12 · Mon Jan 23 2017 16:46:07 GMT+0800 (China Standard Time)

Yes, I have done it,

2017-01-23 00:09:03,391 Host Done 2000/2000 with speed: 0.13/s
59996 2017-01-23 00:09:03,391 Host pixel acc: 80.56%, mean acc: 53.51%, mean iou: 43.35%
59997 2017-01-23 00:09:03,393 Host
59998 [88.35 90.89 97.22 91.05 88.17 89.73 89.76 92.96 75.16 84.42 67.58 76.52
59999  91.54 48.22 52.73 71.40 68.42 60.73 81.58 64.77 89.77 65.26 82.83 72.33
60000  55.09 60.22 72.67 65.47 59.10 39.82 55.46 74.25 46.77 62.72 71.11 62.56
60001  72.27 72.11 37.07 61.37 20.70 24.73 49.68 43.52 63.84 28.29 49.68 78.34
60002  51.60 81.32 72.32 63.67 34.56 31.58 91.14 55.87 95.45 62.98 69.03 31.08
60003  28.88 51.55 50.28 31.45 76.63 89.45 45.26 59.34  8.49 49.39 67.25 76.92
60004  63.46 40.11 67.50 56.36 71.44 24.92 25.10 26.44 90.89 63.24 56.49 23.17
60005  41.79 72.22 23.49 35.65 42.53 78.36 68.38 17.70 51.44 30.31  8.20  9.23
60006  18.01 50.91 32.45 43.55 33.36  8.13 51.54  8.21 19.92 68.98 16.30 66.59
60007  42.89 40.56 55.50 74.12 33.56 54.94 97.54 12.58 66.85 95.17 54.59 66.52
60008  58.16 11.10 25.76 25.41 38.68 45.25 62.01 70.37 54.60 69.70 80.92  5.25
60009  54.33 51.13 47.65 52.59 44.64 14.49 52.82 62.58 20.76 17.08 53.17 54.42
60010  44.54  4.39 47.19 13.36 29.94 29.38]
60011 2017-01-23 00:09:03,394 Host
60012 [73.07 79.78 93.75 76.35 70.95 80.62 80.83 83.85 57.84 68.10 56.20 61.65
60013  76.20 34.57 41.20 53.20 54.55 48.52 69.40 51.92 81.45 50.46 69.64 58.09
60014  37.62 47.88 50.36 58.77 53.30 30.79 36.27 57.64 35.13 44.55 46.74 45.08
60015  58.89 67.18 27.79 48.51 16.81 19.47 41.31 32.17 44.74 21.66 34.45 65.09
60016  41.85 67.57 66.15 35.63 24.72 25.53 70.69 40.21 90.61 50.32 60.27 22.76
60017  16.02 45.28 35.26 29.07 59.97 82.79 32.94 40.58  6.91 41.58 52.18 70.14
60018  47.73 24.67 59.24 42.11 61.66 21.62 23.04 22.69 83.01 52.08 48.08 18.65
60019  30.23 60.74 20.34 27.51 41.63 65.68 52.86 10.04 37.64 21.89  6.15  6.76
60020  17.19 41.53 25.24 38.45 26.77  6.62 40.22  7.57 19.12 55.13 14.72 64.49
60021  32.15 26.62 41.16 53.13 28.58 46.24 75.86 11.27 60.17 79.15 40.42 42.81
60022  48.38  9.34 24.02 21.47 35.34 39.10 59.01 51.34 46.58 62.74 61.09  4.85
60023  44.73 47.76 38.83 35.10 31.67  7.71 41.58 51.51 15.22 10.39 39.97 46.64
60024  34.71  2.90 43.00 12.09 23.53 27.37]
60025 2017-01-23 00:09:03,394 Host Done in 15368.70 s.

And it seems forward is very slow on my K40.

Zifeng Wu · Answer 13 · Mon Jan 23 2017 17:26:46 GMT+0800 (China Standard Time)

For the gap (~0.4%) between IoU scores, please note this Model A1 is different from the Model A2 as reported in our paper. The latter was tuned using the Places 365 data before tuned on ADE20K. Our Model A2 (not released) uses several pooling layers for down-sampling, so it can be incompatible with the newest MXNet, as explained in the README. For the speed, by specifying `--test-flipping --test-steps 2', we 8x the number of forward passes. Using the line_profiler, you will see it also costs a great part of time to interpolate the class-wise scores and compute IoU scores on CPUs. But anyway, we did not claim Model A to be a high efficiency model (See the comparison of speeds in Table 1 of our report).

…

On 01/23/2017 07:16 PM, xiahouzuoxin wrote: Yes, I have done it, |2017-01-23 00:09:03,391 Host Done 2000/2000 with speed: 0.13/s 59996 2017-01-23 00:09:03,391 Host pixel acc: 80.56%, mean acc: 53.51%, mean iou: 43.35% 59997 2017-01-23 00:09:03,393 Host 59998 [88.35 90.89 97.22 91.05 88.17 89.73 89.76 92.96 75.16 84.42 67.58 76.52 59999 91.54 48.22 52.73 71.40 68.42 60.73 81.58 64.77 89.77 65.26 82.83 72.33 60000 55.09 60.22 72.67 65.47 59.10 39.82 55.46 74.25 46.77 62.72 71.11 62.56 60001 72.27 72.11 37.07 61.37 20.70 24.73 49.68 43.52 63.84 28.29 49.68 78.34 60002 51.60 81.32 72.32 63.67 34.56 31.58 91.14 55.87 95.45 62.98 69.03 31.08 60003 28.88 51.55 50.28 31.45 76.63 89.45 45.26 59.34 8.49 49.39 67.25 76.92 60004 63.46 40.11 67.50 56.36 71.44 24.92 25.10 26.44 90.89 63.24 56.49 23.17 60005 41.79 72.22 23.49 35.65 42.53 78.36 68.38 17.70 51.44 30.31 8.20 9.23 60006 18.01 50.91 32.45 43.55 33.36 8.13 51.54 8.21 19.92 68.98 16.30 66.59 60007 42.89 40.56 55.50 74.12 33.56 54.94 97.54 12.58 66.85 95.17 54.59 66.52 60008 58.16 11.10 25.76 25.41 38.68 45.25 62.01 70.37 54.60 69.70 80.92 5.25 60009 54.33 51.13 47.65 52.59 44.64 14.49 52.82 62.58 20.76 17.08 53.17 54.42 60010 44.54 4.39 47.19 13.36 29.94 29.38] 60011 2017-01-23 00:09:03,394 Host 60012 [73.07 79.78 93.75 76.35 70.95 80.62 80.83 83.85 57.84 68.10 56.20 61.65 60013 76.20 34.57 41.20 53.20 54.55 48.52 69.40 51.92 81.45 50.46 69.64 58.09 60014 37.62 47.88 50.36 58.77 53.30 30.79 36.27 57.64 35.13 44.55 46.74 45.08 60015 58.89 67.18 27.79 48.51 16.81 19.47 41.31 32.17 44.74 21.66 34.45 65.09 60016 41.85 67.57 66.15 35.63 24.72 25.53 70.69 40.21 90.61 50.32 60.27 22.76 60017 16.02 45.28 35.26 29.07 59.97 82.79 32.94 40.58 6.91 41.58 52.18 70.14 60018 47.73 24.67 59.24 42.11 61.66 21.62 23.04 22.69 83.01 52.08 48.08 18.65 60019 30.23 60.74 20.34 27.51 41.63 65.68 52.86 10.04 37.64 21.89 6.15 6.76 60020 17.19 41.53 25.24 38.45 26.77 6.62 40.22 7.57 19.12 55.13 14.72 64.49 60021 32.15 26.62 41.16 53.13 28.58 46.24 75.86 11.27 60.17 79.15 40.42 42.81 60022 48.38 9.34 24.02 21.47 35.34 39.10 59.01 51.34 46.58 62.74 61.09 4.85 60023 44.73 47.76 38.83 35.10 31.67 7.71 41.58 51.51 15.22 10.39 39.97 46.64 60024 34.71 2.90 43.00 12.09 23.53 27.37] 60025 2017-01-23 00:09:03,394 Host Done in 15368.70 s. | And it seems forward is very slow on my K40. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#9 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADm2M5MnA6Wns6QopDoV4v7uJ_ZWjTQsks5rVGjPgaJpZM4LjB82>.

-- Dr Zifeng Wu Australian Centre for Visual Technologies School of Computer Science University of Adelaide, Adelaide, SA 5005, Australia