Giters

WXinlong / ASIS

Associatively Segmenting Instances and Semantics in Point Clouds, CVPR 2019

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

reproduce results on Area5 for instance segmentation

FrankCAN opened this issue · comments

commented

Hi,
I retrained your model without any change, and test it on Area5, but the performance result is quite low compared with yours, I just wonder if there are some information I don't know.

My environment is tensorflow 1.6 and python 3.5 and my result is as below:
Instance Segmentation mMUCov: 0.3599536949433257
Instance Segmentation mMWCov: 0.38854278223099514
Instance Segmentation mPrecision: 0.40252245593357444
Instance Segmentation mRecall: 0.31376601611130606

Semantic Segmentation oAcc: 0.8630127024386128
Semantic Segmentation mAcc: 0.5824638078371082
Semantic Segmentation mIoU: 0.5037184939579153

Thanks a lot for your help.

commented

Hi, your results are weird. Can you help me check if the dataset including the annotations are collected and organized correctly. Or could you try again in enrironment with python2.7 and tensorflow1.3.

commented

Hi Xinlong,
Thanks for your response.

I can generate *.npy and *h5 files successfully by following your guidance, so do it mean that the annotations should be no problem and organized correctly?

The difference is just that I only trained data/train_hdf5_file_list_woArea5.txt file instead, but I think train_hdf5_file_list_woArea1~5.txt are totally separate, have no any connection, and they only use for CV test, am I right? Should I have to train train_hdf5_file_list_woArea1 to train_hdf5_file_list_woArea5 completely?

Thanks a lot for your help.

Best Regards
Frank

commented

train_hdf5_file_list_woArea5.txt means the filelist of area 1,2,3,4,6.
The code was tested with Python 2.7. I suggest you use an environment with Python2.7 and TF1.3. Please refer to #12 for more help.

commented

Hi Xinlong,
Thanks for your help, I got expected results, and even a little bit higher than your paper, thanks.

commented

It's weird that I still got unmatched Area5 instance segmentation results when using the pretrained model. I used an environment with python 2.7 and tensorflow 1.3.0 under Ubuntu 16.04. The semantic segmentation results are all good. Any idea why?
Here below are the results that I got:
Instance Segmentation mMUCov: 0.388768637499
Instance Segmentation mMWCov: 0.41931708045
Instance Segmentation mPrecision: 0.49950009677
Instance Segmentation mRecall: 0.357493869746

Semantic Segmentation oAcc: 0.869736688706
Semantic Segmentation mAcc: 0.609961164943
Semantic Segmentation mIoU: 0.534575881062