I set n_input_scans to 8 and retrained salsanext. However, the score on the test set was only 0.58

[yangpan22223]

Hello, I set n_input_scans to 8 and retrained salsanext. However, the score on the test set was only 0.58. What caused this? Can you provide your training parameter settings or log for reference?

[Xieyuanli Chen]

@yangpan22223, thanks for using our code.

Several things you could check:

1. The final result reported in the paper was N=8 and enhanced with semantics. You may check the results after fusing the semantics and a simple way is given here
2. During inferring whether KNN is enabled in the config file.
3. Whether you are using the latest version 1.1, where shuffle and transform are now set to default as True, which will influence the performance.
4. Which graphic and batch_size you are using. Different batch_size will also influence the performance.

All the training parameter settings and logs have already been provided in the collection of downloads.

I hope you have fun with our LiDAR-MOS benchmark and code ;-)

[yangpan22223]

Thanks for the answer.
I have checked my setting:

1. My result was N=8 and enhanced with semantics
2. KNN is enabled when inferring.
3. I used the latest version 1.1
4. I use pytorch framework for training and batch_size=24.
   I have a question, are there any pre-trained models that need to be loaded when training salsanext? My salsanext is trained from scratch.

[Xieyuanli Chen]

There is no pre-trained step. To verify the settings, could you please train it again with 150 epochs?
The randomized initial weights of the network may also influence the results a little bit, but there are multiple users already reproduced similar results as reported in the paper.
What is your account on the benchmark? I could also help you to check the results.

[Xieyuanli Chen]

Since there is no update for a while, I would like to close this issue. Feel free to reopen it, if needed.