AronCao49 / MoPA

[ICRA24] MoPA: Multi-Modal Prior Aided Domain Adaptation for 3D Semantic Segmentation

MoPA

MoPA: Multi-Modal Prior Aided Domain Adaptation for 3D Semantic Segmentation
Haozhi Cao^1,*, Yuecong Xu², Jianfei Yang², Pengyu Yin¹, Shenghai Yuan¹, Lihua Xie¹

[Paper] | [Video]

📜 About MoPA (ICRA'24)

MoPA is a MM-UDA method that aims to alleviate the imbalanced class-wise performance on Rare Objects (ROs) and the lack of 2D dense supervision signals through Valid Ground-based Insertion (VGI) and Segment Anything Mask consistency (SAM consistency). An overall structure is as follows.

Specifically, VGI insert more ROs from the wild with ground truth to guide the recognition of ROs during UDA process without introducing artificial artifacts, while SAM consistency leverage image masks from Segment Anything Model to encourage mask-wise prediction consistency.

👀 Updates

• [2024.03] We are now refactoring our code and evaluating its feasibility. Code will be available shortly.
• [2024.01] Our paper is accepted by ICRA 2024! Check our paper on arxiv here.

✍️ TODO List

• Initial release. 🚀
• Add installation and prerequisite details.
• Add data preparation details.
• Add training details.
• Add evaluation details.

👏 Acknowledgement

We greatly appreciate the contributions of the following public repos:

• torchsparse
• SPVNAS
• SalsaNext
• Patchwork++
• xMUDA

📝 Citation

@article{cao2023mopa,
  title={Mopa: Multi-modal prior aided domain adaptation for 3d semantic segmentation},
  author={Cao, Haozhi and Xu, Yuecong and Yang, Jianfei and Yin, Pengyu and Yuan, Shenghai and Xie, Lihua},
  journal={arXiv preprint arXiv:2309.11839},
  year={2023}
}