zhang-tao-whu / MapTR

MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction

MapTR

Structured Modeling and Learning for Online Vectorized HD Map Construction

Bencheng Liao^1,2,3 *, Shaoyu Chen^1,3 *, Xinggang Wang^{1 📧}, Tianheng Cheng^1,3, Qian Zhang³, Wenyu Liu¹, Chang Huang³

¹ School of EIC, HUST, ² Institute of Artificial Intelligence, HUST, ³ Horizon Robotics

(*) equal contribution, (^📧) corresponding author.

ArXiv Preprint (arXiv 2208.14437)

News

• 11 Nov, 2022: We release an initial version of MapTR.
• 31 Aug, 2022: We released our paper on Arxiv. Code/Models are coming soon. Please stay tuned! ☕️

Introduction

MapTR is a simple, fast and strong online vectorized HD map construction framework.

We present MapTR, a structured end-to-end framework for efficient online vectorized HD map construction. We propose a unified permutation-based modeling approach, ie, modeling map element as a point set with a group of equivalent permutations, which avoids the definition ambiguity of map element and eases learning. We adopt a hierarchical query embedding scheme to flexibly encode structured map information and perform hierarchical bipartite matching for map element learning. MapTR achieves the best performance and efficiency among existing vectorized map construction approaches on nuScenes dataset. In particular, MapTR-nano runs at real-time inference speed ( $25.1$ FPS ) on RTX 3090, $8\times$ faster than the existing state-of-the-art camera-based method while achieving $3.3$ higher mAP. MapTR-tiny significantly outperforms the existing state-of-the-art multi-modality method by $13.5$ mAP while being faster. Qualitative results show that MapTR maintains stable and robust map construction quality in complex and various driving scenes. MapTR is of great application value in autonomous driving.

Models

Results from the paper

Method	Backbone	Lr Schd	mAP	FPS	memroy
MapTR-nano	R18	110ep	44.2	25.1	11907M (bs 24)
MapTR-tiny	R50	24ep	50.3	11.2	10287M (bs 4)
MapTR-tiny	R50	110ep	58.7	11.2	10287M (bs 4)

Notes:

• FPS is measured on NVIDIA RTX3090 GPU with batch size of 1 (containing 6 view images).
• All the experiments are performed on 8 NVIDIA GeForce RTX 3090 GPUs.

Results from this repo, we refactor the code and rerun the experiments, the results are slightly different.

Method	Backbone	Lr Schd	mAP	FPS	memroy	Config	Download
MapTR-nano	R18	110ep	[coming soon]	25.1	11907M (bs 24)	config	[coming soon]
MapTR-tiny	R50	24ep	50.0	11.2	10287M (bs 4)	config	model / log
MapTR-tiny	R50	110ep	59.3	11.2	10287M (bs 4)	config	model / log

Qualitative results on nuScenes val set

MapTR maintains stable and robust map construction quality in various driving scenes.

Sunny&Cloudy

sunny.cloudy_short.mp4

Rainy

rainy_short.mp4

Night

night_short.mp4

Getting Started

• Installation
• Prepare Dataset
• Train and Eval

Catalog

• multi-modal checkpoints
• multi-modal code
• argoverse2 dataset
• Nuscenes dataset
• MapTR checkpoints
• MapTR code
• Initialization

Acknowledgements

MapTR is based on mmdetection3d. It is also greatly inspired by the following outstanding contributions to the open-source community: BEVFusion, BEVFormer, HDMapNet, GKT, VectorMapNet.

Citation

If you find MapTR is useful in your research or applications, please consider giving us a star 🌟 and citing it by the following BibTeX entry.

@article{MapTR,
  title={MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction},
  author={Liao, Bencheng and Chen, Shaoyu and Wang, Xinggang and Cheng, Tianheng, and Zhang, Qian and Liu, Wenyu and Huang, Chang},
  journal={arXiv preprint arXiv:2208.14437},
  year={2022}
}