The official code and benchmark for our paper: Explicit Interaction for Fusion-Based Place Recognition.
This work has been accepted by IROS 2024 π
Jingyi Xu, Junyi Ma, Qi Wu, Zijie Zhou, Yue Wang, Xieyuanli Chen, Wenxian Yu, Ling Pei*.
We follow the installation instructions of our codebase LCPR, which are also posted here.
- Create a conda virtual environment and activate it
git clone git@github.com:BIT-XJY/EINet.git
cd EINet
conda create -n EINet python=3.8
conda activate EINet
- Install other dependencies
pip install -r requirements.txt
- Please download the offical nuScenes dataset, and link it to the data folder.
- nuscenes_occ_infos_train.pkl, and nuscenes_occ_infos_val.pkl are also provided by the previous work. We also use it to train the EINet.
Note that the download data structure should be like:
nuscenes
ββ raw_data
β ββ maps
β β ββ ...
β ββ samples
β β ββ CAM_BACK
β β ββ CAM_BACK_LEFT
β β ββ CAM_BACK_RIGHT
β β ββ CAM_FRONT
β β ββ CAM_FRONT_LEFT
β β ββ CAM_FRONT_RIGHT
β β ββ LIDAR_TOP
β β ββ RADAR_BACK_LEFT
β β ββ RADAR_BACK_RIGHT
β β ββ RADAR_FRONT
β β ββ RADAR_FRONT_LEFT
β β ββ RADAR_FRONT_RIGHT
β ββ sweeps
β β ββ CAM_BACK
β β ββ CAM_BACK_LEFT
β β ββ CAM_BACK_RIGHT
β β ββ CAM_FRONT
β β ββ CAM_FRONT_LEFT
β β ββ CAM_FRONT_RIGHT
β β ββ LIDAR_TOP
β β ββ RADAR_BACK_LEFT
β β ββ RADAR_BACK_RIGHT
β β ββ RADAR_FRONT
β β ββ RADAR_FRONT_LEFT
β β ββ RADAR_FRONT_RIGHT
β ββ v1.0-test
β β ββ attribute.json
β β ββ calibrated_sensor.json
β β ββ ...
β ββ v1.0-traninval
β β ββ attribute.json
β β ββ calibrated_sensor.json
β β ββ ...
We propose the NUSC-PR benchmark to split nuScenes datasets with self-supervised and supervised learning schemes.
- Extract basic information from nuScenes datasets, and split query and database for NUSC-PR.
cd NUSC-PR
cd self_supervised
python generate_basic_infos.py
python split_dataset.py
cd ..
- The data structure with a self-supervised learning scheme should be like:
self_supervised_data
ββ generate_basic_infos
β ββ nuscenes_infos-bs.pkl
β ββ nuscenes_infos-shv.pkl
β ββ nuscenes_infos-son.pkl
β ββ nuscenes_infos-sq.pkl
β ββ nuscenes_infos.pkl
ββ split_dataset
β ββ all_train_query_pos_neg_index_in_infos.pkl
β ββ bs_db_index_in_infos.npy
β ββ bs_test_query_gt_index_in_infos.pkl
β ββ bs_train_query_pos_neg_index_in_infos.pkl
β ββ shv_db_index_in_infos.npy
β ββ shv_test_query_gt_index_in_infos.pkl
β ββ shv_train_query_pos_neg_index_in_infos.pkl
β ββ son_db_index_in_infos.npy
β ββ son_test_query_gt_index_in_infos.pkl
β ββ son_train_query_pos_neg_index_in_infos.pkl
β ββ sq_db_index_in_infos.npy
β ββ sq_test_query_gt_index_in_infos.pkl
β ββ sq_train_query_pos_neg_index_in_infos.pkl
- Extract basic information from nuScenes datasets, and split query and database for NUSC-PR.
cd supervised
python generate_basic_infos.py
python split_dataset.py
python select_pos_neg_samples_by_dis.py
python generate_selected_indicies.py
cd ..
cd ..
- The data structure with a supervised learning scheme should be like:
supervised_data
ββ generate_basic_infos
β ββ nuscenes_infos-bs.pkl
β ββ nuscenes_infos-shv.pkl
β ββ nuscenes_infos-son.pkl
β ββ nuscenes_infos-sq.pkl
β ββ nuscenes_infos.pkl
ββ generate_selected_indicies
β ββ bs_db_index_in_infos.npy
β ββ bs_test_query_gt_index_in_infos.pkl
β ββ bs_train_query_pos_neg_index_in_infos.pkl
β ββ shv_db_index_in_infos.npy
β ββ shv_test_query_gt_index_in_infos.pkl
β ββ shv_train_query_pos_neg_index_in_infos.pkl
β ββ son_db_index_in_infos.npy
β ββ son_test_query_gt_index_in_infos.pkl
β ββ son_train_query_pos_neg_index_in_infos.pkl
β ββ sq_db_index_in_infos.npy
β ββ sq_test_query_gt_index_in_infos.pkl
β ββ sq_train_query_pos_neg_index_in_infos.pkl
ββ select_pos_neg_samples_by_dis
β ββ bs_test_query_gt_tokens.pkl
β ββ bs_train_query_pos_neg_tokens.pkl
β ββ shv_test_query_gt_tokens.pkl
β ββ shv_train_query_pos_neg_tokens.pkl
β ββ son_test_query_gt_tokens.pkl
β ββ son_train_query_pos_neg_tokens.pkl
β ββ sq_test_query_gt_tokens.pkl
β ββ sq_train_query_pos_neg_tokens.pkl
ββ split_dataset
β ββ bs_db_sample_token.npy
β ββ bs_db.npy
β ββ bs_sample_token.npy
β ββ bs_test_query_sample_token.npy
β ββ bs_test_query.npy
β ββ bs_train_query_sample_token.npy
β ββ bs_train_query.npy
β ββ bs_val_query_sample_token.npy
β ββ bs_val_query.npy
β ββ shv_db_sample_token.npy
β ββ shv_db.npy
β ββ shv_sample_token.npy
β ββ shv_test_query_sample_token.npy
β ββ shv_test_query.npy
β ββ shv_train_query_sample_token.npy
β ββ shv_train_query.npy
β ββ shv_val_query_sample_token.npy
β ββ shv_val_query.npy
β ββ son_db_sample_token.npy
β ββ son_db.npy
β ββ son_sample_token.npy
β ββ son_test_query_sample_token.npy
β ββ son_test_query.npy
β ββ son_train_query_sample_token.npy
β ββ son_train_query.npy
β ββ son_val_query_sample_token.npy
β ββ son_val_query.npy
β ββ sq_db_sample_token.npy
β ββ sq_db.npy
β ββ sq_sample_token.npy
β ββ sq_test_query_sample_token.npy
β ββ sq_test_query.npy
β ββ sq_train_query_sample_token.npy
β ββ sq_train_query.npy
β ββ sq_val_query_sample_token.npy
β ββ sq_val_query.npy
- Release the paper
- Release the benchmark NUSC-PR code for EINet
- Release the source code for EINet
- Release our pretrained baseline model
We thank the fantastic works LCPR, ManyDepth, and AutoPlace for their pioneer code release, which provide codebase for this work.