3D Object Detection with Pointformer

This repository contains the code for the paper 3D Object Detection with Pointformer (CVPR 2021) [arXiv]. This work is developed on the top of MMDetection3D toolbox and includes the models and results on SUN RGB-D and ScanNet datasets in the paper.

More models results on KITTI and nuScenes datasets will be released soon.

Installation and Usage

The code is developed with MMDetection3D v0.6.1 and works well with v0.14.0.

Dependencies

• NVIDIA GPU + CUDA 10.2
• Python 3.8 (Recommend to use Anaconda)
• PyTorch == 1.8.0
• mmcv-full == 1.3.7
• mmdet == 2.11.0
• mmsegmentation == 0.13.0

Installation

1. Install dependencies following their guidelines.
2. Clone and install mmdet3d in develop mode.

git clone https://github.com/open-mmlab/mmdetection3d.git
cd mmdetection3d
python setup.py develop

3. Add the files in this repo into the directories in mmdet3d.

Training and Testing

Download the pretrained weights from Google Drive or Tsinghua Cloud and put them in the checkpoints folder. Use votenet_ptr_sunrgbd-3d-10class as an example:

# Training
bash -x tools/dist_train.sh configs/pointformer/votenet_ptr_sunrgbd-3d-10class.py 8

# Testing 
bash tools/dist_test.sh configs/pointformer/votenet_ptr_sunrgbd-3d-10class.py checkpoints/votenet_ptr_sunrgbd-3d-10class.pth 8 --eval mAP

Results

SUN RGB-D

ScanNet

For more details of experimetns please refer to the paper.

Acknowledgement

This code is based on MMDetection3D.

Citation

If you find our work is useful in your research, please consider citing:

@InProceedings{Pan_2021_CVPR,
    author    = {Pan, Xuran and Xia, Zhuofan and Song, Shiji and Li, Li Erran and Huang, Gao},
    title     = {3D Object Detection With Pointformer},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2021},
    pages     = {7463-7472}
}

@misc{pan20203d,
  title={3D Object Detection with Pointformer}, 
  author={Xuran Pan and Zhuofan Xia and Shiji Song and Li Erran Li and Gao Huang},
  year={2020},
  eprint={2012.11409},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}