For academic use only.

Stereo Hybrid Event-Frame (SHEF) Cameras for 3D Perception

Ziwei Wang, Liyuan Pan, Yonhon Ng, Zheyu Zhuang and Robert Mahony

The paper was accepted by the 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021) in Prague, Czech Republic.

Publications

PDF

@inproceedings{wang2021stereo,
  title={Stereo Hybrid Event-Frame (SHEF) Cameras for 3D Perception},
  author={Wang, Ziwei and Pan, Liyuan and Ng, Yonhon and Zhuang, Zheyu and Mahony, Robert},
  booktitle={2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  year={2021},
  organization={IEEE}
}

IROS-video

https://www.youtube.com/watch?v=Azu7rJSPGNc

Data

Events Image Raw Data:

Three scenarios: picnic, complex boxes, and simple boxes. Each scenario includes at least 6 sequences with different camera speeds and lighting conditions.

Notes:

Events are decompressed from .raw to .dat format. To convert raw data to .dat or .csv format, we used the Prophesee tools in Prophesee_tools You can also install the last Prophesee software version follow the instructions on the website If you need, you can find all tools in /usr/share/prophesee_driver/samples or /usr/share/metavision/sdk/driver/samples, depending on what version you are using.

Processed stereo event-frame dataset

Parameters for each sequence

Stereo hybrid event-frame calibration data

Point cloud

UR5 robot arm pose

Notes

1. If you have any questions regarding this code and the corresponding results, please contact [email protected]