Rendering color and depth images for ShapeNet models.

Color & Depth Renderer for ShapeNet

This library includes the tools for rendering multi-view color and depth images of ShapeNet models. Physically based rendering (PBR) is featured based on blender2.79.

Outputs

1. Color image (20 views)

2. Depth image (20 views)

3. Point cloud and normals (Back-projected from color & depth images)

4. Watertight meshes (fused from depth maps)

Install

1. We recommend to install this repository with conda.

   conda env create -f environment.yml
   conda activate renderer
   

2. Install Pyfusion by

   cd ./external/pyfusion
   mkdir build
   cd ./build
   cmake ..
   make
   

   Afterwards, compile the Cython code in ./external/pyfusion by

   cd ./external/pyfusion
   python setup.py build_ext --inplace
   

3. Download & Extract blender2.79b, and specify the path of your blender executable file at ./setting.py by

   g_blender_excutable_path = '../../blender-2.79b-linux-glibc219-x86_64/blender'
   

Usage

1. Normalize ShapeNet models to a unit cube by

   python normalize_shape.py
   

   The ShapeNetCore.v2 dataset is put in ./datasets/ShapeNetCore.v2. Here we only present some samples in this repository.

2. Generate multiple camera viewpoints for rendering by

   python create_viewpoints.py
   

   The camera extrinsic parameters will be saved at ./view_points.txt, or you can customize it in this script.

3. Run renderer to render color and depth images by

   python run_render.py
   

   The rendered images are saved in ./datasets/ShapeNetRenderings. The camera intrinsic and extrinsic parameters are saved in ./datasets/camera_settings. You can change the rendering configurations at ./settings.py, e.g. image sizes and resolution.

4. The back-projected point cloud and corresponding normals can be visualized by

   python visualization/draw_pc_from_depth.py
   

5. Watertight meshes can be obtained by

   python depth_fusion.py
   

   The reconstructed meshes are saved in ./datasets/ShapeNetCore.v2_watertight

Citation

This library is used for data preprocessing in our work SK-PCN. If you find it helpful, please consider citing

@inproceedings{NEURIPS2020_ba036d22,
 author = {Nie, Yinyu and Lin, Yiqun and Han, Xiaoguang and Guo, Shihui and Chang, Jian and Cui, Shuguang and Zhang, Jian.J},
 booktitle = {Advances in Neural Information Processing Systems},
 editor = {H. Larochelle and M. Ranzato and R. Hadsell and M. F. Balcan and H. Lin},
 pages = {16119--16130},
 publisher = {Curran Associates, Inc.},
 title = {Skeleton-bridged Point Completion: From Global Inference to Local Adjustment},
 url = {https://proceedings.neurips.cc/paper/2020/file/ba036d228858d76fb89189853a5503bd-Paper.pdf},
 volume = {33},
 year = {2020}
}

License

This repository is relased under the MIT License.