Self-supervised Outdoor Scene Relighting

This is the implementation of the paper "Self-supervised Outdoor Scene Relighting". The model is implemented in tensorflow.

If you use our code, please cite the following paper:

@inproceedings{yu20relightNet,
    title={Self-supervised Outdoor Scene Relighting},
    author={Yu, Ye and Meka, Abhimitra and Elgharib, Mohamed and Seidel, Hans-Peter and Theobalt, Christian and Smith, William A. P.},
    booktitle={Proc. of the European Conference on Computer Vision (ECCV)},
    year={2020}
}

Evaluation

Dependencies

To run our evaluation code, please create your environment based on following dependencies:

tensorflow 1.12.0
python 3.6
skimage
cv2
PIL
numpy

Pretrained model

Relighting model

• Download our pretrained relighting model from: Link
• Make sure the model files are placed in a folder named "relight_model"

Sky model

• Download our pretrained sky generation model from: Link
• Make sure the model files are placed in a folder named "model_skyGen_net"

Test on time-lapse image pair

The following code performs relighting on a pair of demo time-lapse images, which are placed in "timeLapse_imgs" folders. It uses our pre-trained model to relight "2.jpg" by borrowing the illumination from "1.jpg". The provided mask map can be generated by PSPNet, which you can find on https://github.com/hszhao/PSPNet. The relighting image is saved as timeLapse_rendering.png.

python3 test_timelpase.py

Relight image by time-lapse video

We demonstrate our performance by relighting an input image under illuminations captured by a time-lapse video. Inputs are stored in "timeLapse_illu". Again the mask file can be generated by PSPNet.

python3 test_timelpase_illu.py