CLIPstyler

Official Pytorch implementation of "CLIPstyler:Image Style Transfer with a Single Text Condition"

Environment

Pytorch 1.7.1, Python 3.6

$ conda create -n CLIPstyler python=3.6
$ conda install --yes -c pytorch pytorch=1.7.1 torchvision cudatoolkit=11.0
$ pip install ftfy regex tqdm
$ conda install -c anaconda git
$ pip install git+https://github.com/openai/CLIP.git

Style Transfer with Single-image

To train the model and obtain the image, run

python train_CLIPstyler.py --content_path ./test_set/face.jpg \
--content_name face --exp_name exp1 \
--text "Sketch with black pencil"

To change the style of custom image, please change the --content_path argument

edit the text condition with --text argument

For easy demo, we provide Google Colab .

*Warning : Due to slow computation speed of colab, it may take several minutes in colab environment

Fast Style Transfer

Before training, plase download DIV2K dataset LINK.

We recomment to use Training data of High-Resolution(HR) images.

To train the model, please download the pre-trained vgg encoder & decoder models in LINK.

Please save the downloaded models in ./models directory

Then, run the command

python train_fast.py --content_path $DIV2K_DIR$ \
--name exp1 \
--text "Sketch with black pencil" --test_dir ./test_set

Please set the $DIV2K_DIR$ as the directory in which DIV2K images are saved.

To test the fast style transfer model,

python test_fast.py --test_dir ./test_set --decoder ./model_fast/clip_decoder_iter_200.pth.tar

Change the argument --decoder to other trained models for testing on different text conditions.

We provide several fine-tuned decoders for several text conditions. LINK

To use high-resolution image, please add --hr_dir ./hr_set to test command.

We provide colab notebook for testing fast transfer model