A simple PyTorch Implementation of Generative Adversarial Networks, focusing on anime face drawing.

Related tags

Deep Learningpytorchdatasetgenerative-adversarial-network
Overview

AnimeGAN

A simple PyTorch Implementation of Generative Adversarial Networks, focusing on anime face drawing.

Randomly Generated Images

The images are generated from a DCGAN model trained on 143,000 anime character faces for 100 epochs.

fake_sample_1

Image Interpolation

Manipulating latent codes, enables the transition from images in the first row to the last row.

transition

Original Images

The images are not clean, some outliers can be observed, which degrades the quality of the generated images.

real_sample

Usage

To run the experiment,

$ python main.py --dataRoot path_to_dataset/

The pretrained model for DCGAN are also in this repo, play it inside the jupyter notebook.

anime-faces Dataset

Anime-style images of 126 tags are collected from danbooru.donmai.us using the crawler tool gallery-dl. The images are then processed by a anime face detector python-animeface. The resulting dataset contains ~143,000 anime faces. Note that some of the tags may no longer meaningful after cropping, i.e. the cropped face images under 'uniform' tag may not contain visible parts of uniforms.

How to construct the dataset from scratch ?

Prequisites: gallery-dl, python-animeface

  1. Download anime-style images

    # download 1000 images under the tag "misaka_mikoto"
gallery-dl --images 1000 "https://danbooru.donmai.us/posts?tags=misaka_mikoto"

# in a multi-processing manner
cat tags.txt | \
xargs -n 1 -P 12 -I 'tag' \ 
bash -c ' gallery-dl --images 1000 "https://danbooru.donmai.us/posts?tags=$tag" '

  2. Extract faces from the downloaded images

    import animeface
from PIL import Image

im = Image.open('images/anime_image_misaka_mikoto.png')
faces = animeface.detect(im)
x,y,w,h = faces[0].face.pos
im = im.crop((x,y,x+w,y+h))
im.show() # display

I've cleaned the original dataset, the new version of the dataset has 115085 images in 126 tags. You can access the images from:

Non-commercial use please.

Things I've learned

  1. GANs are really hard to train.
  2. DCGAN generally works well, simply add fully-connected layers causes problems.
  3. In my cases, more layers for G yields better images, in the sense that G should be more powerful than D.
  4. Add noise to D's inputs and labels helps stablize training.
  5. Use differnet input and generate resolution (64x64 vs 96x96), there seems no obvious difference during training, the generated images are also very similar.
  6. Binray Noise as G's input amazingly works, but the images are not as good as those with Gussian Noise, idea credit to @cwhy ['Binary Noise' here I mean a sequence of {-1,1} generated by bernoulli distribution at p=0.5 ]

I did not carefully verify them, if you are looking for some general GAN tips, see @soumith's ganhacks

Others

  1. This project is heavily influenced by chainer-DCGAN and IllustrationGAN, the codes are mostly borrowed from PyTorch DCGAN example, thanks the authors for the clean codes.
  2. Dependencies: pytorch, torchvision
  3. This is a toy project for me to learn PyTorch and GANs, most importantly, for fun! :) Any feedback is welcome.

@jayleicn

Comments
  • KeyError: 'module name can\'t contain

    KeyError: 'module name can\'t contain "."'

    The classes in module.py contains some nn.Module layers whose names contains some'.' in it, so I got error messages like the title, so how could I play it??

    opened by bolin12 2
  • no image under some tags

    no image under some tags

    Hi, The dataset from google drive contains 126 tags. However, some folders are emtpy:

    1girl apron blush collarbone hairclip honma_meiko japanese_clothes monochrome necktie nishizumi_miho purple_eyes scarf school_uniform sunglasses

    Is this normal? Thanks

    opened by samrere 1
  • set_sizes_contiguous is not allowed on a Tensor created from .data or .detach().

    set_sizes_contiguous is not allowed on a Tensor created from .data or .detach().

    input.data.resize_(real_cpu.size()).copy_(real_cpu) RuntimeError: set_sizes_contiguous is not allowed on a Tensor created from .data or .detach(). If your intent is to change the metadata of a Tensor (such as sizes / strides / storage / storage_offset) without autograd tracking the change, remove the .data / .detach() call and wrap the change in a with torch.no_grad(): block. For example, change: x.data.set_(y) to: with torch.no_grad(): x.set_(y)

    opened by athulvingt 0
  • can not run

    can not run

    Traceback (most recent call last): File "main.py", line 6, in import torch File "/Library/Python/2.7/site-packages/torch/init.py", line 81, in from torch._C import * RuntimeError: module compiled against API version 0xa but this version of numpy is 0x9

    opened by xtzero 0
  • How do I start with my own model

    How do I start with my own model

    I would like to know how I can use this image generation to generate my own images from a self made model?

    Where can I read upon on this. I find no concrete info on making the models.

    opened by quintendewilde 0
Owner
Jie Lei 雷杰
UNC CS PhD student, vision+language.
Jie Lei 雷杰
GitHub Repository
code for CVPR paper Zero-shot Instance Segmentation

Code for CVPR2021 paper Zero-shot Instance Segmentation Code requirements python: python3.7 nvidia GPU pytorch1.1.0 GCC =5.4 NCCL 2 the other python

zhengye 86 Dec 13, 2022
Code repository for EMNLP 2021 paper 'Adversarial Attacks on Knowledge Graph Embeddings via Instance Attribution Methods'

Adversarial Attacks on Knowledge Graph Embeddings via Instance Attribution Methods This is the code repository to accompany the EMNLP 2021 paper on ad

Peru Bhardwaj 7 Sep 25, 2022
Like a cowsay but without cows!

Foxsay This is a simple program that generates pictures of a cute fox with a message. It is like a cowsay but without cows! Fox girls are better! Usag

Anastasia Kim 28 Feb 20, 2022
A collection of differentiable SVD methods and also the official implementation of the ICCV21 paper "Why Approximate Matrix Square Root Outperforms Accurate SVD in Global Covariance Pooling?"

Differentiable SVD Introduction This repository contains: The official Pytorch implementation of ICCV21 paper Why Approximate Matrix Square Root Outpe

YueSong 32 Dec 25, 2022
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Website | Documentation | Tutorials | Installation | Release Notes CatBoost is a machine learning method based on gradient boosting over decision tree

CatBoost 6.9k Jan 04, 2023
Code for "LoRA: Low-Rank Adaptation of Large Language Models"

LoRA: Low-Rank Adaptation of Large Language Models This repo contains the implementation of LoRA in GPT-2 and steps to replicate the results in our re

Microsoft 394 Jan 08, 2023
Pytorch implementation of Nueral Style transfer

Nueral Style Transfer Pytorch implementation of Nueral style transfer algorithm , it is used to apply artistic styles to content images . Content is t

Abhinav 9 Oct 15, 2022
Fast and Simple Neural Vocoder, the Multiband RNNMS

Multiband RNN_MS Fast and Simple vocoder, Multiband RNN_MS. Demo Quick training How to Use System Details Results References Demo ToDO: Link super gre

tarepan 5 Jan 11, 2022
TensorFlow2 Classification Model Zoo playing with TensorFlow2 on the CIFAR-10 dataset.

Training CIFAR-10 with TensorFlow2(TF2) TensorFlow2 Classification Model Zoo. I'm playing with TensorFlow2 on the CIFAR-10 dataset. Architectures LeNe

Chia-Hung Yuan 16 Sep 27, 2022
Reliable probability face embeddings

ProbFace, arxiv This is a demo code of training and testing [ProbFace] using Tensorflow. ProbFace is a reliable Probabilistic Face Embeddging (PFE) me

Kaen Chan 34 Dec 31, 2022
Implementation of Heterogeneous Graph Attention Network

HetGAN Implementation of Heterogeneous Graph Attention Network This is the code repository of paper "Prediction of Metro Ridership During the COVID-19

5 Dec 28, 2021
Spectrum is an AI that uses machine learning to generate Rap song lyrics

Spectrum Spectrum is an AI that uses deep learning to generate rap song lyrics. View Demo Report Bug Request Feature Open In Colab About The Project S

39 Dec 16, 2022
git《Commonsense Knowledge Base Completion with Structural and Semantic Context》(AAAI 2020) GitHub: [fig1]

Commonsense Knowledge Base Completion with Structural and Semantic Context Code for the paper Commonsense Knowledge Base Completion with Structural an

AI2 96 Nov 05, 2022
VISSL is FAIR's library of extensible, modular and scalable components for SOTA Self-Supervised Learning with images.

What's New Below we share, in reverse chronological order, the updates and new releases in VISSL. All VISSL releases are available here. [Oct 2021]: V

Meta Research 2.9k Jan 07, 2023
Project code for weakly supervised 3D object detectors using wide-baseline multi-view traffic camera data: WIBAM.

WIBAM (Work in progress) Weakly Supervised Training of Monocular 3D Object Detectors Using Wide Baseline Multi-view Traffic Camera Data 3D object dete

Matthew Howe 10 Aug 24, 2022
QTool: A Low-bit Quantization Toolbox for Deep Neural Networks in Computer Vision

This project provides abundant choices of quantization strategies (such as the quantization algorithms, training schedules and empirical tricks) for quantizing the deep neural networks into low-bit c

Monash Green AI Lab 51 Dec 10, 2022
PyTorch implementation of Histogram Layers from DeepHist: Differentiable Joint and Color Histogram Layers for Image-to-Image Translation

deep-hist PyTorch implementation of Histogram Layers from DeepHist: Differentiable Joint and Color Histogram Layers for Image-to-Image Translation PyT

Winfried Lötzsch 10 Dec 06, 2022
The codes of paper 'Active-LATHE: An Active Learning Algorithm for Boosting the Error exponent for Learning Homogeneous Ising Trees'

Active-LATHE: An Active Learning Algorithm for Boosting the Error exponent for Learning Homogeneous Ising Trees This project contains the codes of pap

0 Apr 20, 2022
A Lightweight Hyperparameter Optimization Tool 🚀

Lightweight Hyperparameter Optimization 🚀 The mle-hyperopt package provides a simple and intuitive API for hyperparameter optimization of your Machin

136 Jan 08, 2023
Official code for CVPR2022 paper: Depth-Aware Generative Adversarial Network for Talking Head Video Generation

📖 Depth-Aware Generative Adversarial Network for Talking Head Video Generation (CVPR 2022) 🔥 If DaGAN is helpful in your photos/projects, please hel

Fa-Ting Hong 503 Jan 04, 2023