Official implementation of Densely connected normalizing flows

Last update: Dec 12, 2022

Related tags

Overview

Densely connected normalizing flows

This repository is the official implementation of NeurIPS 2021 paper Densely connected normalizing flows. Poster available here.

Setup

CUDA 11.1
Python 3.8

pip install -r requirements.txt
pip install -e .

Training

cd ./experiments/image

CIFAR-10:

python train.py --epochs 400 --batch_size 64 --optimizer adamax --lr 1e-3  --gamma 0.9975 --warmup 5000  --eval_every 1 --check_every 10 --dataset cifar10 --augmentation eta --block_conf 6 4 1 --layers_conf  5 6 20  --layer_mid_chnls 48 48 48 --growth_rate 10  --name DF_74_10

python train_more.py --model ./log/cifar10_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 420

ImageNet32:

python train.py --epochs 20 --batch_size 64 --optimizer adamax --lr 1e-3  --gamma 0.95 --warmup 5000  --eval_every 1 --check_every 10 --dataset imagenet32 --augmentation eta --block_conf 6 4 1 --layers_conf  5 6 20  --layer_mid_chnls 48 48 48 --growth_rate 10  --name DF_74_10

python train_more.py --model ./log/imagenet32_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 22

ImageNet64:

python train.py --epochs 10 --batch_size 32 --optimizer adamax --lr 1e-3  --gamma 0.95 --warmup 5000  --eval_every 1 --check_every 10 --dataset imagenet64 --augmentation eta --block_conf 6 4 1 --layers_conf  5 6 20  --layer_mid_chnls 48 48 48 --growth_rate 10  --name DF_74_10

python train_more.py --model ./log/imagenet64_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 11

CelebA:

python train.py --epochs 50 --batch_size 32 --optimizer adamax --lr 1e-3  --gamma 0.95 --warmup 5000  --eval_every 1 --check_every 10 --dataset celeba --augmentation horizontal_flip --block_conf 6 4 1 --layers_conf  5 6 20  --layer_mid_chnls 48 48 48 --growth_rate 10  --name DF_74_10

python train_more.py --model ./log/celeba_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 55

Note: Download instructions for ImageNet and CelebA are defined in denseflow/data/datasets/image/{dataset}.py

Evaluation

CIFAR-10:

python eval_loglik.py --model PATH_TO_MODEL --k 1000 --kbs 50

ImageNet32:

python eval_loglik.py --model PATH_TO_MODEL --k 200 --kbs 50

ImageNet64 and CelebA:

python eval_loglik.py --model PATH_TO_MODEL --k 200 --kbs 25

Model weights

Model weights are stored here.

Samples generation

Generated samples are stored in PATH_TO_MODEL/samples

python eval_sample.py --model PATH_TO_MODEL

Note: PATH_TO_MODEL has to contain check directory.

ImageNet 32x32

ImageNet 64x64

CelebA

Acknowledgements

Significant part of this code benefited from SurVAE [1] code implementation, available under MIT license.

References

[1] Didrik Nielsen, Priyank Jaini, Emiel Hoogeboom, Ole Winther, and Max Welling. Survae flows: Surjections to bridge the gap between vaes and flows. InAdvances in Neural Information Processing Systems 33. Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020.

Official implementation of Densely connected normalizing flows

Related tags

Overview

Densely connected normalizing flows

Setup

Training

Evaluation

Model weights

Samples generation

ImageNet 32x32

ImageNet 64x64

CelebA

Acknowledgements

References

Owner

Matej Grcić

Pytorch-diffusion - A basic PyTorch implementation of 'Denoising Diffusion Probabilistic Models'

TorchIO is a Medical image preprocessing and augmentation toolkit for deep learning. Part of the PyTorch Ecosystem.

Symbolic Parallel Adaptive Importance Sampling for Probabilistic Program Analysis in JAX

Exponential Graph is Provably Efficient for Decentralized Deep Training

Much faster than SORT(Simple Online and Realtime Tracking), a little worse than SORT

O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learning (CoRL 2021)

Self-Correcting Quantum Many-Body Control using Reinforcement Learning with Tensor Networks

A PyTorch-Based Framework for Deep Learning in Computer Vision

using STGCN to achieve egg classification task

Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm

Deep learning library featuring a higher-level API for TensorFlow.

Experimental solutions to selected exercises from the book [Advances in Financial Machine Learning by Marcos Lopez De Prado]

University of Rochester 2021 Summer REU focusing on music sentiment transfer using CycleGAN

Official code repository for the publication "Latent Equilibrium: A unified learning theory for arbitrarily fast computation with arbitrarily slow neurons"

Memoized coduals - Shows that it is possible to implement reverse mode autodiff using a variation on the dual numbers called the codual numbers

Simple, efficient and flexible vision toolbox for mxnet framework.

Benchmark for Answering Existential First Order Queries with Single Free Variable

A PyTorch implementation: "LASAFT-Net-v2: Listen, Attend and Separate by Attentively aggregating Frequency Transformation"

This is the dataset for testing the robustness of various VO/VIO methods

FastReID is a research platform that implements state-of-the-art re-identification algorithms.