Unsupervised Representation Learning by Invariance Propagation

Last update: Jul 06, 2022

Related tags

Deep Learning InvariancePropagation

Overview

Unsupervised Learning by Invariance Propagation

This repository is the official implementation of Unsupervised Learning by Invariance Propagation.

Pretraining on Natual Images

Train on ImageNet

To train the model(s) in the paper, run this command:

python main.py --exp 'your_path' --n_background 4096 --t 0.2 --blur --cos --network 'resnet50' --nonlinearhead 1 --weight_decay 1e-4

Evaluation

To evaluate the model on ImageNet, run:

python -m downstream.linear_classification.linear_classification --gpus '0,1' --exp 'your_exp_path' --pretrained_path 'pretrain_path' --backbone 'resnet50'

Notice that in the paper, to calculate the BFS results, we require to record the id of neighbours of each anchor point. For computational efficiency, we apprximate the BFS results by only concatenating the neighbours of each point, up to L steps. This results may be a little different with the real BFS results due to there exists repeated samples, however it works pretty well, both effectively and efficiently. Pretrained model can be found here.

Train on Cifar

To train the model(s) in cifar10 and cifar100 or svhn, run this command:

# cifar10
python main.py --exp 'your_path' -n_background 4096 --t 0.2 --blur --cos --network 'resnet18_cifar' --nonlinearhead 1 --weight_decay 5e-4 --n_pos 20 --dataset 'cifar10'
# cifar100
python main.py --exp 'your_path' -n_background 4096 --t 0.2 --blur --cos --network 'resnet18_cifar' --nonlinearhead 1 --weight_decay 5e-4 --n_pos 20 --dataset 'cifar100'
# svhn
python main.py --exp 'your_path' -n_background 4096 --t 0.2 --blur --cos --network 'resnet18_cifar' --nonlinearhead 1 --weight_decay 5e-4 --n_pos 20 --dataset 'svhn'

Evaluation

To train the model(s) in cifar10 and cifar100 run this command:

# cifar10
python -m downstream.linear_classification.eval_linear --gpus '0,1' --exp 'your_exp_path' --pretrained_path 'pretrain_path' --backbone 'resnet18_cifar' --dataset 'cifar10'
# cifar100
python -m downstream.linear_classification.eval_linear --gpus '0,1' --exp 'your_exp_path' --pretrained_path 'pretrain_path' --backbone 'resnet18_cifar' --dataset 'cifar100'
# svhn
python -m downstream.linear_classification.eval_linear --gpus '0,1' --exp 'your_exp_path' --pretrained_path 'pretrain_path' --backbone 'resnet18_cifar' --dataset 'svhn'

Pretraining on Defect Classification Dataset

For validate the effectiveness and practicabilities of the proposed algorithms, we can also train and evaluate our method on Defect Detection Dataset.

Train on WM811.

python main.py --gpus '0,1,2' --exp 'output/' --n_background 4096 --t 0.07 --cos --network 'resnet18_wm811' --dataset 'wm811' --nonlinearhead 0 --weight_decay 5e-4

Evaluation

To evaluate the model on WM811, run:

python -m downstream.fine_tune_wm811 --save_folder 'your_output_folder' --model_path 'your_pretrain_model' --model 'resnet18_wm811' --dataset 'wm811' --weight_decay 1e-3 --learning_rate1 0.001 --learning_rate2 0.002 --label_smoothing 0.1 --dropout 0.5

Unsupervised Representation Learning by Invariance Propagation

Related tags

Overview

Unsupervised Learning by Invariance Propagation

Pretraining on Natual Images

Train on ImageNet

Evaluation

Train on Cifar

Evaluation

Pretraining on Defect Classification Dataset

Train on WM811.

Evaluation

Owner

FengWang

🏅 Top 5% in 제2회 연구개발특구 인공지능 경진대회 AI SPARK 챌린지

A Game-Theoretic Perspective on Risk-Sensitive Reinforcement Learning

TensorFlow (Python) implementation of DeepTCN model for multivariate time series forecasting.

Must-read Papers on Physics-Informed Neural Networks.

Coded illumination for improved lensless imaging

Machine learning notebooks in different subjects optimized to run in google collaboratory

Code repository for "Reducing Underflow in Mixed Precision Training by Gradient Scaling" presented at IJCAI '20

GNN-based Recommendation Benchmark

CLIPImageClassifier wraps clip image model from transformers

PantheonRL is a package for training and testing multi-agent reinforcement learning environments.

MediaPipe is a an open-source framework from Google for building multimodal

This is the official implementation of TrivialAugment and a mini-library for the application of multiple image augmentation strategies including RandAugment and TrivialAugment.

Scripts and a shader to get you started on setting up an exported Koikatsu character in Blender.

Crowd-Kit is a powerful Python library that implements commonly-used aggregation methods for crowdsourced annotation and offers the relevant metrics and datasets

Junction Tree Variational Autoencoder for Molecular Graph Generation (ICML 2018)

Mmdetection3d Noted - MMDetection3D is an open source object detection toolbox based on PyTorch

Receptive Field Block Net for Accurate and Fast Object Detection, ECCV 2018

Implementation of Convolutional enhanced image Transformer

EfficientNetV2-with-TPU - Cifar-10 case study

Generalizing Gaze Estimation with Outlier-guided Collaborative Adaptation