Barlow Twins and HSIC

Unofficial Pytorch implementation for Barlow Twins and HSIC_SSL on small datasets (CIFAR10, STL10, and Tiny ImageNet).

Correspondence to:

• Yao-Hung Hubert Tsai ([email protected])

Technical Report

A Note on Connecting Barlow Twins with Negative-Samples-Free Contrastive Learning
Yao-Hung Hubert Tsai, Shaojie Bai, Louis-Philippe Morency, and Ruslan Salakhutdinov

I hope this work will be useful for your research 🥰

Usage

Disclaimer

A large portion of the code is from this repo, which is a great resource for academic development. Note that we do not perform extensive hyper-parameters grid search and hence you may expect a performance boost after tuning some hyper-parameters (e.g., the learning rate).

The official implementation of Barlow Twins can be found here. We have also tried the HSIC_SSL in this official repo and we find similar performance (we tried on ImageNet-1K and CIFAR10) between HSIC_SSL and Barlow Twins' method.

Requirements

• Anaconda
• PyTorch

conda install pytorch torchvision cudatoolkit=10.0 -c pytorch

• thop

pip install thop

Supported Dataset

CIFAR10, STL10, and Tiny_ImageNet.

Train and Linear Evaluation using Barlow Twins

python main.py --lmbda 0.0078125 --corr_zero --batch_size 128 --feature_dim 128 --dataset cifar10
python linear.py --dataset cifar10 --model_path results/0.0078125_128_128_cifar10_model.pth

Train and Linear Evaluation using HSIC

python main.py --lmbda 0.0078125 --corr_neg_one --batch_size 128 --feature_dim 128 --dataset cifar10
python linear.py --dataset cifar10 --model_path results/neg_corr_0.0078125_128_128_cifar10_model.pth