RVT: Robust Vision Transformers

This repository contains PyTorch code for Robust Vision Transformers.

For details see Rethinking the Design Principles of Robust Vision Transformer by Xiaofeng Mao, Gege Qi, Yuefeng Chen, Yuan He and Hui Xue.

Usage

First, clone the repository locally:

git clone https://github.com/vtddggg/Robust-Vision-Transformer.git

Then, install PyTorch 1.7.0+ and torchvision 0.8.1+ and pytorch-image-models 0.3.2:

conda install -c pytorch pytorch torchvision
pip install timm==0.3.2

We use 4 nodes with 8 gpus to train RVT-Ti, RVT-S and RVT-B:

Training RVT-Ti

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_tiny --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-S

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_small --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-B

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_base --data-path /path/to/imagenet --output_dir output --batch-size 32 --dist-eval

If you want to train RVT-Ti*, RVT-S* or RVT-B*, simply add --use_mask and --use_patch_aug to enable positon-aware attention scaling and patch-wise augmentation.

This repository contains PyTorch code for Robust Vision Transformers.

Related tags

Overview

RVT: Robust Vision Transformers

Usage

Training RVT-Ti

Training RVT-S

Training RVT-B

Owner

CARLA: A Python Library to Benchmark Algorithmic Recourse and Counterfactual Explanation Algorithms

Official implementation of the paper "Steganographer Detection via a Similarity Accumulation Graph Convolutional Network"

TorchDistiller - a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and instance segmentation.

Estimating and Exploiting the Aleatoric Uncertainty in Surface Normal Estimation

Deep Latent Force Models

PyTorch Implementation of PortaSpeech: Portable and High-Quality Generative Text-to-Speech

An implementation of IMLE-Net: An Interpretable Multi-level Multi-channel Model for ECG Classification

Classifying audio using Wavelet transform and deep learning

[NeurIPS2021] Code Release of K-Net: Towards Unified Image Segmentation

Inteligência artificial criada para realizar interação social com idosos.

Clustering is a popular approach to detect patterns in unlabeled data

Learning Versatile Neural Architectures by Propagating Network Codes

Fast, flexible and easy to use probabilistic modelling in Python.

This implements one of result networks from Large-scale evolution of image classifiers

A C implementation for creating 2D voronoi diagrams

Useful materials and tutorials for 110-1 NTU DBME5028 (Application of Deep Learning in Medical Imaging)

Faster RCNN with PyTorch

Implementation of "Debiasing Item-to-Item Recommendations With Small Annotated Datasets" (RecSys '20)

Jingju baseline - A baseline model of our project of Beijing opera script generation

Tree LSTM implementation in PyTorch