CATs: Semantic Correspondence with Transformers

Last update: Dec 10, 2021

Related tags

Text Data & NLP CATs

Overview

CATs: Semantic Correspondence with Transformers

For more information, check out the paper on [arXiv].

Training with different backbones and evaluations of them are to be updated soon..

Network

Our model CATs is illustrated below:

Environment Settings

git clone https://github.com/SunghwanHong/CATs
cd CATs

conda create -n CATs python=3.6
conda activate CATs

pip install torch==1.8.0+cu111 torchvision==0.9.0+cu111 torchaudio==0.8.0 -f https://download.pytorch.org/whl/torch_stable.html
pip install -U scikit-image
pip install git+https://github.com/albumentations-team/albumentations
pip install tensorboardX termcolor timm tqdm requests pandas

Evaluation

Download pre-trained weights on Link
All datasets are automatically downloaded into directory specified by argument datapath

Result on SPair-71k: (PCK 49.9%)

  python test.py --pretrained "/path_to_pretrained_model/spair" --benchmark spair

Result on SPair-71k, feature backbone frozen: (PCK 42.4%)

  python test.py --pretrained "/path_to_pretrained_model/spair_frozen" --benchmark spair

Results on PF-PASCAL: (PCK 75.4%, 92.6%, 96.4%)

  python test.py --pretrained "/path_to_pretrained_model/pfpascal" --benchmark pfpascal

Results on PF-PACAL, feature backbone frozen: (PCK 67.5%, 89.1%, 94.9%)

  python test.py --pretrained "/path_to_pretrained_model/pfpascal_frozen" --benchmark pfpascal

BibTeX

If you find this research useful, please consider citing:

@misc{cho2021semantic,
      title={Semantic Correspondence with Transformers}, 
      author={Seokju Cho and Sunghwan Hong and Sangryul Jeon and Yunsung Lee and Kwanghoon Sohn and Seungryong Kim},
      year={2021},
      eprint={2106.02520},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

CATs: Semantic Correspondence with Transformers

Related tags

Overview

CATs: Semantic Correspondence with Transformers

Network

Environment Settings

Evaluation

BibTeX

Owner

Indonesia spellchecker with python

An IVR Chatbot which can exponentially reduce the burden of companies as well as can improve the consumer/end user experience.

A modular Karton Framework service that unpacks common packers like UPX and others using the Qiling Framework.

Code Generation using a large neural network called GPT-J

EdiTTS: Score-based Editing for Controllable Text-to-Speech

Implementation of paper Does syntax matter? A strong baseline for Aspect-based Sentiment Analysis with RoBERTa.

Sentiment Classification using WSD, Maximum Entropy & Naive Bayes Classifiers

An evaluation toolkit for voice conversion models.

Finding Label and Model Errors in Perception Data With Learned Observation Assertions

This is my reading list for my PhD in AI, NLP, Deep Learning and more.

Demo programs for the Talking Head Anime from a Single Image 2: More Expressive project.

PyTranslator é simultaneamente um editor e tradutor de texto com diversos recursos e interface feito com coração e 100% em Python

中文空间语义理解评测

Search with BERT vectors in Solr and Elasticsearch

A PyTorch-based model pruning toolkit for pre-trained language models

Binaural Speech Synthesis

Tensorflow implementation of paper: Learning to Diagnose with LSTM Recurrent Neural Networks.

An implementation of model parallel GPT-3-like models on GPUs, based on the DeepSpeed library. Designed to be able to train models in the hundreds of billions of parameters or larger.

GPT-2 Model for Leetcode Questions in python

NLP techniques such as named entity recognition, sentiment analysis, topic modeling, text classification with Python to predict sentiment and rating of drug from user reviews.