AirCode: A Robust Object Encoding Method

Overview

AirCode

This repo contains source codes for the arXiv preprint "AirCode: A Robust Object Encoding Method"

Demo

Object matching comparison when the objects are non-rigid and the view is changed, left is the result of our method while right is the result of NetVLAD

Relocalization on KITTI datasets

Dependencies

  • Python
  • PyTorch
  • OpenCV
  • Matplotlib
  • NumPy
  • Yaml

Data

Four datasets are used in our experiments.

KITTI Odometry

For relocalization experiment. Three sequences are selected, and they are "00", "05" and "06".

KITTI Tracking

For multi-object matching experiment. Four sequences are selected, and they are "0002", "0003", "0006", "0010".

VOT Datasets

For single-object matching experiment. We select three sequences from VOT2019 datasets and they are "bluecar", "bus6" and "humans_corridor_occ_2_A", because the tracked objects in these sequences are included in coco datasets, which are the data we used to train mask-rcnn.

OTB Datasets

For single-object matching experiment. We select five sequences and they are "BlurBody", "BlurCar2", "Human2", "Human7" and "Liquor".

Examples

Relocalization on KITTI Datasets

  1. Extract object descrptors

    python experiments/place_recogination/online_relocalization.py -c config/experiment_tracking.yaml -g 1 -s PATH_TO_SAVE_MIDDLE_RESULTS -d PATH_TO_DATASET -m PATH_TO_MODELS
    
  2. Compute precision-recall curves

    python experiments/place_recogination/offline_process.py -c config/experiment_tracking.yaml -g 1 -d PATH_TO_DATASET -n PATH_TO_MIDDLE_RESULTS -s PATH_TO_SAVE_RESULTS
    
  3. Compute top-K relocalization results

    python experiments/place_recogination/offline_topK.py -c config/experiment_tracking.yaml -g 1 -d PATH_TO_DATASET -n PATH_TO_MIDDLE_RESULTS -s PATH_TO_SAVE_RESULTS
    

Object Matching on OTB, VOT or KITTI Tracking Datasets

  • Run multi-object matching experiment in KITTI Tracking Datasets Modify the config file and run

    python experiments/object_tracking/object_tracking.py -c config/experiment_tracking.yaml -g 1 -s PATH_TO_SAVE_RESULTS -d PATH_TO_DATASET -m PATH_TO_MODELS 
    
  • Run single-object matching experiment in OTB or VOT Datasets Modify the config file and run

    python experiments/object_tracking/single_object_tracking.py -c config/experiment_tracking.yaml -g 1 -s PATH_TO_SAVE_RESULTS -d PATH_TO_DATASET -m PATH_TO_MODELS 
    
You might also like...
PyTorch implementation of Rethinking Positional Encoding in Language Pre-training
PyTorch implementation of Rethinking Positional Encoding in Language Pre-training

TUPE PyTorch implementation of Rethinking Positional Encoding in Language Pre-training. Quickstart Clone this repository. git clone https://github.com

Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding (CVPR2022)
Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding (CVPR2022)

Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding by Qiaole Dong*, Chenjie Cao*, Yanwei Fu Paper and Supple

A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection
A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection

Confluence: A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection 1. 介绍 用以替代 NMS,在所有 bbox 中挑选出最优的集合。 NMS 仅考虑了 bbox 的得分,然后根据 IOU 来

[ECCVW2020] Robust Long-Term Object Tracking via Improved Discriminative Model Prediction (RLT-DiMP)
[ECCVW2020] Robust Long-Term Object Tracking via Improved Discriminative Model Prediction (RLT-DiMP)

Feel free to visit my homepage Robust Long-Term Object Tracking via Improved Discriminative Model Prediction (RLT-DIMP) [ECCVW2020 paper] Presentation

 Robust Instance Segmentation through Reasoning about Multi-Object Occlusion [CVPR 2021]
Robust Instance Segmentation through Reasoning about Multi-Object Occlusion [CVPR 2021]

Robust Instance Segmentation through Reasoning about Multi-Object Occlusion [CVPR 2021] Abstract Analyzing complex scenes with DNN is a challenging ta

Code release for our paper,
Code release for our paper, "SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo"

SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo Thomas Kollar, Michael Laskey, Kevin Stone, Brijen Thananjeyan

object detection; robust detection; ACM MM21 grand challenge; Security AI Challenger Phase VII
object detection; robust detection; ACM MM21 grand challenge; Security AI Challenger Phase VII

赛题背景 在商品知识产权领域,知识产权体现为在线商品的设计和品牌。不幸的是,在每一天,存在着非法商户通过一些对抗手段干扰商标识别来逃避侵权,这带来了很高的知识产权风险和财务损失。为了促进先进的多媒体人工智能技术的发展,以保护企业来之不易的创作和想法免受恶意使用和剽窃,因此提出了鲁棒性标识检测挑战赛

Code and models for ICCV2021 paper
Code and models for ICCV2021 paper "Robust Object Detection via Instance-Level Temporal Cycle Confusion".

Robust Object Detection via Instance-Level Temporal Cycle Confusion This repo contains the implementation of the ICCV 2021 paper, Robust Object Detect

Official code for 'Robust Siamese Object Tracking for Unmanned Aerial Manipulator' and offical introduction to UAMT100 benchmark
Official code for 'Robust Siamese Object Tracking for Unmanned Aerial Manipulator' and offical introduction to UAMT100 benchmark

SiamSA: Robust Siamese Object Tracking for Unmanned Aerial Manipulator Demo video 📹 Our video on Youtube and bilibili demonstrates the evaluation of

Comments
  • how can I get *.pth files?

    how can I get *.pth files?

    Hello, I am a beginner. When I run python experiments/place_recogination/online_relocalization.py -c config/experiment_tracking.yaml -g 1 -s results/ -d /media/jixingwu/datasetj/KITTI/Odom/data_odometry_color/sequences -m models/, points_model.pth file is needed. So how can I get it? Thank you!

    opened by jixingwu 5
  • Unable to load model under CPU-only configuration

    Unable to load model under CPU-only configuration

    Hi, I want to run object tracking on KITTI tracking datasets with only CPU using the following terminal prompt:

      python experiments/object_tracking/object_tracking.py -c config/experiment_tracking.yaml -g 1 -s ./results -d /data/datasets/SLAM_dataset/training/ -m ./weights
    

    with configuration in object_tracking.py updated with

    configs['use_gpu'] = 0
    

    However, when running with the configuration above with gcn_model.pth, maskrcnn_model.pth, points_model.pth model files in release v2.0.0, the following error occurs:

    (aircode) [email protected]:~/workspace/AirCode$ python experiments/object_tracking/object_tracking.py -c config/experiment_tracking.yaml -g 1 -s ./results -d /data/datasets/SLAM_dataset/training/ -m ./weights
    experiments/object_tracking/object_tracking.py:371: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
      configs = yaml.load(configs)
    Traceback (most recent call last):
      File "experiments/object_tracking/object_tracking.py", line 384, in <module>
        main()
      File "experiments/object_tracking/object_tracking.py", line 381, in main
        show_object_tracking(configs)
      File "experiments/object_tracking/object_tracking.py", line 272, in show_object_tracking
        superpoint_model = build_superpoint_model(configs, requires_grad=False)
      File "./model/build_model.py", line 101, in build_superpoint_model
        model.load_state_dict(model_dict)
      File "/home/yutianc/minicondas/envs/aircode/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1052, in load_state_dict
        self.__class__.__name__, "\n\t".join(error_msgs)))
    RuntimeError: Error(s) in loading state_dict for VggLike:
            Unexpected key(s) in state_dict: "module.pretrained_net.features.0.weight", "module.pretrained_net.features.0.bias", "module.pretrained_net.features.2.weight", "module.pretrained_net.features.2.bias", "module.pretrained_net.features.5.weight", "module.pretrained_net.features.5.bias", "module.pretrained_net.features.7.weight", "module.pretrained_net.features.7.bias", "module.pretrained_net.features.10.weight", "module.pretrained_net.features.10.bias", "module.pretrained_net.features.12.weight", "module.pretrained_net.features.12.bias", "module.pretrained_net.features.14.weight", "module.pretrained_net.features.14.bias", "module.pretrained_net.features.17.weight", "module.pretrained_net.features.17.bias", "module.pretrained_net.features.19.weight", "module.pretrained_net.features.19.bias", "module.pretrained_net.features.21.weight", "module.pretrained_net.features.21.bias", "module.pretrained_net.features.24.weight", "module.pretrained_net.features.24.bias", "module.pretrained_net.features.26.weight", "module.pretrained_net.features.26.bias", "module.pretrained_net.features.28.weight", "module.pretrained_net.features.28.bias", "module.convPa.weight", "module.convPa.bias", "module.bnPa.weight", "module.bnPa.bias", "module.bnPa.running_mean", "module.bnPa.running_var", "module.bnPa.num_batches_tracked", "module.convPb.weight", "module.convPb.bias", "module.bnPb.weight", "module.bnPb.bias", "module.bnPb.running_mean", "module.bnPb.running_var", "module.bnPb.num_batches_tracked", "module.convDa.weight", "module.convDa.bias", "module.bnDa.weight", "module.bnDa.bias", "module.bnDa.running_mean", "module.bnDa.running_var", "module.bnDa.num_batches_tracked", "module.convDb.weight", "module.convDb.bias", "module.bnDb.weight", "module.bnDb.bias", "module.bnDb.running_mean", "module.bnDb.running_var", "module.bnDb.num_batches_tracked".
    

    Running object_tracking.py with CUDA seems to load models successfully. Is there something wrong with the model loading when GPU is disabled?

    opened by MarkChenYutian 4
  • Why RGB image is converted into grayscale image with 3 channels?

    Why RGB image is converted into grayscale image with 3 channels?

    Hi, I'm trying to use AirCode to do object matching on complete KITTI sequences and I'm reading the code in experiments/show_object_matching.py.

    While reading the code, I noticed that the current code is reading RGB image sequence, convert it into grayscale image, and then duplicate the image into 3-channel each with same value (as following):

    https://github.com/wang-chen/AirCode/blob/5e23e9f5322d2e4ee119d5326a6b6112cef0e6bd/experiments/show_object_matching/show_object_matching.py#L172-L176

    I'm a bit unsure about the reason why this operation is performed here as the original RGB image should contain more information about the object comparing to grayscale image. For instance, it should be easier to distinguish objects with different color but similar shape if the RGB value is preserved.

    opened by MarkChenYutian 2
Owner
Chen Wang
I am engaged in delivering simple and efficient source code.
Chen Wang
Consumer Fairness in Recommender Systems: Contextualizing Definitions and Mitigations

Consumer Fairness in Recommender Systems: Contextualizing Definitions and Mitigations This is the repository for the paper Consumer Fairness in Recomm

7 Nov 30, 2022
Creating Multi Task Models With Keras

Creating Multi Task Models With Keras About The Project! I used the keras and Tensorflow Library, To build a Deep Learning Neural Network to Creating

Srajan Chourasia 4 Nov 28, 2022
Code for “ACE-HGNN: Adaptive Curvature ExplorationHyperbolic Graph Neural Network”

ACE-HGNN: Adaptive Curvature Exploration Hyperbolic Graph Neural Network This repository is the implementation of ACE-HGNN in PyTorch. Environment pyt

9 Nov 28, 2022
This is the official Pytorch implementation of "Lung Segmentation from Chest X-rays using Variational Data Imputation", Raghavendra Selvan et al. 2020

README This is the official Pytorch implementation of "Lung Segmentation from Chest X-rays using Variational Data Imputation", Raghavendra Selvan et a

Raghav 42 Dec 15, 2022
GEP (GDB Enhanced Prompt) - a GDB plug-in for GDB command prompt with fzf history search, fish-like autosuggestions, auto-completion with floating window, partial string matching in history, and more!

GEP (GDB Enhanced Prompt) GEP (GDB Enhanced Prompt) is a GDB plug-in which make your GDB command prompt more convenient and flexibility. Why I need th

Alan Li 23 Dec 21, 2022
A strongly-typed genetic programming framework for Python

monkeys "If an army of monkeys were strumming on typewriters they might write all the books in the British Museum." monkeys is a framework designed to

H. Chase Stevens 115 Nov 27, 2022
Framework for training options with different attention mechanism and using them to solve downstream tasks.

Using Attention in HRL Framework for training options with different attention mechanism and using them to solve downstream tasks. Requirements GPU re

5 Nov 03, 2022
(3DV 2021 Oral) Filtering by Cluster Consistency for Large-Scale Multi-Image Matching

Scalable Cluster-Consistency Statistics for Robust Multi-Object Matching (3DV 2021 Oral Presentation) Filtering by Cluster Consistency (FCC) is a very

Yunpeng Shi 11 Sep 28, 2022
The InterScript dataset contains interactive user feedback on scripts generated by a T5-XXL model.

Interscript The Interscript dataset contains interactive user feedback on a T5-11B model generated scripts. Dataset data.json contains the data in an

AI2 8 Dec 01, 2022
BOOKSUM: A Collection of Datasets for Long-form Narrative Summarization

BOOKSUM: A Collection of Datasets for Long-form Narrative Summarization Authors: Wojciech Kryściński, Nazneen Rajani, Divyansh Agarwal, Caiming Xiong,

Salesforce 125 Dec 31, 2022
A Factor Model for Persistence in Investment Manager Performance

Factor-Model-Manager-Performance A Factor Model for Persistence in Investment Manager Performance I apply methods and processes similar to those used

Omid Arhami 1 Dec 01, 2021
This is an example implementation of the paper "Cross Domain Robot Imitation with Invariant Representation".

IR-GAIL This is an example implementation of the paper "Cross Domain Robot Imitation with Invariant Representation". Dependency The experiments are de

Zhao-Heng Yin 1 Jul 14, 2022
CBREN: Convolutional Neural Networks for Constant Bit Rate Video Quality Enhancement

CBREN This is the Pytorch implementation for our IEEE TCSVT paper : CBREN: Convolutional Neural Networks for Constant Bit Rate Video Quality Enhanceme

Zhao Hengrun 3 Nov 04, 2022
An implementation of the Contrast Predictive Coding (CPC) method to train audio features in an unsupervised fashion.

CPC_audio This code implements the Contrast Predictive Coding algorithm on audio data, as described in the paper Unsupervised Pretraining Transfers we

8 Nov 14, 2022
My tensorflow implementation of "A neural conversational model", a Deep learning based chatbot

Deep Q&A Table of Contents Presentation Installation Running Chatbot Web interface Results Pretrained model Improvements Upgrade Presentation This wor

Conchylicultor 2.9k Dec 28, 2022
A Python Reconnection Tool for alt:V

altv-reconnect What? It invokes a reconnect in the altV Client Dev Console. You get to determine when your local client should reconnect when developi

8 Jun 30, 2022
Effective Use of Transformer Networks for Entity Tracking

Effective Use of Transformer Networks for Entity Tracking (EMNLP19) This is a PyTorch implementation of our EMNLP paper on the effectiveness of pre-tr

5 Nov 06, 2021
Efficient neural networks for analog audio effect modeling

micro-TCN Efficient neural networks for audio effect modeling

Christian Steinmetz 94 Dec 29, 2022
Molecular Sets (MOSES): A benchmarking platform for molecular generation models

Molecular Sets (MOSES): A benchmarking platform for molecular generation models Deep generative models are rapidly becoming popular for the discovery

Neelesh C A 3 Oct 14, 2022
A resource for learning about ML, DL, PyTorch and TensorFlow. Feedback always appreciated :)

A resource for learning about ML, DL, PyTorch and TensorFlow. Feedback always appreciated :)

Aladdin Persson 4.7k Jan 08, 2023