Generalized Decision Transformer for Offline Hindsight Information Matching

If you use this codebase for your research, please cite the paper:

@article{furuta2021generalized,
  title={Generalized Decision Transformer for Offline Hindsight Information Matching},
  author={Hiroki Furuta and Yutaka Matsuo and Shixiang Shane Gu},
  journal={arXiv preprint arXiv:2111.10364},
  year={2021}
}

Installation

Experiments require MuJoCo. Follow the instructions in the mujoco-py repo to install. Then, dependencies can be installed with the following command:

conda env create -f conda_env.yml

Downloading datasets

Datasets are stored in the data directory. Install the D4RL repo, following the instructions there. Then, run the following script in order to download the datasets and save them in our format:

python download_d4rl_datasets.py

Run experiments

Run train_cdt.py to train Categorical DT:

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_model True

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_model True

Run eval_cdt.py to eval CDT using saved weights:

python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_rollout True
python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_rollout True

For Bi-directional DT, run train_bdt.py & eval_bdtf.py

python train_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_model True
python eval_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_rollout True

Reference

This repository is developed on top of original Decision Transformer.

Generalized Decision Transformer for Offline Hindsight Information Matching

Related tags

Overview

Generalized Decision Transformer for Offline Hindsight Information Matching

Installation

Downloading datasets

Run experiments

Reference

Owner

Hiroki Furuta

List of awesome things around semantic segmentation 🎉

Learning to Adapt Structured Output Space for Semantic Segmentation, CVPR 2018 (spotlight)

Official Implementation of Few-shot Visual Relationship Co-localization

Algo-burn - Script to configure an Algorand address as a "burn" address for one or more ASA tokens

Code and training data for our ECCV 2016 paper on Unsupervised Learning

CVPR 2021

Software that can generate photos from paintings, turn horses into zebras, perform style transfer, and more.

Official implementation for CVPR 2021 paper: Adaptive Class Suppression Loss for Long-Tail Object Detection

Animation of solving the traveling salesman problem to optimality using mixed-integer programming and iteratively eliminating sub tours

STRIVE: Scene Text Replacement In Videos

Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers [CVPR 2021]

Apache Spark - A unified analytics engine for large-scale data processing

VR Viewport Pose Model for Quantifying and Exploiting Frame Correlations

Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking

Copy Paste positive polyp using poisson image blending for medical image segmentation

Code release for Convolutional Two-Stream Network Fusion for Video Action Recognition

My solutions for Stanford University course CS224W: Machine Learning with Graphs Fall 2021 colabs (GNN, GAT, GraphSAGE, GCN)

USAD - UnSupervised Anomaly Detection on multivariate time series

Pytorch implementation of “Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement”

TCNN Temporal convolutional neural network for real-time speech enhancement in the time domain