banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services.

Last update: Dec 22, 2022

Overview

What's banditml?

banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services. This library is developed by Bandit ML and ex-authors of Facebook's applied reinforcement learning platform, Reagent.

Specifically, this repo contains:

Feature engineering & preprocessing
Model implementations
Model training workflows
Model serving code for Python services

Supported models

Models supported:

Contextual Bandits (small datasets)
- Linear bandit w/ ε-greedy exploration
- Random forest bandit w/ ε-greedy exploration
- Gradient boosted decision tree bandit w/ ε-greedy exploration
Contextual Bandits (medium datasets)
- Neural bandit with ε-greedy exploration
- Neural bandit with UCB-based exploration (via. dropout exploration)
- Neural bandit with UCB-based exploration (via. mixture density networks)
Reinforcement Learning (large datasets)

4 feature types supported:

Numeric: standard floating point features
- e.g. {totalCartValue: 39.99}
Categorical: low-cardinality discrete features
- e.g. {currentlyViewingCategory: "men's jeans"}
ID list: high-cardinality discrete features
- e.g. {productsInCart: ["productId022", "productId109"...]}
- Handled via. learned embedding tables
"Dense" ID list: high-cardinality discrete features, manually mapped to dense feature vectors
- e.g {productId022: [0.5, 1.3, ...], productId109: [1.9, 0.1, ...], ...}

Docs

pip install banditml

Get started

License

GNU General Public License v3.0 or later

See COPYING to see the full text.

You might also like...

Repo for CVPR2021 paper "QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information"

QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information by Masato Tamura, Hiroki Ohashi, and Tomoaki Yosh

105 Dec 23, 2022

Source code and data from the RecSys 2020 article "Carousel Personalization in Music Streaming Apps with Contextual Bandits" by W. Bendada, G. Salha and T. Bontempelli

Carousel Personalization in Music Streaming Apps with Contextual Bandits - RecSys 2020 This repository provides Python code and data to reproduce expe

48 Jan 2, 2023

UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus

UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus General info This is

71 Oct 25, 2022

Generate Contextual Directory Wordlist For Target Org

PathPermutor Generate Contextual Directory Wordlist For Target Org This script generates contextual wordlist for any target org based on the set of UR

8 Jun 23, 2021

ICCV2021 - Mining Contextual Information Beyond Image for Semantic Segmentation

Introduction The official repository for "Mining Contextual Information Beyond Image for Semantic Segmentation". Our full code has been merged into ss

55 Nov 9, 2022

[2021 MultiMedia] CONQUER: Contextual Query-aware Ranking for Video Corpus Moment Retrieval

CONQUER: Contexutal Query-aware Ranking for Video Corpus Moment Retreival PyTorch implementation of CONQUER: Contexutal Query-aware Ranking for Video

23 Dec 26, 2022

Official implementation of NeurIPS 2021 paper "Contextual Similarity Aggregation with Self-attention for Visual Re-ranking"

CSA: Contextual Similarity Aggregation with Self-attention for Visual Re-ranking PyTorch training code for CSA (Contextual Similarity Aggregation). We

19 Oct 21, 2022

Official Pytorch implementation for Deep Contextual Video Compression, NeurIPS 2021

Introduction Official Pytorch implementation for Deep Contextual Video Compression, NeurIPS 2021 Prerequisites Python 3.8 and conda, get Conda CUDA 11

51 Dec 3, 2022

Code and data for ImageCoDe, a contextual vison-and-language benchmark

ImageCoDe This repository contains code and data for ImageCoDe: Image Retrieval from Contextual Descriptions. Data All collected descriptions for the

27 Dec 2, 2022

Comments

Adapting ABTest data to contextual bandit setting

Hi, and thanks for open sourcing this project.

I wanted to dive into it by testing some ABTesting data with the implemented neural bandit.

In my setting I have only 2 choices, 121 features as context, a reward range of [0.0, 120], and only 11% rows have non-zero reward. After training for a few epoch I see the testing loss decreasing a bit. But at test time, scores of the two choices are always equals, and the ucb_scores always equal to 0.

opened by virgile-blg 0
Model input dimension does not update when keeping top n features

Setting : Neural Bandit

When setting keep_only_top_n to True, the model keeps the original number of features, resulting in a Pytorch matmul error for the first linear layer:

RuntimeError: mat1 and mat2 shapes cannot be multiplied (256x10 and 121x64)

opened by virgile-blg 0

banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services.

Related tags

Overview

What's banditml?

Supported models

Docs

License

You might also like...

Repo for CVPR2021 paper "QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information"

Source code and data from the RecSys 2020 article "Carousel Personalization in Music Streaming Apps with Contextual Bandits" by W. Bendada, G. Salha and T. Bontempelli

UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus

Generate Contextual Directory Wordlist For Target Org

ICCV2021 - Mining Contextual Information Beyond Image for Semantic Segmentation

[2021 MultiMedia] CONQUER: Contextual Query-aware Ranking for Video Corpus Moment Retrieval

Official implementation of NeurIPS 2021 paper "Contextual Similarity Aggregation with Self-attention for Visual Re-ranking"

Official Pytorch implementation for Deep Contextual Video Compression, NeurIPS 2021

Code and data for ImageCoDe, a contextual vison-and-language benchmark

Comments

Adapting ABTest data to contextual bandit setting

Model input dimension does not update when keeping top n features

Releases(1.0.2)

1.0.2(Jun 4, 2021)

Owner

Bandit ML

An End-to-End Machine Learning Library to Optimize AUC (AUROC, AUPRC).

Cleaned up code for DSTC 10: SIMMC 2.0 track: subtask 2: multimodal coreference resolution

[BMVC 2021] Official PyTorch Implementation of Self-supervised learning of Image Scale and Orientation Estimation

Repositorio de los Laboratorios de Análisis Numérico / Análisis Numérico I de FAMAF, UNC.

Implementation of "With a Little Help from my Temporal Context: Multimodal Egocentric Action Recognition, BMVC, 2021" in PyTorch

A list of all named GANs!

Pytorch code for paper "Image Compressed Sensing Using Non-local Neural Network" TMM 2021.

Binary Passage Retriever (BPR) - an efficient passage retriever for open-domain question answering

FPSAutomaticAiming——基于YOLOV5的FPS类游戏自动瞄准AI

Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields.

Use .csv files to record, play and evaluate motion capture data.

DeLag: Detecting Latency Degradation Patterns in Service-based Systems

Memory Efficient Attention (O(sqrt(n)) for Jax and PyTorch

Code for "On the Effects of Batch and Weight Normalization in Generative Adversarial Networks"

A BaSiC Tool for Background and Shading Correction of Optical Microscopy Images

Semi-supervised Semantic Segmentation with Directional Context-aware Consistency (CVPR 2021)

A PyTorch-based library for fast prototyping and sharing of deep neural network models.

African language Speech Recognition - Speech-to-Text

Seeing Dynamic Scene in the Dark: High-Quality Video Dataset with Mechatronic Alignment (ICCV2021)

Sign Language Transformers (CVPR'20)