• only with multiple labeling functions (LFs)², i.e. without any labeled training data!
• in an end-to-end manner, i.e. directly train and evaluate your neural net (end-model from here on), there's no need to train a separate label model any more as in Snorkel & co,
• with better test set performance and enhanced robustness against correlated or inaccurate LFs than prior methods like Snorkel

¹ This includes learning from crowdsourced labels or annotations!
² LFs are labeling heuristics, that output noisy labels for (subsets of) the training data (e.g. crowdworkers or keyword detectors).

Getting Started

This library assumes familiarity with (multi-source) weak supervision, if that's not the case you may want to first learn its basics in e.g. this overview slides from Stanford or this Snorkel tutorial.

That being said, have a look at our examples and the notebooks therein showing you how to use Weasel for your own dataset, LF set, or end-model. E.g.:

• A high-level starter tutorial, with few code, many explanations and including Snorkel as a baseline (so that if you are familiar with Snorkel you can see the similarities and differences to Weasel).

• See how the whole WeaSEL pipeline works with all details, necessary steps and definitions for a new dataset & custom end-model. This notebook will probably make you learn the most about WeaSEL and how to apply it to your own problem.

• A realistic ML experiment script with all that's part of a ML pipeline, including logging to Weight&Biases, arbitrary callbacks, and eventually retrieving your fully trained end-model.

Reproducibility

Please have a look at the research code branch, which operates on pure PyTorch.

Installation

conda create --name weasel python=3.7  # or other python version >=3.7
conda activate weasel  

python -m pip install git+https://github.com/autonlab/weasel#egg=weasel[all]

git clone https://github.com/autonlab/weasel.git
cd weasel
pip install -e .[all]

python -m pip install git+https://github.com/autonlab/weasel

Note: Weasel is under active development, some uncovered edge cases might exist, and any feedback is very welcomed!

Apply WeaSEL to your own problem

Configuration with Hydra

Optional: This template config will help you get started with your own application, an analogous config is used in this tutorial script that you may want to check out.

Pre-defined or custom downstream models & Baselines

Please have a look at the detailed instructions in this Readme.

Using your own dataset and/or labeling heuristics

Please have a look at the detailed instructions in this Readme.

Citation

@article{cachay2021endtoend,
  author={R{\"u}hling Cachay, Salva and Boecking, Benedikt and Dubrawski, Artur},
  journal={Advances in Neural Information Processing Systems}, 
  title={End-to-End Weak Supervision},
  year={2021}
}