LPAQA: Language model Prompt And Query Archive

This repository contains data and code for the paper How Can We Know What Language Models Know?

Install

Our repository is based on LAMA. Please download our fork from here and follow the instructions to set up the environment and download pre-trained language models.

# clone LPAQA
git clone https://github.com/jzbjyb/LPAQA.git LPAQA
# clone LAMA
git clone https://github.com/jzbjyb/LAMA.git LAMA
pushd LAMA && git reset --hard b6b1885c64de5981f249a8b65de25cb0802b4bd5 && rm -rf .git && popd
mv LAMA/* LPAQA/ && rm -rf LAMA
# follow the instructions to install
cd LPAQA
./setup.sh

Retrieve factual knowledge from LMs

For example, to query the owner of MSN (Microsoft is the answer), you can either use manually created prompts (x is owned by y):

python lama/eval_generation.py --lm bert --t "MSN is owned by [MASK]."

or use LPAQA that ensembles a diversity of prompts:

# mined prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/mine
# paraphrased prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/paraphrase

Language model Prompt And Query Archive

Related tags

Overview

LPAQA: Language model Prompt And Query Archive

Install

Retrieve factual knowledge from LMs

Owner

Amazing-Python-Scripts - 🚀 Curated collection of Amazing Python scripts from Basics to Advance with automation task scripts.

A Simple Key-Value Data-store written in Python

ZeroGen: Efficient Zero-shot Learning via Dataset Generation

Hl classification bc - A Network-Based High-Level Data Classification Algorithm Using Betweenness Centrality

PyBrain - Another Python Machine Learning Library.

Code for ViTAS_Vision Transformer Architecture Search

MonoRCNN is a monocular 3D object detection method for automonous driving

Rotary Transformer

[CVPR 2022 Oral] Versatile Multi-Modal Pre-Training for Human-Centric Perception

A benchmark dataset for emulating atmospheric radiative transfer in weather and climate models with machine learning (NeurIPS 2021 Datasets and Benchmarks Track)

The implementation of 'Image synthesis via semantic composition'.

Code for "Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo"

DI-HPC is an acceleration operator component for general algorithm modules in reinforcement learning algorithms

Real time Human Detection Counting

Black-Box-Tuning - Black-Box Tuning for Language-Model-as-a-Service

Implementation of the ICCV'21 paper Temporally-Coherent Surface Reconstruction via Metric-Consistent Atlases

Unofficial pytorch implementation of 'Image Inpainting for Irregular Holes Using Partial Convolutions'

Create animations for the optimization trajectory of neural nets

Repo for CVPR2021 paper "QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information"

Cancer Drug Response Prediction via a Hybrid Graph Convolutional Network