Human Action Controller - A human action controller running on different platforms.

Related tags

Deep Learninghac
Overview

Human Action Controller (HAC)

Goal

A human action controller running on different platforms.

Fun Easy-to-use
Accurate Anywhere

Fun Examples

Mouse Control

Mouse Control

Keyboard Control

Keyboard Control

Playing Game

Pikachu

Enhancing interaction

Gather Town

Solutions provided by HAC

Platform Module Progress Comment
PC / Win10 Mouse Control V
PC / Win10 Keyboard Control V
PC / Ubuntu Mouse Control
PC / Ubuntu Keyboard Control

Getting started

Installation

$ pip install pyhac

Run the demo of mouse control

$ git clone https://github.com/dabit-lucas/hac.git
$ cd hac
$ python demo.py

Recording custom actions

$ python recording.py -d {action name} -k True

Press key "r" to start recording, the data will be saved into ./data

Training a custom module

Here is an example of a config file of action set,

{
    "actions": [
        "r_five",
        "r_zero",
        "l_five",
        "l_zero",
        "two_index_fingers_up",
        "two_index_fingers_down",
        "33",
        "55",
        "sit"
    ],
    "type": "gesture_only"
}

These actions form a model by running a training process:

$ python train.py --conf {path_of_action} --model_name {name_of_model}

The generated model will become a module. Take mouse control module as an exmaple, it can create mappings among actions and controls by the following code:

mouse_module = hac.add_module("mouse_control")
hac.set_init_module(mouse_module)

# create mapping between controls and actions
mouse_module.add_mouse_mapping("mouse_left_down", ["r_five", "r_zero"])
mouse_module.add_mouse_mapping("mouse_left_up", "r_five")
mouse_module.add_mouse_mapping("mouse_right_down", ["l_five", "l_zero"])
mouse_module.add_mouse_mapping("mouse_right_up", "l_five")
mouse_module.add_mouse_mapping("right_move_diff", ["r_five", "r_five"])
mouse_module.add_mouse_mapping("right_move_diff", ["r_zero", "r_zero"])
mouse_module.add_mouse_mapping("left_move_diff", ["l_five", "l_five"])
mouse_module.add_mouse_mapping("left_move_diff", ["l_zero", "l_zero"])
mouse_module.add_mouse_mapping("roll_up", "two_index_fingers_up")
mouse_module.add_mouse_mapping("roll_down", "two_index_fingers_down")

If the five gesture with a right hand shows in consecutive two frames ["r_five", "r_five"], then do control right_move_diff, which means moving the mouse cursor. The above description can be represented by the following code:

mouse_module.add_mouse_mapping("right_move_diff", ["r_five", "r_five"])

Development guideline

The structure of HAC

Community

Welcome to ask any question in issues.

Contributing

Any contribution is welcomed. Please fork this repo and summit a pull request.

Owner
GitHub Repository
GB-CosFace: Rethinking Softmax-based Face Recognition from the Perspective of Open Set Classification

GB-CosFace: Rethinking Softmax-based Face Recognition from the Perspective of Open Set Classification This is the official pytorch implementation of t

Alibaba Cloud 5 Nov 14, 2022
ZEBRA: Zero Evidence Biometric Recognition Assessment

ZEBRA: Zero Evidence Biometric Recognition Assessment license: LGPLv3 - please reference our paper version: 2020-06-11 author: Andreas Nautsch (EURECO

Voice Privacy Challenge 2 Dec 12, 2021
A Simple Example for Imitation Learning with Dataset Aggregation (DAGGER) on Torcs Env

Imitation Learning with Dataset Aggregation (DAGGER) on Torcs Env This repository implements a simple algorithm for imitation learning: DAGGER. In thi

Hao 66 Nov 23, 2022
Collaborative forensic timeline analysis

Timesketch Table of Contents About Timesketch Getting started Community Contributing About Timesketch Timesketch is an open-source tool for collaborat

Google 2.1k Dec 28, 2022
Semi-supervised Domain Adaptation via Minimax Entropy

Semi-supervised Domain Adaptation via Minimax Entropy (ICCV 2019) Install pip install -r requirements.txt The code is written for Pytorch 0.4.0, but s

Vision and Learning Group 243 Jan 09, 2023
Mitsuba 2: A Retargetable Forward and Inverse Renderer

Mitsuba Renderer 2 Documentation Mitsuba 2 is a research-oriented rendering system written in portable C++17. It consists of a small set of core libra

Mitsuba Physically Based Renderer 2k Jan 07, 2023
Step by Step on how to create an vision recognition model using LOBE.ai, export the model and run the model in an Azure Function

Step by Step on how to create an vision recognition model using LOBE.ai, export the model and run the model in an Azure Function

El Bruno 3 Mar 30, 2022
STBP is a way to train SNN with datasets by Backward propagation.

Spiking neural network (SNN), compared with depth neural network (DNN), has faster processing speed, lower energy consumption and more biological interpretability, which is expected to approach Stron

Ling Zhang 18 Dec 09, 2022
Fibonacci Method Gradient Descent

An implementation of the Fibonacci method for gradient descent, featuring a TKinter GUI for inputting the function / parameters to be examined and a matplotlib plot of the function and results.

Emma 1 Jan 28, 2022
Multi-modal co-attention for drug-target interaction annotation and Its Application to SARS-CoV-2

CoaDTI Multi-modal co-attention for drug-target interaction annotation and Its Application to SARS-CoV-2 Abstract Environment The test was conducted i

Layne_Huang 7 Nov 14, 2022
Seq2seq - Sequence to Sequence Learning with Keras

Seq2seq Sequence to Sequence Learning with Keras Hi! You have just found Seq2Seq. Seq2Seq is a sequence to sequence learning add-on for the python dee

Fariz Rahman 3.1k Dec 18, 2022
Official code repository for A Simple Long-Tailed Rocognition Baseline via Vision-Language Model.

This is the official code repository for A Simple Long-Tailed Rocognition Baseline via Vision-Language Model.

peng gao 42 Nov 26, 2022
Bare bones use-case for deploying a containerized web app (built in streamlit) on AWS.

Containerized Streamlit web app This repository is featured in a 3-part series on Deploying web apps with Streamlit, Docker, and AWS. Checkout the blo

Collin Prather 62 Jan 02, 2023
GNN-based Recommendation Benchmark

GRecX A Fair Benchmark for GNN-based Recommendation Homepage and Documentation Homepage: Documentation: Paper: GRecX: An Efficient and Unified Benchma

73 Oct 17, 2022
Tackling Obstacle Tower Challenge using PPO & A2C combined with ICM.

Obstacle Tower Challenge using Deep Reinforcement Learning Unity Obstacle Tower is a challenging realistic 3D, third person perspective and procedural

Zhuoyu Feng 5 Feb 10, 2022
Style-based Point Generator with Adversarial Rendering for Point Cloud Completion (CVPR 2021)

Style-based Point Generator with Adversarial Rendering for Point Cloud Completion (CVPR 2021) An efficient PyTorch library for Point Cloud Completion.

Microsoft 119 Jan 02, 2023
Source code and data from the RecSys 2020 article "Carousel Personalization in Music Streaming Apps with Contextual Bandits" by W. Bendada, G. Salha and T. Bontempelli

Carousel Personalization in Music Streaming Apps with Contextual Bandits - RecSys 2020 This repository provides Python code and data to reproduce expe

Deezer 48 Jan 02, 2023
A machine learning benchmark of in-the-wild distribution shifts, with data loaders, evaluators, and default models.

WILDS is a benchmark of in-the-wild distribution shifts spanning diverse data modalities and applications, from tumor identification to wildlife monitoring to poverty mapping.

P-Lambda 437 Dec 30, 2022
Covid19-Forecasting - An interactive website that tracks, models and predicts COVID-19 Cases

Covid-Tracker This is an interactive website that tracks, models and predicts CO

Adam Lahmadi 1 Feb 01, 2022
Official PyTorch code for Mutual Affine Network for Spatially Variant Kernel Estimation in Blind Image Super-Resolution (MANet, ICCV2021)

Mutual Affine Network for Spatially Variant Kernel Estimation in Blind Image Super-Resolution (MANet, ICCV2021) This repository is the official PyTorc

Jingyun Liang 139 Dec 29, 2022