The aim of this task is to predict someone's English proficiency based on a text input.

Last update: Dec 13, 2021

Overview

English_proficiency_prediction_NLP

The aim of this task is to predict someone's English proficiency based on a text input.

Using the The NICT JLE Corpus available here : https://alaginrc.nict.go.jp/nict_jle/index_E.html

The source of the corpus data is the transcripts of the audio-recorded speech samples of 1,281 participants (1.2 million words, 300 hours in total) of English oral proficiency interview test. Each participant got a SST (Standard Speaking Test) score between 1 (low proficiency) and 9 (high proficiency) based on this test.

The goal is to build a machine learning algorithm for predicting the SST score of each participant based on their transcript.

Steps:

1 - Pre-process the dataset: extract the participant transcript (all tags). Inside participant transcript, you can remove all other tags and extract only English words.

2 - Process the dataset: extract features with the Bag of Word (BoW) technique

3 - Train a classifier to predict the SST score

4 - Compute the accuracy of your system (the number of participant classified correctly) and plot the confusion matrix.

5 - Try to improve your system (for example you can try to use GloVe instead of BoW).

The aim of this task is to predict someone's English proficiency based on a text input.

Related tags

Overview

English_proficiency_prediction_NLP

Owner

Just a basic Telegram AI chat bot written in Python using Pyrogram.

A Survey of Natural Language Generation in Task-Oriented Dialogue System (TOD): Recent Advances and New Frontiers

Implementation for paper BLEU: a Method for Automatic Evaluation of Machine Translation

A Pytorch implementation of "Splitter: Learning Node Representations that Capture Multiple Social Contexts" (WWW 2019).

This repository details the steps in creating a Part of Speech tagger using Trigram Hidden Markov Models and the Viterbi Algorithm without using external libraries.

PhoNLP: A BERT-based multi-task learning toolkit for part-of-speech tagging, named entity recognition and dependency parsing

Source code and dataset for ACL 2019 paper "ERNIE: Enhanced Language Representation with Informative Entities"

A simple implementation of N-gram language model.

Use Google's BERT for named entity recognition （CoNLL-2003 as the dataset）.

PyTranslator é simultaneamente um editor e tradutor de texto com diversos recursos e interface feito com coração e 100% em Python

A very simple framework for state-of-the-art Natural Language Processing (NLP)

Maha is a text processing library specially developed to deal with Arabic text.

texlive expressions for documents

Contains links to publicly available datasets for modeling health outcomes using speech and language.

Codes for processing meeting summarization datasets AMI and ICSI.

Official PyTorch code for ClipBERT, an efficient framework for end-to-end learning on image-text and video-text tasks

MASS: Masked Sequence to Sequence Pre-training for Language Generation

Creating a python chatbot that Starbucks users can text to place an order + help cut wait time of a normal coffee.

topic modeling on unstructured data in Space news articles retrieved from the Guardian (UK) newspaper using API

BERN2: an advanced neural biomedical namedentity recognition and normalization tool