This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

Owner
신재욱
신재욱
It's a simple tool for test vulnerability shellshock

Shellshock, also known as Bashdoor, is a family of security bugs in the Unix Bash shell, the first of which was disclosed on 24 September 2014. Shellshock could enable an attacker to cause Bash to ex

Mr. Cl0wn - H4ck1ng C0d3r 88 Dec 23, 2022
NS-Defacer: a auto html injecter, In other words It's a auto defacer to deface a lot of websites in less time

Overview NS-Defacer is a auto html injecter, In other words It's a auto defacer

NightSec 10 Nov 19, 2022
APKLeaks - Scanning APK file for URIs, endpoints & secrets.

APKLeaks - Scanning APK file for URIs, endpoints & secrets.

dw1 3.5k Jan 09, 2023
A black hole for Internet advertisements

Network-wide ad blocking via your own Linux hardware The Pi-hole® is a DNS sinkhole that protects your devices from unwanted content, without installi

Pi-hole 40.3k Jan 09, 2023
CVE-2021-26855 SSRF Exchange Server

CVE-2021-26855 Brute Force EMail Exchange Server Timeline: Monday, March 8, 2021: Update Dumping content...(I'm not done, can u guy help me done this

lulz 117 Nov 28, 2022
An easy-to-use wrapper for NTFS-3G on macOS

ezNTFS ezNTFS is an easy-to-use wrapper for NTFS-3G on macOS. ezNTFS can be used as a menu bar app, or via the CLI in the terminal. Installation To us

Matthew Go 34 Dec 01, 2022
聚合Github上已有的Poc或者Exp,CVE信息来自CVE官网。Auto Collect Poc Or CVE from Github by CVE ID.

PocOrExp in Github 聚合Github上已有的Poc或者Exp,CVE信息来自CVE官网 注意:只通过通用的CVE号聚合,因此对于MS17-010等Windows编号漏洞以及著名的有绰号的漏洞,还是自己检索一下比较好 Usage python3 exp.py -h usage: ex

567 Dec 30, 2022
Uma ferramenta de segurança da informação escrita em python3,capaz de dar acesso total ao computador de alguém!

shell-reverse Uma ferramenta de segurança da informação escrita em python3, capaz de dar acesso total ao computador de alguém! A cybersecurity tool wr

Marcus Vinícius Ribeiro Andrade 1 Nov 03, 2021
A honeypot for the Log4Shell vulnerability (CVE-2021-44228)

Log4Pot A honeypot for the Log4Shell vulnerability (CVE-2021-44228). License: GPLv3.0 Features Listen on various ports for Log4Shell exploitation. Det

Thomas Patzke 79 Dec 27, 2022
Early days of an Asset Discovery tool.

Please star this project! Written in Python Report Bug . Request Feature DISCLAIMER This project is in its early days, everything you see here is almo

grag1337 3 Dec 20, 2022
This little tool is to calculate a MurmurHash value of a favicon to hunt phishing websites on the Shodan platform.

MurMurHash This little tool is to calculate a MurmurHash value of a favicon to hunt phishing websites on the Shodan platform. What is MurMurHash? Murm

Viral Maniar 87 Dec 31, 2022
This is a Crypto asset tracker that I built to aid my personal journey in cryptocurrencies.

Wallet Tracker This is a Crypto asset tracker that I built to aid my personal journey in cryptocurrencies. build docker build -t wallet-tracker . run

2 Mar 21, 2022
Cracker - Tools CRACK FACEBOOK DAN INSTAGRAM DENGAN FITUR BANYAK

CLOME TO TOOLS ME 😁 FITUR TOOLS RESULTS INSTALASI ____/-- INSTALLASI /+/+/+/ t

Jeeck X Nano 3 Jan 08, 2022
A hack for writing switch statements with type annotations in Python.

py_annotation_switch A hack for writing switch statements in type annotations for Python. Why should I use this? You most definitely should not use th

6 Oct 17, 2021
ProxyLogon Pre-Auth SSRF To Arbitrary File Write

ProxyLogon Pre-Auth SSRF To Arbitrary File Write For Education and Research Usage: C:\python proxylogon.py mail.evil.corp lulz 117 Nov 28, 2022

telegram bug that discloses user's hidden phone number (still unpatched) (exploit included)

CVE-2019-15514 Type: Information Disclosure Affected Users, Versions, Devices: All Telegram Users Still not fixed/unpatched. brute.py is available exp

Gray Programmerz 66 Dec 08, 2022
C++ fully undetected shellcode launcher

charlotte c++ fully undetected shellcode launcher ;) releasing this to celebrate the birth of my newborn description 13/05/2021: c++ shellcode launche

894 Dec 25, 2022
BurpSuite Extension: Log4j2 RCE Scanner

Log4j2 RCE Scanner 作者:[email protected]元亨实验室 声明:由于传播、利用本项目所提供的信息而造成的任何直接或者间接的后果及损失,均由使用者本人负责,项目作者不为此承担任何责

ᴋᴇʏ 87 Dec 29, 2021
Yara Based Detection Engine for web browsers

Yobi Yara Based Detection for web browsers System Requirements Yobi requires python3 and and right now supports only firefox and other Gecko-based bro

imp0rtp3 44 Nov 20, 2022
一款Web在线自动免杀工具

一款利用加载器以及Python反序列化绕过AV的在线免杀工具 因为打包方式的局限性,不能跨平台,若要生成exe格式的只能在Windows下运行本项目 打包速度有点慢,提交后稍等一会 开发环境及运行 前端使用Bootstrap框架,后端使用Django框架 。

yhy 172 Nov 28, 2022