Sharpened Cosine Similarity

A layer implementation for PyTorch

Install

At your command line:

git clone https://github.com/brohrer/sharpened_cosine_similarity_torch.git

You'll need to install or upgrade PyTorch if you haven't already. If python3 is the command you use to invoke Python at your command line:

python3 -m pip install torch torchvision --upgrade

Demo

Run the Fashion MNIST demo to see sharpened cosine similarity in action.

cd sharpened_cosine_similarity_torch
python3 demo_fashion_mnist.py

When you run this it will take a few extra minutes the first time through to download and extract the Fashion MNIST data set. Its less than 100MB when fully extracted.

I run this entirely on laptop CPUs. I have a dual-core i7 that takes about 90 seconds per epoch and an 8-core i7 that takes about 45 seconds per epoch. Your mileage may vary.

Monitor

You can check on the status of your runs at any time. In another console navigate to the smae directory and run

python3 show_results.py

This will give a little console summary like this

testing errors for version test
mean  : 14.08%
stddev: 0.1099%
stderr: 0.03887%
n runs: 8

and drop a couple of plots like this in the plots directory showing how the classification error on the test data set decreases with each pass through the training data set.

The demo will keep running for a long time if you let it. Kill it when you get bored of it. If you want to pick the sequence of runs back up, re-run the demo and it will load all the results it's generated so far and append to them.

Track

If you'd like to experiment with the sharpened cosine similarity code, the demo, or with other data sets, you can keep track of each new run by adding a version argument at the command line.

To start a run with version string "v37" run

python3 demo_fashion_mnist.py v37

To check on its progress

python3 show_results.py v37

The version string can be arbitrarily descriptive, for example "3_scs_layer_2_fully_connected_layer_learning_rate_003", but keep it alphanumeric with underscores.

Credit where it's due

Based on and copy/pasted heavily from code from Ze Wang and code from Oliver Batchelor and the TensorFlow implementation and blog post from Raphael Pisoni.