Rethinking Nearest Neighbors for Visual Classification

arXiv

Environment settings

Check out scripts/env_setup.sh

Setup data

Download the following fine-grained datasets and ImageNet.

• Oxford Flowers
• CUB200 2011
• Stanford Dogs
• Stanford Cars

In current version, you need to modify each data file under knn/data/finetune/*.py

Experiments

The numerical experiment results with corresponding hyper-parameters can be found here:

• Natural world binary classification: linear-eval
• Fine-grained object classification: linear-eval, fine-tune
• ImageNet classification: linear-eval

To use the code in this repo, here are some key configs:

• DATA.FEATURE: specify which representation to use. FEATURES.md includes more details
• DATA.BATCH_SIZE: ViT-based backbone requires a smaller batchsize
• RUN_N_TIMES: ensure only run once in case duplicated submision
• MODEL.TYPE: base or joint training
• OUTPUT_DIR: output dir of the final model and logs
• SOLVER.BASE_LR: learning rate for the experiment
• SOLVER.WEIGHT_DECAY: weight decay value for the experiment
• MODEL.KNN_LAMBDA: alpha in Eq 4

Linear evaluation

See script/run_linear.sh and script/run_newt.sh

End-to-end finetuning

See script/run_finetune.sh

License

This repo are released under the CC-BY-NC 4.0 license. See LICENSE for additional details.

Acknowledgement

We thank the researchers who propose NEWT for providing the features for the datasets.