SALOD

Source code of our work: "Benchmarking Deep Models for Salient Object Detection".
In this works, we propose a new benchmark for SALient Object Detection (SALOD) methods.

We re-implement 14 methods using same settings, including input size, data loader and evaluation metrics (thanks to Metrics). Hyperparameters of optimizer are different because of various network structures and objective functions. We try our best to tune the optimizer for these models to achieve the best performance one-by-one. Some other networks are debugging now, it is welcome for your contributions on these networks to obtain better performance.

Properties

1. A unify interface for new models. To develop a new network, you only need to 1) set configs; 2) define network; 3) define loss function. See methods/template.
2. We build a new dataset by collecting several prevalent datasets in SOD task.
3. Easy to adopt different backbones (Available backbones: ResNet-50, VGG-16, MobileNet-v2, EfficientNet-B0, GhostNet, Res2Net)
4. Testing all networks on your own device. By input the name of network, you can test all available methods in our benchmark. Comparisons includes FPS, GFLOPs, model size and multiple effectiveness metrics.
5. We implement a loss factory that you can change the loss functions using command line parameters.

Available Methods:

Usage

# model_name: lower-cased method name. E.g. poolnet, egnet, gcpa, dhsnet or minet.
python3 train.py model_name --gpus=0

python3 test.py model_name --gpus=0 --weight=path_to_weight 

python3 test_fps.py model_name --gpus=0

# To evaluate generated maps:
python3 eval.py --pre_path=path_to_maps

Results

We report benchmark results here.
More results please refer to Reproduction, Few-shot and Generalization.

Notice: please contact us if you get better results.

VGG16-based:

ResNet50-based:

Create New Model

To create a new model, you can copy the template folder and modify it as you want.

cp -r ./methods/template ./methods/new_name

More details please refer to python files in template floder.

Loss Factory

We supply a Loss Factory for an easier way to tune the loss functions. You can set --loss and --lw parameters to use it.

Here are some examples:

loss_dict = {'b': BCE, 's': SSIM, 'i': IOU, 'd': DICE, 'e': Edge, 'c': CTLoss}

python train.py ... --loss=bd
# loss = 1 * bce_loss + 1 * dice_loss

python train.py ... --loss=bs --lw=0.3,0.7
# loss = 0.3 * bce_loss + 0.7 * ssim_loss

python train.py ... --loss=bsid --lw=0.3,0.1,0.5,0.2
# loss = 0.3 * bce_loss + 0.1 * ssim_loss + 0.5 * iou_loss + 0.2 * dice_loss