Awesome Semantic Segmentation

List of awesome things around semantic segmentation 🎉

Semantic segmentation is a computer vision task in which we label specific regions of an image according to what's being shown. Semantic segmentation awswers for the question: "What's in this image, and where in the image is it located?".

Semantic segmentation is a critical module in robotics related applications, especially autonomous driving, remote sensing. Most of the research on semantic segmentation is focused on improving the accuracy with less attention paid to computationally efficient solutions.

The recent appoarch in semantic segmentation is using deep neural network, specifically Fully Convolutional Network (a.k.a FCN). We can follow the trend of semantic segmenation approach at: paper-with-code.

Evaluate metrics: mIOU, accuracy, speed,...

State-Of-The-Art (SOTA) methods of Semantic Segmentation

Variants

• FCN with VGG(Resnet, Densenet) backbone: pytorch
• The easiest implementation of fully convolutional networks (FCN8s VGG): pytorch
• TernausNet (UNet model with VGG11 encoder pre-trained on Kaggle Carvana dataset paper: pytorch
• TernausNetV2: Fully Convolutional Network for Instance Segmentation: pytorch

Review list of Semantic Segmentation

• Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey 2020 (University of Gour Banga,India) ⭐ ⭐ ⭐ ⭐ ⭐
• A peek of Semantic Segmentation 2018 (mc.ai) ⭐ ⭐ ⭐ ⭐
• Semantic Segmentation guide 2018 (towardds) ⭐ ⭐ ⭐ ⭐
• An overview of semantic image segmentation (jeremyjordan.me) ⭐ ⭐ ⭐ ⭐ ⭐
• Recent progress in semantic image segmentation 2018 (arxiv, towardsdatascience) ⭐ ⭐ ⭐ ⭐
• A 2017 Guide to Semantic Segmentation Deep Learning Review (blog.qure.ai) ⭐ ⭐ ⭐ ⭐ ⭐
• Review popular network architecture (medium-towardds) ⭐ ⭐ ⭐ ⭐ ⭐
• Lecture 11 - Detection and Segmentation - CS231n (slide, vid): ⭐ ⭐ ⭐ ⭐ ⭐
• A Survey of Semantic Segmentation 2016 (arxiv) ⭐ ⭐ ⭐ ⭐ ⭐

Case studies

• Dstl Satellite Imagery Competition, 3rd Place Winners' Interview: Vladimir & Sergey: Blog, Code
• Carvana Image Masking Challenge–1st Place Winner's Interview: Blog, Code
• Data Science Bowl 2017, Predicting Lung Cancer: Solution Write-up, Team Deep Breath: Blog
• MICCAI 2017 Robotic Instrument Segmentation: Code and explain
• 2018 Data Science Bowl Find the nuclei in divergent images to advance medical discovery: 1st place, 2nd, 3rd, 4th, 5th, 10th
• Airbus Ship Detection Challenge: 4th place, 6th

Most used loss functions

• Pixel-wise cross entropy loss:
• Dice loss: which is pretty nice for balancing dataset
• Focal loss:
• Lovasz-Softmax loss:

Datasets

• Visual Object Classes Challenge 2012 (VOC2012): 400+ classes of real-world data
• COCO Dataset: 164k images, 72 classes: 80 thing classes, 91 stuff classes and 1 class 'unlabeled'
• Cityscapes: This dataset consists of segmentation ground truths for roads, lanes, vehicles and objects on road. The dataset contains 30 classes and of 50 cities collected over different environmental and weather conditions
• PASCAL-Context
• ADE20K: 20k+ images
• Semantic3d
• CamVid
• lartpang/awesome-segmentation-saliency-dataset
• Kaggle

Frameworks for segmentation

• Semantic Segmentation in PyTorch (by yassouali): Semantic segmentation models, datasets and losses implemented in PyTorch.
• Semantic Segmentation Suite (by George Seif): Semantic Segmentation Suite in TensorFlow. Implement, train, and test new Semantic Segmentation models easily!
• Segmentation Training Pipeline: Research Pipeline for image masking/segmentation in Keras
• Tramac/awesome-semantic-segmentation-pytorch Semantic Segmentation on PyTorch (include FCN, PSPNet, Deeplabv3, Deeplabv3+, DANet, DenseASPP, BiSeNet, EncNet, DUNet, ICNet, ENet, OCNet, CCNet, PSANet, CGNet, ESPNet, LEDNet, DFANet)
• CSAILVision/semantic-segmentation-pytorch Pytorch implementation for Semantic Segmentation/Scene Parsing on MIT ADE20K dataset
• divamgupta/image-segmentation-keras Implementation of Segnet, FCN, UNet , PSPNet and other models in Keras.

Related techniques

• Atrous/ Dilated Convolution
• Transpose Convolution (Deconvolution, Upconvolution)
• Unpooling
• A technical report on convolution arithmetic in the context of deep learning
• CRF

Feel free to show your ❤️ by giving a star ⭐

🎁 Check Out the List of Contributors - Feel free to add your details here!