🌾 PASTIS 🌾 Panoptic Agricultural Satellite TIme Series (optical and radar)

The PASTIS Dataset

• Dataset presentation

PASTIS is a benchmark dataset for panoptic and semantic segmentation of agricultural parcels from satellite time series. It contains 2,433 patches within the French metropolitan territory with panoptic annotations (instance index + semantic labelfor each pixel). Each patch is a Sentinel-2 multispectral image time series of variable lentgh.

We propose an official 5 fold split provided in the dataset's metadata, and evaluated several of the top-performing image time series networks. You are welcome to use our numbers and to submit your own entries to the leaderboard!

• Dataset in numbers

• 🔥 NEW: Radar extension (PASTIS-R)

We also propose an extended version of PASTIS which contains all radar observations of Sentinel-1 for all 2433 patches in addition to the Sentinel-2 images. For each patch, approximately 70 observations of Sentinel-1 in ascending orbit, and 70 observations in descending orbit are added to the dataset. The PASTIS-R extension can thus be used to evaluate optical-radar fusion methods for parcel-based classification, semantic segmentation, and panoptic segmentation.
For more details on PASTIS-R, refer to our recent paper on multi-modal fusion with attention-based models (link coming soon).

Usage

• Download

The dataset can be downloaded from zenodo in different formats:

1. PASTIS (29 GB zipped) : The original PASTIS dataset for semantic and panoptic segmentation on Sentinel-2 time series (format used for the ICCV 2021 paper).
2. PASTIS-R (54 GB zipped) : The extended version with Sentinel-1 observations.
3. PASTIS-R (pixel-set format) (27 GB zipped) : The PASTIS-R dataset prepared in pixel-set format for parcel-based classification only. See this repo and paper for more details on this format.

• Data loading

This repository also contains a PyTorch dataset class in code/dataloader.py that can be readily used to load data for training models on PASTIS and PASTIS-R. For the pixel-set dataset, use the dataloader in code/dataloader_pixelset.py. The time series contained in PASTIS have variable lengths. The code/collate.py contains a pad_collate function that you can use in the pytorch dataloader to temporally pad shorter sequences. The demo.ipynb notebook shows how to use these classes and methods to load data from PASTIS.

• Metrics

A PyTorch implementation is also given in code/panoptic_metrics.py to compute the panoptic metrics. In order to use these metrics, the model's output should contain an instance prediction and a semantic prediction. The first one allocates an instance id to each pixel of the image, and the latter a semantic label.

Leaderboard

Please open an issue to submit new entries. Do mention if the work has been published and wether the code accessible for reproducibility. We require that at least a preprint is available to present the method used.

Semantic Segmentation

Optical only (PASTIS)

Optical+Radar fusion (PASTIS-R)

Panoptic Segmentation

Optical only (PASTIS)

Optical+Radar fusion (PASTIS-R)

Documentation

The agricultural parcels are grouped into 18 different crop classes as shown in the table below. The backgroud class corresponds to non-agricultural land, and the void label for parcels that are mostly outside their patch.

Additional information about the dataset can be found in the documentation/pastis-documentation.pdf document.

References

If you use PASTIS please cite the related paper:

@article{garnot2021panoptic,
  title={Panoptic Segmentation of Satellite Image Time Series
with Convolutional Temporal Attention Networks},
  author={Sainte Fare Garnot, Vivien  and Landrieu, Loic },
  journal={ICCV},
  year={2021}
}

For the PASTIS-R optical-radar fusion dataset, please also cite this paper:

@article{garnot2021mmfusion,
  title={Multi-Modal Temporal Attention Models for Crop Mapping from Satellite Time Series},
  author={Sainte Fare Garnot, Vivien  and Landrieu, Loic and Chehata, Nesrine },
  journal={arxiv},
  year={2021}
}

Credits

• The satellite imagery used in PASTIS was retrieved from THEIA: "Value-added data processed by the CNES for the Theia www.theia.land.fr data cluster using Copernicus data. The treatments use algorithms developed by Theia’s Scientific Expertise Centres. "

• The annotations used in PASTIS stem from the French land parcel identification system produced by IGN, the French mapping agency.

• This work was partly supported by ASP, the French Payment Agency.

• We also thank Zenodo for hosting the datasets.