Hyperspectral-Image-Super-Resolution-Benchmark

A list of hyperspectral image super-resolution resources collected by Junjun Jiang. If you find that important resources are not included, please feel free to contact me.

According to whether or not to use auxiliary information (PAN image/RGB image/multispectral images), hyperspectral image super-resolution techniques can be divided into two classes: hyperspectral image super-resolution (fusion) and single hyperspectral image super-resolution. The former could be roughly categorized as follows: 1) Bayesian based approaches, 2) Tensor based approaches, 3) Matrix factorization based approaches, and 4) Deep Learning based approaches.

================================================================================

Pioneer Work and Technique Review

• Unmixing based multisensor multiresolution image fusion, TGRS1999, B. Zhukov et al.

• Application of the stochastic mixing model to hyperspectral resolution enhancement, TGRS2004, M. T. Eismann et al.

• Resolution enhancement of hyperspectral imagery using maximum a posteriori estimation with a stochastic mixing model, Ph.D. dissertation, 2004, M. T. Eismann et al.

• MAP estimation for hyperspectral image resolution enhancement using an auxiliary sensor, TIP2004, R. C. Hardie et al.

• Hyperspectral resolution enhancement using high-resolution multispectral imagery with arbitrary response functions, TGRS2005, M. T. Eismann et al.

• Hyperspectral pansharpening: a review. GRSM2015, L. Loncan et al. [PDF] [Code]

• Hyperspectral and multispectral data fusion: A comparative review of the recent literature, GRSM2017, N. Yokoya,et al. [PDF] [Code]

================================================================================

Hyperspectral Image Super-Resolution (Fusion)

1) Bayesian based approaches

• Blind Image Fusion for Hyperspectral Imaging with the Directional Total Variation, Inverse Problems, 2018, Leon Bungert et al. [PDF] [Code]

• Bayesian sparse representation for hyperspectral image super resolution, CVPR2015, N. Akhtar et al. [PDF] [Code]

• Hysure: A convex formulation for hyperspectral image superresolution via subspace-based regularization, TGRS2015, M. Simoes et al. [PDF] [Code]

• Hyperspectral and multispectral image fusion based on a sparse representation, TGRS2015, Q. Wei et al. [PDF] [Code]

• Bayesian fusion of multi-band images, Jstar2015, W. Qi et al. [PDF] [Code]

• Noise-resistant wavelet-based Bayesian fusion of multispectral and hyperspectral images, TGRS2009, Y. Zhang et al. [PDF]

• Weighted Low-rank Tensor Recovery for Hyperspectral Image Restoration, arXiv2018, Yi Chang et al. [PDF]

2) Tensor based approaches

• Hyperspectral image superresolution via non-local sparse tensor factorization, CVPR2017, R. Dian et al. [PDF]

• Spatial–Spectral-Graph-Regularized Low-Rank Tensor Decomposition for Multispectral and Hyperspectral Image Fusion, Jstars2018, K. Zhang et al. [PDF]

• Fusing Hyperspectral and Multispectral Images via Coupled Sparse Tensor Factorization, TIP2108, S. Li et al. [PDF] [Code]

• Hyperspectral Super-Resolution: A Coupled Tensor Factorization Approach, arXiv2018, Charilaos I. Kanatsoulis et al. [PDF]

• Nonlocal Patch Tensor Sparse Representation for Hyperspectral Image Super-Resolution, TIP2019, Yang Xu et al. [PDF] [Web]

• Learning a Low Tensor-Train Rank Representation for Hyperspectral Image Super-Resolution, TNNLS2019, Renwei Dian et al. [PDF] [Web]

• Nonnegative and Nonlocal Sparse Tensor Factorization-Based Hyperspectral Image Super-Resolution, IEEE TGRS2020, Wei Wan et al. [PDF]

• Nonlocal Coupled Tensor CP Decomposition for Hyperspectral and Multispectral Image Fusion, IEEE TGRS2020, Xu Yang et al. [PDF]

• Hyperspectral Super-Resolution via Coupled Tensor Ring Factorization, IEEE TGRS2020, Wei He et al. [PDF]

• Spatial-Spectral Structured Sparse Low-Rank Representation for Hyperspectral Image Super-Resolution, IEEE TIP2021, Jize Xue et al., [PDF]

3) Matrix factorization based approaches

• High-resolution hyperspectral imaging via matrix factorization, CVPR2011, R. Kawakami et al. [PDF] [Code]

• Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion, TGRS2012, N. Yokoya et al. [PDF] [Code]

• Sparse spatio-spectral representation for hyperspectral image super-resolution, ECCV2014, N. Akhtar et al. [PDF] [Code]

• Hyper-sharpening: A first approach on SIM-GA data, Jstars2015, M. Selva et al.

• Hyperspectral super-resolution by coupled spectral unmixing, ICCV2015, C Lanaras. [PDF] [Code]

• RGB-guided hyperspectral image upsampling, CVPR2015, H. Kwon et al. [PDF] [Code]

• Multiband image fusion based on spectral unmixing, TGRS2016, Q. Wei et al. [PDF] [Code]

• Hyperspectral image super-resolution via non-negative structured sparse representation, TIP2016, W. Dong, et al. [PDF] [Code]

• Hyperspectral super-resolution of locally low rank images from complementary multisource data, TIP2016, M. A. Veganzones et al. [PDF]

• Multispectral and hyperspectral image fusion based on group spectral embedding and low-rank factorization, TGRS2017, K. Zhang et al.

• Hyperspectral Image Super-Resolution Based on Spatial and Spectral Correlation Fusion, TRGS2018, C. Yi et al.

• Self-Similarity Constrained Sparse Representation for Hyperspectral Image Super-Resolution, TIP2108, X. Han et al.

• Exploiting Clustering Manifold Structure for Hyperspectral Imagery Super-Resolution, TIP2018, L. Zhang et al. [Code]

• Hyperspectral Image Super-Resolution With a Mosaic RGB Image, TIP2018, Y. Fu et al. [PDF]

• Fusing Hyperspectral and Multispectral Images via Coupled Sparse Tensor Factorization, TIP2018, S. Li et al. [PDF][Code]

• Multispectral Image Super-Resolution via RGB Image Fusion and Radiometric Calibration, TIP2019, Zhi-Wei Pan et al. [PDF] [Web]

• Hyperspectral Image Super-resolution via Subspace-Based Low Tensor Multi-Rank Regularization, TIP2019, Renwei Dian et al. [PDF]

• Hyperspectral Image Super-Resolution With Optimized RGB Guidance, Ying Fu et al., CVPR2019. [PDF]

• Super-Resolution for Hyperspectral and Multispectral Image Fusion Accounting for Seasonal Spectral Variability, TIP2020, R.A. Borsoi et al. [PDF]

• A Truncated Matrix Decomposition for Hyperspectral Image Super-Resolution, TIP2020, Jianjun Liu et al. [PDF]

4) Deep Learning based approaches

• Deep Residual Convolutional Neural Network for Hyperspectral Image Super-Resolution, ICIG2017, C. Wang et al. [PDF]

• SSF-CNN: Spatial and Spectral Fusion with CNN for Hyperspectral Image Super-Resolution, ICIP2018, X. Han et al. [PDF]

• Deep Hyperspectral Image Sharpening, TNNLS2018, R. Dian et al. [PDF] [Code]

• HSI-DeNet: Hyperspectral Image Restoration via Convolutional Neural Network, TGRS2018, Y. Chang et al. [Web]

• Unsupervised Sparse Dirichlet-Net for Hyperspectral Image Super-Resolution, CVPR2018, Y. Qu et al. [PDF] [Code]

• Deep Hyperspectral Prior: Denoising, Inpainting, Super-Resolution, arXiv2019, Oleksii Sidorov et al. [PDF] [Code]

• Multi-level and Multi-scale Spatial and Spectral Fusion CNN for Hyperspectral Image Super-resolution, ICCVW 2019, Xianhua Han et al. [PDF]

• Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net, CVPR2019, Xie Qi et al. [PDF] [Web]

• Hyperspectral Image Reconstruction Using Deep External and Internal Learning,ICCV2019, Zhang Tao et al. [PDF] [Web]

• Deep Blind Hyperspectral Image Super-Resolution, IEEE TNNLS 2020, Lei Zhang et al. [Pdf]

• Deep Recursive Network for Hyperspectral Image Super-Resolution, IEEE TCI2020, Wei Wei, et al. [PDF][Web]

• Unsupervised Recurrent Hyperspectral Imagery Super-Resolution Using Pixel-Aware Refinement, IEEE TGRS2021, Wei Wei, et al. [PDF][Web]

• A Band Divide-and-Conquer Multispectral and Hyperspectral Image Fusion Method, IEEE TGRS 2021, Weiwei Sun et al. [Pdf]

• Hyperspectral Image Super-Resolution via Deep Progressive Zero-Centric Residual Learning, IEEE TIP 2021, Zhiyu Zhu et al. [Pdf]

5) Simulations registration and super-resolution approaches

• An Integrated Approach to Registration and Fusion of Hyperspectral and Multispectral Images, TRGS 2019, Yuan Zhou et al.

• Deep Blind Hyperspectral Image Fusion, ICCV2019, Wu Wang et al. [PDF]

================================================================================

Single Hyperspectral Image Super-Resolution

• Super-resolution reconstruction of hyperspectral images, TIP2005, T. Akgun et al.

• Enhanced self-training superresolution mapping technique for hyperspectral imagery, GRSL2011, F. A. Mianji et al.

• A super-resolution reconstruction algorithm for hyperspectral images. Signal Process. 2012, H. Zhang et al.

• Super-resolution hyperspectral imaging with unknown blurring by low-rank and group-sparse modeling, ICIP2014, H. Huang et al.

• Super-resolution mapping via multi-dictionary based sparse representation, ICASSP2016, H. Huang et al.

• Super-resolution: An efficient method to improve spatial resolution of hyperspectral images, IGARSS2016, A. Villa, J. Chanussot et al.

• Hyperspectral image super resolution reconstruction with a joint spectral-spatial sub-pixel mapping model, IGARSS2016, X. Xu et al.

• Hyperspectral image super-resolution by spectral mixture analysis and spatial–spectral group sparsity, GRSL2016, J. Li et al.

• Super-resolution reconstruction of hyperspectral images via low rank tensor modeling and total variation regularization, IGARSS2016, S. He et al. [PDF]

• Hyperspectral image super-resolution by spectral difference learning and spatial error correction, GRSL2017, J. Hu et al.

• Super-Resolution for Remote Sensing Images via Local–Global Combined Network, GRSL2017, J. Hu et al.

• Hyperspectral image superresolution by transfer learning, Jstars2017, Y. Yuan et al. [PDF]

• Hyperspectral image super-resolution using deep convolutional neural network, Neurocomputing, 2017, Sen Lei et al. [PDF]

• Hyperspectral image super-resolution via nonlocal low-rank tensor approximation and total variation regularization, Remote Sensing, 2017, Yao Wang et al. [PDF]

• Hyperspectral Image Spatial Super-Resolution via 3D Full Convolutional Neural Network, Remote Sensing, 2017, Saohui Mei et al. [PDF] [Code]

• A MAP-Based Approach for Hyperspectral Imagery Super-Resolution, TIP2018, Hasan Irmak et al.

• Single Hyperspectral Image Super-resolution with Grouped Deep Recursive Residual Network, BigMM2018, Yong Li et al. [PDF] [Code]

• Hyperspectral image super-resolution with spectral–spatial network, IJRS2018, Jinrang Jia et al. [PDF]

• Separable-spectral convolution and inception network for hyperspectral image super-resolution, IJMLC 2019, Ke Zheng et al.

• Hyperspectral Image Super-Resolution Using Deep Feature Matrix Factorization, IEEE TGRS 2019, Weiying Xie et al. [PDF]

• Deep Hyperspectral Prior Single-Image Denoising, Inpainting, Super-Resolution, ICCVW2019, Oleksii Sidorov et al. [PDF]

• Spatial-Spectral Residual Network for Hyperspectral Image Super-Resolution, arXiv2020, Qi Wang et al. [PDF]

• CNN-Based Super-Resolution of Hyperspectral Images, IEEE TGRS 2020, P. V. Arun et al. [PDF]

• Hyperspectral Image Super-Resolution via Intrafusion Network, IEEE TGRS 2020, Jing Hu et al. [PDF]

• Mixed 2D/3D Convolutional Network for Hyperspectral Image Super-Resolution, Remote Sensing 2020, Qiang Li et al. [Code][Pdf]

• Hyperspectral Image Super-Resolution by Band Attention Through Adversarial Learning, IEEE TGRS 2020, Jiaojiao Li et al. [Pdf]

• Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery, IEEE TCI 2020, Junjun Jiang et al. [Code][Pdf] 【This is our method and achieves state-of-the-art performance for Single Hyperspectral Image Super-Resolution】

• Bidirectional 3D Quasi-Recurrent Neural Networkfor Hyperspectral Image Super-Resolution, IEEE JStars 2021, Ying Fu et al. [Web][Pdf]

• Hyperspectral Image Super-Resolution Using Spectrum and Feature Context, IEEE TIM 2021, Qi Wang et al. [Web][Pdf]

• Hyperspectral Image Super-Resolution with Spectral Mixup and Heterogeneous Datasets, arXiv2021, Ke Li et al. [Pdf]

• A Spectral Grouping and Attention-Driven Residual Dense Network for Hyperspectral Image Super-Resolution, IEEE TGRS 2021, Denghong Liu et al. [Web][Pdf]

• Spatial-Spectral Feedback Network for Super-Resolution of Hyperspectral Imagery, arXiv 2021, Enhai Liu et al. [Web][Pdf]

• Exploring the Relationship Between 2D/3D Convolution for Hyperspectral Image Super-Resolution, IEEE TGRS 2021, Qi Wang et al. [Web][Pdf]

================================================================================

Databases

• CAVE dataset
• Harvard dataset
• iCVL dataset
• NUS datase
• NTIRE18 dataset
• Chikusei dataset
• Indian Pines, Salinas, KSC et al.

================================================================================

Image Quality Measurement

• Peak Signal to Noise Ratio (PSNR)
• Root Mean Square Error (RMSE)
• Structural SIMilarity index (SSIM)
• Spectral Angle Mapper (SAM)
• Erreur Relative Globale Adimensionnelle de Synthèse (ERGAS)
• Universal Image Quality Index (UIQI)