Cutting-edge super-resolution (SR) and downscaling techniques are creating a revolution in Earth Observation (EO) and Earth science, as set out in a workshop held recently in ESA/ESRIN, in Frascati (Italy). The first ESA forum of this kind – called SUREDOS24 (Super-Resolution and Downscaling for Earth Observation and Earth Science) - provided a deep dive into how recent SR techniques are leveraging deep learning and artificial intelligence to push the boundaries of remote sensing.

As European New Space continues to bolster the global offering of very high resolution data, ESA is taking steps to support the potential of SR techniques to enhance remote sensing data, while ensuring its validity and reliability for scientific and commercial use. At SUREDOS24, which was co-hosted by the ESA Earth Observation Programme Green Solution Division and Φ-lab, remote sensing experts gathered to discover the latest in SR and its value for real-world applications.

SR techniques can be applied across multispectral, hyperspectral, thermal, infra-red, video, Synthetic Aperture Radar (SAR), radiometer, and scatterometer data, with one common goal – to reconstruct data from lower resolution sources into higher resolution. SR techniques have the potential to enhance the spatial detail of lower resolution imagery, bridging the gap between sensor generations and types. The methods are equally useful to improve the spectral or temporal information of data for applications where finer details are critical.

Advanced deep learning algorithms and Artificial Intelligence (AI) techniques are pushing boundaries to achieve higher resolution enhancement in satellite imagery. The potential for these enhanced data is boundless, with application areas including climate modelling, hydrology, ecosystems, agriculture, urban development, weather forecasting and air quality.

Improve resolution while maintaining radiometric value

Peggy Fischer, mission manager for ESA’s Third Party Mission (TPM) programme, made a presentation on how the TPM programme supports advancements in super resolution, and showcased several TPM missions that can provide super resolution data, such as WorldView-3, Pléiades Neo and GEOSAT-2. These “off-the-shelf” datasets are provided by the commercial providers and are fit for immediate use.

For more than 45 years, ESA’s Earthnet programme has provided the framework to access EO data from non-ESA missions, via the TPM programme, for research and pre-operational applications development. TPM data, combined with data from ESA missions, can exploit the synergy between all sources of data to better meet the growing needs of user communities.

“The objective of SR development is to improve satellite imagery resolution by making use of AI and machine learning techniques, while maintaining the radiometric value of the original data,” says Fischer.

Fischer showed how Maxar’s WorldView-3 native optical 30 cm and 50/60 cm resolution products can be processed with Maxar HD Technology to generate respectively, 15 cm and 30 cm products. The HD technique increases the number of pixels and improves the visual clarity of images, providing precise edges and reconstructed details.

Pléiades Neo, following on from its predecessor, Pléiades, is another TPM mission offering very high resolution 30 cm imagery. Fischer showed how super resolution techniques can improve Pléiades Neo imagery further, allowing smaller objects to be detected, without affecting the geolocation accuracy. Imagery collected with a native 30 cm spatial resolution is processed using a proprietary algorithm that enhances the level of detail, while maintaining the native ground sampling distance. Colours of the image are brighter, with sharper detail visible.

Finally, Fischer showcased GEOSAT-2 super resolution products that preserve radiometric quality and can be used for characterisation of change, precision agriculture and other scientific applications.

At SUREDOS24, a range of presentations spanning three days aired the challenges and inspiration that come hand-in-hand with SR. One key takeaway is that together with new possibilities, SR brings great responsibility. As data are enhanced, ensuring its validity and reliability for scientific and commercial use becomes paramount. As well as uncertainty quantification in AI, quality control and validation of SR/downscaling products are essential for reliable adoption and downstream services.