Showing 1–50 of 60 results for author: Saux, B L

Quantum Latent Diffusion Models

Authors: Francesca De Falco, Andrea Ceschini, Alessandro Sebastianelli, Bertrand Le Saux, Massimo Panella

Abstract: The introduction of quantum concepts is increasingly making its way into generative machine learning models. However, while there are various implementations of quantum Generative Adversarial Networks, the integration of quantum elements into diffusion models remains an open and challenging task. In this work, we propose a potential version of a quantum diffusion model that leverages the establish… ▽ More The introduction of quantum concepts is increasingly making its way into generative machine learning models. However, while there are various implementations of quantum Generative Adversarial Networks, the integration of quantum elements into diffusion models remains an open and challenging task. In this work, we propose a potential version of a quantum diffusion model that leverages the established idea of classical latent diffusion models. This involves using a traditional autoencoder to reduce images, followed by operations with variational circuits in the latent space. To effectively assess the benefits brought by quantum computing, the images generated by the quantum latent diffusion model have been compared to those generated by a classical model with a similar number of parameters, evaluated in terms of quantitative metrics. The results demonstrate an advantage in using a quantum version, as evidenced by obtaining better metrics for the images generated by the quantum version compared to those obtained by the classical version. Furthermore, quantum models continue to outperform even when considering small percentages of the dataset for training, demonstrating the quantum's ability to extract features more effectively even in a few shot learning scenario. △ Less

Submitted 19 January, 2025; originally announced January 2025.

Comments: 40 pages, 15 figures, 5 tables

Journal ref: Quantum Machine Intelligence, Vol. 6, No. 2, Article No. 85, pp. 1-20, ISSN: 2524-4906, Springer Nature, Switzerland, December 2024

IceCloudNet: 3D reconstruction of cloud ice from Meteosat SEVIRI

Authors: Kai Jeggle, Mikolaj Czerkawski, Federico Serva, Bertrand Le Saux, David Neubauer, Ulrike Lohmann

Abstract: IceCloudNet is a novel method based on machine learning able to predict high-quality vertically resolved cloud ice water contents (IWC) and ice crystal number concentrations (N$_\textrm{ice}$). The predictions come at the spatio-temporal coverage and resolution of geostationary satellite observations (SEVIRI) and the vertical resolution of active satellite retrievals (DARDAR). IceCloudNet consists… ▽ More IceCloudNet is a novel method based on machine learning able to predict high-quality vertically resolved cloud ice water contents (IWC) and ice crystal number concentrations (N$_\textrm{ice}$). The predictions come at the spatio-temporal coverage and resolution of geostationary satellite observations (SEVIRI) and the vertical resolution of active satellite retrievals (DARDAR). IceCloudNet consists of a ConvNeXt-based U-Net and a 3D PatchGAN discriminator model and is trained by predicting DARDAR profiles from co-located SEVIRI images. Despite the sparse availability of DARDAR data due to its narrow overpass, IceCloudNet is able to predict cloud occurrence, spatial structure, and microphysical properties with high precision. The model has been applied to ten years of SEVIRI data, producing a dataset of vertically resolved IWC and N$_\textrm{ice}$ of clouds containing ice with a 3 kmx3 kmx240 mx15 minute resolution in a spatial domain of 30°W to 30°E and 30°S to 30°N. The produced dataset increases the availability of vertical cloud profiles, for the period when DARDAR is available, by more than six orders of magnitude and moreover, IceCloudNet is able to produce vertical cloud profiles beyond the lifetime of the recently ended satellite missions underlying DARDAR. △ Less

Submitted 5 October, 2024; originally announced October 2024.

Comments: his paper was submitted to Artificial Intelligence for the Earth Systems

ACM Class: J.2

From Graphs to Qubits: A Critical Review of Quantum Graph Neural Networks

Authors: Andrea Ceschini, Francesco Mauro, Francesca De Falco, Alessandro Sebastianelli, Alessio Verdone, Antonello Rosato, Bertrand Le Saux, Massimo Panella, Paolo Gamba, Silvia L. Ullo

Abstract: Quantum Graph Neural Networks (QGNNs) represent a novel fusion of quantum computing and Graph Neural Networks (GNNs), aimed at overcoming the computational and scalability challenges inherent in classical GNNs that are powerful tools for analyzing data with complex relational structures but suffer from limitations such as high computational complexity and over-smoothing in large-scale applications… ▽ More Quantum Graph Neural Networks (QGNNs) represent a novel fusion of quantum computing and Graph Neural Networks (GNNs), aimed at overcoming the computational and scalability challenges inherent in classical GNNs that are powerful tools for analyzing data with complex relational structures but suffer from limitations such as high computational complexity and over-smoothing in large-scale applications. Quantum computing, leveraging principles like superposition and entanglement, offers a pathway to enhanced computational capabilities. This paper critically reviews the state-of-the-art in QGNNs, exploring various architectures. We discuss their applications across diverse fields such as high-energy physics, molecular chemistry, finance and earth sciences, highlighting the potential for quantum advantage. Additionally, we address the significant challenges faced by QGNNs, including noise, decoherence, and scalability issues, proposing potential strategies to mitigate these problems. This comprehensive review aims to provide a foundational understanding of QGNNs, fostering further research and development in this promising interdisciplinary field. △ Less

Submitted 12 August, 2024; originally announced August 2024.

Comments: 21 pages, 9 figures, 2 tables. arXiv admin note: text overlap with arXiv:1909.12264 by other authors

Quanv4EO: Empowering Earth Observation by means of Quanvolutional Neural Networks

Authors: Alessandro Sebastianelli, Francesco Mauro, Giulia Ciabatti, Dario Spiller, Bertrand Le Saux, Paolo Gamba, Silvia Ullo

Abstract: A significant amount of remotely sensed data is generated daily by many Earth observation (EO) spaceborne and airborne sensors over different countries of our planet. Different applications use those data, such as natural hazard monitoring, global climate change, urban planning, and more. Many challenges are brought by the use of these big data in the context of remote sensing applications. In rec… ▽ More A significant amount of remotely sensed data is generated daily by many Earth observation (EO) spaceborne and airborne sensors over different countries of our planet. Different applications use those data, such as natural hazard monitoring, global climate change, urban planning, and more. Many challenges are brought by the use of these big data in the context of remote sensing applications. In recent years, employment of machine learning (ML) and deep learning (DL)-based algorithms have allowed a more efficient use of these data but the issues in managing, processing, and efficiently exploiting them have even increased since classical computers have reached their limits. This article highlights a significant shift towards leveraging quantum computing techniques in processing large volumes of remote sensing data. The proposed Quanv4EO model introduces a quanvolution method for preprocessing multi-dimensional EO data. First its effectiveness is demonstrated through image classification tasks on MNIST and Fashion MNIST datasets, and later on, its capabilities on remote sensing image classification and filtering are shown. Key findings suggest that the proposed model not only maintains high precision in image classification but also shows improvements of around 5\% in EO use cases compared to classical approaches. Moreover, the proposed framework stands out for its reduced parameter size and the absence of training quantum kernels, enabling better scalability for processing massive datasets. These advancements underscore the promising potential of quantum computing in addressing the limitations of classical algorithms in remote sensing applications, offering a more efficient and effective alternative for image data classification and analysis. △ Less

Submitted 24 July, 2024; originally announced July 2024.

Comments: Submitted to IEEE Transactions on Geoscience and Remote Sensing

In Search of Quantum Advantage: Estimating the Number of Shots in Quantum Kernel Methods

Authors: Artur Miroszewski, Marco Fellous Asiani, Jakub Mielczarek, Bertrand Le Saux, Jakub Nalepa

Abstract: Quantum Machine Learning (QML) has gathered significant attention through approaches like Quantum Kernel Machines. While these methods hold considerable promise, their quantum nature presents inherent challenges. One major challenge is the limited resolution of estimated kernel values caused by the finite number of circuit runs performed on a quantum device. In this study, we propose a comprehensi… ▽ More Quantum Machine Learning (QML) has gathered significant attention through approaches like Quantum Kernel Machines. While these methods hold considerable promise, their quantum nature presents inherent challenges. One major challenge is the limited resolution of estimated kernel values caused by the finite number of circuit runs performed on a quantum device. In this study, we propose a comprehensive system of rules and heuristics for estimating the required number of circuit runs in quantum kernel methods. We introduce two critical effects that necessitate an increased measurement precision through additional circuit runs: the spread effect and the concentration effect. The effects are analyzed in the context of fidelity and projected quantum kernels. To address these phenomena, we develop an approach for estimating desired precision of kernel values, which, in turn, is translated into the number of circuit runs. Our methodology is validated through extensive numerical simulations, focusing on the problem of exponential value concentration. We stress that quantum kernel methods should not only be considered from the machine learning performance perspective, but also from the context of the resource consumption. The results provide insights into the possible benefits of quantum kernel methods, offering a guidance for their application in quantum machine learning tasks. △ Less

Submitted 22 July, 2024; originally announced July 2024.

Comments: 18 + 13 pages, 8 figures. This manuscript is a first release that will be improved in future versions. We wanted to provide this preview now as we recently became aware of extensive modifications in arXiv:2208.11060

Evaluating and Benchmarking Foundation Models for Earth Observation and Geospatial AI

Authors: Nikolaos Dionelis, Casper Fibaek, Luke Camilleri, Andreas Luyts, Jente Bosmans, Bertrand Le Saux

Abstract: When we are primarily interested in solving several problems jointly with a given prescribed high performance accuracy for each target application, then Foundation Models should for most cases be used rather than problem-specific models. We focus on the specific Computer Vision application of Foundation Models for Earth Observation (EO) and geospatial AI. These models can solve important problems… ▽ More When we are primarily interested in solving several problems jointly with a given prescribed high performance accuracy for each target application, then Foundation Models should for most cases be used rather than problem-specific models. We focus on the specific Computer Vision application of Foundation Models for Earth Observation (EO) and geospatial AI. These models can solve important problems we are tackling, including for example land cover classification, crop type mapping, flood segmentation, building density estimation, and road regression segmentation. In this paper, we show that for a limited number of labelled data, Foundation Models achieve improved performance compared to problem-specific models. In this work, we also present our proposed evaluation benchmark for Foundation Models for EO. Benchmarking the generalization performance of Foundation Models is important as it has become difficult to standardize a fair comparison across the many different models that have been proposed recently. We present the results using our evaluation benchmark for EO Foundation Models and show that Foundation Models are label efficient in the downstream tasks and help us solve problems we are tackling in EO and remote sensing. △ Less

Submitted 26 June, 2024; originally announced June 2024.

Comments: 5 pages, 2 figures, Submitted

Latent Style-based Quantum GAN for high-quality Image Generation

Authors: Su Yeon Chang, Supanut Thanasilp, Bertrand Le Saux, Sofia Vallecorsa, Michele Grossi

Abstract: Quantum generative modeling is among the promising candidates for achieving a practical advantage in data analysis. Nevertheless, one key challenge is to generate large-size images comparable to those generated by their classical counterparts. In this work, we take an initial step in this direction and introduce the Latent Style-based Quantum GAN (LaSt-QGAN), which employs a hybrid classical-quant… ▽ More Quantum generative modeling is among the promising candidates for achieving a practical advantage in data analysis. Nevertheless, one key challenge is to generate large-size images comparable to those generated by their classical counterparts. In this work, we take an initial step in this direction and introduce the Latent Style-based Quantum GAN (LaSt-QGAN), which employs a hybrid classical-quantum approach in training Generative Adversarial Networks (GANs) for arbitrary complex data generation. This novel approach relies on powerful classical auto-encoders to map a high-dimensional original image dataset into a latent representation. The hybrid classical-quantum GAN operates in this latent space to generate an arbitrary number of fake features, which are then passed back to the auto-encoder to reconstruct the original data. Our LaSt-QGAN can be successfully trained on realistic computer vision datasets beyond the standard MNIST, namely Fashion MNIST (fashion products) and SAT4 (Earth Observation images) with 10 qubits, resulting in a comparable performance (and even better in some metrics) with the classical GANs. Moreover, we analyze the barren plateau phenomena within this context of the continuous quantum generative model using a polynomial depth circuit and propose a method to mitigate the detrimental effect during the training of deep-depth networks. Through empirical experiments and theoretical analysis, we demonstrate the potential of LaSt-QGAN for the practical usage in the context of image generation and open the possibility of applying it to a larger dataset in the future. △ Less

Submitted 4 June, 2024; originally announced June 2024.

Comments: 28 pages, 22 figures

A Semantic Segmentation-guided Approach for Ground-to-Aerial Image Matching

Authors: Francesco Pro, Nikolaos Dionelis, Luca Maiano, Bertrand Le Saux, Irene Amerini

Abstract: Nowadays the accurate geo-localization of ground-view images has an important role across domains as diverse as journalism, forensics analysis, transports, and Earth Observation. This work addresses the problem of matching a query ground-view image with the corresponding satellite image without GPS data. This is done by comparing the features from a ground-view image and a satellite one, innovativ… ▽ More Nowadays the accurate geo-localization of ground-view images has an important role across domains as diverse as journalism, forensics analysis, transports, and Earth Observation. This work addresses the problem of matching a query ground-view image with the corresponding satellite image without GPS data. This is done by comparing the features from a ground-view image and a satellite one, innovatively leveraging the corresponding latter's segmentation mask through a three-stream Siamese-like network. The proposed method, Semantic Align Net (SAN), focuses on limited Field-of-View (FoV) and ground panorama images (images with a FoV of 360°). The novelty lies in the fusion of satellite images in combination with their semantic segmentation masks, aimed at ensuring that the model can extract useful features and focus on the significant parts of the images. This work shows how SAN through semantic analysis of images improves the performance on the unlabelled CVUSA dataset for all the tested FoVs. △ Less

Submitted 23 May, 2024; v1 submitted 17 April, 2024; originally announced April 2024.

Comments: 6 pages, 2 figures, 2 tables, Submitted to IGARSS 2024

Learning from Unlabelled Data with Transformers: Domain Adaptation for Semantic Segmentation of High Resolution Aerial Images

Authors: Nikolaos Dionelis, Francesco Pro, Luca Maiano, Irene Amerini, Bertrand Le Saux

Abstract: Data from satellites or aerial vehicles are most of the times unlabelled. Annotating such data accurately is difficult, requires expertise, and is costly in terms of time. Even if Earth Observation (EO) data were correctly labelled, labels might change over time. Learning from unlabelled data within a semi-supervised learning framework for segmentation of aerial images is challenging. In this pape… ▽ More Data from satellites or aerial vehicles are most of the times unlabelled. Annotating such data accurately is difficult, requires expertise, and is costly in terms of time. Even if Earth Observation (EO) data were correctly labelled, labels might change over time. Learning from unlabelled data within a semi-supervised learning framework for segmentation of aerial images is challenging. In this paper, we develop a new model for semantic segmentation of unlabelled images, the Non-annotated Earth Observation Semantic Segmentation (NEOS) model. NEOS performs domain adaptation as the target domain does not have ground truth semantic segmentation masks. The distribution inconsistencies between the target and source domains are due to differences in acquisition scenes, environment conditions, sensors, and times. Our model aligns the learned representations of the different domains to make them coincide. The evaluation results show that NEOS is successful and outperforms other models for semantic segmentation of unlabelled data. △ Less

Submitted 17 April, 2024; originally announced April 2024.

Comments: 6 pages, 7 figures, Submitted to IGARSS 2024

Neural Plasticity-Inspired Multimodal Foundation Model for Earth Observation

Authors: Zhitong Xiong, Yi Wang, Fahong Zhang, Adam J. Stewart, Joëlle Hanna, Damian Borth, Ioannis Papoutsis, Bertrand Le Saux, Gustau Camps-Valls, Xiao Xiang Zhu

Abstract: The development of foundation models has revolutionized our ability to interpret the Earth's surface using satellite observational data. Traditional models have been siloed, tailored to specific sensors or data types like optical, radar, and hyperspectral, each with its own unique characteristics. This specialization hinders the potential for a holistic analysis that could benefit from the combine… ▽ More The development of foundation models has revolutionized our ability to interpret the Earth's surface using satellite observational data. Traditional models have been siloed, tailored to specific sensors or data types like optical, radar, and hyperspectral, each with its own unique characteristics. This specialization hinders the potential for a holistic analysis that could benefit from the combined strengths of these diverse data sources. Our novel approach introduces the Dynamic One-For-All (DOFA) model, leveraging the concept of neural plasticity in brain science to integrate various data modalities into a single framework adaptively. This dynamic hypernetwork, adjusting to different wavelengths, enables a single versatile Transformer jointly trained on data from five sensors to excel across 12 distinct Earth observation tasks, including sensors never seen during pretraining. DOFA's innovative design offers a promising leap towards more accurate, efficient, and unified Earth observation analysis, showcasing remarkable adaptability and performance in harnessing the potential of multimodal Earth observation data. △ Less

Submitted 7 June, 2024; v1 submitted 22 March, 2024; originally announced March 2024.

Comments: 36 pages, 7 figures

Towards Efficient Quantum Hybrid Diffusion Models

Authors: Francesca De Falco, Andrea Ceschini, Alessandro Sebastianelli, Bertrand Le Saux, Massimo Panella

Abstract: In this paper, we propose a new methodology to design quantum hybrid diffusion models, derived from classical U-Nets with ResNet and Attention layers. Specifically, we propose two possible different hybridization schemes combining quantum computing's superior generalization with classical networks' modularity. In the first one, we acted at the vertex: ResNet convolutional layers are gradually repl… ▽ More In this paper, we propose a new methodology to design quantum hybrid diffusion models, derived from classical U-Nets with ResNet and Attention layers. Specifically, we propose two possible different hybridization schemes combining quantum computing's superior generalization with classical networks' modularity. In the first one, we acted at the vertex: ResNet convolutional layers are gradually replaced with variational circuits to create Quantum ResNet blocks. In the second proposed architecture, we extend the hybridization to the intermediate level of the encoder, due to its higher sensitivity in the feature extraction process. In order to conduct an in-depth analysis of the potential advantages stemming from the integration of quantum layers, images generated by quantum hybrid diffusion models are compared to those generated by classical models, and evaluated in terms of several quantitative metrics. The results demonstrate an advantage in using a hybrid quantum diffusion models, as they generally synthesize better-quality images and converges faster. Moreover, they show the additional advantage of having a lower number of parameters to train compared to the classical one, with a reduction that depends on the extent to which the vertex is hybridized. △ Less

Submitted 25 February, 2024; originally announced February 2024.

Comments: 23 pages, 14 figures (many with subfigures), 2 tables

Journal ref: KI - Künstliche Intelligenz, Vol. 38, No. 4, pp. 311-326, 2024

A Hybrid MLP-Quantum approach in Graph Convolutional Neural Networks for Oceanic Nino Index (ONI) prediction

Authors: Francesco Mauro, Alessandro Sebastianelli, Bertrand Le Saux, Paolo Gamba, Silvia Liberata Ullo

Abstract: This paper explores an innovative fusion of Quantum Computing (QC) and Artificial Intelligence (AI) through the development of a Hybrid Quantum Graph Convolutional Neural Network (HQGCNN), combining a Graph Convolutional Neural Network (GCNN) with a Quantum Multilayer Perceptron (MLP). The study highlights the potentialities of GCNNs in handling global-scale dependencies and proposes the HQGCNN fo… ▽ More This paper explores an innovative fusion of Quantum Computing (QC) and Artificial Intelligence (AI) through the development of a Hybrid Quantum Graph Convolutional Neural Network (HQGCNN), combining a Graph Convolutional Neural Network (GCNN) with a Quantum Multilayer Perceptron (MLP). The study highlights the potentialities of GCNNs in handling global-scale dependencies and proposes the HQGCNN for predicting complex phenomena such as the Oceanic Nino Index (ONI). Preliminary results suggest the model potential to surpass state-of-the-art (SOTA). The code will be made available with the paper publication. △ Less

Submitted 29 January, 2024; originally announced January 2024.

PhilEO Bench: Evaluating Geo-Spatial Foundation Models

Authors: Casper Fibaek, Luke Camilleri, Andreas Luyts, Nikolaos Dionelis, Bertrand Le Saux

Abstract: Massive amounts of unlabelled data are captured by Earth Observation (EO) satellites, with the Sentinel-2 constellation generating 1.6 TB of data daily. This makes Remote Sensing a data-rich domain well suited to Machine Learning (ML) solutions. However, a bottleneck in applying ML models to EO is the lack of annotated data as annotation is a labour-intensive and costly process. As a result, resea… ▽ More Massive amounts of unlabelled data are captured by Earth Observation (EO) satellites, with the Sentinel-2 constellation generating 1.6 TB of data daily. This makes Remote Sensing a data-rich domain well suited to Machine Learning (ML) solutions. However, a bottleneck in applying ML models to EO is the lack of annotated data as annotation is a labour-intensive and costly process. As a result, research in this domain has focused on Self-Supervised Learning and Foundation Model approaches. This paper addresses the need to evaluate different Foundation Models on a fair and uniform benchmark by introducing the PhilEO Bench, a novel evaluation framework for EO Foundation Models. The framework comprises of a testbed and a novel 400 GB Sentinel-2 dataset containing labels for three downstream tasks, building density estimation, road segmentation, and land cover classification. We present experiments using our framework evaluating different Foundation Models, including Prithvi and SatMAE, at multiple n-shots and convergence rates. △ Less

Submitted 15 January, 2024; v1 submitted 9 January, 2024; originally announced January 2024.

Comments: 6 pages, 5 figures, Submitted to IGARSS 2024

The curse of language biases in remote sensing VQA: the role of spatial attributes, language diversity, and the need for clear evaluation

Authors: Christel Chappuis, Eliot Walt, Vincent Mendez, Sylvain Lobry, Bertrand Le Saux, Devis Tuia

Abstract: Remote sensing visual question answering (RSVQA) opens new opportunities for the use of overhead imagery by the general public, by enabling human-machine interaction with natural language. Building on the recent advances in natural language processing and computer vision, the goal of RSVQA is to answer a question formulated in natural language about a remote sensing image. Language understanding i… ▽ More Remote sensing visual question answering (RSVQA) opens new opportunities for the use of overhead imagery by the general public, by enabling human-machine interaction with natural language. Building on the recent advances in natural language processing and computer vision, the goal of RSVQA is to answer a question formulated in natural language about a remote sensing image. Language understanding is essential to the success of the task, but has not yet been thoroughly examined in RSVQA. In particular, the problem of language biases is often overlooked in the remote sensing community, which can impact model robustness and lead to wrong conclusions about the performances of the model. Thus, the present work aims at highlighting the problem of language biases in RSVQA with a threefold analysis strategy: visual blind models, adversarial testing and dataset analysis. This analysis focuses both on model and data. Moreover, we motivate the use of more informative and complementary evaluation metrics sensitive to the issue. The gravity of language biases in RSVQA is then exposed for all of these methods with the training of models discarding the image data and the manipulation of the visual input during inference. Finally, a detailed analysis of question-answer distribution demonstrates the root of the problem in the data itself. Thanks to this analytical study, we observed that biases in remote sensing are more severe than in standard VQA, likely due to the specifics of existing remote sensing datasets for the task, e.g. geographical similarities and sparsity, as well as a simpler vocabulary and question generation strategies. While new, improved and less-biased datasets appear as a necessity for the development of the promising field of RSVQA, we demonstrate that more informed, relative evaluation metrics remain much needed to transparently communicate results of future RSVQA methods. △ Less

Submitted 28 November, 2023; originally announced November 2023.

Quantum Machine Learning for Remote Sensing: Exploring potential and challenges

Authors: Artur Miroszewski, Jakub Nalepa, Bertrand Le Saux, Jakub Mielczarek

Abstract: The industry of quantum technologies is rapidly expanding, offering promising opportunities for various scientific domains. Among these emerging technologies, Quantum Machine Learning (QML) has attracted considerable attention due to its potential to revolutionize data processing and analysis. In this paper, we investigate the application of QML in the field of remote sensing. It is believed that… ▽ More The industry of quantum technologies is rapidly expanding, offering promising opportunities for various scientific domains. Among these emerging technologies, Quantum Machine Learning (QML) has attracted considerable attention due to its potential to revolutionize data processing and analysis. In this paper, we investigate the application of QML in the field of remote sensing. It is believed that QML can provide valuable insights for analysis of data from space. We delve into the common beliefs surrounding the quantum advantage in QML for remote sensing and highlight the open challenges that need to be addressed. To shed light on the challenges, we conduct a study focused on the problem of kernel value concentration, a phenomenon that adversely affects the runtime of quantum computers. Our findings indicate that while this issue negatively impacts quantum computer performance, it does not entirely negate the potential quantum advantage in QML for remote sensing. △ Less

Submitted 13 November, 2023; originally announced November 2023.

Comments: 2 pages, 2 figures. Presented at the Big Data from Space 2023 conference

Estimating optical vegetation indices and biophysical variables for temperate forests with Sentinel-1 SAR data using machine learning techniques: A case study for Czechia

Authors: Daniel Paluba, Bertrand Le Saux, Přemysl Stych

Abstract: Current optical vegetation indices (VIs) for monitoring forest ecosystems are well established and widely used in various applications, but can be limited by atmospheric effects such as clouds. In contrast, synthetic aperture radar (SAR) data can offer insightful and systematic forest monitoring with complete time series (TS) due to signal penetration through clouds and day and night image acquisi… ▽ More Current optical vegetation indices (VIs) for monitoring forest ecosystems are well established and widely used in various applications, but can be limited by atmospheric effects such as clouds. In contrast, synthetic aperture radar (SAR) data can offer insightful and systematic forest monitoring with complete time series (TS) due to signal penetration through clouds and day and night image acquisitions. This study aims to address the limitations of optical satellite data by using SAR data as an alternative for estimating optical VIs for forests through machine learning (ML). While this approach is less direct and likely only feasible through the power of ML, it raises the scientific question of whether enough relevant information is contained in the SAR signal to accurately estimate VIs. This work covers the estimation of TS of four VIs (LAI, FAPAR, EVI and NDVI) using multitemporal Sentinel-1 SAR and ancillary data. The study focused on both healthy and disturbed temperate forest areas in Czechia for the year 2021, while ground truth labels generated from Sentinel-2 multispectral data. This was enabled by creating a paired multi-modal TS dataset in Google Earth Engine (GEE), including temporally and spatially aligned Sentinel-1, Sentinel-2, DEM, weather and land cover datasets. The inclusion of DEM-derived auxiliary features and additional meteorological information, further improved the results. In the comparison of ML models, the traditional ML algorithms, RFR and XGBoost slightly outperformed the AutoML approach, auto-sklearn, for all VIs, achieving high accuracies ($R^2$ between 70-86%) and low errors (0.055-0.29 of MAE). In general, up to 240 measurements per year and a spatial resolution of 20 m can be achieved using estimated SAR-based VIs with high accuracy. A great advantage of the SAR-based VI is the ability to detect abrupt forest changes with sub-weekly temporal accuracy. △ Less

Submitted 27 August, 2024; v1 submitted 13 November, 2023; originally announced November 2023.

Comments: Revised version of the preprint, based on comments from the reviewers. Full research article. 23 pages, 10 figures, 7 tables

ACM Class: I.4.8; I.4.9

Diffusion Models for Earth Observation Use-cases: from cloud removal to urban change detection

Authors: Fulvio Sanguigni, Mikolaj Czerkawski, Lorenzo Papa, Irene Amerini, Bertrand Le Saux

Abstract: The advancements in the state of the art of generative Artificial Intelligence (AI) brought by diffusion models can be highly beneficial in novel contexts involving Earth observation data. After introducing this new family of generative models, this work proposes and analyses three use cases which demonstrate the potential of diffusion-based approaches for satellite image data. Namely, we tackle c… ▽ More The advancements in the state of the art of generative Artificial Intelligence (AI) brought by diffusion models can be highly beneficial in novel contexts involving Earth observation data. After introducing this new family of generative models, this work proposes and analyses three use cases which demonstrate the potential of diffusion-based approaches for satellite image data. Namely, we tackle cloud removal and inpainting, dataset generation for change-detection tasks, and urban replanning. △ Less

Submitted 10 November, 2023; originally announced November 2023.

Comments: Presented at Big Data from Space 2023 (BiDS)

Report number: JRC135493

Journal ref: Proceedings of the 2023 conference on Big Data from Space, Soille, P., Lumnitz, S. and Albani, S. editor(s), Publications Office of the European Union, Luxembourg, 2023

Super-resolved rainfall prediction with physics-aware deep learning

Authors: S. Moran, B. Demir, F. Serva, B. Le Saux

Abstract: Rainfall prediction at the kilometre-scale up to a few hours in the future is key for planning and safety. But it is challenging given the complex influence of climate change on cloud processes and the limited skill of weather models at this scale. Following the set-up proposed by the \emph{weather4cast} challenge of NeurIPS, we build a two-step deep-learning solution for predicting rainfall occur… ▽ More Rainfall prediction at the kilometre-scale up to a few hours in the future is key for planning and safety. But it is challenging given the complex influence of climate change on cloud processes and the limited skill of weather models at this scale. Following the set-up proposed by the \emph{weather4cast} challenge of NeurIPS, we build a two-step deep-learning solution for predicting rainfall occurrence at ground radar high spatial resolution starting from coarser resolution weather satellite images. Our approach is designed to predict future satellite images with a physics-aware ConvLSTM network, which is then converted into precipitation maps through a U-Net. We find that our two-step pipeline outperforms the baseline model and we quantify the benefits of including physical information. We find that local-scale rainfall predictions with good accuracy starting from satellite radiances can be obtained for up to 4 hours in the future. △ Less

Submitted 24 October, 2023; originally announced October 2023.

Comments: Accepted at Big Data from Space 2023 (BiDS); 4 pages, 3 figures

Journal ref: Proceedings of the 2023 conference on Big Data from Space

Deep-Learning-based Change Detection with Spaceborne Hyperspectral PRISMA data

Authors: J. F. Amieva, A. Austoni, M. A. Brovelli, L. Ansalone, P. Naylor, F. Serva, B. Le Saux

Abstract: Change detection (CD) methods have been applied to optical data for decades, while the use of hyperspectral data with a fine spectral resolution has been rarely explored. CD is applied in several sectors, such as environmental monitoring and disaster management. Thanks to the PRecursore IperSpettrale della Missione operativA (PRISMA), hyperspectral-from-space CD is now possible. In this work, we a… ▽ More Change detection (CD) methods have been applied to optical data for decades, while the use of hyperspectral data with a fine spectral resolution has been rarely explored. CD is applied in several sectors, such as environmental monitoring and disaster management. Thanks to the PRecursore IperSpettrale della Missione operativA (PRISMA), hyperspectral-from-space CD is now possible. In this work, we apply standard and deep-learning (DL) CD methods to different targets, from natural to urban areas. We propose a pipeline starting from coregistration, followed by CD with a full-spectrum algorithm and by a DL network developed for optical data. We find that changes in vegetation and built environments are well captured. The spectral information is valuable to identify subtle changes and the DL methods are less affected by noise compared to the statistical method, but atmospheric effects and the lack of reliable ground truth represent a major challenge to hyperspectral CD. △ Less

Submitted 20 October, 2023; originally announced October 2023.

Comments: Accepted at Big Data from Space 2023 (BiDS); 4 pages, 4 figures

Journal ref: Proceedings of the 2023 conference on Big Data from Space

IceCloudNet: Cirrus and mixed-phase cloud prediction from SEVIRI input learned from sparse supervision

Authors: Kai Jeggle, Mikolaj Czerkawski, Federico Serva, Bertrand Le Saux, David Neubauer, Ulrike Lohmann

Abstract: Clouds containing ice particles play a crucial role in the climate system. Yet they remain a source of great uncertainty in climate models and future climate projections. In this work, we create a new observational constraint of regime-dependent ice microphysical properties at the spatio-temporal coverage of geostationary satellite instruments and the quality of active satellite retrievals. We ach… ▽ More Clouds containing ice particles play a crucial role in the climate system. Yet they remain a source of great uncertainty in climate models and future climate projections. In this work, we create a new observational constraint of regime-dependent ice microphysical properties at the spatio-temporal coverage of geostationary satellite instruments and the quality of active satellite retrievals. We achieve this by training a convolutional neural network on three years of SEVIRI and DARDAR data sets. This work will enable novel research to improve ice cloud process understanding and hence, reduce uncertainties in a changing climate and help assess geoengineering methods for cirrus clouds. △ Less

Submitted 5 October, 2023; originally announced October 2023.

Comments: A Preprint. Submitted to Tackling Climate Change with Machine Learning: workshop at NeurIPS 2023

Journal ref: NeurIPS 2023 Workshop: Tackling Climate Change with Machine Learning

Approximately Equivariant Quantum Neural Network for $p4m$ Group Symmetries in Images

Authors: Su Yeon Chang, Michele Grossi, Bertrand Le Saux, Sofia Vallecorsa

Abstract: Quantum Neural Networks (QNNs) are suggested as one of the quantum algorithms which can be efficiently simulated with a low depth on near-term quantum hardware in the presence of noises. However, their performance highly relies on choosing the most suitable architecture of Variational Quantum Algorithms (VQAs), and the problem-agnostic models often suffer issues regarding trainability and generali… ▽ More Quantum Neural Networks (QNNs) are suggested as one of the quantum algorithms which can be efficiently simulated with a low depth on near-term quantum hardware in the presence of noises. However, their performance highly relies on choosing the most suitable architecture of Variational Quantum Algorithms (VQAs), and the problem-agnostic models often suffer issues regarding trainability and generalization power. As a solution, the most recent works explore Geometric Quantum Machine Learning (GQML) using QNNs equivariant with respect to the underlying symmetry of the dataset. GQML adds an inductive bias to the model by incorporating the prior knowledge on the given dataset and leads to enhancing the optimization performance while constraining the search space. This work proposes equivariant Quantum Convolutional Neural Networks (EquivQCNNs) for image classification under planar $p4m$ symmetry, including reflectional and $90^\circ$ rotational symmetry. We present the results tested in different use cases, such as phase detection of the 2D Ising model and classification of the extended MNIST dataset, and compare them with those obtained with the non-equivariant model, proving that the equivariance fosters better generalization of the model. △ Less

Submitted 3 October, 2023; originally announced October 2023.

Comments: 7 pages, 6 figures, To be published as part of the IEEE Quantum Week 2023 Proceedings

Cloud Detection in Multispectral Satellite Images Using Support Vector Machines With Quantum Kernels

Authors: Artur Miroszewski, Jakub Mielczarek, Filip Szczepanek, Grzegorz Czelusta, Bartosz Grabowski, Bertrand Le Saux, Jakub Nalepa

Abstract: Support vector machines (SVMs) are a well-established classifier effectively deployed in an array of pattern recognition and classification tasks. In this work, we consider extending classic SVMs with quantum kernels and applying them to satellite data analysis. The design and implementation of SVMs with quantum kernels (hybrid SVMs) is presented. It consists of the Quantum Kernel Estimation (QKE)… ▽ More Support vector machines (SVMs) are a well-established classifier effectively deployed in an array of pattern recognition and classification tasks. In this work, we consider extending classic SVMs with quantum kernels and applying them to satellite data analysis. The design and implementation of SVMs with quantum kernels (hybrid SVMs) is presented. It consists of the Quantum Kernel Estimation (QKE) procedure combined with a classic SVM training routine. The pixel data are mapped to the Hilbert space using ZZ-feature maps acting on the parameterized ansatz state. The parameters are optimized to maximize the kernel target alignment. We approach the problem of cloud detection in satellite image data, which is one of the pivotal steps in both on-the-ground and on-board satellite image analysis processing chains. The experiments performed over the benchmark Landsat-8 multispectral dataset revealed that the simulated hybrid SVM successfully classifies satellite images with accuracy on par with classic SVMs. △ Less

Submitted 14 July, 2023; originally announced July 2023.

Comments: Prepared for IGARSS 2023 Proceedings, 4 pages, 2 figures

Optimizing Kernel-Target Alignment for cloud detection in multispectral satellite images

Authors: Artur Miroszewski, Jakub Mielczarek, Filip Szczepanek, Grzegorz Czelusta, Bartosz Grabowski, Bertrand Le Saux, Jakub Nalepa

Abstract: The optimization of Kernel-Target Alignment (TA) has been recently proposed as a way to reduce the number of hardware resources in quantum classifiers. It allows to exchange highly expressive and costly circuits to moderate size, task oriented ones. In this work we propose a simple toy model to study the optimization landscape of the Kernel-Target Alignment. We find that for underparameterized cir… ▽ More The optimization of Kernel-Target Alignment (TA) has been recently proposed as a way to reduce the number of hardware resources in quantum classifiers. It allows to exchange highly expressive and costly circuits to moderate size, task oriented ones. In this work we propose a simple toy model to study the optimization landscape of the Kernel-Target Alignment. We find that for underparameterized circuits the optimization landscape possess either many local extrema or becomes flat with narrow global extremum. We find the dependence of the width of the global extremum peak on the amount of data introduced to the model. The experimental study was performed using multispectral satellite data, and we targeted the cloud detection task, being one of the most fundamental and important image analysis tasks in remote sensing. △ Less

Submitted 26 June, 2023; originally announced June 2023.

Comments: Prepared for IGARSS 2023 Proceedings, 4 pages, 4 figures

Squeezing nnU-Nets with Knowledge Distillation for On-Board Cloud Detection

Authors: Bartosz Grabowski, Maciej Ziaja, Michal Kawulok, Piotr Bosowski, Nicolas Longépé, Bertrand Le Saux, Jakub Nalepa

Abstract: Cloud detection is a pivotal satellite image pre-processing step that can be performed both on the ground and on board a satellite to tag useful images. In the latter case, it can reduce the amount of data to downlink by pruning the cloudy areas, or to make a satellite more autonomous through data-driven acquisition re-scheduling. We approach this task with nnU-Nets, a self-reconfigurable framewor… ▽ More Cloud detection is a pivotal satellite image pre-processing step that can be performed both on the ground and on board a satellite to tag useful images. In the latter case, it can reduce the amount of data to downlink by pruning the cloudy areas, or to make a satellite more autonomous through data-driven acquisition re-scheduling. We approach this task with nnU-Nets, a self-reconfigurable framework able to perform meta-learning of a segmentation network over various datasets. Unfortunately, such models are commonly memory-inefficient due to their (very) large architectures. To benefit from them in on-board processing, we compress nnU-Nets with knowledge distillation into much smaller and compact U-Nets. Our experiments, performed over Sentinel-2 and Landsat-8 images revealed that nnU-Nets deliver state-of-the-art performance without any manual design. Our approach was ranked within the top 7% best solutions (across 847 teams) in the On Cloud N: Cloud Cover Detection Challenge, where we reached the Jaccard index of 0.882 over more than 10k unseen Sentinel-2 images (the winners obtained 0.897, the baseline U-Net with the ResNet-34 backbone: 0.817, and the classic Sentinel-2 image thresholding: 0.652). Finally, we showed that knowledge distillation enables to elaborate dramatically smaller (almost 280x) U-Nets when compared to nnU-Nets while still maintaining their segmentation capabilities. △ Less

Submitted 16 June, 2023; originally announced June 2023.

Comments: arXiv admin note: substantial text overlap with arXiv:2210.13659

Artificial intelligence to advance Earth observation: : A review of models, recent trends, and pathways forward

Authors: Devis Tuia, Konrad Schindler, Begüm Demir, Xiao Xiang Zhu, Mrinalini Kochupillai, Sašo Džeroski, Jan N. van Rijn, Holger H. Hoos, Fabio Del Frate, Mihai Datcu, Volker Markl, Bertrand Le Saux, Rochelle Schneider, Gustau Camps-Valls

Abstract: Earth observation (EO) is a prime instrument for monitoring land and ocean processes, studying the dynamics at work, and taking the pulse of our planet. This article gives a bird's eye view of the essential scientific tools and approaches informing and supporting the transition from raw EO data to usable EO-based information. The promises, as well as the current challenges of these developments, a… ▽ More Earth observation (EO) is a prime instrument for monitoring land and ocean processes, studying the dynamics at work, and taking the pulse of our planet. This article gives a bird's eye view of the essential scientific tools and approaches informing and supporting the transition from raw EO data to usable EO-based information. The promises, as well as the current challenges of these developments, are highlighted under dedicated sections. Specifically, we cover the impact of (i) Computer vision; (ii) Machine learning; (iii) Advanced processing and computing; (iv) Knowledge-based AI; (v) Explainable AI and causal inference; (vi) Physics-aware models; (vii) User-centric approaches; and (viii) the much-needed discussion of ethical and societal issues related to the massive use of ML technologies in EO. △ Less

Submitted 16 September, 2024; v1 submitted 15 May, 2023; originally announced May 2023.

Journal ref: IEEE Geoscience and Remote Sensing Magazine, 2024

A Single-Step Multiclass SVM based on Quantum Annealing for Remote Sensing Data Classification

Authors: Amer Delilbasic, Bertrand Le Saux, Morris Riedel, Kristel Michielsen, Gabriele Cavallaro

Abstract: In recent years, the development of quantum annealers has enabled experimental demonstrations and has increased research interest in applications of quantum annealing, such as in quantum machine learning and in particular for the popular quantum SVM. Several versions of the quantum SVM have been proposed, and quantum annealing has been shown to be effective in them. Extensions to multiclass proble… ▽ More In recent years, the development of quantum annealers has enabled experimental demonstrations and has increased research interest in applications of quantum annealing, such as in quantum machine learning and in particular for the popular quantum SVM. Several versions of the quantum SVM have been proposed, and quantum annealing has been shown to be effective in them. Extensions to multiclass problems have also been made, which consist of an ensemble of multiple binary classifiers. This work proposes a novel quantum SVM formulation for direct multiclass classification based on quantum annealing, called Quantum Multiclass SVM (QMSVM). The multiclass classification problem is formulated as a single Quadratic Unconstrained Binary Optimization (QUBO) problem solved with quantum annealing. The main objective of this work is to evaluate the feasibility, accuracy, and time performance of this approach. Experiments have been performed on the D-Wave Advantage quantum annealer for a classification problem on remote sensing data. The results indicate that, despite the memory demands of the quantum annealer, QMSVM can achieve accuracy that is comparable to standard SVM methods and, more importantly, it scales much more efficiently with the number of training examples, resulting in nearly constant time. This work shows an approach for bringing together classical and quantum computation, solving practical problems in remote sensing with current hardware. △ Less

Submitted 21 March, 2023; originally announced March 2023.

Comments: 12 pages, 10 figures, 3 tables. Submitted to IEEE JSTARS

Analysis of COVID-19 first wave in the US based on demographic, mobility, and environmental variables

Authors: Dario Spiller, Gabriele Santin, Alessandro Sebastianelli, Lorenzo Lucchini, Riccardo Gallotti, Brennan Lake, Silvia Liberata Ullo, Bertrand Le Saux, Bruno Lepri

Abstract: COVID-19 had a strong and disruptive impact on our society, and yet further analyses on most relevant factors explaining the spread of the pandemic are needed. Interdisciplinary studies linking epidemiological, mobility, environmental, and socio-demographic data analysis can help understanding how historical conditions, concurrent social policies and environmental factors impacted on the evolution… ▽ More COVID-19 had a strong and disruptive impact on our society, and yet further analyses on most relevant factors explaining the spread of the pandemic are needed. Interdisciplinary studies linking epidemiological, mobility, environmental, and socio-demographic data analysis can help understanding how historical conditions, concurrent social policies and environmental factors impacted on the evolution of the pandemic crisis. This work deals with a regression analysis linking COVID-19 mortality to socio-demographic, mobility, and environmental data in the US during the first half of 2020, i.e., during the COVID-19 pandemic first wave. This study can provide very useful insights about risk factors enhancing mortality rates before non-pharmaceutical interventions or vaccination campaigns took place. Our cross-sectional ecological regression analysis demonstrates that, when considering the entire US area, the socio-demographic variables globally play the most important role with respect to environmental and mobility variables in describing COVID-19 mortality. Compared to the complete generalized linear model considering all socio-demographic, mobility, and environmental data, the regression based only on socio-demographic data provides a better approximation and proves to be a better explanatory model when compared to the mobility-based and environmental-based models. However, when looking at single entries within each of the three groups, we see that the mobility data can become relevant descriptive predictors at local scale, as in New Jersey where the time spent at work is one of the most relevant explanatory variables, while environmental data play contradictory roles. △ Less

Submitted 22 February, 2023; originally announced February 2023.

Comments: Submitted to the Scientific Reports, COVID-19 Collection, for possible publication

Detecting Clouds in Multispectral Satellite Images Using Quantum-Kernel Support Vector Machines

Authors: Artur Miroszewski, Jakub Mielczarek, Grzegorz Czelusta, Filip Szczepanek, Bartosz Grabowski, Bertrand Le Saux, Jakub Nalepa

Abstract: Support vector machines (SVMs) are a well-established classifier effectively deployed in an array of classification tasks. In this work, we consider extending classical SVMs with quantum kernels and applying them to satellite data analysis. The design and implementation of SVMs with quantum kernels (hybrid SVMs) are presented. Here, the pixels are mapped to the Hilbert space using a family of para… ▽ More Support vector machines (SVMs) are a well-established classifier effectively deployed in an array of classification tasks. In this work, we consider extending classical SVMs with quantum kernels and applying them to satellite data analysis. The design and implementation of SVMs with quantum kernels (hybrid SVMs) are presented. Here, the pixels are mapped to the Hilbert space using a family of parameterized quantum feature maps (related to quantum kernels). The parameters are optimized to maximize the kernel target alignment. The quantum kernels have been selected such that they enabled analysis of numerous relevant properties while being able to simulate them with classical computers on a real-life large-scale dataset. Specifically, we approach the problem of cloud detection in the multispectral satellite imagery, which is one of the pivotal steps in both on-the-ground and on-board satellite image analysis processing chains. The experiments performed over the benchmark Landsat-8 multispectral dataset revealed that the simulated hybrid SVM successfully classifies satellite images with accuracy comparable to the classical SVM with the RBF kernel for large datasets. Interestingly, for large datasets, the high accuracy was also observed for the simple quantum kernels, lacking quantum entanglement. △ Less

Submitted 16 February, 2023; originally announced February 2023.

Comments: 12 pages, 10 figures

Self-Configuring nnU-Nets Detect Clouds in Satellite Images

Authors: Bartosz Grabowski, Maciej Ziaja, Michal Kawulok, Nicolas Longépé, Bertrand Le Saux, Jakub Nalepa

Abstract: Cloud detection is a pivotal satellite image pre-processing step that can be performed both on the ground and on board a satellite to tag useful images. In the latter case, it can help to reduce the amount of data to downlink by pruning the cloudy areas, or to make a satellite more autonomous through data-driven acquisition re-scheduling of the cloudy areas. We approach this important task with nn… ▽ More Cloud detection is a pivotal satellite image pre-processing step that can be performed both on the ground and on board a satellite to tag useful images. In the latter case, it can help to reduce the amount of data to downlink by pruning the cloudy areas, or to make a satellite more autonomous through data-driven acquisition re-scheduling of the cloudy areas. We approach this important task with nnU-Nets, a self-reconfigurable framework able to perform meta-learning of a segmentation network over various datasets. Our experiments, performed over Sentinel-2 and Landsat-8 multispectral images revealed that nnU-Nets deliver state-of-the-art cloud segmentation performance without any manual design. Our approach was ranked within the top 7% best solutions (across 847 participating teams) in the On Cloud N: Cloud Cover Detection Challenge, where we reached the Jaccard index of 0.882 over more than 10k unseen Sentinel-2 image patches (the winners obtained 0.897, whereas the baseline U-Net with the ResNet-34 backbone used as an encoder: 0.817, and the classic Sentinel-2 image thresholding: 0.652). △ Less

Submitted 24 October, 2022; originally announced October 2022.

A Multibranch Convolutional Neural Network for Hyperspectral Unmixing

Authors: Lukasz Tulczyjew, Michal Kawulok, Nicolas Longépé, Bertrand Le Saux, Jakub Nalepa

Abstract: Hyperspectral unmixing remains one of the most challenging tasks in the analysis of such data. Deep learning has been blooming in the field and proved to outperform other classic unmixing techniques, and can be effectively deployed onboard Earth observation satellites equipped with hyperspectral imagers. In this letter, we follow this research pathway and propose a multi-branch convolutional neura… ▽ More Hyperspectral unmixing remains one of the most challenging tasks in the analysis of such data. Deep learning has been blooming in the field and proved to outperform other classic unmixing techniques, and can be effectively deployed onboard Earth observation satellites equipped with hyperspectral imagers. In this letter, we follow this research pathway and propose a multi-branch convolutional neural network that benefits from fusing spectral, spatial, and spectral-spatial features in the unmixing process. The results of our experiments, backed up with the ablation study, revealed that our techniques outperform others from the literature and lead to higher-quality fractional abundance estimation. Also, we investigated the influence of reducing the training sets on the capabilities of all algorithms and their robustness against noise, as capturing large and representative ground-truth sets is time-consuming and costly in practice, especially in emerging Earth observation scenarios. △ Less

Submitted 3 August, 2022; originally announced August 2022.

Comments: 14 pages (including supplementary material), published in IEEE Geoscience and Remote Sensing Letters

Journal ref: IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1-5, 2022, Art no. 6011105

Graph Neural Networks Extract High-Resolution Cultivated Land Maps from Sentinel-2 Image Series

Authors: Lukasz Tulczyjew, Michal Kawulok, Nicolas Longépé, Bertrand Le Saux, Jakub Nalepa

Abstract: Maintaining farm sustainability through optimizing the agricultural management practices helps build more planet-friendly environment. The emerging satellite missions can acquire multi- and hyperspectral imagery which captures more detailed spectral information concerning the scanned area, hence allows us to benefit from subtle spectral features during the analysis process in agricultural applicat… ▽ More Maintaining farm sustainability through optimizing the agricultural management practices helps build more planet-friendly environment. The emerging satellite missions can acquire multi- and hyperspectral imagery which captures more detailed spectral information concerning the scanned area, hence allows us to benefit from subtle spectral features during the analysis process in agricultural applications. We introduce an approach for extracting 2.5 m cultivated land maps from 10 m Sentinel-2 multispectral image series which benefits from a compact graph convolutional neural network. The experiments indicate that our models not only outperform classical and deep machine learning techniques through delivering higher-quality segmentation maps, but also dramatically reduce the memory footprint when compared to U-Nets (almost 8k trainable parameters of our models, with up to 31M parameters of U-Nets). Such memory frugality is pivotal in the missions which allow us to uplink a model to the AI-powered satellite once it is in orbit, as sending large nets is impossible due to the time constraints. △ Less

Submitted 3 August, 2022; originally announced August 2022.

Comments: 7 pages (including supplementary material), published in IEEE Geoscience and Remote Sensing Letters

Journal ref: IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1-5, 2022, Art no. 5513105

Rapid training of quantum recurrent neural networks

Authors: Michał Siemaszko, Adam Buraczewski, Bertrand Le Saux, Magdalena Stobińska

Abstract: Time series prediction is essential for human activities in diverse areas. A common approach to this task is to harness Recurrent Neural Networks (RNNs). However, while their predictions are quite accurate, their learning process is complex and, thus, time and energy consuming. Here, we propose to extend the concept of RRNs by including continuous-variable quantum resources in it, and to use a qua… ▽ More Time series prediction is essential for human activities in diverse areas. A common approach to this task is to harness Recurrent Neural Networks (RNNs). However, while their predictions are quite accurate, their learning process is complex and, thus, time and energy consuming. Here, we propose to extend the concept of RRNs by including continuous-variable quantum resources in it, and to use a quantum-enhanced RNN to overcome these obstacles. The design of the Continuous-Variable Quantum RNN (CV-QRNN) is rooted in the continuous-variable quantum computing paradigm. By performing extensive numerical simulations, we demonstrate that the quantum network is capable of learning-time dependence of several types of temporal data, and that it converges to the optimal weights in fewer epochs than a classical network. Furthermore, for a small number of trainable parameters, it can achieve lower losses than its classical counterpart. CV-QRNN can be implemented using commercially available quantum-photonic hardware. △ Less

Submitted 7 March, 2023; v1 submitted 1 July, 2022; originally announced July 2022.

Report number: 5 (31)

Journal ref: Quantum Mach. Intell. 5, 31 (2023)

Self-supervised learning -- A way to minimize time and effort for precision agriculture?

Authors: Michael L. Marszalek, Bertrand Le Saux, Pierre-Philippe Mathieu, Artur Nowakowski, Daniel Springer

Abstract: Machine learning, satellites or local sensors are key factors for a sustainable and resource-saving optimisation of agriculture and proved its values for the management of agricultural land. Up to now, the main focus was on the enlargement of data which were evaluated by means of supervised learning methods. Nevertheless, the need for labels is also a limiting and time-consuming factor, while in c… ▽ More Machine learning, satellites or local sensors are key factors for a sustainable and resource-saving optimisation of agriculture and proved its values for the management of agricultural land. Up to now, the main focus was on the enlargement of data which were evaluated by means of supervised learning methods. Nevertheless, the need for labels is also a limiting and time-consuming factor, while in contrast, ongoing technological development is already providing an ever-increasing amount of unlabeled data. Self-supervised learning (SSL) could overcome this limitation and incorporate existing unlabeled data. Therefore, a crop type data set was utilized to conduct experiments with SSL and compare it to supervised methods. A unique feature of our data set from 2016 to 2018 was a divergent climatological condition in 2018 that reduced yields and affected the spectral fingerprint of the plants. Our experiments focused on predicting 2018 using SLL without or a few labels to clarify whether new labels should be collected for an unknown year. Despite these challenging conditions, the results showed that SSL contributed to higher accuracies. We believe that the results will encourage further improvements in the field of precision farming, why the SSL framework and data will be published (Marszalek, 2021). △ Less

Submitted 5 April, 2022; originally announced April 2022.

Comments: Accepted for ISPRS Archives 2022

Beyond Ansätze: Learning Quantum Circuits as Unitary Operators

Authors: Bálint Máté, Bertrand Le Saux, Maxwell Henderson

Abstract: This paper explores the advantages of optimizing quantum circuits on $N$ wires as operators in the unitary group $U(2^N)$. We run gradient-based optimization in the Lie algebra $\mathfrak u(2^N)$ and use the exponential map to parametrize unitary matrices. We argue that $U(2^N)$ is not only more general than the search space induced by an ansatz, but in ways easier to work with on classical comput… ▽ More This paper explores the advantages of optimizing quantum circuits on $N$ wires as operators in the unitary group $U(2^N)$. We run gradient-based optimization in the Lie algebra $\mathfrak u(2^N)$ and use the exponential map to parametrize unitary matrices. We argue that $U(2^N)$ is not only more general than the search space induced by an ansatz, but in ways easier to work with on classical computers. The resulting approach is quick, ansatz-free and provides an upper bound on performance over all ansätze on $N$ wires. △ Less

Submitted 3 March, 2022; v1 submitted 1 March, 2022; originally announced March 2022.

DIAL: Deep Interactive and Active Learning for Semantic Segmentation in Remote Sensing

Authors: Gaston Lenczner, Adrien Chan-Hon-Tong, Bertrand Le Saux, Nicola Luminari, Guy Le Besnerais

Abstract: We propose in this article to build up a collaboration between a deep neural network and a human in the loop to swiftly obtain accurate segmentation maps of remote sensing images. In a nutshell, the agent iteratively interacts with the network to correct its initially flawed predictions. Concretely, these interactions are annotations representing the semantic labels. Our methodological contributio… ▽ More We propose in this article to build up a collaboration between a deep neural network and a human in the loop to swiftly obtain accurate segmentation maps of remote sensing images. In a nutshell, the agent iteratively interacts with the network to correct its initially flawed predictions. Concretely, these interactions are annotations representing the semantic labels. Our methodological contribution is twofold. First, we propose two interactive learning schemes to integrate user inputs into deep neural networks. The first one concatenates the annotations with the other network's inputs. The second one uses the annotations as a sparse ground-truth to retrain the network. Second, we propose an active learning strategy to guide the user towards the most relevant areas to annotate. To this purpose, we compare different state-of-the-art acquisition functions to evaluate the neural network uncertainty such as ConfidNet, entropy or ODIN. Through experiments on three remote sensing datasets, we show the effectiveness of the proposed methods. Notably, we show that active learning based on uncertainty estimation enables to quickly lead the user towards mistakes and that it is thus relevant to guide the user interventions. △ Less

Submitted 4 January, 2022; originally announced January 2022.

Weakly-supervised continual learning for class-incremental segmentation

Authors: Gaston Lenczner, Adrien Chan-Hon-Tong, Nicola Luminari, Bertrand Le Saux

Abstract: Transfer learning is a powerful way to adapt existing deep learning models to new emerging use-cases in remote sensing. Starting from a neural network already trained for semantic segmentation, we propose to modify its label space to swiftly adapt it to new classes under weak supervision. To alleviate the background shift and the catastrophic forgetting problems inherent to this form of continual… ▽ More Transfer learning is a powerful way to adapt existing deep learning models to new emerging use-cases in remote sensing. Starting from a neural network already trained for semantic segmentation, we propose to modify its label space to swiftly adapt it to new classes under weak supervision. To alleviate the background shift and the catastrophic forgetting problems inherent to this form of continual learning, we compare different regularization terms and leverage a pseudo-label strategy. We experimentally show the relevance of our approach on three public remote sensing datasets. Code is open-source and released in this repository: https://github.com/alteia-ai/ICSS}{https://github.com/alteia-ai/ICSS. △ Less

Submitted 15 June, 2022; v1 submitted 4 January, 2022; originally announced January 2022.

Weakly Supervised Change Detection Using Guided Anisotropic Difusion

Authors: Rodrigo Caye Daudt, Bertrand Le Saux, Alexandre Boulch, Yann Gousseau

Abstract: Large scale datasets created from crowdsourced labels or openly available data have become crucial to provide training data for large scale learning algorithms. While these datasets are easier to acquire, the data are frequently noisy and unreliable, which is motivating research on weakly supervised learning techniques. In this paper we propose original ideas that help us to leverage such datasets… ▽ More Large scale datasets created from crowdsourced labels or openly available data have become crucial to provide training data for large scale learning algorithms. While these datasets are easier to acquire, the data are frequently noisy and unreliable, which is motivating research on weakly supervised learning techniques. In this paper we propose original ideas that help us to leverage such datasets in the context of change detection. First, we propose the guided anisotropic diffusion (GAD) algorithm, which improves semantic segmentation results using the input images as guides to perform edge preserving filtering. We then show its potential in two weakly-supervised learning strategies tailored for change detection. The first strategy is an iterative learning method that combines model optimisation and data cleansing using GAD to extract the useful information from a large scale change detection dataset generated from open vector data. The second one incorporates GAD within a novel spatial attention layer that increases the accuracy of weakly supervised networks trained to perform pixel-level predictions from image-level labels. Improvements with respect to state-of-the-art are demonstrated on 4 different public datasets. △ Less

Submitted 31 December, 2021; originally announced December 2021.

Comments: Machine Learning Journal 2021. arXiv admin note: substantial text overlap with arXiv:1904.08208

How to find a good image-text embedding for remote sensing visual question answering?

Authors: Christel Chappuis, Sylvain Lobry, Benjamin Kellenberger, Bertrand Le Saux, Devis Tuia

Abstract: Visual question answering (VQA) has recently been introduced to remote sensing to make information extraction from overhead imagery more accessible to everyone. VQA considers a question (in natural language, therefore easy to formulate) about an image and aims at providing an answer through a model based on computer vision and natural language processing methods. As such, a VQA model needs to join… ▽ More Visual question answering (VQA) has recently been introduced to remote sensing to make information extraction from overhead imagery more accessible to everyone. VQA considers a question (in natural language, therefore easy to formulate) about an image and aims at providing an answer through a model based on computer vision and natural language processing methods. As such, a VQA model needs to jointly consider visual and textual features, which is frequently done through a fusion step. In this work, we study three different fusion methodologies in the context of VQA for remote sensing and analyse the gains in accuracy with respect to the model complexity. Our findings indicate that more complex fusion mechanisms yield an improved performance, yet that seeking a trade-of between model complexity and performance is worthwhile in practice. △ Less

Submitted 24 September, 2021; originally announced September 2021.

Comments: 10 pages, 4 figures, presented in the MACLEAN workshop during ECML PKDD 2021

On Circuit-based Hybrid Quantum Neural Networks for Remote Sensing Imagery Classification

Authors: Alessandro Sebastianelli, Daniela A. Zaidenberg, Dario Spiller, Bertrand Le Saux, Silvia Liberata Ullo

Abstract: This article aims to investigate how circuit-based hybrid Quantum Convolutional Neural Networks (QCNNs) can be successfully employed as image classifiers in the context of remote sensing. The hybrid QCNNs enrich the classical architecture of CNNs by introducing a quantum layer within a standard neural network. The novel QCNN proposed in this work is applied to the Land Use and Land Cover (LULC) cl… ▽ More This article aims to investigate how circuit-based hybrid Quantum Convolutional Neural Networks (QCNNs) can be successfully employed as image classifiers in the context of remote sensing. The hybrid QCNNs enrich the classical architecture of CNNs by introducing a quantum layer within a standard neural network. The novel QCNN proposed in this work is applied to the Land Use and Land Cover (LULC) classification, chosen as an Earth Observation (EO) use case, and tested on the EuroSAT dataset used as reference benchmark. The results of the multiclass classification prove the effectiveness of the presented approach, by demonstrating that the QCNN performances are higher than the classical counterparts. Moreover, investigation of various quantum circuits shows that the ones exploiting quantum entanglement achieve the best classification scores. This study underlines the potentialities of applying quantum computing to an EO case study and provides the theoretical and experimental background for futures investigations. △ Less

Submitted 1 December, 2021; v1 submitted 20 September, 2021; originally announced September 2021.

Comments: Submitted to the JSTARS special issue on "Quantum resources for Earth Observation" for possible publication

Pix2Point: Learning Outdoor 3D Using Sparse Point Clouds and Optimal Transport

Authors: Rémy Leroy, Pauline Trouvé-Peloux, Frédéric Champagnat, Bertrand Le Saux, Marcela Carvalho

Abstract: Good quality reconstruction and comprehension of a scene rely on 3D estimation methods. The 3D information was usually obtained from images by stereo-photogrammetry, but deep learning has recently provided us with excellent results for monocular depth estimation. Building up a sufficiently large and rich training dataset to achieve these results requires onerous processing. In this paper, we addre… ▽ More Good quality reconstruction and comprehension of a scene rely on 3D estimation methods. The 3D information was usually obtained from images by stereo-photogrammetry, but deep learning has recently provided us with excellent results for monocular depth estimation. Building up a sufficiently large and rich training dataset to achieve these results requires onerous processing. In this paper, we address the problem of learning outdoor 3D point cloud from monocular data using a sparse ground-truth dataset. We propose Pix2Point, a deep learning-based approach for monocular 3D point cloud prediction, able to deal with complete and challenging outdoor scenes. Our method relies on a 2D-3D hybrid neural network architecture, and a supervised end-to-end minimisation of an optimal transport divergence between point clouds. We show that, when trained on sparse point clouds, our simple promising approach achieves a better coverage of 3D outdoor scenes than efficient monocular depth methods. △ Less

Submitted 30 July, 2021; originally announced July 2021.

Comments: 5 pages, 2 figures, to be published in 2021 International Conference on Machine Vision Applications

ACM Class: I.4.5; I.2.10

Advantages and Bottlenecks of Quantum Machine Learning for Remote Sensing

Authors: Daniela A. Zaidenberg, Alessandro Sebastianelli, Dario Spiller, Bertrand Le Saux, Silvia Liberata Ullo

Abstract: This concept paper aims to provide a brief outline of quantum computers, explore existing methods of quantum image classification techniques, so focusing on remote sensing applications, and discuss the bottlenecks of performing these algorithms on currently available open source platforms. Initial results demonstrate feasibility. Next steps include expanding the size of the quantum hidden layer an… ▽ More This concept paper aims to provide a brief outline of quantum computers, explore existing methods of quantum image classification techniques, so focusing on remote sensing applications, and discuss the bottlenecks of performing these algorithms on currently available open source platforms. Initial results demonstrate feasibility. Next steps include expanding the size of the quantum hidden layer and increasing the variety of output image options. △ Less

Submitted 30 June, 2021; v1 submitted 26 January, 2021; originally announced January 2021.

Comments: Submitted and accepted for IEEE IGARSS2021

Street to Cloud: Improving Flood Maps With Crowdsourcing and Semantic Segmentation

Authors: Veda Sunkara, Matthew Purri, Bertrand Le Saux, Jennifer Adams

Abstract: To address the mounting destruction caused by floods in climate-vulnerable regions, we propose Street to Cloud, a machine learning pipeline for incorporating crowdsourced ground truth data into the segmentation of satellite imagery of floods. We propose this approach as a solution to the labor-intensive task of generating high-quality, hand-labeled training data, and demonstrate successes and fail… ▽ More To address the mounting destruction caused by floods in climate-vulnerable regions, we propose Street to Cloud, a machine learning pipeline for incorporating crowdsourced ground truth data into the segmentation of satellite imagery of floods. We propose this approach as a solution to the labor-intensive task of generating high-quality, hand-labeled training data, and demonstrate successes and failures of different plausible crowdsourcing approaches in our model. Street to Cloud leverages community reporting and machine learning to generate novel, near-real time insights into the extent of floods to be used for emergency response. △ Less

Submitted 5 November, 2020; originally announced November 2020.

Comments: 5 pages, 2 figures, Tackling Climate Change with Machine Learning workshop at NeurIPS 2020

Semi-Supervised Semantic Segmentation in Earth Observation: The MiniFrance Suite, Dataset Analysis and Multi-task Network Study

Authors: Javiera Castillo-Navarro, Bertrand Le Saux, Alexandre Boulch, Nicolas Audebert, Sébastien Lefèvre

Abstract: The development of semi-supervised learning techniques is essential to enhance the generalization capacities of machine learning algorithms. Indeed, raw image data are abundant while labels are scarce, therefore it is crucial to leverage unlabeled inputs to build better models. The availability of large databases have been key for the development of learning algorithms with high level performance.… ▽ More The development of semi-supervised learning techniques is essential to enhance the generalization capacities of machine learning algorithms. Indeed, raw image data are abundant while labels are scarce, therefore it is crucial to leverage unlabeled inputs to build better models. The availability of large databases have been key for the development of learning algorithms with high level performance. Despite the major role of machine learning in Earth Observation to derive products such as land cover maps, datasets in the field are still limited, either because of modest surface coverage, lack of variety of scenes or restricted classes to identify. We introduce a novel large-scale dataset for semi-supervised semantic segmentation in Earth Observation, the MiniFrance suite. MiniFrance has several unprecedented properties: it is large-scale, containing over 2000 very high resolution aerial images, accounting for more than 200 billions samples (pixels); it is varied, covering 16 conurbations in France, with various climates, different landscapes, and urban as well as countryside scenes; and it is challenging, considering land use classes with high-level semantics. Nevertheless, the most distinctive quality of MiniFrance is being the only dataset in the field especially designed for semi-supervised learning: it contains labeled and unlabeled images in its training partition, which reproduces a life-like scenario. Along with this dataset, we present tools for data representativeness analysis in terms of appearance similarity and a thorough study of MiniFrance data, demonstrating that it is suitable for learning and generalizes well in a semi-supervised setting. Finally, we present semi-supervised deep architectures based on multi-task learning and the first experiments on MiniFrance. △ Less

Submitted 15 October, 2020; originally announced October 2020.

Interactive Learning for Semantic Segmentation in Earth Observation

Authors: Gaston Lenczner, Adrien Chan-Hon-Tong, Nicola Luminari, Bertrand Le Saux, Guy Le Besnerais

Abstract: Dense pixel-wise classification maps output by deep neural networks are of extreme importance for scene understanding. However, these maps are often partially inaccurate due to a variety of possible factors. Therefore, we propose to interactively refine them within a framework named DISCA (Deep Image Segmentation with Continual Adaptation). It consists of continually adapting a neural network to a… ▽ More Dense pixel-wise classification maps output by deep neural networks are of extreme importance for scene understanding. However, these maps are often partially inaccurate due to a variety of possible factors. Therefore, we propose to interactively refine them within a framework named DISCA (Deep Image Segmentation with Continual Adaptation). It consists of continually adapting a neural network to a target image using an interactive learning process with sparse user annotations as ground-truth. We show through experiments on three datasets using synthesized annotations the benefits of the approach, reaching an IoU improvement up to 4.7% for ten sampled clicks. Finally, we exhibit that our approach can be particularly rewarding when it is faced to additional issues such as domain adaptation. △ Less

Submitted 23 September, 2020; originally announced September 2020.

Comments: 8 pages, 4 Figures, ECML-PKDD Workshop MACLEAN

DISIR: Deep Image Segmentation with Interactive Refinement

Authors: Gaston Lenczner, Bertrand Le Saux, Nicola Luminari, Adrien Chan Hon Tong, Guy Le Besnerais

Abstract: This paper presents an interactive approach for multi-class segmentation of aerial images. Precisely, it is based on a deep neural network which exploits both RGB images and annotations. Starting from an initial output based on the image only, our network then interactively refines this segmentation map using a concatenation of the image and user annotations. Importantly, user annotations modify t… ▽ More This paper presents an interactive approach for multi-class segmentation of aerial images. Precisely, it is based on a deep neural network which exploits both RGB images and annotations. Starting from an initial output based on the image only, our network then interactively refines this segmentation map using a concatenation of the image and user annotations. Importantly, user annotations modify the inputs of the network - not its weights - enabling a fast and smooth process. Through experiments on two public aerial datasets, we show that user annotations are extremely rewarding: each click corrects roughly 5000 pixels. We analyze the impact of different aspects of our framework such as the representation of the annotations, the volume of training data or the network architecture. Code is available at https://github.com/delair-ai/DISIR. △ Less

Submitted 20 August, 2020; v1 submitted 31 March, 2020; originally announced March 2020.

Comments: 8 pages, 12 figures. Published in the ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences

Journal ref: XXIV ISPRS Congress, Commission II (Volume V-2-2020)

Multi-Task Learning of Height and Semantics from Aerial Images

Authors: Marcela Carvalho, Bertrand Le Saux, Pauline Trouvé-Peloux, Frédéric Champagnat, Andrés Almansa

Abstract: Aerial or satellite imagery is a great source for land surface analysis, which might yield land use maps or elevation models. In this investigation, we present a neural network framework for learning semantics and local height together. We show how this joint multi-task learning benefits to each task on the large dataset of the 2018 Data Fusion Contest. Moreover, our framework also yields an uncer… ▽ More Aerial or satellite imagery is a great source for land surface analysis, which might yield land use maps or elevation models. In this investigation, we present a neural network framework for learning semantics and local height together. We show how this joint multi-task learning benefits to each task on the large dataset of the 2018 Data Fusion Contest. Moreover, our framework also yields an uncertainty map which allows assessing the prediction of the model. Code is available at https://github.com/marcelampc/mtl_aerial_images . △ Less

Submitted 18 November, 2019; originally announced November 2019.

Comments: Published IEEE Geoscience and Remote Sensing Letters. Code https://github.com/marcelampc/mtl_aerial_images

Technical Report: Co-learning of geometry and semantics for online 3D mapping

Authors: Marcela Carvalho, Maxime Ferrera, Alexandre Boulch, Julien Moras, Bertrand Le Saux, Pauline Trouvé-Peloux

Abstract: This paper is a technical report about our submission for the ECCV 2018 3DRMS Workshop Challenge on Semantic 3D Reconstruction \cite{Tylecek2018rms}. In this paper, we address 3D semantic reconstruction for autonomous navigation using co-learning of depth map and semantic segmentation. The core of our pipeline is a deep multi-task neural network which tightly refines depth and also produces accur… ▽ More This paper is a technical report about our submission for the ECCV 2018 3DRMS Workshop Challenge on Semantic 3D Reconstruction \cite{Tylecek2018rms}. In this paper, we address 3D semantic reconstruction for autonomous navigation using co-learning of depth map and semantic segmentation. The core of our pipeline is a deep multi-task neural network which tightly refines depth and also produces accurate semantic segmentation maps. Its inputs are an image and a raw depth map produced from a pair of images by standard stereo vision. The resulting semantic 3D point clouds are then merged in order to create a consistent 3D mesh, in turn used to produce dense semantic 3D reconstruction maps. The performances of each step of the proposed method are evaluated on the dataset and multiple tasks of the 3DRMS Challenge, and repeatedly surpass state-of-the-art approaches. △ Less

Submitted 4 November, 2019; originally announced November 2019.

Distance transform regression for spatially-aware deep semantic segmentation

Authors: Nicolas Audebert, Alexandre Boulch, Bertrand Le Saux, Sébastien Lefèvre

Abstract: Understanding visual scenes relies more and more on dense pixel-wise classification obtained via deep fully convolutional neural networks. However, due to the nature of the networks, predictions often suffer from blurry boundaries and ill-segmented shapes, fueling the need for post-processing. This work introduces a new semantic segmentation regularization based on the regression of a distance tra… ▽ More Understanding visual scenes relies more and more on dense pixel-wise classification obtained via deep fully convolutional neural networks. However, due to the nature of the networks, predictions often suffer from blurry boundaries and ill-segmented shapes, fueling the need for post-processing. This work introduces a new semantic segmentation regularization based on the regression of a distance transform. After computing the distance transform on the label masks, we train a FCN in a multi-task setting in both discrete and continuous spaces by learning jointly classification and distance regression. This requires almost no modification of the network structure and adds a very low overhead to the training process. Learning to approximate the distance transform back-propagates spatial cues that implicitly regularizes the segmentation. We validate this technique with several architectures on various datasets, and we show significant improvements compared to competitive baselines. △ Less

Submitted 4 September, 2019; originally announced September 2019.

Guided Anisotropic Diffusion and Iterative Learning for Weakly Supervised Change Detection

Authors: Rodrigo Caye Daudt, Bertrand Le Saux, Alexandre Boulch, Yann Gousseau

Abstract: Large scale datasets created from user labels or openly available data have become crucial to provide training data for large scale learning algorithms. While these datasets are easier to acquire, the data are frequently noisy and unreliable, which is motivating research on weakly supervised learning techniques. In this paper we propose an iterative learning method that extracts the useful informa… ▽ More Large scale datasets created from user labels or openly available data have become crucial to provide training data for large scale learning algorithms. While these datasets are easier to acquire, the data are frequently noisy and unreliable, which is motivating research on weakly supervised learning techniques. In this paper we propose an iterative learning method that extracts the useful information from a large scale change detection dataset generated from open vector data to train a fully convolutional network which surpasses the performance obtained by naive supervised learning. We also propose the guided anisotropic diffusion algorithm, which improves semantic segmentation results using the input images as guides to perform edge preserving filtering, and is used in conjunction with the iterative training method to improve results. △ Less

Submitted 17 April, 2019; originally announced April 2019.

Comments: Accepted at CVPR 2019 Workshops

Urban Change Detection for Multispectral Earth Observation Using Convolutional Neural Networks

Authors: Rodrigo Caye Daudt, Bertrand Le Saux, Alexandre Boulch, Yann Gousseau

Abstract: The Copernicus Sentinel-2 program now provides multispectral images at a global scale with a high revisit rate. In this paper we explore the usage of convolutional neural networks for urban change detection using such multispectral images. We first present the new change detection dataset that was used for training the proposed networks, which will be openly available to serve as a benchmark. The… ▽ More The Copernicus Sentinel-2 program now provides multispectral images at a global scale with a high revisit rate. In this paper we explore the usage of convolutional neural networks for urban change detection using such multispectral images. We first present the new change detection dataset that was used for training the proposed networks, which will be openly available to serve as a benchmark. The Onera Satellite Change Detection (OSCD) dataset is composed of pairs of multispectral aerial images, and the changes were manually annotated at pixel level. We then propose two architectures to detect changes, Siamese and Early Fusion, and compare the impact of using different numbers of spectral channels as inputs. These architectures are trained from scratch using the provided dataset. △ Less

Submitted 19 October, 2018; originally announced October 2018.

Comments: To appear inProc. IGARSS 2018, July 22-27, 2018, Valencia, Spain