Showing 1–31 of 31 results for author: Silva, B

TerraTrace: Temporal Signature Land Use Mapping System

Authors: Angela Busheska, Vikram Iyer, Bruno Silva, Peder Olsen, Ranveer Chandra, Vaishnavi Ranganathan

Abstract: Understanding land use over time is critical to tracking events related to climate change, like deforestation. However, satellite-based remote sensing tools which are used for monitoring struggle to differentiate vegetation types in farms and orchards from forests. We observe that metrics such as the Normalized Difference Vegetation Index (NDVI), based on plant photosynthesis, have unique temporal… ▽ More Understanding land use over time is critical to tracking events related to climate change, like deforestation. However, satellite-based remote sensing tools which are used for monitoring struggle to differentiate vegetation types in farms and orchards from forests. We observe that metrics such as the Normalized Difference Vegetation Index (NDVI), based on plant photosynthesis, have unique temporal signatures that reflect agricultural practices and seasonal cycles. We analyze yearly NDVI changes on 20 farms for 10 unique crops. Initial results show that NDVI curves are coherent with agricultural practices, are unique to each crop, consistent globally, and can differentiate farms from forests. We develop a novel longitudinal NDVI dataset for the state of California from 2020-2023 with 500~m resolution and over 70 million points. We use this to develop the TerraTrace platform, an end-to-end analytic tool that classifies land use using NDVI signatures and allows users to query the system through an LLM chatbot and graphical interface. △ Less

Submitted 25 February, 2025; originally announced February 2025.

Machine Learning for Spectrum Sharing: A Survey

Authors: Francisco R. V. Guimarães, José Mairton B. da Silva Jr., Charles Casimiro Cavalcante, Gabor Fodor, Mats Bengtsson, Carlo Fischione

Abstract: The 5th generation (5G) of wireless systems is being deployed with the aim to provide many sets of wireless communication services, such as low data rates for a massive amount of devices, broadband, low latency, and industrial wireless access. Such an aim is even more complex in the next generation wireless systems (6G) where wireless connectivity is expected to serve any connected intelligent uni… ▽ More The 5th generation (5G) of wireless systems is being deployed with the aim to provide many sets of wireless communication services, such as low data rates for a massive amount of devices, broadband, low latency, and industrial wireless access. Such an aim is even more complex in the next generation wireless systems (6G) where wireless connectivity is expected to serve any connected intelligent unit, such as software robots and humans interacting in the metaverse, autonomous vehicles, drones, trains, or smart sensors monitoring cities, buildings, and the environment. Because of the wireless devices will be orders of magnitude denser than in 5G cellular systems, and because of their complex quality of service requirements, the access to the wireless spectrum will have to be appropriately shared to avoid congestion, poor quality of service, or unsatisfactory communication delays. Spectrum sharing methods have been the objective of intense study through model-based approaches, such as optimization or game theories. However, these methods may fail when facing the complexity of the communication environments in 5G, 6G, and beyond. Recently, there has been significant interest in the application and development of data-driven methods, namely machine learning methods, to handle the complex operation of spectrum sharing. In this survey, we provide a complete overview of the state-of-theart of machine learning for spectrum sharing. First, we map the most prominent methods that we encounter in spectrum sharing. Then, we show how these machine learning methods are applied to the numerous dimensions and sub-problems of spectrum sharing, such as spectrum sensing, spectrum allocation, spectrum access, and spectrum handoff. We also highlight several open questions and future trends. △ Less

Submitted 28 November, 2024; originally announced November 2024.

Comments: Published at NOW Foundations and Trends in Networking

Journal ref: Foundations and Trends in Networking: Vol. 14: No. 1-2, pp 1-159, 2024

Iterative Occlusion-Aware Light Field Depth Estimation using 4D Geometrical Cues

Authors: Rui Lourenço, Lucas Thomaz, Eduardo A. B. Silva, Sergio M. M. Faria

Abstract: Light field cameras and multi-camera arrays have emerged as promising solutions for accurately estimating depth by passively capturing light information. This is possible because the 3D information of a scene is embedded in the 4D light field geometry. Commonly, depth estimation methods extract this information relying on gradient information, heuristic-based optimisation models, or learning-based… ▽ More Light field cameras and multi-camera arrays have emerged as promising solutions for accurately estimating depth by passively capturing light information. This is possible because the 3D information of a scene is embedded in the 4D light field geometry. Commonly, depth estimation methods extract this information relying on gradient information, heuristic-based optimisation models, or learning-based approaches. This paper focuses mainly on explicitly understanding and exploiting 4D geometrical cues for light field depth estimation. Thus, a novel method is proposed, based on a non-learning-based optimisation approach for depth estimation that explicitly considers surface normal accuracy and occlusion regions by utilising a fully explainable 4D geometric model of the light field. The 4D model performs depth/disparity estimation by determining the orientations and analysing the intersections of key 2D planes in 4D space, which are the images of 3D-space points in the 4D light field. Experimental results show that the proposed method outperforms both learning-based and non-learning-based state-of-the-art methods in terms of surface normal angle accuracy, achieving a Median Angle Error on planar surfaces, on average, 26.3$\%$ lower than the state-of-the-art, and still being competitive with state-of-the-art methods in terms of MSE ${\times}$ 100 and Badpix 0.07. △ Less

Submitted 14 May, 2025; v1 submitted 4 March, 2024; originally announced March 2024.

PyKoopman: A Python Package for Data-Driven Approximation of the Koopman Operator

Authors: Shaowu Pan, Eurika Kaiser, Brian M. de Silva, J. Nathan Kutz, Steven L. Brunton

Abstract: PyKoopman is a Python package for the data-driven approximation of the Koopman operator associated with a dynamical system. The Koopman operator is a principled linear embedding of nonlinear dynamics and facilitates the prediction, estimation, and control of strongly nonlinear dynamics using linear systems theory. In particular, PyKoopman provides tools for data-driven system identification for un… ▽ More PyKoopman is a Python package for the data-driven approximation of the Koopman operator associated with a dynamical system. The Koopman operator is a principled linear embedding of nonlinear dynamics and facilitates the prediction, estimation, and control of strongly nonlinear dynamics using linear systems theory. In particular, PyKoopman provides tools for data-driven system identification for unforced and actuated systems that build on the equation-free dynamic mode decomposition (DMD) and its variants. In this work, we provide a brief description of the mathematical underpinnings of the Koopman operator, an overview and demonstration of the features implemented in PyKoopman (with code examples), practical advice for users, and a list of potential extensions to PyKoopman. Software is available at http://github.com/dynamicslab/pykoopman △ Less

Submitted 22 June, 2023; originally announced June 2023.

Comments: 16 pages

eXplainable Artificial Intelligence on Medical Images: A Survey

Authors: Matteus Vargas Simão da Silva, Rodrigo Reis Arrais, Jhessica Victoria Santos da Silva, Felipe Souza Tânios, Mateus Antonio Chinelatto, Natalia Backhaus Pereira, Renata De Paris, Lucas Cesar Ferreira Domingos, Rodrigo Dória Villaça, Vitor Lopes Fabris, Nayara Rossi Brito da Silva, Ana Claudia Akemi Matsuki de Faria, Jose Victor Nogueira Alves da Silva, Fabiana Cristina Queiroz de Oliveira Marucci, Francisco Alves de Souza Neto, Danilo Xavier Silva, Vitor Yukio Kondo, Claudio Filipi Gonçalves dos Santos

Abstract: Over the last few years, the number of works about deep learning applied to the medical field has increased enormously. The necessity of a rigorous assessment of these models is required to explain these results to all people involved in medical exams. A recent field in the machine learning area is explainable artificial intelligence, also known as XAI, which targets to explain the results of such… ▽ More Over the last few years, the number of works about deep learning applied to the medical field has increased enormously. The necessity of a rigorous assessment of these models is required to explain these results to all people involved in medical exams. A recent field in the machine learning area is explainable artificial intelligence, also known as XAI, which targets to explain the results of such black box models to permit the desired assessment. This survey analyses several recent studies in the XAI field applied to medical diagnosis research, allowing some explainability of the machine learning results in several different diseases, such as cancers and COVID-19. △ Less

Submitted 12 May, 2023; originally announced May 2023.

Computing Functions Over-the-Air Using Digital Modulations

Authors: Saeed Razavikia, Jose Mairton Barros da Silva Jr, Carlo Fischione

Abstract: Over-the-air computation (AirComp) is a known technique in which wireless devices transmit values by analog amplitude modulation so that a function of these values is computed over the communication channel at a common receiver. The physical reason is the superposition properties of the electromagnetic waves, which naturally return sums of analog values. Consequently, the applications of AirComp a… ▽ More Over-the-air computation (AirComp) is a known technique in which wireless devices transmit values by analog amplitude modulation so that a function of these values is computed over the communication channel at a common receiver. The physical reason is the superposition properties of the electromagnetic waves, which naturally return sums of analog values. Consequently, the applications of AirComp are almost entirely restricted to analog communication systems. However, the use of digital communications for over-the-air computations would have several benefits, such as error correction, synchronization, acquisition of channel state information, and easier adoption by current digital communication systems. Nevertheless, a common belief is that digital modulations are generally unfeasible for computation tasks because the overlapping of digitally modulated signals returns signals that seem to be meaningless for these tasks. This paper breaks through such a belief and proposes a fundamentally new computing method, named ChannelComp, for performing over-the-air computations by any digital modulation. In particular, we propose digital modulation formats that allow us to compute a wider class of functions than AirComp can compute, and we propose a feasibility optimization problem that ascertains the optimal digital modulation for computing functions over-the-air. The simulation results verify the superior performance of ChannelComp in comparison to AirComp, particularly for the product functions, with around 10 dB improvement of the computation error. △ Less

Submitted 20 March, 2023; v1 submitted 1 March, 2023; originally announced March 2023.

Comments: submitted version to the IEEE ICC conference

SurgT challenge: Benchmark of Soft-Tissue Trackers for Robotic Surgery

Authors: Joao Cartucho, Alistair Weld, Samyakh Tukra, Haozheng Xu, Hiroki Matsuzaki, Taiyo Ishikawa, Minjun Kwon, Yong Eun Jang, Kwang-Ju Kim, Gwang Lee, Bizhe Bai, Lueder Kahrs, Lars Boecking, Simeon Allmendinger, Leopold Muller, Yitong Zhang, Yueming Jin, Sophia Bano, Francisco Vasconcelos, Wolfgang Reiter, Jonas Hajek, Bruno Silva, Estevao Lima, Joao L. Vilaca, Sandro Queiros , et al. (1 additional authors not shown)

Abstract: This paper introduces the ``SurgT: Surgical Tracking" challenge which was organised in conjunction with MICCAI 2022. There were two purposes for the creation of this challenge: (1) the establishment of the first standardised benchmark for the research community to assess soft-tissue trackers; and (2) to encourage the development of unsupervised deep learning methods, given the lack of annotated da… ▽ More This paper introduces the ``SurgT: Surgical Tracking" challenge which was organised in conjunction with MICCAI 2022. There were two purposes for the creation of this challenge: (1) the establishment of the first standardised benchmark for the research community to assess soft-tissue trackers; and (2) to encourage the development of unsupervised deep learning methods, given the lack of annotated data in surgery. A dataset of 157 stereo endoscopic videos from 20 clinical cases, along with stereo camera calibration parameters, have been provided. Participants were assigned the task of developing algorithms to track the movement of soft tissues, represented by bounding boxes, in stereo endoscopic videos. At the end of the challenge, the developed methods were assessed on a previously hidden test subset. This assessment uses benchmarking metrics that were purposely developed for this challenge, to verify the efficacy of unsupervised deep learning algorithms in tracking soft-tissue. The metric used for ranking the methods was the Expected Average Overlap (EAO) score, which measures the average overlap between a tracker's and the ground truth bounding boxes. Coming first in the challenge was the deep learning submission by ICVS-2Ai with a superior EAO score of 0.617. This method employs ARFlow to estimate unsupervised dense optical flow from cropped images, using photometric and regularization losses. Second, Jmees with an EAO of 0.583, uses deep learning for surgical tool segmentation on top of a non-deep learning baseline method: CSRT. CSRT by itself scores a similar EAO of 0.563. The results from this challenge show that currently, non-deep learning methods are still competitive. The dataset and benchmarking tool created for this challenge have been made publicly available at https://surgt.grand-challenge.org/. △ Less

Submitted 30 August, 2023; v1 submitted 6 February, 2023; originally announced February 2023.

Physical Layer Security Techniques Applied to Vehicle-to-Everything Networks

Authors: Leonardo Barbosa da Silva, Evelio Martín Garcia Fernández, Ândrei Camponogara

Abstract: Physical Layer Security (PLS) is an emerging concept in the field of secrecy for wireless communications that can be used alongside cryptography to prevent unauthorized devices from eavesdropping a legitimate transmission. It offers low computational cost and overhead by injecting an interfering signal in the wiretap channels of potential eavesdroppers. This paper discusses the benefits of the Art… ▽ More Physical Layer Security (PLS) is an emerging concept in the field of secrecy for wireless communications that can be used alongside cryptography to prevent unauthorized devices from eavesdropping a legitimate transmission. It offers low computational cost and overhead by injecting an interfering signal in the wiretap channels of potential eavesdroppers. This paper discusses the benefits of the Artificial Noise and Cooperative Jamming techniques in the context of Vehicle-to-everything (V2X) networks, which require secure data exchange with small latency. The simulations indicate that messages can be safely delivered even with devices that have low available power. △ Less

Submitted 12 January, 2023; originally announced January 2023.

Comments: 5 pages, 6 figures

Mixed Coherent and Non-Coherent Transmission for Multi-CPU Cell-Free Systems

Authors: Roberto P. Antonioli, Iran M. Braga Jr., Gabor Fodor, Yuri C. B. Silva, Walter C. Freitas Jr

Abstract: Existing works on cell-free systems consider either coherent or non-coherent downlink data transmission and a network deployment with a single central processing unit (CPU). While it is known that coherent transmission outperforms noncoherent transmission when assuming unlimited fronthaul links, the former requires a perfect timing synchronization, which is practically not viable over a large netw… ▽ More Existing works on cell-free systems consider either coherent or non-coherent downlink data transmission and a network deployment with a single central processing unit (CPU). While it is known that coherent transmission outperforms noncoherent transmission when assuming unlimited fronthaul links, the former requires a perfect timing synchronization, which is practically not viable over a large network. Furthermore, relying on a single CPU for geographically large cell-free networks is not scalable. Thus, to realize the expected gains of cellfree systems in practice, alternative transmission strategies for realistic multi-CPU cell-free systems are required. Therefore, this paper proposes a novel downlink data transmission scheme that combines and generalizes the existing coherent and non-coherent transmissions. The proposed transmission scheme, named mixed transmission, works based on the realistic assumption that only the access points (APs) controlled by a same CPU are synchronized, and thus transmit in a coherent fashion, while APs from different CPUs require no synchronism and transmit in a non-coherent manner. We also propose extensions of existing clustering algorithms for multi-CPU cell-free systems with mixed transmission. Simulation results show that the combination of the proposed clustering algorithms with mixed transmission have the potential to perform close to the ideal coherent transmission. △ Less

Submitted 28 December, 2022; originally announced December 2022.

Comments: Submitted for possible publication in IEEE conference

A Distributed Game-Theoretic Solution for Power Management in the Uplink of Cell-Free Systems

Authors: Juno V. Saraiva, Roberto P. Antonioli, Gábor Fodor, Walter C. Freitas Jr., Yuri C. B. Silva

Abstract: This paper investigates cell-free massive multiple input multiple output systems with a particular focus on uplink power allocation. In these systems, uplink power control is highly non-trivial, since a single user terminal is associated with multiple intended receiving base stations. In addition, in cell-free systems, distributed power control schemes that address the inherent spectral and energy… ▽ More This paper investigates cell-free massive multiple input multiple output systems with a particular focus on uplink power allocation. In these systems, uplink power control is highly non-trivial, since a single user terminal is associated with multiple intended receiving base stations. In addition, in cell-free systems, distributed power control schemes that address the inherent spectral and energy efficiency targets are desirable. By utilizing tools from game theory, we formulate our proposal as a noncooperative game, and using the best-response dynamics, we obtain a distributed power control mechanism. To ensure that this power control game converges to a Nash equilibrium, we apply the theory of potential games. Differently from existing gamebased schemes, interestingly, our proposed potential function has a scalar parameter that controls the power usage of the users. Numerical results confirm that the proposed approach improves the use of the energy stored in the battery of user terminals and balances between spectral and energy efficiency. △ Less

Submitted 28 December, 2022; originally announced December 2022.

Comments: Accepted at IEEE Globecom 2022

Efficient Battery Usage in Wireless-Powered Cell-Free Systems with Self-Energy Recycling

Authors: Iran M. Braga Jr., Roberto P. Antonioli, Gabor Fodor, Yuri C. B. Silva, Walter C. Freitas Jr

Abstract: This paper investigates wireless-powered cell-free systems, in which the users send their uplink data signal while simultaneously harvesting energy from network nodes and user terminals - including the transmitting user terminal itself - by performing self-energy recycling. In this rather general setting, a closed-form lower bound of the amount of harvested energy and the achieved signal-to-interf… ▽ More This paper investigates wireless-powered cell-free systems, in which the users send their uplink data signal while simultaneously harvesting energy from network nodes and user terminals - including the transmitting user terminal itself - by performing self-energy recycling. In this rather general setting, a closed-form lower bound of the amount of harvested energy and the achieved signal-to-interference-plus-noise ratio expressions are derived. Then, to improve the energy efficiency, we formulate the problem of minimizing the users' battery energy usage while satisfying minimum data rate requirements. Due to the non-convexity of the problem, a novel alternating optimization algorithm is proposed, and its proof of convergence is provided. Finally, numerical results show that the proposed method is more efficient than a state-of-art algorithm in terms of battery energy usage and outage rate. △ Less

Submitted 28 December, 2022; originally announced December 2022.

Comments: Accepted as a correspondance at IEEE TVT

Federated Learning Using Three-Operator ADMM

Authors: Shashi Kant, José Mairton B. da Silva Jr., Gabor Fodor, Bo Göransson, Mats Bengtsson, Carlo Fischione

Abstract: Federated learning (FL) has emerged as an instance of distributed machine learning paradigm that avoids the transmission of data generated on the users' side. Although data are not transmitted, edge devices have to deal with limited communication bandwidths, data heterogeneity, and straggler effects due to the limited computational resources of users' devices. A prominent approach to overcome such… ▽ More Federated learning (FL) has emerged as an instance of distributed machine learning paradigm that avoids the transmission of data generated on the users' side. Although data are not transmitted, edge devices have to deal with limited communication bandwidths, data heterogeneity, and straggler effects due to the limited computational resources of users' devices. A prominent approach to overcome such difficulties is FedADMM, which is based on the classical two-operator consensus alternating direction method of multipliers (ADMM). The common assumption of FL algorithms, including FedADMM, is that they learn a global model using data only on the users' side and not on the edge server. However, in edge learning, the server is expected to be near the base station and have direct access to rich datasets. In this paper, we argue that leveraging the rich data on the edge server is much more beneficial than utilizing only user datasets. Specifically, we show that the mere application of FL with an additional virtual user node representing the data on the edge server is inefficient. We propose FedTOP-ADMM, which generalizes FedADMM and is based on a three-operator ADMM-type technique that exploits a smooth cost function on the edge server to learn a global model parallel to the edge devices. Our numerical experiments indicate that FedTOP-ADMM has substantial gain up to 33\% in communication efficiency to reach a desired test accuracy with respect to FedADMM, including a virtual user on the edge server. △ Less

Submitted 25 March, 2024; v1 submitted 8 November, 2022; originally announced November 2022.

Comments: accepted to IEEE Journal of Selected Topics in Signal Processing, 2022

Blind Asynchronous Over-the-Air Federated Edge Learning

Authors: Saeed Razavikia, Jaume Anguera Peris, Jose Mairton B. da Silva Jr, Carlo Fischione

Abstract: Federated Edge Learning (FEEL) is a distributed machine learning technique where each device contributes to training a global inference model by independently performing local computations with their data. More recently, FEEL has been merged with over-the-air computation (OAC), where the global model is calculated over the air by leveraging the superposition of analog signals. However, when implem… ▽ More Federated Edge Learning (FEEL) is a distributed machine learning technique where each device contributes to training a global inference model by independently performing local computations with their data. More recently, FEEL has been merged with over-the-air computation (OAC), where the global model is calculated over the air by leveraging the superposition of analog signals. However, when implementing FEEL with OAC, there is the challenge on how to precode the analog signals to overcome any time misalignment at the receiver. In this work, we propose a novel synchronization-free method to recover the parameters of the global model over the air without requiring any prior information about the time misalignments. For that, we construct a convex optimization based on the norm minimization problem to directly recover the global model by solving a convex semi-definite program. The performance of the proposed method is evaluated in terms of accuracy and convergence via numerical experiments. We show that our proposed algorithm is close to the ideal synchronized scenario by $10\%$, and performs $4\times$ better than the simple case where no recovering method is used. △ Less

Submitted 31 October, 2022; originally announced October 2022.

Model-Based Reinforcement Learning with SINDy

Authors: Rushiv Arora, Bruno Castro da Silva, Eliot Moss

Abstract: We draw on the latest advancements in the physics community to propose a novel method for discovering the governing non-linear dynamics of physical systems in reinforcement learning (RL). We establish that this method is capable of discovering the underlying dynamics using significantly fewer trajectories (as little as one rollout with $\leq 30$ time steps) than state of the art model learning alg… ▽ More We draw on the latest advancements in the physics community to propose a novel method for discovering the governing non-linear dynamics of physical systems in reinforcement learning (RL). We establish that this method is capable of discovering the underlying dynamics using significantly fewer trajectories (as little as one rollout with $\leq 30$ time steps) than state of the art model learning algorithms. Further, the technique learns a model that is accurate enough to induce near-optimal policies given significantly fewer trajectories than those required by model-free algorithms. It brings the benefits of model-based RL without requiring a model to be developed in advance, for systems that have physics-based dynamics. To establish the validity and applicability of this algorithm, we conduct experiments on four classic control tasks. We found that an optimal policy trained on the discovered dynamics of the underlying system can generalize well. Further, the learned policy performs well when deployed on the actual physical system, thus bridging the model to real system gap. We further compare our method to state-of-the-art model-based and model-free approaches, and show that our method requires fewer trajectories sampled on the true physical system compared other methods. Additionally, we explored approximate dynamics models and found that they also can perform well. △ Less

Submitted 30 August, 2022; originally announced August 2022.

Comments: 8 pages, 1 figure, 1 table, 1 algorithm, presented at the Decision Awareness in Reinforcement Learning workshop held at the International Conference on Machine Learning, 22 July 2022, Baltimore MD, USA

PySINDy: A comprehensive Python package for robust sparse system identification

Authors: Alan A. Kaptanoglu, Brian M. de Silva, Urban Fasel, Kadierdan Kaheman, Andy J. Goldschmidt, Jared L. Callaham, Charles B. Delahunt, Zachary G. Nicolaou, Kathleen Champion, Jean-Christophe Loiseau, J. Nathan Kutz, Steven L. Brunton

Abstract: Automated data-driven modeling, the process of directly discovering the governing equations of a system from data, is increasingly being used across the scientific community. PySINDy is a Python package that provides tools for applying the sparse identification of nonlinear dynamics (SINDy) approach to data-driven model discovery. In this major update to PySINDy, we implement several advanced feat… ▽ More Automated data-driven modeling, the process of directly discovering the governing equations of a system from data, is increasingly being used across the scientific community. PySINDy is a Python package that provides tools for applying the sparse identification of nonlinear dynamics (SINDy) approach to data-driven model discovery. In this major update to PySINDy, we implement several advanced features that enable the discovery of more general differential equations from noisy and limited data. The library of candidate terms is extended for the identification of actuated systems, partial differential equations (PDEs), and implicit differential equations. Robust formulations, including the integral form of SINDy and ensembling techniques, are also implemented to improve performance for real-world data. Finally, we provide a range of new optimization algorithms, including several sparse regression techniques and algorithms to enforce and promote inequality constraints and stability. Together, these updates enable entirely new SINDy model discovery capabilities that have not been reported in the literature, such as constrained PDE identification and ensembling with different sparse regression optimizers. △ Less

Submitted 25 January, 2022; v1 submitted 12 November, 2021; originally announced November 2021.

Signal-carrying speckle in Optical Coherence Tomography: a methodological review on biomedical applications

Authors: Vania Bastos Silva, Danilo Andrade De Jesus, Stefan Klein, Theo van Walsum, João Cardoso, Luisa Sánchez Brea, Pedro G. Vaz

Abstract: Significance: Speckle has historically been considered a source of noise in coherent light imaging. However, a number of works in optical coherence tomography (OCT) imaging have shown that speckle patterns may contain relevant information regarding sub-resolution and structural properties of the tissues from which it is originated. Aim: The objective of this work is to provide a comprehensive ov… ▽ More Significance: Speckle has historically been considered a source of noise in coherent light imaging. However, a number of works in optical coherence tomography (OCT) imaging have shown that speckle patterns may contain relevant information regarding sub-resolution and structural properties of the tissues from which it is originated. Aim: The objective of this work is to provide a comprehensive overview of the methods developed for retrieving speckle information in biomedical OCT applications. Approach: PubMed and Scopus databases were used to perform a systematic review on studies published until April 2021. From 134-screened studies, 37 were eligible for this review. Results: The studies have been clustered according to the nature of their analysis, namely static or dynamic, and all features were described and analysed. The results show that features retrieved from speckle can be used successfully in different applications, such as classification and segmentation. However, the results also show that speckle analysis is highly application-dependant, and the best approach varies between applications. Conclusions: Several of the reviewed analysis were only performed in a theoretical context or using phantoms, showing that signal-carrying speckle analysis in OCT imaging is still in its early stage, and further work is needed to validate its applicability and reproducibility in a clinical context. △ Less

Submitted 30 August, 2021; originally announced August 2021.

Encoder-Decoder Architectures for Clinically Relevant Coronary Artery Segmentation

Authors: João Lourenço Silva, Miguel Nobre Menezes, Tiago Rodrigues, Beatriz Silva, Fausto J. Pinto, Arlindo L. Oliveira

Abstract: Coronary X-ray angiography is a crucial clinical procedure for the diagnosis and treatment of coronary artery disease, which accounts for roughly 16% of global deaths every year. However, the images acquired in these procedures have low resolution and poor contrast, making lesion detection and assessment challenging. Accurate coronary artery segmentation not only helps mitigate these problems, but… ▽ More Coronary X-ray angiography is a crucial clinical procedure for the diagnosis and treatment of coronary artery disease, which accounts for roughly 16% of global deaths every year. However, the images acquired in these procedures have low resolution and poor contrast, making lesion detection and assessment challenging. Accurate coronary artery segmentation not only helps mitigate these problems, but also allows the extraction of relevant anatomical features for further analysis by quantitative methods. Although automated segmentation of coronary arteries has been proposed before, previous approaches have used non-optimal segmentation criteria, leading to less useful results. Most methods either segment only the major vessel, discarding important information from the remaining ones, or segment the whole coronary tree based mostly on contrast information, producing a noisy output that includes vessels that are not relevant for diagnosis. We adopt a better-suited clinical criterion and segment vessels according to their clinical relevance. Additionally, we simultaneously perform catheter segmentation, which may be useful for diagnosis due to the scale factor provided by the catheter's known diameter, and is a task that has not yet been performed with good results. To derive the optimal approach, we conducted an extensive comparative study of encoder-decoder architectures trained on a combination of focal loss and a variant of generalized dice loss. Based on the EfficientNet and the UNet++ architectures, we propose a line of efficient and high-performance segmentation models using a new decoder architecture, the EfficientUNet++, whose best-performing version achieved average dice scores of 0.8904 and 0.7526 for the artery and catheter classes, respectively, and an average generalized dice score of 0.9234. △ Less

Submitted 21 June, 2021; originally announced June 2021.

XCycles Backprojection Acoustic Super-Resolution

Authors: Feras Almasri, Jurgen Vandendriessche, Laurent Segers, Bruno da Silva, An Braeken, Kris Steenhaut, Abdellah Touhafi, Olivier Debeir

Abstract: The computer vision community has paid much attention to the development of visible image super-resolution (SR) using deep neural networks (DNNs) and has achieved impressive results. The advancement of non-visible light sensors, such as acoustic imaging sensors, has attracted much attention, as they allow people to visualize the intensity of sound waves beyond the visible spectrum. However, becaus… ▽ More The computer vision community has paid much attention to the development of visible image super-resolution (SR) using deep neural networks (DNNs) and has achieved impressive results. The advancement of non-visible light sensors, such as acoustic imaging sensors, has attracted much attention, as they allow people to visualize the intensity of sound waves beyond the visible spectrum. However, because of the limitations imposed on acquiring acoustic data, new methods for improving the resolution of the acoustic images are necessary. At this time, there is no acoustic imaging dataset designed for the SR problem. This work proposed a novel backprojection model architecture for the acoustic image super-resolution problem, together with Acoustic Map Imaging VUB-ULB Dataset (AMIVU). The dataset provides large simulated and real captured images at different resolutions. The proposed XCycles BackProjection model (XCBP), in contrast to the feedforward model approach, fully uses the iterative correction procedure in each cycle to reconstruct the residual error correction for the encoded features in both low- and high-resolution space. The proposed approach was evaluated on the dataset and showed high outperformance compared to the classical interpolation operators and to the recent feedforward state-of-the-art models. It also contributed to a drastically reduced sub-sampling error produced during the data acquisition. △ Less

Submitted 19 May, 2021; originally announced May 2021.

Journal ref: Sensors 2021, 21, 3453

Simultaneous Wireless Information and Power Transfer for Federated Learning

Authors: José Mairton B. da Silva Jr., Konstantinos Ntougias, Ioannis Krikidis, Gábor Fodor, Carlo Fischione

Abstract: In the Internet of Things, learning is one of most prominent tasks. In this paper, we consider an Internet of Things scenario where federated learning is used with simultaneous transmission of model data and wireless power. We investigate the trade-off between the number of communication rounds and communication round time while harvesting energy to compensate the energy expenditure. We formulate… ▽ More In the Internet of Things, learning is one of most prominent tasks. In this paper, we consider an Internet of Things scenario where federated learning is used with simultaneous transmission of model data and wireless power. We investigate the trade-off between the number of communication rounds and communication round time while harvesting energy to compensate the energy expenditure. We formulate and solve an optimization problem by considering the number of local iterations on devices, the time to transmit-receive the model updates, and to harvest sufficient energy. Numerical results indicate that maximum ratio transmission and zero-forcing beamforming for the optimization of the local iterations on devices substantially boost the test accuracy of the learning task. Moreover, maximum ratio transmission instead of zero-forcing provides the best test accuracy and communication round time trade-off for various energy harvesting percentages. Thus, it is possible to learn a model quickly with few communication rounds without depleting the battery. △ Less

Submitted 21 July, 2021; v1 submitted 26 April, 2021; originally announced April 2021.

Comments: Accepted to appear in the IEEE International Workshop on Signal Processing Advances in Wireless Communications (SPAWC) in Lucca, Italy, Sep. 2021

Joint Resource Allocation and Transceiver Design for Sum-Rate Maximization under Latency Constraints in Multicell MU-MIMO Systems

Authors: Iran M. Braga Jr., Roberto P. Antonioli, Gabor Fodor, Yuri C. B. Silva, Carlos F. M. e Silva, Walter C. Freitas Jr

Abstract: Due to the continuous advancements of orthogonal frequency division multiplexing (OFDM) and multiple antenna techniques, multiuser multiple input multiple output (MU-MIMO) OFDM is a key enabler of both fourth and fifth generation networks. In this paper, we consider the problem of weighted sum-rate maximization under latency constraints in finite buffer multicell MU-MIMO OFDM systems. Unlike previ… ▽ More Due to the continuous advancements of orthogonal frequency division multiplexing (OFDM) and multiple antenna techniques, multiuser multiple input multiple output (MU-MIMO) OFDM is a key enabler of both fourth and fifth generation networks. In this paper, we consider the problem of weighted sum-rate maximization under latency constraints in finite buffer multicell MU-MIMO OFDM systems. Unlike previous works, the optimization variables include the transceiver beamforming vectors, the scheduled packet size and the resources in the frequency and power domains. This problem is motivated by the observation that multicell MU-MIMO OFDM systems serve multiple quality of service classes and the system performance depends critically on both the transceiver design and the scheduling algorithm. Since this problem is non-convex, we resort to the max-plus queuing method and successive convex approximation. We propose both centralized and decentralized solutions, in which practical design aspects, such as signaling overhead, are considered. Finally, we compare the proposed framework with state-of-the-art algorithms in relevant scenarios, assuming a realistic channel model with space, frequency and time correlations. Numerical results indicate that our design provides significant gains over designs based on the wide-spread saturated buffers assumption, while also outperforming algorithms that consider a finite-buffer model. △ Less

Submitted 29 March, 2021; originally announced March 2021.

Comments: Accepted at IEEE Transactions on Communications

PySensors: A Python Package for Sparse Sensor Placement

Authors: Brian M. de Silva, Krithika Manohar, Emily Clark, Bingni W. Brunton, Steven L. Brunton, J. Nathan Kutz

Abstract: PySensors is a Python package for selecting and placing a sparse set of sensors for classification and reconstruction tasks. Specifically, PySensors implements algorithms for data-driven sparse sensor placement optimization for reconstruction (SSPOR) and sparse sensor placement optimization for classification (SSPOC). In this work we provide a brief description of the mathematical algorithms and t… ▽ More PySensors is a Python package for selecting and placing a sparse set of sensors for classification and reconstruction tasks. Specifically, PySensors implements algorithms for data-driven sparse sensor placement optimization for reconstruction (SSPOR) and sparse sensor placement optimization for classification (SSPOC). In this work we provide a brief description of the mathematical algorithms and theory for sparse sensor optimization, along with an overview and demonstration of the features implemented in PySensors (with code examples). We also include practical advice for user and a list of potential extensions to PySensors. Software is available at https://github.com/dynamicslab/pysensors. △ Less

Submitted 20 February, 2021; originally announced February 2021.

Full-Duplex and Dynamic-TDD: Pushing the Limits of Spectrum Reuse in Multi-Cell Communications

Authors: José Mairton B. da Silva Jr., Gustav Wikström, Ratheesh K. Mungara, Carlo Fischione

Abstract: Although in cellular networks full-duplex and dynamic time-division duplexing promise increased spectrum efficiency, their potential is so far challenged by increased interference. While previous studies have shown that self-interference can be suppressed to a sufficient level, we show that the cross-link interference for both duplexing modes, especially from base station to base station, is the r… ▽ More Although in cellular networks full-duplex and dynamic time-division duplexing promise increased spectrum efficiency, their potential is so far challenged by increased interference. While previous studies have shown that self-interference can be suppressed to a sufficient level, we show that the cross-link interference for both duplexing modes, especially from base station to base station, is the remaining challenge in multi-cell networks, restricting the uplink performance. Using beamforming techniques of low-complexity, we show that this interference can be mitigated, and that full-duplex and dynamic time-division duplexing can substantially increase the capacity of multi-cell networks. Our results suggest that if we can control the cross link interference in full-duplex, then we can almost double the multi cell network capacity as well as user throughput. Therefore, the techniques in this paper have the potentiality to enable a smooth introduction of full-duplex into cellular systems. △ Less

Submitted 21 October, 2020; originally announced October 2020.

Comments: 15 pages, 6 figures. Accepted to IEEE Wireless Communications - Special Issue on Full Duplex Communications Theory, Standardization and Practice

Wireless for Machine Learning

Authors: Henrik Hellström, José Mairton B. da Silva Jr, Mohammad Mohammadi Amiri, Mingzhe Chen, Viktoria Fodor, H. Vincent Poor, Carlo Fischione

Abstract: As data generation increasingly takes place on devices without a wired connection, machine learning (ML) related traffic will be ubiquitous in wireless networks. Many studies have shown that traditional wireless protocols are highly inefficient or unsustainable to support ML, which creates the need for new wireless communication methods. In this survey, we give an exhaustive review of the state-of… ▽ More As data generation increasingly takes place on devices without a wired connection, machine learning (ML) related traffic will be ubiquitous in wireless networks. Many studies have shown that traditional wireless protocols are highly inefficient or unsustainable to support ML, which creates the need for new wireless communication methods. In this survey, we give an exhaustive review of the state-of-the-art wireless methods that are specifically designed to support ML services over distributed datasets. Currently, there are two clear themes within the literature, analog over-the-air computation and digital radio resource management optimized for ML. This survey gives a comprehensive introduction to these methods, reviews the most important works, highlights open problems, and discusses application scenarios. △ Less

Submitted 9 June, 2022; v1 submitted 31 August, 2020; originally announced August 2020.

Automatic Detection of Aedes aegypti Breeding Grounds Based on Deep Networks with Spatio-Temporal Consistency

Authors: Wesley L. Passos, Gabriel M. Araujo, Amaro A. de Lima, Sergio L. Netto, Eduardo A. B. da Silva

Abstract: Every year, the Aedes aegypti mosquito infects millions of people with diseases such as dengue, zika, chikungunya, and urban yellow fever. The main form to combat these diseases is to avoid mosquito reproduction by searching for and eliminating the potential mosquito breeding grounds. In this work, we introduce a comprehensive dataset of aerial videos, acquired with an unmanned aerial vehicle, con… ▽ More Every year, the Aedes aegypti mosquito infects millions of people with diseases such as dengue, zika, chikungunya, and urban yellow fever. The main form to combat these diseases is to avoid mosquito reproduction by searching for and eliminating the potential mosquito breeding grounds. In this work, we introduce a comprehensive dataset of aerial videos, acquired with an unmanned aerial vehicle, containing possible mosquito breeding sites. All frames of the video dataset were manually annotated with bounding boxes identifying all objects of interest. This dataset was employed to develop an automatic detection system of such objects based on deep convolutional networks. We propose the exploitation of the temporal information contained in the videos by the incorporation, in the object detection pipeline, of a spatio-temporal consistency module that can register the detected objects, minimizing most false-positive and false-negative occurrences. Also, we experimentally show that using videos is more beneficial than only composing a mosaic using the frames. Using the ResNet-50-FPN as a backbone, we achieve F$_1$-scores of 0.65 and 0.77 on the object-level detection of `tires' and `water tanks', respectively, illustrating the system capabilities to properly locate potential mosquito breeding objects. △ Less

Submitted 27 November, 2021; v1 submitted 29 July, 2020; originally announced July 2020.

Physics-informed machine learning for sensor fault detection with flight test data

Authors: Brian M. de Silva, Jared Callaham, Jonathan Jonker, Nicholas Goebel, Jennifer Klemisch, Darren McDonald, Nathan Hicks, J. Nathan Kutz, Steven L. Brunton, Aleksandr Y. Aravkin

Abstract: We develop data-driven algorithms to fully automate sensor fault detection in systems governed by underlying physics. The proposed machine learning method uses a time series of typical behavior to approximate the evolution of measurements of interest by a linear time-invariant system. Given additional data from related sensors, a Kalman observer is used to maintain a separate real-time estimate of… ▽ More We develop data-driven algorithms to fully automate sensor fault detection in systems governed by underlying physics. The proposed machine learning method uses a time series of typical behavior to approximate the evolution of measurements of interest by a linear time-invariant system. Given additional data from related sensors, a Kalman observer is used to maintain a separate real-time estimate of the measurement of interest. Sustained deviation between the measurements and the estimate is used to detect anomalous behavior. A decision tree, informed by integrating other sensor measurement values, is used to determine the amount of deviation required to identify a sensor fault. We validate the method by applying it to three test systems exhibiting various types of sensor faults: commercial flight test data, an unsteady aerodynamics model with dynamic stall, and a model for longitudinal flight dynamics forced by atmospheric turbulence. In the latter two cases we test fault detection for several prototypical failure modes. The combination of a learned dynamical model with the automated decision tree accurately detects sensor faults in each case. △ Less

Submitted 23 June, 2020; originally announced June 2020.

Comments: 21 pages, 10 figures, submitted to AIAA

Hippocampus Segmentation on Epilepsy and Alzheimer's Disease Studies with Multiple Convolutional Neural Networks

Authors: Diedre Carmo, Bruna Silva, Clarissa Yasuda, Letícia Rittner, Roberto Lotufo

Abstract: Hippocampus segmentation on magnetic resonance imaging is of key importance for the diagnosis, treatment decision and investigation of neuropsychiatric disorders. Automatic segmentation is an active research field, with many recent models using deep learning. Most current state-of-the art hippocampus segmentation methods train their methods on healthy or Alzheimer's disease patients from public da… ▽ More Hippocampus segmentation on magnetic resonance imaging is of key importance for the diagnosis, treatment decision and investigation of neuropsychiatric disorders. Automatic segmentation is an active research field, with many recent models using deep learning. Most current state-of-the art hippocampus segmentation methods train their methods on healthy or Alzheimer's disease patients from public datasets. This raises the question whether these methods are capable of recognizing the hippocampus on a different domain, that of epilepsy patients with hippocampus resection. In this paper we present a state-of-the-art, open source, ready-to-use, deep learning based hippocampus segmentation method. It uses an extended 2D multi-orientation approach, with automatic pre-processing and orientation alignment. The methodology was developed and validated using HarP, a public Alzheimer's disease hippocampus segmentation dataset. We test this methodology alongside other recent deep learning methods, in two domains: The HarP test set and an in-house epilepsy dataset, containing hippocampus resections, named HCUnicamp. We show that our method, while trained only in HarP, surpasses others from the literature in both the HarP test set and HCUnicamp in Dice. Additionally, Results from training and testing in HCUnicamp volumes are also reported separately, alongside comparisons between training and testing in epilepsy and Alzheimer's data and vice versa. Although current state-of-the-art methods, including our own, achieve upwards of 0.9 Dice in HarP, all tested methods, including our own, produced false positives in HCUnicamp resection regions, showing that there is still room for improvement for hippocampus segmentation methods when resection is involved. △ Less

Submitted 10 February, 2021; v1 submitted 14 January, 2020; originally announced January 2020.

Comments: Code is available at https://github.com/dscarmo/e2dhipseg Published in Heliyon: https://www.sciencedirect.com/science/article/pii/S2405844021003315

Journal ref: Heliyon, Volume 7, Issue 2, 2021

A Multistage Method for SCMA Codebook Design Based on MDS Codes

Authors: Bruno Fontana da Silva, Danilo Silva, Bartolomeu F. Uchôa-Filho, Didier Le Ruyet

Abstract: Sparse Code Multiple Access (SCMA) has been recently proposed for the future generation of wireless communication standards. SCMA system design involves specifying several parameters. In order to simplify the procedure, most works consider a multistage design approach. Two main stages are usually emphasized in these methods: sparse signatures design (equivalently, resource allocation) and codebook… ▽ More Sparse Code Multiple Access (SCMA) has been recently proposed for the future generation of wireless communication standards. SCMA system design involves specifying several parameters. In order to simplify the procedure, most works consider a multistage design approach. Two main stages are usually emphasized in these methods: sparse signatures design (equivalently, resource allocation) and codebook design. In this paper, we present a novel SCMA codebook design method. The proposed method considers SCMA codebooks structured with an underlying vector space obtained from classical block codes. In particular, when using maximum distance separable (MDS) codes, our proposed design provides maximum signal-space diversity with a relatively small alphabet. The use of small alphabets also helps to maintain desired properties in the codebooks, such as low peak-to-average power ratio and low-complexity detection. △ Less

Submitted 7 May, 2019; originally announced May 2019.

Comments: Submitted to IEEE Wireless Communication Letters

Extended 2D Consensus Hippocampus Segmentation

Authors: Diedre Carmo, Bruna Silva, Clarissa Yasuda, Letícia Rittner, Roberto Lotufo

Abstract: Hippocampus segmentation plays a key role in diagnosing various brain disorders such as Alzheimer's disease, epilepsy, multiple sclerosis, cancer, depression and others. Nowadays, segmentation is still mainly performed manually by specialists. Segmentation done by experts is considered to be a gold-standard when evaluating automated methods, buts it is a time consuming and arduos task, requiring s… ▽ More Hippocampus segmentation plays a key role in diagnosing various brain disorders such as Alzheimer's disease, epilepsy, multiple sclerosis, cancer, depression and others. Nowadays, segmentation is still mainly performed manually by specialists. Segmentation done by experts is considered to be a gold-standard when evaluating automated methods, buts it is a time consuming and arduos task, requiring specialized personnel. In recent years, efforts have been made to achieve reliable automated segmentation. For years the best performing authomatic methods were multi atlas based with around 80-85% Dice coefficient and very time consuming, but machine learning methods are recently rising with promising time and accuracy performance. A method for volumetric hippocampus segmentation is presented, based on the consensus of tri-planar U-Net inspired fully convolutional networks (FCNNs), with some modifications, including residual connections, VGG weight transfers, batch normalization and a patch extraction technique employing data from neighbor patches. A study on the impact of our modifications to the classical U-Net architecture was performed. Our method achieves cutting edge performance in our dataset, with around 96% volumetric Dice accuracy in our test data. In a public validation dataset, HARP, we achieve 87.48% DICE. GPU execution time is in the order of seconds per volume, and source code is publicly available. Also, masks are shown to be similar to other recent state-of-the-art hippocampus segmentation methods in a third dataset, without manual annotations. △ Less

Submitted 13 May, 2020; v1 submitted 12 February, 2019; originally announced February 2019.

Comments: This was published as an extended abstract in MIDL 2019 [arXiv:1907.08612]. An alpha version of the code is available at https://github.com/dscarmo/e2dhipseg. More experiments on improvements to the method and code are ongoing. Future updates are to be expected. A new, more complete paper is published in arXiv:2001.05058

Report number: MIDL/2019/ExtendedAbstract/Sygx97DaKV

How to Split UL/DL Antennas in Full-Duplex Cellular Networks

Authors: José Mairton B. da Silva Jr., Hadi Ghauch, Gábor Fodor, Carlo Fischione

Abstract: To further improve the potential of full-duplex communications, networks may employ multiple antennas at the base station or user equipment. To this end, networks that employ current radios usually deal with self-interference and multi-user interference by beamforming techniques. Although previous works investigated beamforming design to improve spectral efficiency, the fundamental question of how… ▽ More To further improve the potential of full-duplex communications, networks may employ multiple antennas at the base station or user equipment. To this end, networks that employ current radios usually deal with self-interference and multi-user interference by beamforming techniques. Although previous works investigated beamforming design to improve spectral efficiency, the fundamental question of how to split the antennas at a base station between uplink and downlink in full-duplex networks has not been investigated rigorously. This paper addresses this question by posing antenna splitting as a binary nonlinear optimization problem to minimize the sum mean squared error of the received data symbols. It is shown that this is an NP-hard problem. This combinatorial problem is dealt with by equivalent formulations, iterative convex approximations, and a binary relaxation. The proposed algorithm is guaranteed to converge to a stationary solution of the relaxed problem with much smaller complexity than exhaustive search. Numerical results indicate that the proposed solution is close to the optimal in both high and low self-interference capable scenarios, while the usually assumed antenna splitting is far from optimal. For large number of antennas, a simple antenna splitting is close to the proposed solution. This reveals that the importance of antenna splitting is inversely proportional with the number of antennas. △ Less

Submitted 23 May, 2018; v1 submitted 11 January, 2018; originally announced January 2018.

Comments: 7 pages, 4 figures. Accepted to IEEE ICC 2018 Workshop on Full-Duplex Communications for Future Wireless Networks

Reconstruction of Electrical Impedance Tomography Using Fish School Search, Non-Blind Search, and Genetic Algorithm

Authors: Valter Augusto de Freitas Barbosa, Reiga Ramalho Ribeiro, Allan Rivalles Souza Feitosa, Victor Luiz Bezerra Araújo da Silva, Arthur Diego Dias Rocha, Rafaela Covello de Freitas, Ricardo Emmanuel de Souza, Wellington Pinheiro dos Santos

Abstract: Electrical Impedance Tomography (EIT) is a noninvasive imaging technique that does not use ionizing radiation, with application both in environmental sciences and in health. Image reconstruction is performed by solving an inverse problem and ill-posed. Evolutionary Computation and Swarm Intelligence have become a source of methods for solving inverse problems. Fish School Search (FSS) is a promisi… ▽ More Electrical Impedance Tomography (EIT) is a noninvasive imaging technique that does not use ionizing radiation, with application both in environmental sciences and in health. Image reconstruction is performed by solving an inverse problem and ill-posed. Evolutionary Computation and Swarm Intelligence have become a source of methods for solving inverse problems. Fish School Search (FSS) is a promising search and optimization method, based on the dynamics of schools of fish. In this article the authors present a method for reconstruction of EIT images based on FSS and Non-Blind Search (NBS). The method was evaluated using numerical phantoms consisting of electrical conductivity images with subjects in the center, between the center and the edge and on the edge of a circular section, with meshes of 415 finite elements. The authors performed 20 simulations for each configuration. Results showed that both FSS and FSS-NBS were able to converge faster than genetic algorithms. △ Less

Submitted 3 December, 2017; originally announced December 2017.

Journal ref: International Journal of Swarm Intelligence Research, Volume 8, Issue 2, 2017

A Compression-Inspired Framework for Macro Discovery

Authors: Francisco M. Garcia, Bruno C. da Silva, Philip S. Thomas

Abstract: In this paper we consider the problem of how a reinforcement learning agent tasked with solving a set of related Markov decision processes can use knowledge acquired early in its lifetime to improve its ability to more rapidly solve novel, but related, tasks. One way of exploiting this experience is by identifying recurrent patterns in trajectories obtained from well-performing policies. We propos… ▽ More In this paper we consider the problem of how a reinforcement learning agent tasked with solving a set of related Markov decision processes can use knowledge acquired early in its lifetime to improve its ability to more rapidly solve novel, but related, tasks. One way of exploiting this experience is by identifying recurrent patterns in trajectories obtained from well-performing policies. We propose a three-step framework in which an agent 1) generates a set of candidate open-loop macros by compressing trajectories drawn from near-optimal policies; 2) evaluates the value of each macro; and 3) selects a maximally diverse subset of macros that spans the space of policies typically required for solving the set of related tasks. Our experiments show that extending the original primitive action-set of the agent with the identified macros allows it to more rapidly learn an optimal policy in unseen, but similar MDPs. △ Less

Submitted 22 February, 2019; v1 submitted 24 November, 2017; originally announced November 2017.

Comments: Accepted as Extended Abstract, AAMAS, 2019