Showing 1–50 of 82 results for author: Evans, B

Modelling bounded rational decision-making through Wasserstein constraints

Authors: Benjamin Patrick Evans, Leo Ardon, Sumitra Ganesh

Abstract: Modelling bounded rational decision-making through information constrained processing provides a principled approach for representing departures from rationality within a reinforcement learning framework, while still treating decision-making as an optimization process. However, existing approaches are generally based on Entropy, Kullback-Leibler divergence, or Mutual Information. In this work, we… ▽ More Modelling bounded rational decision-making through information constrained processing provides a principled approach for representing departures from rationality within a reinforcement learning framework, while still treating decision-making as an optimization process. However, existing approaches are generally based on Entropy, Kullback-Leibler divergence, or Mutual Information. In this work, we highlight issues with these approaches when dealing with ordinal action spaces. Specifically, entropy assumes uniform prior beliefs, missing the impact of a priori biases on decision-makings. KL-Divergence addresses this, however, has no notion of "nearness" of actions, and additionally, has several well known potentially undesirable properties such as the lack of symmetry, and furthermore, requires the distributions to have the same support (e.g. positive probability for all actions). Mutual information is often difficult to estimate. Here, we propose an alternative approach for modeling bounded rational RL agents utilising Wasserstein distances. This approach overcomes the aforementioned issues. Crucially, this approach accounts for the nearness of ordinal actions, modeling "stickiness" in agent decisions and unlikeliness of rapidly switching to far away actions, while also supporting low probability actions, zero-support prior distributions, and is simple to calculate directly. △ Less

Submitted 1 April, 2025; originally announced April 2025.

Comments: Accepted at RLDM 2025

Towards deployment-centric multimodal AI beyond vision and language

Authors: Xianyuan Liu, Jiayang Zhang, Shuo Zhou, Thijs L. van der Plas, Avish Vijayaraghavan, Anastasiia Grishina, Mengdie Zhuang, Daniel Schofield, Christopher Tomlinson, Yuhan Wang, Ruizhe Li, Louisa van Zeeland, Sina Tabakhi, Cyndie Demeocq, Xiang Li, Arunav Das, Orlando Timmerman, Thomas Baldwin-McDonald, Jinge Wu, Peizhen Bai, Zahraa Al Sahili, Omnia Alwazzan, Thao N. Do, Mohammod N. I. Suvon, Angeline Wang , et al. (23 additional authors not shown)

Abstract: Multimodal artificial intelligence (AI) integrates diverse types of data via machine learning to improve understanding, prediction, and decision-making across disciplines such as healthcare, science, and engineering. However, most multimodal AI advances focus on models for vision and language data, while their deployability remains a key challenge. We advocate a deployment-centric workflow that in… ▽ More Multimodal artificial intelligence (AI) integrates diverse types of data via machine learning to improve understanding, prediction, and decision-making across disciplines such as healthcare, science, and engineering. However, most multimodal AI advances focus on models for vision and language data, while their deployability remains a key challenge. We advocate a deployment-centric workflow that incorporates deployment constraints early to reduce the likelihood of undeployable solutions, complementing data-centric and model-centric approaches. We also emphasise deeper integration across multiple levels of multimodality and multidisciplinary collaboration to significantly broaden the research scope beyond vision and language. To facilitate this approach, we identify common multimodal-AI-specific challenges shared across disciplines and examine three real-world use cases: pandemic response, self-driving car design, and climate change adaptation, drawing expertise from healthcare, social science, engineering, science, sustainability, and finance. By fostering multidisciplinary dialogue and open research practices, our community can accelerate deployment-centric development for broad societal impact. △ Less

Submitted 4 April, 2025; originally announced April 2025.

Large Language Models Are Effective Human Annotation Assistants, But Not Good Independent Annotators

Authors: Feng Gu, Zongxia Li, Carlos Rafael Colon, Benjamin Evans, Ishani Mondal, Jordan Lee Boyd-Graber

Abstract: Event annotation is important for identifying market changes, monitoring breaking news, and understanding sociological trends. Although expert annotators set the gold standards, human coding is expensive and inefficient. Unlike information extraction experiments that focus on single contexts, we evaluate a holistic workflow that removes irrelevant documents, merges documents about the same event,… ▽ More Event annotation is important for identifying market changes, monitoring breaking news, and understanding sociological trends. Although expert annotators set the gold standards, human coding is expensive and inefficient. Unlike information extraction experiments that focus on single contexts, we evaluate a holistic workflow that removes irrelevant documents, merges documents about the same event, and annotates the events. Although LLM-based automated annotations are better than traditional TF-IDF-based methods or Event Set Curation, they are still not reliable annotators compared to human experts. However, adding LLMs to assist experts for Event Set Curation can reduce the time and mental effort required for Variable Annotation. When using LLMs to extract event variables to assist expert annotators, they agree more with the extracted variables than fully automated LLMs for annotation. △ Less

Submitted 5 April, 2025; v1 submitted 9 March, 2025; originally announced March 2025.

Comments: 9 pages, 4 figures

SHAZAM: Self-Supervised Change Monitoring for Hazard Detection and Mapping

Authors: Samuel Garske, Konrad Heidler, Bradley Evans, KC Wong, Xiao Xiang Zhu

Abstract: The increasing frequency of environmental hazards due to climate change underscores the urgent need for effective monitoring systems. Current approaches either rely on expensive labelled datasets, struggle with seasonal variations, or require multiple observations for confirmation (which delays detection). To address these challenges, this work presents SHAZAM - Self-Supervised Change Monitoring f… ▽ More The increasing frequency of environmental hazards due to climate change underscores the urgent need for effective monitoring systems. Current approaches either rely on expensive labelled datasets, struggle with seasonal variations, or require multiple observations for confirmation (which delays detection). To address these challenges, this work presents SHAZAM - Self-Supervised Change Monitoring for Hazard Detection and Mapping. SHAZAM uses a lightweight conditional UNet to generate expected images of a region of interest (ROI) for any day of the year, allowing for the direct modelling of normal seasonal changes and the ability to distinguish potential hazards. A modified structural similarity measure compares the generated images with actual satellite observations to compute region-level anomaly scores and pixel-level hazard maps. Additionally, a theoretically grounded seasonal threshold eliminates the need for dataset-specific optimisation. Evaluated on four diverse datasets that contain bushfires (wildfires), burned regions, extreme and out-of-season snowfall, floods, droughts, algal blooms, and deforestation, SHAZAM achieved F1 score improvements of between 0.066 and 0.234 over existing methods. This was achieved primarily through more effective hazard detection (higher recall) while using only 473K parameters. SHAZAM demonstrated superior mapping capabilities through higher spatial resolution and improved ability to suppress background features while accentuating both immediate and gradual hazards. SHAZAM has been established as an effective and generalisable solution for hazard detection and mapping across different geographical regions and a diverse range of hazards. The Python code is available at: https://github.com/WiseGamgee/SHAZAM △ Less

Submitted 28 February, 2025; originally announced March 2025.

Comments: 20 pages, 9 figures, 3 tables, code available at: https://github.com/WiseGamgee/SHAZAM

ADAGE: A generic two-layer framework for adaptive agent based modelling

Authors: Benjamin Patrick Evans, Sihan Zeng, Sumitra Ganesh, Leo Ardon

Abstract: Agent-based models (ABMs) are valuable for modelling complex, potentially out-of-equilibria scenarios. However, ABMs have long suffered from the Lucas critique, stating that agent behaviour should adapt to environmental changes. Furthermore, the environment itself often adapts to these behavioural changes, creating a complex bi-level adaptation problem. Recent progress integrating multi-agent rein… ▽ More Agent-based models (ABMs) are valuable for modelling complex, potentially out-of-equilibria scenarios. However, ABMs have long suffered from the Lucas critique, stating that agent behaviour should adapt to environmental changes. Furthermore, the environment itself often adapts to these behavioural changes, creating a complex bi-level adaptation problem. Recent progress integrating multi-agent reinforcement learning into ABMs introduces adaptive agent behaviour, beginning to address the first part of this critique, however, the approaches are still relatively ad hoc, lacking a general formulation, and furthermore, do not tackle the second aspect of simultaneously adapting environmental level characteristics in addition to the agent behaviours. In this work, we develop a generic two-layer framework for ADaptive AGEnt based modelling (ADAGE) for addressing these problems. This framework formalises the bi-level problem as a Stackelberg game with conditional behavioural policies, providing a consolidated framework for adaptive agent-based modelling based on solving a coupled set of non-linear equations. We demonstrate how this generic approach encapsulates several common (previously viewed as distinct) ABM tasks, such as policy design, calibration, scenario generation, and robust behavioural learning under one unified framework. We provide example simulations on multiple complex economic and financial environments, showing the strength of the novel framework under these canonical settings, addressing long-standing critiques of traditional ABMs. △ Less

Submitted 16 January, 2025; originally announced January 2025.

Comments: Accepted at the 2025 International Conference on Autonomous Agents and Multiagent Systems (AAMAS)

Decentralized Convergence to Equilibrium Prices in Trading Networks

Authors: Edwin Lock, Benjamin Patrick Evans, Eleonora Kreacic, Sujay Bhatt, Alec Koppel, Sumitra Ganesh, Paul W. Goldberg

Abstract: We propose a decentralized market model in which agents can negotiate bilateral contracts. This builds on a similar, but centralized, model of trading networks introduced by Hatfield et al. in 2013. Prior work has established that fully-substitutable preferences guarantee the existence of competitive equilibria which can be centrally computed. Our motivation comes from the fact that prices in mark… ▽ More We propose a decentralized market model in which agents can negotiate bilateral contracts. This builds on a similar, but centralized, model of trading networks introduced by Hatfield et al. in 2013. Prior work has established that fully-substitutable preferences guarantee the existence of competitive equilibria which can be centrally computed. Our motivation comes from the fact that prices in markets such as over-the-counter markets and used car markets arise from decentralized negotiation among agents, which has left open an important question as to whether equilibrium prices can emerge from agent-to-agent bilateral negotiations. We design a best response dynamic intended to capture such negotiations between market participants. We assume fully substitutable preferences for market participants. In this setting, we provide proofs of convergence for sparse markets (covering many real world markets of interest), and experimental results for more general cases, demonstrating that prices indeed reach equilibrium, quickly, via bilateral negotiations. Our best response dynamic, and its convergence behavior, forms an important first step in understanding how decentralized markets reach, and retain, equilibrium. △ Less

Submitted 28 January, 2025; v1 submitted 18 December, 2024; originally announced December 2024.

Comments: Extended version of paper accepted at AAAI'25

(Blind) Users Really Do Heed Aural Telephone Scam Warnings

Authors: Filipo Sharevski, Jennifer Vander Loop, Bill Evans, Alexander Ponticello

Abstract: This paper reports on a study exploring how two groups of individuals, legally blind (n=36) and sighted ones (n=36), react to aural telephone scam warnings in naturalistic settings. As spoofing a CallerID is trivial, communicating the context of an incoming call instead offers a better possibility to warn a receiver about a potential scam. Usually, such warnings are visual in nature and fail to ca… ▽ More This paper reports on a study exploring how two groups of individuals, legally blind (n=36) and sighted ones (n=36), react to aural telephone scam warnings in naturalistic settings. As spoofing a CallerID is trivial, communicating the context of an incoming call instead offers a better possibility to warn a receiver about a potential scam. Usually, such warnings are visual in nature and fail to cater to users with visual disabilities. To address this exclusion, we developed an aural variant of telephone scam warnings and tested them in three conditions: baseline (no warning), short warning, and contextual warning that preceded the scam's content. We tested the two most common scam scenarios: fraud (interest rate reduction) and identity theft (social security number) by cold-calling participants and recording their action, and debriefing and obtaining consent afterward. Only two participants "pressed one" as the scam demanded, both from the legally blind group that heard the contextual warning for the social security scenario. Upon close inspection, we learned that one of them did so because of accessibility issues with their screen reader and the other did so intentionally because the warning convinced them to waste the scammer's time, so they don't scam vulnerable people. Both the legally blind and the sighted participants found the contextual warnings as powerful usable security cues that, together with STIR/SHAKEN indicators like "Scam Likely", would provide robust protection against any type of scam. We also discussed the potential privacy implications of the contextual warnings and collected recommendations for usably accessible implementation. △ Less

Submitted 5 December, 2024; originally announced December 2024.

Simulate and Optimise: A two-layer mortgage simulator for designing novel mortgage assistance products

Authors: Leo Ardon, Benjamin Patrick Evans, Deepeka Garg, Annapoorani Lakshmi Narayanan, Makada Henry-Nickie, Sumitra Ganesh

Abstract: We develop a novel two-layer approach for optimising mortgage relief products through a simulated multi-agent mortgage environment. While the approach is generic, here the environment is calibrated to the US mortgage market based on publicly available census data and regulatory guidelines. Through the simulation layer, we assess the resilience of households to exogenous income shocks, while the op… ▽ More We develop a novel two-layer approach for optimising mortgage relief products through a simulated multi-agent mortgage environment. While the approach is generic, here the environment is calibrated to the US mortgage market based on publicly available census data and regulatory guidelines. Through the simulation layer, we assess the resilience of households to exogenous income shocks, while the optimisation layer explores strategies to improve the robustness of households to these shocks by making novel mortgage assistance products available to households. Households in the simulation are adaptive, learning to make mortgage-related decisions (such as product enrolment or strategic foreclosures) that maximize their utility, balancing their available liquidity and equity. We show how this novel two-layer simulation approach can successfully design novel mortgage assistance products to improve household resilience to exogenous shocks, and balance the costs of providing such products through post-hoc analysis. Previously, such analysis could only be conducted through expensive pilot studies involving real participants, demonstrating the benefit of the approach for designing and evaluating financial products. △ Less

Submitted 1 November, 2024; originally announced November 2024.

Comments: Accepted at the 5th ACM International Conference on AI in Finance

ERX: A Fast Real-Time Anomaly Detection Algorithm for Hyperspectral Line Scanning

Authors: Samuel Garske, Bradley Evans, Christopher Artlett, KC Wong

Abstract: Detecting unexpected objects (anomalies) in real time has great potential for monitoring, managing, and protecting the environment. Hyperspectral line-scan cameras are a low-cost solution that enhance confidence in anomaly detection over RGB and multispectral imagery. However, existing line-scan algorithms are too slow when using small computers (e.g. those onboard a drone or small satellite), do… ▽ More Detecting unexpected objects (anomalies) in real time has great potential for monitoring, managing, and protecting the environment. Hyperspectral line-scan cameras are a low-cost solution that enhance confidence in anomaly detection over RGB and multispectral imagery. However, existing line-scan algorithms are too slow when using small computers (e.g. those onboard a drone or small satellite), do not adapt to changing scenery, or lack robustness against geometric distortions. This paper introduces the Exponentially moving RX algorithm (ERX) to address these issues, and compares it with four existing RX-based anomaly detection methods for hyperspectral line scanning. Three large and more complex datasets are also introduced to better assess the practical challenges when using line-scan cameras (two hyperspectral and one multispectral). ERX was evaluated using a Jetson Xavier NX edge computing module (6-core CPU, 8GB RAM, 20W power draw), achieving the best combination of speed and detection performance. ERX was 9 times faster than the next-best algorithm on the dataset with the highest number of bands (108 band), with an average speed of 561 lines per second on the Jetson. It achieved a 29.3% AUC improvement over the next-best algorithm on the most challenging dataset, while showing greater adaptability through consistently high AUC scores regardless of the camera's starting location. ERX performed robustly across all datasets, achieving an AUC of 0.941 on a drone-collected hyperspectral line scan dataset without geometric corrections (a 16.9% improvement over existing algorithms). This work enables future research on the detection of anomalous objects in real time, adaptive and automatic threshold selection, and real-time field tests. The datasets and the Python code are openly available at: https://github.com/WiseGamgee/HyperAD, promoting accessibility and future work. △ Less

Submitted 23 December, 2024; v1 submitted 27 August, 2024; originally announced August 2024.

Comments: 17 pages, 13 figures, 4 tables, code and datasets accessible at https://github.com/WiseGamgee/HyperAD

CRIS: Collaborative Refinement Integrated with Segmentation for Polyp Segmentation

Authors: Ankush Gajanan Arudkar, Bernard J. E. Evans

Abstract: Accurate detection of colorectal cancer and early prevention heavily rely on precise polyp identification during gastrointestinal colonoscopy. Due to limited data, many current state-of-the-art deep learning methods for polyp segmentation often rely on post-processing of masks to reduce noise and enhance results. In this study, we propose an approach that integrates mask refinement and binary sema… ▽ More Accurate detection of colorectal cancer and early prevention heavily rely on precise polyp identification during gastrointestinal colonoscopy. Due to limited data, many current state-of-the-art deep learning methods for polyp segmentation often rely on post-processing of masks to reduce noise and enhance results. In this study, we propose an approach that integrates mask refinement and binary semantic segmentation, leveraging a novel collaborative training strategy that surpasses current widely-used refinement strategies. We demonstrate the superiority of our approach through comprehensive evaluation on established benchmark datasets and its successful application across various medical image segmentation architectures. △ Less

Submitted 29 May, 2024; originally announced May 2024.

Generalizing Multi-Step Inverse Models for Representation Learning to Finite-Memory POMDPs

Authors: Lili Wu, Ben Evans, Riashat Islam, Raihan Seraj, Yonathan Efroni, Alex Lamb

Abstract: Discovering an informative, or agent-centric, state representation that encodes only the relevant information while discarding the irrelevant is a key challenge towards scaling reinforcement learning algorithms and efficiently applying them to downstream tasks. Prior works studied this problem in high-dimensional Markovian environments, when the current observation may be a complex object but is s… ▽ More Discovering an informative, or agent-centric, state representation that encodes only the relevant information while discarding the irrelevant is a key challenge towards scaling reinforcement learning algorithms and efficiently applying them to downstream tasks. Prior works studied this problem in high-dimensional Markovian environments, when the current observation may be a complex object but is sufficient to decode the informative state. In this work, we consider the problem of discovering the agent-centric state in the more challenging high-dimensional non-Markovian setting, when the state can be decoded from a sequence of past observations. We establish that generalized inverse models can be adapted for learning agent-centric state representation for this task. Our results include asymptotic theory in the deterministic dynamics setting as well as counter-examples for alternative intuitive algorithms. We complement these findings with a thorough empirical study on the agent-centric state discovery abilities of the different alternatives we put forward. Particularly notable is our analysis of past actions, where we show that these can be a double-edged sword: making the algorithms more successful when used correctly and causing dramatic failure when used incorrectly. △ Less

Submitted 22 April, 2024; originally announced April 2024.

Adapting to time: Why nature may have evolved a diverse set of neurons

Authors: Karim G. Habashy, Benjamin D. Evans, Dan F. M. Goodman, Jeffrey S. Bowers

Abstract: Brains have evolved diverse neurons with varying morphologies and dynamics that impact temporal information processing. In contrast, most neural network models use homogeneous units that vary only in spatial parameters (weights and biases). To explore the importance of temporal parameters, we trained spiking neural networks on tasks with varying temporal complexity, holding different parameter sub… ▽ More Brains have evolved diverse neurons with varying morphologies and dynamics that impact temporal information processing. In contrast, most neural network models use homogeneous units that vary only in spatial parameters (weights and biases). To explore the importance of temporal parameters, we trained spiking neural networks on tasks with varying temporal complexity, holding different parameter subsets constant. We found that adapting conduction delays is crucial for solving all test conditions under tight resource constraints. Remarkably, these tasks can be solved using only temporal parameters (delays and time constants) with constant weights. In more complex spatio-temporal tasks, an adaptable bursting parameter was essential. Overall, allowing adaptation of both temporal and spatial parameters enhances network robustness to noise, a vital feature for biological brains and neuromorphic computing systems. Our findings suggest that rich and adaptable dynamics may be the key for solving temporally structured tasks efficiently in evolving organisms, which would help explain the diverse physiological properties of biological neurons. △ Less

Submitted 12 January, 2025; v1 submitted 22 April, 2024; originally announced April 2024.

Comments: 19 pages, 6 figures

ACM Class: K.3.2; I.2.m

Journal ref: PLoS Comput Biol 20(12): e1012673 (2024)

MindSet: Vision. A toolbox for testing DNNs on key psychological experiments

Authors: Valerio Biscione, Dong Yin, Gaurav Malhotra, Marin Dujmovic, Milton L. Montero, Guillermo Puebla, Federico Adolfi, Rachel F. Heaton, John E. Hummel, Benjamin D. Evans, Karim Habashy, Jeffrey S. Bowers

Abstract: Multiple benchmarks have been developed to assess the alignment between deep neural networks (DNNs) and human vision. In almost all cases these benchmarks are observational in the sense they are composed of behavioural and brain responses to naturalistic images that have not been manipulated to test hypotheses regarding how DNNs or humans perceive and identify objects. Here we introduce the toolbo… ▽ More Multiple benchmarks have been developed to assess the alignment between deep neural networks (DNNs) and human vision. In almost all cases these benchmarks are observational in the sense they are composed of behavioural and brain responses to naturalistic images that have not been manipulated to test hypotheses regarding how DNNs or humans perceive and identify objects. Here we introduce the toolbox MindSet: Vision, consisting of a collection of image datasets and related scripts designed to test DNNs on 30 psychological findings. In all experimental conditions, the stimuli are systematically manipulated to test specific hypotheses regarding human visual perception and object recognition. In addition to providing pre-generated datasets of images, we provide code to regenerate these datasets, offering many configurable parameters which greatly extend the dataset versatility for different research contexts, and code to facilitate the testing of DNNs on these image datasets using three different methods (similarity judgments, out-of-distribution classification, and decoder method), accessible at https://github.com/MindSetVision/mindset-vision. We test ResNet-152 on each of these methods as an example of how the toolbox can be used. △ Less

Submitted 8 April, 2024; originally announced April 2024.

Unifying F1TENTH Autonomous Racing: Survey, Methods and Benchmarks

Authors: Benjamin David Evans, Raphael Trumpp, Marco Caccamo, Felix Jahncke, Johannes Betz, Hendrik Willem Jordaan, Herman Arnold Engelbrecht

Abstract: The F1TENTH autonomous driving platform, consisting of 1:10-scale remote-controlled cars, has evolved into a well-established education and research platform. The many publications and real-world competitions span many domains, from classical path planning to novel learning-based algorithms. Consequently, the field is wide and disjointed, hindering direct comparison of developed methods and making… ▽ More The F1TENTH autonomous driving platform, consisting of 1:10-scale remote-controlled cars, has evolved into a well-established education and research platform. The many publications and real-world competitions span many domains, from classical path planning to novel learning-based algorithms. Consequently, the field is wide and disjointed, hindering direct comparison of developed methods and making it difficult to assess the state-of-the-art. Therefore, we aim to unify the field by surveying current approaches, describing common methods, and providing benchmark results to facilitate clear comparisons and establish a baseline for future work. This research aims to survey past and current work with F1TENTH vehicles in the classical and learning categories and explain the different solution approaches. We describe particle filter localisation, trajectory optimisation and tracking, model predictive contouring control, follow-the-gap, and end-to-end reinforcement learning. We provide an open-source evaluation of benchmark methods and investigate overlooked factors of control frequency and localisation accuracy for classical methods as well as reward signal and training map for learning methods. The evaluation shows that the optimisation and tracking method achieves the fastest lap times, followed by the online planning approach. Finally, our work identifies and outlines the relevant research aspects to help motivate future work in the F1TENTH domain. △ Less

Submitted 25 April, 2024; v1 submitted 28 February, 2024; originally announced February 2024.

Comments: 12 pages, 18 figures. Sumbitted for publication

A Heterogeneous Agent Model of Mortgage Servicing: An Income-based Relief Analysis

Authors: Deepeka Garg, Benjamin Patrick Evans, Leo Ardon, Annapoorani Lakshmi Narayanan, Jared Vann, Udari Madhushani, Makada Henry-Nickie, Sumitra Ganesh

Abstract: Mortgages account for the largest portion of household debt in the United States, totaling around \$12 trillion nationwide. In times of financial hardship, alleviating mortgage burdens is essential for supporting affected households. The mortgage servicing industry plays a vital role in offering this assistance, yet there has been limited research modelling the complex relationship between househo… ▽ More Mortgages account for the largest portion of household debt in the United States, totaling around \$12 trillion nationwide. In times of financial hardship, alleviating mortgage burdens is essential for supporting affected households. The mortgage servicing industry plays a vital role in offering this assistance, yet there has been limited research modelling the complex relationship between households and servicers. To bridge this gap, we developed an agent-based model that explores household behavior and the effectiveness of relief measures during financial distress. Our model represents households as adaptive learning agents with realistic financial attributes. These households experience exogenous income shocks, which may influence their ability to make mortgage payments. Mortgage servicers provide relief options to these households, who then choose the most suitable relief based on their unique financial circumstances and individual preferences. We analyze the impact of various external shocks and the success of different mortgage relief strategies on specific borrower subgroups. Through this analysis, we show that our model can not only replicate real-world mortgage studies but also act as a tool for conducting a broad range of what-if scenario analyses. Our approach offers fine-grained insights that can inform the development of more effective and inclusive mortgage relief solutions. △ Less

Submitted 29 February, 2024; v1 submitted 27 February, 2024; originally announced February 2024.

Comments: AAAI 2024 - AI in Finance for Social Impact

Does mapping elites illuminate search spaces? A large-scale user study of MAP--Elites applied to human--AI collaborative design

Authors: Sean P. Walton, Ben J. Evans, Alma A. M. Rahat, James Stovold, Jakub Vincalek

Abstract: Two studies of a human-AI collaborative design tool were carried out in order to understand the influence design recommendations have on the design process. The tool investigated is based on an evolutionary algorithm attempting to design a virtual car to travel as far as possible in a fixed time. Participants were able to design their own cars, make recommendations to the algorithm and view sets o… ▽ More Two studies of a human-AI collaborative design tool were carried out in order to understand the influence design recommendations have on the design process. The tool investigated is based on an evolutionary algorithm attempting to design a virtual car to travel as far as possible in a fixed time. Participants were able to design their own cars, make recommendations to the algorithm and view sets of recommendations from the algorithm. The algorithm-recommended sets were designs which had been previously tested; some sets were simply randomly picked and other sets were picked using MAP-Elites. In the first study 808 design sessions were recorded as part of a science outreach program, each with analytical data of how each participant used the tool. To provide context to this quantitative data, a smaller double-blind lab study was also carried out with 12 participants. In the lab study the same quantitative data from the large scale study was collected alongside responses to interview questions. Although there is some evidence that the MAP-Elites provide higher-quality individual recommendations, neither study provides convincing evidence that these recommendations have a more positive influence on the design process than simply a random selection of designs. In fact, it seems that providing a combination of MAP-Elites and randomly selected recommendations is beneficial to the process. Furthermore, simply viewing recommendations from the MAP-Elites had a positive influence on engagement in the design task and the quality of the final design produced. Our findings are significant both for researchers designing new mixed-initiative tools, and those who wish to evaluate existing tools. Most significantly, we found that metrics researchers currently use to evaluate the success of human-AI collaborative algorithms do not measure the full influence these algorithms have on the design process. △ Less

Submitted 30 January, 2024; originally announced February 2024.

Comments: 17 pages

ACM Class: I.2.0; J.6; G.1.6

Learning and Calibrating Heterogeneous Bounded Rational Market Behaviour with Multi-Agent Reinforcement Learning

Authors: Benjamin Patrick Evans, Sumitra Ganesh

Abstract: Agent-based models (ABMs) have shown promise for modelling various real world phenomena incompatible with traditional equilibrium analysis. However, a critical concern is the manual definition of behavioural rules in ABMs. Recent developments in multi-agent reinforcement learning (MARL) offer a way to address this issue from an optimisation perspective, where agents strive to maximise their utilit… ▽ More Agent-based models (ABMs) have shown promise for modelling various real world phenomena incompatible with traditional equilibrium analysis. However, a critical concern is the manual definition of behavioural rules in ABMs. Recent developments in multi-agent reinforcement learning (MARL) offer a way to address this issue from an optimisation perspective, where agents strive to maximise their utility, eliminating the need for manual rule specification. This learning-focused approach aligns with established economic and financial models through the use of rational utility-maximising agents. However, this representation departs from the fundamental motivation for ABMs: that realistic dynamics emerging from bounded rationality and agent heterogeneity can be modelled. To resolve this apparent disparity between the two approaches, we propose a novel technique for representing heterogeneous processing-constrained agents within a MARL framework. The proposed approach treats agents as constrained optimisers with varying degrees of strategic skills, permitting departure from strict utility maximisation. Behaviour is learnt through repeated simulations with policy gradients to adjust action likelihoods. To allow efficient computation, we use parameterised shared policy learning with distributions of agent skill levels. Shared policy learning avoids the need for agents to learn individual policies yet still enables a spectrum of bounded rational behaviours. We validate our model's effectiveness using real-world data on a range of canonical $n$-agent settings, demonstrating significantly improved predictive capability. △ Less

Submitted 1 February, 2024; originally announced February 2024.

Comments: Accepted as a full paper at AAMAS 2024

High-performance Racing on Unmapped Tracks using Local Maps

Authors: Benjamin David Evans, Hendrik Willem Jordaan, Herman Arnold Engelbrecht

Abstract: Map-based methods for autonomous racing estimate the vehicle's location, which is used to follow a high-level plan. While map-based optimisation methods demonstrate high-performance results, they are limited by requiring a map of the environment. In contrast, mapless methods can operate in unmapped contexts since they directly process raw sensor data (often LiDAR) to calculate commands. However, a… ▽ More Map-based methods for autonomous racing estimate the vehicle's location, which is used to follow a high-level plan. While map-based optimisation methods demonstrate high-performance results, they are limited by requiring a map of the environment. In contrast, mapless methods can operate in unmapped contexts since they directly process raw sensor data (often LiDAR) to calculate commands. However, a major limitation in mapless methods is poor performance due to a lack of optimisation. In response, we propose the local map framework that uses easily extractable, low-level features to build local maps of the visible region that form the input to optimisation-based controllers. Our local map generation extracts the visible racetrack boundaries and calculates a centreline and track widths used for planning. We evaluate our method for simulated F1Tenth autonomous racing using a two-stage trajectory optimisation and tracking strategy and a model predictive controller. Our method achieves lap times that are 8.8% faster than the Follow-The-Gap method and 3.22% faster than end-to-end neural networks due to the optimisation resulting in a faster speed profile. The local map planner is 3.28% slower than global methods that have access to an entire map of the track that can be used for planning. Critically, our approach enables high-speed autonomous racing on unmapped tracks, achieving performance similar to global methods without requiring a track map. △ Less

Submitted 31 January, 2024; originally announced January 2024.

Comments: 6 pages, 14 figures. Submitted to IV 2024

A Framework for Scalable Ambient Air Pollution Concentration Estimation

Authors: Liam J Berrisford, Lucy S Neal, Helen J Buttery, Benjamin R Evans, Ronaldo Menezes

Abstract: Ambient air pollution remains a critical issue in the United Kingdom, where data on air pollution concentrations form the foundation for interventions aimed at improving air quality. However, the current air pollution monitoring station network in the UK is characterized by spatial sparsity, heterogeneous placement, and frequent temporal data gaps, often due to issues such as power outages. We int… ▽ More Ambient air pollution remains a critical issue in the United Kingdom, where data on air pollution concentrations form the foundation for interventions aimed at improving air quality. However, the current air pollution monitoring station network in the UK is characterized by spatial sparsity, heterogeneous placement, and frequent temporal data gaps, often due to issues such as power outages. We introduce a scalable data-driven supervised machine learning model framework designed to address temporal and spatial data gaps by filling missing measurements. This approach provides a comprehensive dataset for England throughout 2018 at a 1kmx1km hourly resolution. Leveraging machine learning techniques and real-world data from the sparsely distributed monitoring stations, we generate 355,827 synthetic monitoring stations across the study area, yielding data valued at approximately \pounds70 billion. Validation was conducted to assess the model's performance in forecasting, estimating missing locations, and capturing peak concentrations. The resulting dataset is of particular interest to a diverse range of stakeholders engaged in downstream assessments supported by outdoor air pollution concentration data for NO2, O3, PM10, PM2.5, and SO2. This resource empowers stakeholders to conduct studies at a higher resolution than was previously possible. △ Less

Submitted 16 January, 2024; originally announced January 2024.

Comments: Main: 27 pages, 11 figures, 6 tables. Supplementary: 32 pages, 21 figures, 11 tables

FourCastNeXt: Optimizing FourCastNet Training for Limited Compute

Authors: Edison Guo, Maruf Ahmed, Yue Sun, Rui Yang, Harrison Cook, Tennessee Leeuwenburg, Ben Evans

Abstract: FourCastNeXt is an optimization of FourCastNet - a global machine learning weather forecasting model - that performs with a comparable level of accuracy and can be trained using around 5% of the original FourCastNet computational requirements. This technical report presents strategies for model optimization that maintain similar performance as measured by the root-mean-square error (RMSE) of the m… ▽ More FourCastNeXt is an optimization of FourCastNet - a global machine learning weather forecasting model - that performs with a comparable level of accuracy and can be trained using around 5% of the original FourCastNet computational requirements. This technical report presents strategies for model optimization that maintain similar performance as measured by the root-mean-square error (RMSE) of the modelled variables. By providing a model with very low comparative training costs, FourCastNeXt makes Neural Earth System Modelling much more accessible to researchers looking to conduct training experiments and ablation studies. FourCastNeXt training and inference code are available at https://github.com/nci/FourCastNeXt △ Less

Submitted 20 March, 2024; v1 submitted 10 January, 2024; originally announced January 2024.

Comments: Major revision. All prior content (text, figures, table) has been updated. Additionally, new text, tables and figures have been added. Updated title. Updated author list

PcLast: Discovering Plannable Continuous Latent States

Authors: Anurag Koul, Shivakanth Sujit, Shaoru Chen, Ben Evans, Lili Wu, Byron Xu, Rajan Chari, Riashat Islam, Raihan Seraj, Yonathan Efroni, Lekan Molu, Miro Dudik, John Langford, Alex Lamb

Abstract: Goal-conditioned planning benefits from learned low-dimensional representations of rich observations. While compact latent representations typically learned from variational autoencoders or inverse dynamics enable goal-conditioned decision making, they ignore state reachability, hampering their performance. In this paper, we learn a representation that associates reachable states together for effe… ▽ More Goal-conditioned planning benefits from learned low-dimensional representations of rich observations. While compact latent representations typically learned from variational autoencoders or inverse dynamics enable goal-conditioned decision making, they ignore state reachability, hampering their performance. In this paper, we learn a representation that associates reachable states together for effective planning and goal-conditioned policy learning. We first learn a latent representation with multi-step inverse dynamics (to remove distracting information), and then transform this representation to associate reachable states together in $\ell_2$ space. Our proposals are rigorously tested in various simulation testbeds. Numerical results in reward-based settings show significant improvements in sampling efficiency. Further, in reward-free settings this approach yields layered state abstractions that enable computationally efficient hierarchical planning for reaching ad hoc goals with zero additional samples. △ Less

Submitted 10 June, 2024; v1 submitted 6 November, 2023; originally announced November 2023.

Comments: Accepted at ICML 2024

See to Touch: Learning Tactile Dexterity through Visual Incentives

Authors: Irmak Guzey, Yinlong Dai, Ben Evans, Soumith Chintala, Lerrel Pinto

Abstract: Equipping multi-fingered robots with tactile sensing is crucial for achieving the precise, contact-rich, and dexterous manipulation that humans excel at. However, relying solely on tactile sensing fails to provide adequate cues for reasoning about objects' spatial configurations, limiting the ability to correct errors and adapt to changing situations. In this paper, we present Tactile Adaptation f… ▽ More Equipping multi-fingered robots with tactile sensing is crucial for achieving the precise, contact-rich, and dexterous manipulation that humans excel at. However, relying solely on tactile sensing fails to provide adequate cues for reasoning about objects' spatial configurations, limiting the ability to correct errors and adapt to changing situations. In this paper, we present Tactile Adaptation from Visual Incentives (TAVI), a new framework that enhances tactile-based dexterity by optimizing dexterous policies using vision-based rewards. First, we use a contrastive-based objective to learn visual representations. Next, we construct a reward function using these visual representations through optimal-transport based matching on one human demonstration. Finally, we use online reinforcement learning on our robot to optimize tactile-based policies that maximize the visual reward. On six challenging tasks, such as peg pick-and-place, unstacking bowls, and flipping slender objects, TAVI achieves a success rate of 73% using our four-fingered Allegro robot hand. The increase in performance is 108% higher than policies using tactile and vision-based rewards and 135% higher than policies without tactile observational input. Robot videos are best viewed on our project website: https://see-to-touch.github.io/. △ Less

Submitted 21 September, 2023; originally announced September 2023.

Creating Multi-Level Skill Hierarchies in Reinforcement Learning

Authors: Joshua B. Evans, Özgür Şimşek

Abstract: What is a useful skill hierarchy for an autonomous agent? We propose an answer based on a graphical representation of how the interaction between an agent and its environment may unfold. Our approach uses modularity maximisation as a central organising principle to expose the structure of the interaction graph at multiple levels of abstraction. The result is a collection of skills that operate at… ▽ More What is a useful skill hierarchy for an autonomous agent? We propose an answer based on a graphical representation of how the interaction between an agent and its environment may unfold. Our approach uses modularity maximisation as a central organising principle to expose the structure of the interaction graph at multiple levels of abstraction. The result is a collection of skills that operate at varying time scales, organised into a hierarchy, where skills that operate over longer time scales are composed of skills that operate over shorter time scales. The entire skill hierarchy is generated automatically, with no human intervention, including the skills themselves (their behaviour, when they can be called, and when they terminate) as well as the hierarchical dependency structure between them. In a wide range of environments, this approach generates skill hierarchies that are intuitively appealing and that considerably improve the learning performance of the agent. △ Less

Submitted 17 January, 2024; v1 submitted 16 June, 2023; originally announced June 2023.

Comments: 20 pages, 10 figures. Accepted at NeurIPS 2023

High-speed Autonomous Racing using Trajectory-aided Deep Reinforcement Learning

Authors: Benjamin David Evans, Herman Arnold Engelbrecht, Hendrik Willem Jordaan

Abstract: The classical method of autonomous racing uses real-time localisation to follow a precalculated optimal trajectory. In contrast, end-to-end deep reinforcement learning (DRL) can train agents to race using only raw LiDAR scans. While classical methods prioritise optimization for high-performance racing, DRL approaches have focused on low-performance contexts with little consideration of the speed p… ▽ More The classical method of autonomous racing uses real-time localisation to follow a precalculated optimal trajectory. In contrast, end-to-end deep reinforcement learning (DRL) can train agents to race using only raw LiDAR scans. While classical methods prioritise optimization for high-performance racing, DRL approaches have focused on low-performance contexts with little consideration of the speed profile. This work addresses the problem of using end-to-end DRL agents for high-speed autonomous racing. We present trajectory-aided learning (TAL) that trains DRL agents for high-performance racing by incorporating the optimal trajectory (racing line) into the learning formulation. Our method is evaluated using the TD3 algorithm on four maps in the open-source F1Tenth simulator. The results demonstrate that our method achieves a significantly higher lap completion rate at high speeds compared to the baseline. This is due to TAL training the agent to select a feasible speed profile of slowing down in the corners and roughly tracking the optimal trajectory. △ Less

Submitted 12 June, 2023; originally announced June 2023.

Comments: 7 pages, 16 figures. Submitted for review

Learning-Based One-Bit Maximum Likelihood Detection for Massive MIMO Systems: Dithering-Aided Adaptive Approach

Authors: Yunseong Cho, Jinseok Choi, Brian L. Evans

Abstract: In this paper, we propose a learning-based detection framework for uplink massive multiple-input and multiple-output (MIMO) systems with one-bit analog-to-digital converters. The learning-based detection only requires counting the occurrences of the quantized outputs of -1 and +1 for estimating a likelihood probability at each antenna. Accordingly, the key advantage of this approach is to perform… ▽ More In this paper, we propose a learning-based detection framework for uplink massive multiple-input and multiple-output (MIMO) systems with one-bit analog-to-digital converters. The learning-based detection only requires counting the occurrences of the quantized outputs of -1 and +1 for estimating a likelihood probability at each antenna. Accordingly, the key advantage of this approach is to perform maximum likelihood detection without explicit channel estimation which has been one of the primary challenges of one-bit quantized systems. However, due to the quasi-deterministic reception in the high signal-to-noise ratio (SNR) regime, one-bit observations in the high SNR regime are biased to either +1 or -1, and thus, the learning requires excessive training to estimate the small likelihood probabilities. To address this drawback, we propose a dither-and-learning technique to estimate likelihood functions from dithered signals. First, we add a dithering signal to artificially decrease the SNR and then infer the likelihood function from the quantized dithered signals by using an SNR estimate derived from a deep neural network-based estimator which is trained offline. We extend our technique by developing an adaptive dither-and-learning method that updates the dithering power according to the patterns observed in the quantized dithered signals. The proposed framework is also applied to channel-coded MIMO systems by computing a bit-wise and user-wise log-likelihood ratio from the refined likelihood probabilities. Simulation results validate the performance of the proposed methods in both uncoded and coded systems. △ Less

Submitted 21 March, 2024; v1 submitted 16 April, 2023; originally announced April 2023.

Comments: Accepted for publication in IEEE Transactions on Vehicular Technologies

Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Play

Authors: Irmak Guzey, Ben Evans, Soumith Chintala, Lerrel Pinto

Abstract: Teaching dexterity to multi-fingered robots has been a longstanding challenge in robotics. Most prominent work in this area focuses on learning controllers or policies that either operate on visual observations or state estimates derived from vision. However, such methods perform poorly on fine-grained manipulation tasks that require reasoning about contact forces or about objects occluded by the… ▽ More Teaching dexterity to multi-fingered robots has been a longstanding challenge in robotics. Most prominent work in this area focuses on learning controllers or policies that either operate on visual observations or state estimates derived from vision. However, such methods perform poorly on fine-grained manipulation tasks that require reasoning about contact forces or about objects occluded by the hand itself. In this work, we present T-Dex, a new approach for tactile-based dexterity, that operates in two phases. In the first phase, we collect 2.5 hours of play data, which is used to train self-supervised tactile encoders. This is necessary to bring high-dimensional tactile readings to a lower-dimensional embedding. In the second phase, given a handful of demonstrations for a dexterous task, we learn non-parametric policies that combine the tactile observations with visual ones. Across five challenging dexterous tasks, we show that our tactile-based dexterity models outperform purely vision and torque-based models by an average of 1.7X. Finally, we provide a detailed analysis on factors critical to T-Dex including the importance of play data, architectures, and representation learning. △ Less

Submitted 21 March, 2023; originally announced March 2023.

Comments: Video and code can be accessed here: https://tactile-dexterity.github.io/

Adaptive Learning-Based Detection for One-Bit Quantized Massive MIMO Systems

Authors: Yunseong Cho, Jinseok Choi, Brian L. Evans

Abstract: We propose an adaptive learning-based framework for uplink massive multiple-input multiple-output (MIMO) systems with one-bit analog-to-digital converters. Learning-based detection does not need to estimate channels, which overcomes a key drawback in one-bit quantized systems. During training, learning-based detection suffers at high signal-to-noise ratio (SNR) because observations will be biased… ▽ More We propose an adaptive learning-based framework for uplink massive multiple-input multiple-output (MIMO) systems with one-bit analog-to-digital converters. Learning-based detection does not need to estimate channels, which overcomes a key drawback in one-bit quantized systems. During training, learning-based detection suffers at high signal-to-noise ratio (SNR) because observations will be biased to +1 or -1 which leads to many zero-valued empirical likelihood functions. At low SNR, observations vary frequently in value but the high noise power makes capturing the effect of the channel difficult. To address these drawbacks, we propose an adaptive dithering-and-learning method. During training, received values are mixed with dithering noise whose statistics are known to the base station, and the dithering noise power is updated for each antenna element depending on the observed pattern of the output. We then use the refined probabilities in the one-bit maximum likelihood detection rule. Simulation results validate the detection performance of the proposed method vs. our previous method using fixed dithering noise power as well as zero-forcing and optimal ML detection both of which assume perfect channel knowledge. △ Less

Submitted 13 November, 2022; originally announced November 2022.

Comments: 6 pages 6 figures

Bypassing the Simulation-to-reality Gap: Online Reinforcement Learning using a Supervisor

Authors: Benjamin David Evans, Johannes Betz, Hongrui Zheng, Herman A. Engelbrecht, Rahul Mangharam, Hendrik W. Jordaan

Abstract: Deep reinforcement learning (DRL) is a promising method to learn control policies for robots only from demonstration and experience. To cover the whole dynamic behaviour of the robot, DRL training is an active exploration process typically performed in simulation environments. Although this simulation training is cheap and fast, applying DRL algorithms to real-world settings is difficult. If agent… ▽ More Deep reinforcement learning (DRL) is a promising method to learn control policies for robots only from demonstration and experience. To cover the whole dynamic behaviour of the robot, DRL training is an active exploration process typically performed in simulation environments. Although this simulation training is cheap and fast, applying DRL algorithms to real-world settings is difficult. If agents are trained until they perform safely in simulation, transferring them to physical systems is difficult due to the sim-to-real gap caused by the difference between the simulation dynamics and the physical robot. In this paper, we present a method of online training a DRL agent to drive autonomously on a physical vehicle by using a model-based safety supervisor. Our solution uses a supervisory system to check if the action selected by the agent is safe or unsafe and ensure that a safe action is always implemented on the vehicle. With this, we can bypass the sim-to-real problem while training the DRL algorithm safely, quickly, and efficiently. We compare our method with conventional learning in simulation and on a physical vehicle. We provide a variety of real-world experiments where we train online a small-scale vehicle to drive autonomously with no prior simulation training. The evaluation results show that our method trains agents with improved sample efficiency while never crashing, and the trained agents demonstrate better driving performance than those trained in simulation. △ Less

Submitted 13 July, 2023; v1 submitted 22 September, 2022; originally announced September 2022.

Comments: 7 Pages, 10 Figures, 1 Table

Coordinated Per-Antenna Power Minimization for Multicell Massive MIMO Systems with Low-Resolution Data Converters

Authors: Yunseong Cho, Jinseok Choi, Brian L. Evans

Abstract: A multicell-coordinated beamforming solution for massive multiple-input multiple-output orthogonal frequency-division multiplexing (OFDM) systems is presented when employing low-resolution data converters and per-antenna level constraints. For a more realistic deployment, we aim to find the downlink (DL) beamformer that minimizes the maximum power on transmit antenna array of each basestation unde… ▽ More A multicell-coordinated beamforming solution for massive multiple-input multiple-output orthogonal frequency-division multiplexing (OFDM) systems is presented when employing low-resolution data converters and per-antenna level constraints. For a more realistic deployment, we aim to find the downlink (DL) beamformer that minimizes the maximum power on transmit antenna array of each basestation under received signal quality constraints while minimizing per-antenna transmit power. We show that strong duality holds between the primal DL formulation and its manageable Lagrangian dual problem which can be interpreted as the virtual uplink (UL) problem with adjustable noise covariance matrices. For a fixed set of noise covariance matrices, we claim that the virtual UL solution is effectively used to compute the DL beamformer and noise covariance matrices can be subsequently updated with an associated subgradient. Our primary contributions are then (1) formulating the quantized DL OFDM antenna power minimax problem and deriving its associated dual problem, (2) showing strong duality and interpreting the dual as a virtual quantized UL OFDM problem, and (3) developing an iterative minimax algorithm based on the dual problem. Simulations validate the proposed algorithm in terms of the maximum antenna transmit power and peak-to-average-power ratio. △ Less

Submitted 10 September, 2023; v1 submitted 8 August, 2022; originally announced August 2022.

Comments: submitted for possible IEEE journal publication

Bounded strategic reasoning explains crisis emergence in multi-agent market games

Authors: Benjamin Patrick Evans, Mikhail Prokopenko

Abstract: The efficient market hypothesis (EMH), based on rational expectations and market equilibrium, is the dominant perspective for modelling economic markets. However, the most notable critique of the EMH is the inability to model periods of out-of-equilibrium behaviour in the absence of any significant external news. When such dynamics emerge endogenously, the traditional economic frameworks provide n… ▽ More The efficient market hypothesis (EMH), based on rational expectations and market equilibrium, is the dominant perspective for modelling economic markets. However, the most notable critique of the EMH is the inability to model periods of out-of-equilibrium behaviour in the absence of any significant external news. When such dynamics emerge endogenously, the traditional economic frameworks provide no explanation for such behaviour and the deviation from equilibrium. This work offers an alternate perspective explaining the endogenous emergence of punctuated out-of-equilibrium dynamics based on bounded rational agents. In a concise market entrance game, we show how boundedly rational strategic reasoning can lead to endogenously emerging crises, exhibiting fat tails in "returns". We also show how other common stylised facts of economic markets, such as clustered volatility, can be explained due to agent diversity (or lack thereof) and the varying learning updates across the agents. This work explains various stylised facts and crisis emergence in economic markets, in the absence of any external news, based purely on agent interactions and bounded rational reasoning. △ Less

Submitted 11 June, 2022; originally announced June 2022.

Comments: 10 pages + 7 page appendix, 7 figures

Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulation

Authors: Sridhar Pandian Arunachalam, Sneha Silwal, Ben Evans, Lerrel Pinto

Abstract: Optimizing behaviors for dexterous manipulation has been a longstanding challenge in robotics, with a variety of methods from model-based control to model-free reinforcement learning having been previously explored in literature. Perhaps one of the most powerful techniques to learn complex manipulation strategies is imitation learning. However, collecting and learning from demonstrations in dexter… ▽ More Optimizing behaviors for dexterous manipulation has been a longstanding challenge in robotics, with a variety of methods from model-based control to model-free reinforcement learning having been previously explored in literature. Perhaps one of the most powerful techniques to learn complex manipulation strategies is imitation learning. However, collecting and learning from demonstrations in dexterous manipulation is quite challenging. The complex, high-dimensional action-space involved with multi-finger control often leads to poor sample efficiency of learning-based methods. In this work, we propose 'Dexterous Imitation Made Easy' (DIME) a new imitation learning framework for dexterous manipulation. DIME only requires a single RGB camera to observe a human operator and teleoperate our robotic hand. Once demonstrations are collected, DIME employs standard imitation learning methods to train dexterous manipulation policies. On both simulation and real robot benchmarks we demonstrate that DIME can be used to solve complex, in-hand manipulation tasks such as 'flipping', 'spinning', and 'rotating' objects with the Allegro hand. Our framework along with pre-collected demonstrations is publicly available at https://nyu-robot-learning.github.io/dime. △ Less

Submitted 24 March, 2022; originally announced March 2022.

Comments: The first two authors contributed equally

Context is Everything: Implicit Identification for Dynamics Adaptation

Authors: Ben Evans, Abitha Thankaraj, Lerrel Pinto

Abstract: Understanding environment dynamics is necessary for robots to act safely and optimally in the world. In realistic scenarios, dynamics are non-stationary and the causal variables such as environment parameters cannot necessarily be precisely measured or inferred, even during training. We propose Implicit Identification for Dynamics Adaptation (IIDA), a simple method to allow predictive models to ad… ▽ More Understanding environment dynamics is necessary for robots to act safely and optimally in the world. In realistic scenarios, dynamics are non-stationary and the causal variables such as environment parameters cannot necessarily be precisely measured or inferred, even during training. We propose Implicit Identification for Dynamics Adaptation (IIDA), a simple method to allow predictive models to adapt to changing environment dynamics. IIDA assumes no access to the true variations in the world and instead implicitly infers properties of the environment from a small amount of contextual data. We demonstrate IIDA's ability to perform well in unseen environments through a suite of simulated experiments on MuJoCo environments and a real robot dynamic sliding task. In general, IIDA significantly reduces model error and results in higher task performance over commonly used methods. Our code and robot videos are at https://bennevans.github.io/iida/ △ Less

Submitted 10 March, 2022; originally announced March 2022.

Comments: Accepted at ICRA 2022

BAM: Bayes with Adaptive Memory

Authors: Josue Nassar, Jennifer Brennan, Ben Evans, Kendall Lowrey

Abstract: Online learning via Bayes' theorem allows new data to be continuously integrated into an agent's current beliefs. However, a naive application of Bayesian methods in non stationary environments leads to slow adaptation and results in state estimates that may converge confidently to the wrong parameter value. A common solution when learning in changing environments is to discard/downweight past dat… ▽ More Online learning via Bayes' theorem allows new data to be continuously integrated into an agent's current beliefs. However, a naive application of Bayesian methods in non stationary environments leads to slow adaptation and results in state estimates that may converge confidently to the wrong parameter value. A common solution when learning in changing environments is to discard/downweight past data; however, this simple mechanism of "forgetting" fails to account for the fact that many real-world environments involve revisiting similar states. We propose a new framework, Bayes with Adaptive Memory (BAM), that takes advantage of past experience by allowing the agent to choose which past observations to remember and which to forget. We demonstrate that BAM generalizes many popular Bayesian update rules for non-stationary environments. Through a variety of experiments, we demonstrate the ability of BAM to continuously adapt in an ever-changing world. △ Less

Submitted 8 February, 2022; v1 submitted 4 February, 2022; originally announced February 2022.

Comments: International Conference on Learning Representations (ICLR), 2022

Coordinated Beamforming in Quantized Massive MIMO Systems with Per-Antenna Constraints

Authors: Yunseong Cho, Jinseok Choi, Brian L. Evans

Abstract: In this work, we present a solution for coordinated beamforming for large-scale downlink (DL) communication systems with low-resolution data converters when employing a per-antenna power constraint that limits the maximum antenna power to alleviate hardware cost. To this end, we formulate and solve the antenna power minimax problem for the coarsely quantized DL system with target signal-to-interfe… ▽ More In this work, we present a solution for coordinated beamforming for large-scale downlink (DL) communication systems with low-resolution data converters when employing a per-antenna power constraint that limits the maximum antenna power to alleviate hardware cost. To this end, we formulate and solve the antenna power minimax problem for the coarsely quantized DL system with target signal-to-interference-plus-noise ratio requirements. We show that the associated Lagrangian dual with uncertain noise covariance matrices achieves zero duality gap and that the dual solution can be used to obtain the primal DL solution. Using strong duality, we propose an iterative algorithm to determine the optimal dual solution, which is used to compute the optimal DL beamformer. We further update the noise covariance matrices using the optimal DL solution with an associated subgradient and perform projection onto the feasible domain. Through simulation, we evaluate the proposed method in maximum antenna power consumption and peak-to-average power ratio which are directly related to hardware efficiency. △ Less

Submitted 18 October, 2021; originally announced October 2021.

Comments: 6 pages, 3 figures, 1 table, submitted to WCNC 2022

Bounded rationality for relaxing best response and mutual consistency: The Quantal Hierarchy model of decision-making

Authors: Benjamin Patrick Evans, Mikhail Prokopenko

Abstract: While game theory has been transformative for decision-making, the assumptions made can be overly restrictive in certain instances. In this work, we investigate some of the underlying assumptions of rationality, such as mutual consistency and best response, and consider ways to relax these assumptions using concepts from level-$k$ reasoning and quantal response equilibrium (QRE) respectively. Spec… ▽ More While game theory has been transformative for decision-making, the assumptions made can be overly restrictive in certain instances. In this work, we investigate some of the underlying assumptions of rationality, such as mutual consistency and best response, and consider ways to relax these assumptions using concepts from level-$k$ reasoning and quantal response equilibrium (QRE) respectively. Specifically, we propose an information-theoretic two-parameter model called the Quantal Hierarchy model, which can relax both mutual consistency and best response while still approximating level-$k$, QRE, or typical Nash equilibrium behaviour in the limiting cases. The model is based on a recursive form of the variational free energy principle, representing higher-order reasoning as (pseudo) sequential decision-making in extensive-form game tree. This representation enables us to treat simultaneous games in a similar manner to sequential games, where reasoning resources deplete throughout the game-tree. Bounds in player processing abilities are captured as information costs, where future branches of reasoning are discounted, implying a hierarchy of players where lower-level players have fewer processing resources. We demonstrate the effectiveness of the Quantal Hierarchy model in several canonical economic games, {both simultaneous and sequential}, using out-of-sample modelling. △ Less

Submitted 15 March, 2023; v1 submitted 30 June, 2021; originally announced June 2021.

Comments: 31 pages, 20 figures

Adaptive Self-Interference Cancellation for Full-Duplex Wireless Communication Systems

Authors: Elyes Balti, Brian L. Evans

Abstract: In this letter, we consider single-cell, single-user systems wherein uplink and downlink user equipment communicate with a full-duplex relay. Due to the near-far problem, the self-interference (SI) can be 100-1000x the received signal power. In this context, we consider the adaptive Least Mean Squares (LMS) algorithm to estimate the SI channel and then subtract the SI from the desired received sig… ▽ More In this letter, we consider single-cell, single-user systems wherein uplink and downlink user equipment communicate with a full-duplex relay. Due to the near-far problem, the self-interference (SI) can be 100-1000x the received signal power. In this context, we consider the adaptive Least Mean Squares (LMS) algorithm to estimate the SI channel and then subtract the SI from the desired received signal before the analog-to-digital converter (ADC). We measure the robustness of this technique in terms of bit error rate (BER) and spectral efficiency. △ Less

Submitted 1 June, 2021; v1 submitted 3 April, 2021; originally announced April 2021.

Reward Signal Design for Autonomous Racing

Authors: Benjamin Evans, Herman A. Engelbrecht, Hendrik W. Jordaan

Abstract: Reinforcement learning (RL) has shown to be a valuable tool in training neural networks for autonomous motion planning. The application of RL to a specific problem is dependent on a reward signal to quantify how good or bad a certain action is. This paper addresses the problem of reward signal design for robotic control in the context of local planning for autonomous racing. We aim to design rewar… ▽ More Reinforcement learning (RL) has shown to be a valuable tool in training neural networks for autonomous motion planning. The application of RL to a specific problem is dependent on a reward signal to quantify how good or bad a certain action is. This paper addresses the problem of reward signal design for robotic control in the context of local planning for autonomous racing. We aim to design reward signals that are able to perform well in multiple, competing, continuous metrics. Three different methodologies of position-based, velocity-based, and action-based rewards are considered and evaluated in the context of F1/10th racing. A novel method of rewarding the agent on its state relative to an optimal trajectory is presented. Agents are trained and tested in simulation and the behaviors generated by the reward signals are compared to each other on the basis of average lap time and completion rate. The results indicate that a reward based on the distance and velocity relative to a minimum curvature trajectory produces the fastest lap times. △ Less

Submitted 26 August, 2021; v1 submitted 18 March, 2021; originally announced March 2021.

Comments: 6 pages, 10 Figures, This work has been submitted to the IEEE for possible publication

Learning the Subsystem of Local Planning for Autonomous Racing

Authors: Benjamin Evans, Hendrik W. Jordaan, Herman A. Engelbrecht

Abstract: The problem of autonomous racing is to navigate through a race course as quickly as possible while not colliding with any obstacles. We approach the autonomous racing problem with the added constraint of not maintaining an updated obstacle map of the environment. Several current approaches to this problem use end-to-end learning systems where an agent replaces the entire navigation pipeline. This… ▽ More The problem of autonomous racing is to navigate through a race course as quickly as possible while not colliding with any obstacles. We approach the autonomous racing problem with the added constraint of not maintaining an updated obstacle map of the environment. Several current approaches to this problem use end-to-end learning systems where an agent replaces the entire navigation pipeline. This paper presents a hierarchical planning architecture that combines a high level planner and path following system with a reinforcement learning agent that learns that subsystem of obstacle avoidance. The novel "modification planner" uses the path follower to track the global plan and the deep reinforcement learning agent to modify the references generated by the path follower to avoid obstacles. Importantly, our architecture does not require an updated obstacle map and only 10 laser range finders to avoid obstacles. The modification planner is evaluated in the context of F1/10th autonomous racing and compared to a end-to-end learning baseline, the Follow the Gap Method and an optimisation based planner. The results show that the modification planner can achieve faster average times compared to the baseline end-to-end planner and a 94% success rate which is similar to the baseline. △ Less

Submitted 26 August, 2021; v1 submitted 22 February, 2021; originally announced February 2021.

Comments: 7 pages, 11 figures, This work has been submitted to the IEEE for possible publication

A maximum entropy model of bounded rational decision-making with prior beliefs and market feedback

Authors: Benjamin Patrick Evans, Mikhail Prokopenko

Abstract: Bounded rationality is an important consideration stemming from the fact that agents often have limits on their processing abilities, making the assumption of perfect rationality inapplicable to many real tasks. We propose an information-theoretic approach to the inference of agent decisions under Smithian competition. The model explicitly captures the boundedness of agents (limited in their infor… ▽ More Bounded rationality is an important consideration stemming from the fact that agents often have limits on their processing abilities, making the assumption of perfect rationality inapplicable to many real tasks. We propose an information-theoretic approach to the inference of agent decisions under Smithian competition. The model explicitly captures the boundedness of agents (limited in their information-processing capacity) as the cost of information acquisition for expanding their prior beliefs. The expansion is measured as the Kullblack-Leibler divergence between posterior decisions and prior beliefs. When information acquisition is free, the homo economicus agent is recovered, while in cases when information acquisition becomes costly, agents instead revert to their prior beliefs. The maximum entropy principle is used to infer least-biased decisions based upon the notion of Smithian competition formalised within the Quantal Response Statistical Equilibrium framework. The incorporation of prior beliefs into such a framework allowed us to systematically explore the effects of prior beliefs on decision-making in the presence of market feedback, as well as importantly adding a temporal interpretation to the framework. We verified the proposed model using Australian housing market data, showing how the incorporation of prior knowledge alters the resulting agent decisions. Specifically, it allowed for the separation of past beliefs and utility maximisation behaviour of the agent as well as the analysis into the evolution of agent beliefs. △ Less

Submitted 23 May, 2021; v1 submitted 18 February, 2021; originally announced February 2021.

Comments: 39 pages, 15 figures

It's the Journey Not the Destination: Building Genetic Algorithms Practitioners Can Trust

Authors: Jakub Vincalek, Sean Walton, Ben Evans

Abstract: Genetic algorithms have been developed for decades by researchers in academia and perform well in engineering applications, yet their uptake in industry remains limited. In order to understand why this is the case, the opinions of users of engineering design tools were gathered. The results from a survey showing the attitudes of engineers and students with design experience with respect to optimis… ▽ More Genetic algorithms have been developed for decades by researchers in academia and perform well in engineering applications, yet their uptake in industry remains limited. In order to understand why this is the case, the opinions of users of engineering design tools were gathered. The results from a survey showing the attitudes of engineers and students with design experience with respect to optimisation algorithms are presented. A survey was designed to answer two research questions: To what extent is there a pre-existing sentiment (negative or positive) among students, engineers, and managers towards genetic algorithm-based design? and What are the requirements of practitioners with regards to design optimisation and the design optimisation process? A total of 23 participants (N = 23) took part in the 3-part mixed methods survey. Thematic analysis was conducted on the open-ended questions. A common thread throughout participants responses is that there is a question of trust towards genetic algorithms within industry. Perhaps surprising is that the key to gaining this trust is not producing good results, but creating algorithms which explain the process they take in reaching a result. Participants have expressed a desire to continue to remain in the design loop. This is at odds with the motivation of a portion of the genetic algorithms community of removing humans from the loop. It is clear we need to take a different approach to increase industrial uptake. Based on this, the following recommendations have been made to increase their use in industry: an increase of transparency and explainability of genetic algorithms, an increased focus on user experience, better communication between developers and engineers, and visualising algorithm behaviour. △ Less

Submitted 13 October, 2020; originally announced October 2020.

Comments: 10 pages, 4 figures, submitted to IEEE Transactions on Evolutionary Computation

The impact of social influence in Australian real-estate: market forecasting with a spatial agent-based model

Authors: Benjamin Patrick Evans, Kirill Glavatskiy, Michael S. Harré, Mikhail Prokopenko

Abstract: Housing markets are inherently spatial, yet many existing models fail to capture this spatial dimension. Here we introduce a new graph-based approach for incorporating a spatial component in a large-scale urban housing agent-based model (ABM). The model explicitly captures several social and economic factors that influence the agents' decision-making behaviour (such as fear of missing out, their t… ▽ More Housing markets are inherently spatial, yet many existing models fail to capture this spatial dimension. Here we introduce a new graph-based approach for incorporating a spatial component in a large-scale urban housing agent-based model (ABM). The model explicitly captures several social and economic factors that influence the agents' decision-making behaviour (such as fear of missing out, their trend following aptitude, and the strength of their submarket outreach), and interprets these factors in spatial terms. The proposed model is calibrated and validated with the housing market data for the Greater Sydney region. The ABM simulation results not only include predictions for the overall market, but also produce area-specific forecasting at the level of local government areas within Sydney as arising from individual buy and sell decisions. In addition, the simulation results elucidate agent preferences in submarkets, highlighting differences in agent behaviour, for example, between first-time home buyers and investors, and between both local and overseas investors. △ Less

Submitted 9 February, 2021; v1 submitted 15 September, 2020; originally announced September 2020.

Comments: 25 pages + 31 page appendix

Coordinated Multicell Beamforming and Power Allocation for Massive MIMO with Low-Resolution ADC/DAC

Authors: Yunseong Cho, Jinseok Choi, Brian L. Evans

Abstract: In this work, we present a solution for coordinated beamforming and power allocation when base stations employ a massive number of antennas equipped with low-resolution analog-to-digital and digital-to-analog converters. We address total power minimization problems of the coarsely quantized uplink (UL) and downlink (DL) communication systems with target signal-to-interference-plus-noise ratio (SIN… ▽ More In this work, we present a solution for coordinated beamforming and power allocation when base stations employ a massive number of antennas equipped with low-resolution analog-to-digital and digital-to-analog converters. We address total power minimization problems of the coarsely quantized uplink (UL) and downlink (DL) communication systems with target signal-to-interference-plus-noise ratio (SINR) constraints. By combining the UL problem with minimum mean square error combiners and deriving the Lagrangian dual of the DL problem, we prove UL-DL duality and show there is no duality gap even with coarse data converters. Inspired by strong duality, we devise an iterative algorithm to determine the optimal UL transmit powers, and then linearly amplify the UL combiners with proper weights to acquire the optimal DL precoder. Simulation results validate strong duality and evaluate the proposed method in terms of total power consumption and achieved SINR. △ Less

Submitted 7 March, 2021; v1 submitted 25 May, 2020; originally announced May 2020.

Comments: 6 pages, 4 figures, IEEE ICC 2021

Quantized Massive MIMO Systems with Multicell Coordinated Beamforming and Power Control

Authors: Jinseok Choi, Yunseong Cho, Brian L. Evans

Abstract: In this paper, we investigate a coordinated multipoint (CoMP) beamforming and power control problem for base stations (BSs) with a massive number of antenna arrays under coarse quantization at low-resolution analog-to-digital converters (ADCs) and digital-to-analog converter (DACs). Unlike high-resolution ADC and DAC systems, non-negligible quantization noise that needs to be considered in CoMP de… ▽ More In this paper, we investigate a coordinated multipoint (CoMP) beamforming and power control problem for base stations (BSs) with a massive number of antenna arrays under coarse quantization at low-resolution analog-to-digital converters (ADCs) and digital-to-analog converter (DACs). Unlike high-resolution ADC and DAC systems, non-negligible quantization noise that needs to be considered in CoMP design makes the problem more challenging. We first formulate total power minimization problems of both uplink (UL) and downlink (DL) systems subject to signal-to-interference-and-noise ratio (SINR) constraints. We then derive strong duality for the UL and DL problems under coarse quantization systems. Leveraging the duality, we propose a framework that is directed toward a twofold aim: to discover the optimal transmit powers in UL by developing iterative algorithm in a distributed manner and to obtain the optimal precoder in DL as a scaled instance of UL combiner. Under homogeneous transmit power and SINR constraints per cell, we further derive a deterministic solution for the UL CoMP problem by analyzing the lower bound of the SINR. Lastly, we extend the derived result to wideband orthogonal frequency-division multiplexing systems to optimize transmit power and beamformer for all subcarriers. Simulation results validate the theoretical results and proposed algorithms. △ Less

Submitted 15 May, 2020; originally announced May 2020.

Comments: 30 pages, 4 figures, submitted to IEEE Transactions on Communications

Compressed-Sensing based Beam Detection in 5G NR Initial Access

Authors: Junmo Sung, Brian L. Evans

Abstract: To support millimeter wave (mmWave) frequency bands in cellular communications, both the base station and the mobile platform utilize large antenna arrays to steer narrow beams towards each other to compensate the path loss and improve communication performance. The time-frequency resource allocated for initial access, however, is limited, which gives rise to need for efficient approaches for beam… ▽ More To support millimeter wave (mmWave) frequency bands in cellular communications, both the base station and the mobile platform utilize large antenna arrays to steer narrow beams towards each other to compensate the path loss and improve communication performance. The time-frequency resource allocated for initial access, however, is limited, which gives rise to need for efficient approaches for beam detection. For hybrid analog-digital beamforming (HB) architectures, which are used to reduce power consumption, we propose a compressed sensing (CS) based approach for 5G initial access beam detection that is for a HB architecture and that is compliant with the 3GPP standard. The CS-based approach is compared with the exhaustive search in terms of beam detection accuracy and by simulation is shown to outperform. Up to 256 antennas are considered, and the importance of a careful codebook design is reaffirmed. △ Less

Submitted 2 May, 2020; originally announced May 2020.

Comments: 5 pages, 6 figures, SPAWC2020

Advanced Receiver Architectures for Millimeter Wave Communications with Low-Resolution ADCs

Authors: Jinseok Choi, Gilwon Lee, Ahmed Alkhateeb, Alan Gatherer, Naofal Al-Dhahir, Brian L. Evans

Abstract: Employing low-resolution analog-to-digital converters (ADCs) for millimeter wave receivers with large antenna arrays provides opportunity to efficiently reduce power consumption of the receiver. Reducing ADC resolution, however, results in performance degradation due to non-negligible quantization error. In addition, the large number of radio frequency (RF) chains is still not desirable. According… ▽ More Employing low-resolution analog-to-digital converters (ADCs) for millimeter wave receivers with large antenna arrays provides opportunity to efficiently reduce power consumption of the receiver. Reducing ADC resolution, however, results in performance degradation due to non-negligible quantization error. In addition, the large number of radio frequency (RF) chains is still not desirable. Accordingly, conventional low-resolution ADC systems require more efficient designs to minimize the cost and complexity while maximizing performance. In this article, we discuss advanced low-resolution ADC receiver architectures that further improve the spectral and energy efficiency tradeoff. To reduce both the numbers of RF chains and ADC bits, hybrid analog-and-digital beamforming is jointly considered with low-resolution ADCs. We explore the challenges in designing such receivers and present key insights on how the advanced architectures overcome such challenges. As an alternative low-resolution ADC receiver, we also introduce receivers with learning-based detection. The receiver does not require explicit channel estimation, thereby is suitable for one-bit ADC systems. Finally, future challenges and research issues are discussed. △ Less

Submitted 10 May, 2020; v1 submitted 7 March, 2020; originally announced March 2020.

Comments: Accepted to IEEE Communications Magazine (7 pages, 5 figures)

An Adaptive and Near Parameter-free Evolutionary Computation Approach Towards True Automation in AutoML

Authors: Benjamin Patrick Evans, Bing Xue, Mengjie Zhang

Abstract: A common claim of evolutionary computation methods is that they can achieve good results without the need for human intervention. However, one criticism of this is that there are still hyperparameters which must be tuned in order to achieve good performance. In this work, we propose a near "parameter-free" genetic programming approach, which adapts the hyperparameter values throughout evolution wi… ▽ More A common claim of evolutionary computation methods is that they can achieve good results without the need for human intervention. However, one criticism of this is that there are still hyperparameters which must be tuned in order to achieve good performance. In this work, we propose a near "parameter-free" genetic programming approach, which adapts the hyperparameter values throughout evolution without ever needing to be specified manually. We apply this to the area of automated machine learning (by extending TPOT), to produce pipelines which can effectively be claimed to be free from human input, and show that the results are competitive with existing state-of-the-art which use hand-selected hyperparameter values. Pipelines begin with a randomly chosen estimator and evolve to competitive pipelines automatically. This work moves towards a truly automatic approach to AutoML. △ Less

Submitted 28 January, 2020; originally announced January 2020.

Comments: 18 pages (single column), 2 figures

Improving generalisation of AutoML systems with dynamic fitness evaluations

Authors: Benjamin Patrick Evans, Bing Xue, Mengjie Zhang

Abstract: A common problem machine learning developers are faced with is overfitting, that is, fitting a pipeline too closely to the training data that the performance degrades for unseen data. Automated machine learning aims to free (or at least ease) the developer from the burden of pipeline creation, but this overfitting problem can persist. In fact, this can become more of a problem as we look to iterat… ▽ More A common problem machine learning developers are faced with is overfitting, that is, fitting a pipeline too closely to the training data that the performance degrades for unseen data. Automated machine learning aims to free (or at least ease) the developer from the burden of pipeline creation, but this overfitting problem can persist. In fact, this can become more of a problem as we look to iteratively optimise the performance of an internal cross-validation (most often \textit{k}-fold). While this internal cross-validation hopes to reduce this overfitting, we show we can still risk overfitting to the particular folds used. In this work, we aim to remedy this problem by introducing dynamic fitness evaluations which approximate repeated \textit{k}-fold cross-validation, at little extra cost over single \textit{k}-fold, and far lower cost than typical repeated \textit{k}-fold. The results show that when time equated, the proposed fitness function results in significant improvement over the current state-of-the-art baseline method which uses an internal single \textit{k}-fold. Furthermore, the proposed extension is very simple to implement on top of existing evolutionary computation methods, and can provide essentially a free boost in generalisation/testing performance. △ Less

Submitted 23 January, 2020; originally announced January 2020.

Comments: 19 pages, 4 figures

Deep Learning Predictive Band Switching in Wireless Networks

Authors: Faris B. Mismar, Ahmad AlAmmouri, Ahmed Alkhateeb, Jeffrey G. Andrews, Brian L. Evans

Abstract: In cellular systems, the user equipment (UE) can request a change in the frequency band when its rate drops below a threshold on the current band. The UE is then instructed by the base station (BS) to measure the quality of candidate bands, which requires a measurement gap in the data transmission, thus lowering the data rate. We propose an online-learning based band switching approach that does n… ▽ More In cellular systems, the user equipment (UE) can request a change in the frequency band when its rate drops below a threshold on the current band. The UE is then instructed by the base station (BS) to measure the quality of candidate bands, which requires a measurement gap in the data transmission, thus lowering the data rate. We propose an online-learning based band switching approach that does not require any measurement gap. Our proposed classifier-based band switching policy instead exploits spatial and spectral correlation between radio frequency signals in different bands based on knowledge of the UE location. We focus on switching between a lower (e.g., 3.5 GHz) band and a millimeter wave band (e.g., 28 GHz), and design and evaluate two classification models that are trained on a ray-tracing dataset. A key insight is that measurement gaps are overkill, in that only the relative order of the bands is necessary for band selection, rather than a full channel estimate. Our proposed machine learning based policies achieve roughly 30% improvement in mean effective rates over those of the industry standard policy, while achieving misclassification errors well below 0.5% and maintaining resilience against blockage uncertainty. △ Less

Submitted 1 September, 2020; v1 submitted 2 October, 2019; originally announced October 2019.

Comments: 31 pages, 15 figures, revised and resubmitted to IEEE Transactions on Wireless Communications on October 2, 2019, March 9, 2020, July 2, 2020, and September 1, 2020

Journal ref: 2020 IEEE Transactions on Wireless Communications

Genetic Programming and Gradient Descent: A Memetic Approach to Binary Image Classification

Authors: Benjamin Patrick Evans, Harith Al-Sahaf, Bing Xue, Mengjie Zhang

Abstract: Image classification is an essential task in computer vision, which aims to categorise a set of images into different groups based on some visual criteria. Existing methods, such as convolutional neural networks, have been successfully utilised to perform image classification. However, such methods often require human intervention to design a model. Furthermore, such models are difficult to interp… ▽ More Image classification is an essential task in computer vision, which aims to categorise a set of images into different groups based on some visual criteria. Existing methods, such as convolutional neural networks, have been successfully utilised to perform image classification. However, such methods often require human intervention to design a model. Furthermore, such models are difficult to interpret and it is challenging to analyse the patterns of different classes. This paper presents a hybrid (memetic) approach combining genetic programming (GP) and Gradient-based optimisation for image classification to overcome the limitations mentioned. The performance of the proposed method is compared to a baseline version (without local search) on four binary classification image datasets to provide an insight into the usefulness of local search mechanisms for enhancing the performance of GP. △ Less

Submitted 28 September, 2019; originally announced September 2019.

Hybrid Beamformer Codebook Design and Ordering for Compressive mmWave Channel Estimation

Authors: Junmo Sung, Brian L. Evans

Abstract: In millimeter wave (mmWave) communication systems, beamforming with large antenna arrays is critical to overcome high path losses. Separating all-digital beamforming into analog and digital stages can provide the large reduction in power consumption and small loss in spectral efficiency needed for practical implementations. Developing algorithms with this favorable tradeoff is challenging due to t… ▽ More In millimeter wave (mmWave) communication systems, beamforming with large antenna arrays is critical to overcome high path losses. Separating all-digital beamforming into analog and digital stages can provide the large reduction in power consumption and small loss in spectral efficiency needed for practical implementations. Developing algorithms with this favorable tradeoff is challenging due to the additional degrees of freedom in the analog stage and its accompanying hardware constraints. In hybrid beamforming systems, for example, channel estimation algorithms do not directly observe the channels, face a high channel count, and operate at low SNR before transmit-receive beam alignment. Since mmWave channels are sparse in time and beam domains, many compressed sensing (CS) channel estimation algorithms have been developed that randomly configure the analog beamformers, digital beamformers, and/or pilot symbols. In this paper, we propose to design deterministic beamformers and pilot symbols for open-loop channel estimation. We use CS approaches that rely on low coherence for their recovery guarantees, and hence seek to minimize the mutual coherence of the compressed sensing matrix. We also propose a precoder column ordering to design the pilot symbols. Simulation results show that our beamformer designs reduce channel estimation error over competing methods. △ Less

Submitted 21 September, 2019; originally announced September 2019.