Showing 1–44 of 44 results for author: Moran, B

Splatting Physical Scenes: End-to-End Real-to-Sim from Imperfect Robot Data

Authors: Ben Moran, Mauro Comi, Arunkumar Byravan, Steven Bohez, Tom Erez, Zhibin Li, Leonard Hasenclever

Abstract: Creating accurate, physical simulations directly from real-world robot motion holds great value for safe, scalable, and affordable robot learning, yet remains exceptionally challenging. Real robot data suffers from occlusions, noisy camera poses, dynamic scene elements, which hinder the creation of geometrically accurate and photorealistic digital twins of unseen objects. We introduce a novel real… ▽ More Creating accurate, physical simulations directly from real-world robot motion holds great value for safe, scalable, and affordable robot learning, yet remains exceptionally challenging. Real robot data suffers from occlusions, noisy camera poses, dynamic scene elements, which hinder the creation of geometrically accurate and photorealistic digital twins of unseen objects. We introduce a novel real-to-sim framework tackling all these challenges at once. Our key insight is a hybrid scene representation merging the photorealistic rendering of 3D Gaussian Splatting with explicit object meshes suitable for physics simulation within a single representation. We propose an end-to-end optimization pipeline that leverages differentiable rendering and differentiable physics within MuJoCo to jointly refine all scene components - from object geometry and appearance to robot poses and physical parameters - directly from raw and imprecise robot trajectories. This unified optimization allows us to simultaneously achieve high-fidelity object mesh reconstruction, generate photorealistic novel views, and perform annotation-free robot pose calibration. We demonstrate the effectiveness of our approach both in simulation and on challenging real-world sequences using an ALOHA 2 bi-manual manipulator, enabling more practical and robust real-to-simulation pipelines. △ Less

Submitted 9 June, 2025; v1 submitted 4 June, 2025; originally announced June 2025.

Comments: Updated version correcting inadvertent omission in author list

Proc4Gem: Foundation models for physical agency through procedural generation

Authors: Yixin Lin, Jan Humplik, Sandy H. Huang, Leonard Hasenclever, Francesco Romano, Stefano Saliceti, Daniel Zheng, Jose Enrique Chen, Catarina Barros, Adrian Collister, Matt Young, Adil Dostmohamed, Ben Moran, Ken Caluwaerts, Marissa Giustina, Joss Moore, Kieran Connell, Francesco Nori, Nicolas Heess, Steven Bohez, Arunkumar Byravan

Abstract: In robot learning, it is common to either ignore the environment semantics, focusing on tasks like whole-body control which only require reasoning about robot-environment contacts, or conversely to ignore contact dynamics, focusing on grounding high-level movement in vision and language. In this work, we show that advances in generative modeling, photorealistic rendering, and procedural generation… ▽ More In robot learning, it is common to either ignore the environment semantics, focusing on tasks like whole-body control which only require reasoning about robot-environment contacts, or conversely to ignore contact dynamics, focusing on grounding high-level movement in vision and language. In this work, we show that advances in generative modeling, photorealistic rendering, and procedural generation allow us to tackle tasks requiring both. By generating contact-rich trajectories with accurate physics in semantically-diverse simulations, we can distill behaviors into large multimodal models that directly transfer to the real world: a system we call Proc4Gem. Specifically, we show that a foundation model, Gemini, fine-tuned on only simulation data, can be instructed in language to control a quadruped robot to push an object with its body to unseen targets in unseen real-world environments. Our real-world results demonstrate the promise of using simulation to imbue foundation models with physical agency. Videos can be found at our website: https://sites.google.com/view/proc4gem △ Less

Submitted 11 March, 2025; originally announced March 2025.

Multi-Action Restless Bandits with Weakly Coupled Constraints: Simultaneous Learning and Control

Authors: Jing Fu, Bill Moran, José Niño-Mora

Abstract: We study a system with finitely many groups of multi-action bandit processes, each of which is a Markov decision process (MDP) with finite state and action spaces and potentially different transition matrices when taking different actions. The bandit processes of the same group share the same state and action spaces and, given the same action that is taken, the same transition matrix. All the band… ▽ More We study a system with finitely many groups of multi-action bandit processes, each of which is a Markov decision process (MDP) with finite state and action spaces and potentially different transition matrices when taking different actions. The bandit processes of the same group share the same state and action spaces and, given the same action that is taken, the same transition matrix. All the bandit processes across various groups are subject to multiple weakly coupled constraints over their state and action variables. Unlike the past studies that focused on the offline case, we consider the online case without assuming full knowledge of transition matrices and reward functions a priori and propose an effective scheme that enables simultaneous learning and control. We prove the convergence of the relevant processes in both the timeline and the number of the bandit processes, referred to as the convergence in the time and the magnitude dimensions. Moreover, we prove that the relevant processes converge exponentially fast in the magnitude dimension, leading to exponentially diminishing performance deviation between the proposed online algorithms and offline optimality. △ Less

Submitted 4 December, 2024; originally announced December 2024.

Comments: 70 pages,0 figure

MSC Class: 90B36 (Primary) 90B15; 90B22 (Secondary)

Channel nonlocality under decoherence

Authors: Albert Rico, Moisés Bermejo Morán, Fereshte Shahbeigi, Karol Życzkowski

Abstract: The implementation of realistic quantum devices requires a solid understanding of the nonlocal resources present in quantum channels, and the effects of decoherence on them. Here we quantify nonlocality of bipartite quantum channels and identify its component resisting the effects of dephasing noise. Despite its classical nature, we demonstrate that the latter plays a relevant role in performing q… ▽ More The implementation of realistic quantum devices requires a solid understanding of the nonlocal resources present in quantum channels, and the effects of decoherence on them. Here we quantify nonlocality of bipartite quantum channels and identify its component resisting the effects of dephasing noise. Despite its classical nature, we demonstrate that the latter plays a relevant role in performing quantum protocols, such as state transformations and quantum coding for noisy communication. In the converse direction, we show that simulating certain stochastic processes with quantum channels undergoing decoherence has a communication advantage with respect to their classical simulation. △ Less

Submitted 19 August, 2024; originally announced August 2024.

CRAG -- Comprehensive RAG Benchmark

Authors: Xiao Yang, Kai Sun, Hao Xin, Yushi Sun, Nikita Bhalla, Xiangsen Chen, Sajal Choudhary, Rongze Daniel Gui, Ziran Will Jiang, Ziyu Jiang, Lingkun Kong, Brian Moran, Jiaqi Wang, Yifan Ethan Xu, An Yan, Chenyu Yang, Eting Yuan, Hanwen Zha, Nan Tang, Lei Chen, Nicolas Scheffer, Yue Liu, Nirav Shah, Rakesh Wanga, Anuj Kumar , et al. (2 additional authors not shown)

Abstract: Retrieval-Augmented Generation (RAG) has recently emerged as a promising solution to alleviate Large Language Model (LLM)'s deficiency in lack of knowledge. Existing RAG datasets, however, do not adequately represent the diverse and dynamic nature of real-world Question Answering (QA) tasks. To bridge this gap, we introduce the Comprehensive RAG Benchmark (CRAG), a factual question answering bench… ▽ More Retrieval-Augmented Generation (RAG) has recently emerged as a promising solution to alleviate Large Language Model (LLM)'s deficiency in lack of knowledge. Existing RAG datasets, however, do not adequately represent the diverse and dynamic nature of real-world Question Answering (QA) tasks. To bridge this gap, we introduce the Comprehensive RAG Benchmark (CRAG), a factual question answering benchmark of 4,409 question-answer pairs and mock APIs to simulate web and Knowledge Graph (KG) search. CRAG is designed to encapsulate a diverse array of questions across five domains and eight question categories, reflecting varied entity popularity from popular to long-tail, and temporal dynamisms ranging from years to seconds. Our evaluation of this benchmark highlights the gap to fully trustworthy QA. Whereas most advanced LLMs achieve <=34% accuracy on CRAG, adding RAG in a straightforward manner improves the accuracy only to 44%. State-of-the-art industry RAG solutions only answer 63% of questions without any hallucination. CRAG also reveals much lower accuracy in answering questions regarding facts with higher dynamism, lower popularity, or higher complexity, suggesting future research directions. The CRAG benchmark laid the groundwork for a KDD Cup 2024 challenge and attracted thousands of participants and submissions. We commit to maintaining CRAG to serve research communities in advancing RAG solutions and general QA solutions. CRAG is available at https://github.com/facebookresearch/CRAG/. △ Less

Submitted 1 November, 2024; v1 submitted 7 June, 2024; originally announced June 2024.

Comments: NeurIPS 2024 Datasets and Benchmarks Track

Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learning

Authors: Dhruva Tirumala, Markus Wulfmeier, Ben Moran, Sandy Huang, Jan Humplik, Guy Lever, Tuomas Haarnoja, Leonard Hasenclever, Arunkumar Byravan, Nathan Batchelor, Neil Sreendra, Kushal Patel, Marlon Gwira, Francesco Nori, Martin Riedmiller, Nicolas Heess

Abstract: We apply multi-agent deep reinforcement learning (RL) to train end-to-end robot soccer policies with fully onboard computation and sensing via egocentric RGB vision. This setting reflects many challenges of real-world robotics, including active perception, agile full-body control, and long-horizon planning in a dynamic, partially-observable, multi-agent domain. We rely on large-scale, simulation-b… ▽ More We apply multi-agent deep reinforcement learning (RL) to train end-to-end robot soccer policies with fully onboard computation and sensing via egocentric RGB vision. This setting reflects many challenges of real-world robotics, including active perception, agile full-body control, and long-horizon planning in a dynamic, partially-observable, multi-agent domain. We rely on large-scale, simulation-based data generation to obtain complex behaviors from egocentric vision which can be successfully transferred to physical robots using low-cost sensors. To achieve adequate visual realism, our simulation combines rigid-body physics with learned, realistic rendering via multiple Neural Radiance Fields (NeRFs). We combine teacher-based multi-agent RL and cross-experiment data reuse to enable the discovery of sophisticated soccer strategies. We analyze active-perception behaviors including object tracking and ball seeking that emerge when simply optimizing perception-agnostic soccer play. The agents display equivalent levels of performance and agility as policies with access to privileged, ground-truth state. To our knowledge, this paper constitutes a first demonstration of end-to-end training for multi-agent robot soccer, mapping raw pixel observations to joint-level actions, that can be deployed in the real world. Videos of the game-play and analyses can be seen on our website https://sites.google.com/view/vision-soccer . △ Less

Submitted 3 May, 2024; originally announced May 2024.

Replay across Experiments: A Natural Extension of Off-Policy RL

Authors: Dhruva Tirumala, Thomas Lampe, Jose Enrique Chen, Tuomas Haarnoja, Sandy Huang, Guy Lever, Ben Moran, Tim Hertweck, Leonard Hasenclever, Martin Riedmiller, Nicolas Heess, Markus Wulfmeier

Abstract: Replaying data is a principal mechanism underlying the stability and data efficiency of off-policy reinforcement learning (RL). We present an effective yet simple framework to extend the use of replays across multiple experiments, minimally adapting the RL workflow for sizeable improvements in controller performance and research iteration times. At its core, Replay Across Experiments (RaE) involve… ▽ More Replaying data is a principal mechanism underlying the stability and data efficiency of off-policy reinforcement learning (RL). We present an effective yet simple framework to extend the use of replays across multiple experiments, minimally adapting the RL workflow for sizeable improvements in controller performance and research iteration times. At its core, Replay Across Experiments (RaE) involves reusing experience from previous experiments to improve exploration and bootstrap learning while reducing required changes to a minimum in comparison to prior work. We empirically show benefits across a number of RL algorithms and challenging control domains spanning both locomotion and manipulation, including hard exploration tasks from egocentric vision. Through comprehensive ablations, we demonstrate robustness to the quality and amount of data available and various hyperparameter choices. Finally, we discuss how our approach can be applied more broadly across research life cycles and can increase resilience by reloading data across random seeds or hyperparameter variations. △ Less

Submitted 28 November, 2023; v1 submitted 27 November, 2023; originally announced November 2023.

Score-based generative models learn manifold-like structures with constrained mixing

Authors: Li Kevin Wenliang, Ben Moran

Abstract: How do score-based generative models (SBMs) learn the data distribution supported on a low-dimensional manifold? We investigate the score model of a trained SBM through its linear approximations and subspaces spanned by local feature vectors. During diffusion as the noise decreases, the local dimensionality increases and becomes more varied between different sample sequences. Importantly, we find… ▽ More How do score-based generative models (SBMs) learn the data distribution supported on a low-dimensional manifold? We investigate the score model of a trained SBM through its linear approximations and subspaces spanned by local feature vectors. During diffusion as the noise decreases, the local dimensionality increases and becomes more varied between different sample sequences. Importantly, we find that the learned vector field mixes samples by a non-conservative field within the manifold, although it denoises with normal projections as if there is an energy function in off-manifold directions. At each noise level, the subspace spanned by the local features overlap with an effective density function. These observations suggest that SBMs can flexibly mix samples with the learned score field while carefully maintaining a manifold-like structure of the data distribution. △ Less

Submitted 16 November, 2023; originally announced November 2023.

Comments: NeurIPS 2022 Workshop on Score-Based Methods

Localising two sub-diffraction emitters in 3D using quantum correlation microscopy

Authors: Shuo Li, Wenchao Li, Qiang Sun, Bill Moran, Timothy C. Brown, Brant C. Gibson, Andrew D. Greentree

Abstract: The localisation of fluorophores is an important aspect of the determination of the biological function of cellular systems. Quantum correlation microscopy (QCM) is a promising technique for providing diffraction unlimited emitter localisation that can be used with either confocal or widefield modalities. However, so far, QCM has not been applied to three dimensional localisation problems. Here we… ▽ More The localisation of fluorophores is an important aspect of the determination of the biological function of cellular systems. Quantum correlation microscopy (QCM) is a promising technique for providing diffraction unlimited emitter localisation that can be used with either confocal or widefield modalities. However, so far, QCM has not been applied to three dimensional localisation problems. Here we show that quantum correlation microscopy provides diffraction-unlimited three-dimensional localisation for two emitters within a single diffraction-limited spot. By introducing a two-stage maximum likelihood estimator, our modelling shows that localisation precision scales as $1/\sqrt{t}$ where $t$ is the total detection time. Diffraction unlimited localisation is achieved using both intensity and photon correlation from Hanbury Brown and Twiss measurements at as few as four measurement locations. △ Less

Submitted 4 October, 2023; originally announced October 2023.

Uncertainty relations from state polynomial optimization

Authors: Moisés Bermejo Morán, Felix Huber

Abstract: Uncertainty relations are a fundamental feature of quantum mechanics. How can these relations be found systematically? Here we develop a semidefinite programming hierarchy for additive uncertainty relations in the variances of non-commuting observables. Our hierarchy is built on the state polynomial optimization framework, also known as scalar extension. The hierarchy is complete, in the sense tha… ▽ More Uncertainty relations are a fundamental feature of quantum mechanics. How can these relations be found systematically? Here we develop a semidefinite programming hierarchy for additive uncertainty relations in the variances of non-commuting observables. Our hierarchy is built on the state polynomial optimization framework, also known as scalar extension. The hierarchy is complete, in the sense that it converges to tight uncertainty relations. We improve upon upper bounds for all 1292 additive uncertainty relations on up to nine operators for which a tight bound is not known. The bounds are dimension-free and depend entirely on the algebraic relations among the operators. The techniques apply to a range of scenarios, including Pauli, Heisenberg-Weyl, and fermionic operators, and generalize to higher order moments and multiplicative uncertainty relations. △ Less

Submitted 12 November, 2024; v1 submitted 1 October, 2023; originally announced October 2023.

Comments: 9 pages, 4 tables. Reference to Eq 23 added. Final version

Journal ref: Physical Review Letters 132, 200202 (2024)

Estimation of the number of single-photon emitters for multiple fluorophores with the same spectral signature

Authors: Wenchao Li, Shuo Li, Timothy C. Brown, Qiang Sun, Xuezhi Wang, Vladislav V. Yakovlev, Allison Kealy, Bill Moran, Andrew D. Greentree

Abstract: Fluorescence microscopy is of vital importance for understanding biological function. However most fluorescence experiments are only qualitative inasmuch as the absolute number of fluorescent particles can often not be determined. Additionally, conventional approaches to measuring fluorescence intensity cannot distinguish between two or more fluorophores that are excited and emit in the same spect… ▽ More Fluorescence microscopy is of vital importance for understanding biological function. However most fluorescence experiments are only qualitative inasmuch as the absolute number of fluorescent particles can often not be determined. Additionally, conventional approaches to measuring fluorescence intensity cannot distinguish between two or more fluorophores that are excited and emit in the same spectral window, as only the total intensity in a spectral window can be obtained. Here we show that, by using photon number resolving experiments, we are able to determine the number of emitters and their probability of emission for a number of different species, all with the same measured spectral signature. We illustrate our ideas by showing the determination of the number of emitters per species and the probability of photon collection from that species, for one, two, and three otherwise unresolvable fluorophores. The convolution Binomial model is presented to model the counted photons emitted by multiple species. And then the Expectation-Maximization (EM) algorithm is used to match the measured photon counts to the expected convolution Binomial distribution function. In applying the EM algorithm, to leverage the problem of being trapped in a sub-optimal solution, the moment method is introduced in finding the initial guess of the EM algorithm. Additionally, the associated Cramér-Rao lower bound is derived and compared with the simulation results. △ Less

Submitted 12 February, 2024; v1 submitted 8 June, 2023; originally announced June 2023.

Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning

Authors: Tuomas Haarnoja, Ben Moran, Guy Lever, Sandy H. Huang, Dhruva Tirumala, Jan Humplik, Markus Wulfmeier, Saran Tunyasuvunakool, Noah Y. Siegel, Roland Hafner, Michael Bloesch, Kristian Hartikainen, Arunkumar Byravan, Leonard Hasenclever, Yuval Tassa, Fereshteh Sadeghi, Nathan Batchelor, Federico Casarini, Stefano Saliceti, Charles Game, Neil Sreendra, Kushal Patel, Marlon Gwira, Andrea Huber, Nicole Hurley , et al. (3 additional authors not shown)

Abstract: We investigate whether Deep Reinforcement Learning (Deep RL) is able to synthesize sophisticated and safe movement skills for a low-cost, miniature humanoid robot that can be composed into complex behavioral strategies in dynamic environments. We used Deep RL to train a humanoid robot with 20 actuated joints to play a simplified one-versus-one (1v1) soccer game. The resulting agent exhibits robust… ▽ More We investigate whether Deep Reinforcement Learning (Deep RL) is able to synthesize sophisticated and safe movement skills for a low-cost, miniature humanoid robot that can be composed into complex behavioral strategies in dynamic environments. We used Deep RL to train a humanoid robot with 20 actuated joints to play a simplified one-versus-one (1v1) soccer game. The resulting agent exhibits robust and dynamic movement skills such as rapid fall recovery, walking, turning, kicking and more; and it transitions between them in a smooth, stable, and efficient manner. The agent's locomotion and tactical behavior adapts to specific game contexts in a way that would be impractical to manually design. The agent also developed a basic strategic understanding of the game, and learned, for instance, to anticipate ball movements and to block opponent shots. Our agent was trained in simulation and transferred to real robots zero-shot. We found that a combination of sufficiently high-frequency control, targeted dynamics randomization, and perturbations during training in simulation enabled good-quality transfer. Although the robots are inherently fragile, basic regularization of the behavior during training led the robots to learn safe and effective movements while still performing in a dynamic and agile way -- well beyond what is intuitively expected from the robot. Indeed, in experiments, they walked 181% faster, turned 302% faster, took 63% less time to get up, and kicked a ball 34% faster than a scripted baseline, while efficiently combining the skills to achieve the longer term objectives. △ Less

Submitted 11 April, 2024; v1 submitted 26 April, 2023; originally announced April 2023.

Comments: Project website: https://sites.google.com/view/op3-soccer

Bell inequalities with overlapping measurements

Authors: Moisés Bermejo Morán, Alejandro Pozas-Kerstjens, Felix Huber

Abstract: Which nonlocal correlations can be obtained, when a party has access to more than one subsystem? While traditionally nonlocality deals with spacelike separated parties, this question becomes important with quantum technologies that connect devices by means of small shared systems. Here we study Bell inequalities where measurements of different parties can have overlap. This allows to accommodate p… ▽ More Which nonlocal correlations can be obtained, when a party has access to more than one subsystem? While traditionally nonlocality deals with spacelike separated parties, this question becomes important with quantum technologies that connect devices by means of small shared systems. Here we study Bell inequalities where measurements of different parties can have overlap. This allows to accommodate problems in quantum information such as the existence of quantum error correction codes in the framework of non-locality. The scenarios considered show an interesting behaviour with respect to Hilbert space dimension, overlap, and symmetry. △ Less

Submitted 31 August, 2023; v1 submitted 3 March, 2023; originally announced March 2023.

Comments: 9 pages, 3 figures. Accepted version

Journal ref: Phys. Rev. Lett. 131, 080201 (2023)

Quantum diamond magnetometry for navigation in GNSS denied environments

Authors: Xuezhi Wang, Wenchao Li, Bill Moran, Brant C. Gibson, Liam T. Hall, David Simpson, Allison N. Kealy, Andrew D. Greentree

Abstract: In this paper, a probabilistic method for map matching localisation based on magnetometery measurement and total magnetic intensity maps is described. We show that the method is able to effectively address the challenge issues associated with map matching using geophysical maps and provides a mechanism of handling map measurement ambiguity and a way of evaluating the underlying quality. Furthermor… ▽ More In this paper, a probabilistic method for map matching localisation based on magnetometery measurement and total magnetic intensity maps is described. We show that the method is able to effectively address the challenge issues associated with map matching using geophysical maps and provides a mechanism of handling map measurement ambiguity and a way of evaluating the underlying quality. Furthermore, the effectiveness of the magnetometery map matching localisation is demonstrated using the simulation of removing position drift of an inertial navigation system, that arises in INS over a long duration, by the magnetometery aiding in the absence GNSS positioning. Simulation results using online maps verified the robustness and effectiveness of the proposed algorithm, particularly, the aiding precision will be getting better if a high sensitivity magnetometer is used. △ Less

Submitted 13 February, 2023; originally announced February 2023.

Comments: 16 pages, submitted to the International Association of Geodesy Symposia for Regular article consideration

SkillS: Adaptive Skill Sequencing for Efficient Temporally-Extended Exploration

Authors: Giulia Vezzani, Dhruva Tirumala, Markus Wulfmeier, Dushyant Rao, Abbas Abdolmaleki, Ben Moran, Tuomas Haarnoja, Jan Humplik, Roland Hafner, Michael Neunert, Claudio Fantacci, Tim Hertweck, Thomas Lampe, Fereshteh Sadeghi, Nicolas Heess, Martin Riedmiller

Abstract: The ability to effectively reuse prior knowledge is a key requirement when building general and flexible Reinforcement Learning (RL) agents. Skill reuse is one of the most common approaches, but current methods have considerable limitations.For example, fine-tuning an existing policy frequently fails, as the policy can degrade rapidly early in training. In a similar vein, distillation of expert be… ▽ More The ability to effectively reuse prior knowledge is a key requirement when building general and flexible Reinforcement Learning (RL) agents. Skill reuse is one of the most common approaches, but current methods have considerable limitations.For example, fine-tuning an existing policy frequently fails, as the policy can degrade rapidly early in training. In a similar vein, distillation of expert behavior can lead to poor results when given sub-optimal experts. We compare several common approaches for skill transfer on multiple domains including changes in task and system dynamics. We identify how existing methods can fail and introduce an alternative approach to mitigate these problems. Our approach learns to sequence existing temporally-extended skills for exploration but learns the final policy directly from the raw experience. This conceptual split enables rapid adaptation and thus efficient data collection but without constraining the final solution.It significantly outperforms many classical methods across a suite of evaluation tasks and we use a broad set of ablations to highlight the importance of differentc omponents of our method. △ Less

Submitted 11 January, 2023; v1 submitted 24 November, 2022; originally announced November 2022.

Estimating and detecting random processes on the unit circle

Authors: Changrong Liu, S. Suvorova, R. J. Evans, B. Moran, A. Melatos

Abstract: The problem of detecting a sinusoidal signal with randomly varying frequency has a long history. It is one of the core problems in signal processing, arising in many applications including, for example, underwater acoustic frequency line tracking, demodulation of FM radio communications, laser phase drift in optical communications and, recently, continuous gravitational wave astronomy. In this pap… ▽ More The problem of detecting a sinusoidal signal with randomly varying frequency has a long history. It is one of the core problems in signal processing, arising in many applications including, for example, underwater acoustic frequency line tracking, demodulation of FM radio communications, laser phase drift in optical communications and, recently, continuous gravitational wave astronomy. In this paper we describe a Markov Chain Monte Carlo based procedure to compute a specific detection posterior density. We demonstrate via simulation that our approach results in an up to $25$ percent higher detection rate than Hidden Markov Model based solutions, which are generally considered to be the leading techniques for these problems. △ Less

Submitted 14 November, 2022; originally announced November 2022.

Comments: 6 pages, 4 figures, 12th IFAC Symposium on Nonlinear Control Systems

Bayesian Detection of a Sinusoidal Signal with Randomly Varying Frequency

Authors: Changrong Liu, S. Suvorova, R. J. Evans, B. Moran, A. Melatos

Abstract: The problem of detecting a sinusoidal signal with randomly varying frequency has a long history. It is one of the core problems in signal processing, arising in many applications including, for example, underwater acoustic frequency line tracking, demodulation of FM radio communications, laser phase drift in optical communications and, recently, continuous gravitational wave astronomy. In this pap… ▽ More The problem of detecting a sinusoidal signal with randomly varying frequency has a long history. It is one of the core problems in signal processing, arising in many applications including, for example, underwater acoustic frequency line tracking, demodulation of FM radio communications, laser phase drift in optical communications and, recently, continuous gravitational wave astronomy. In this paper we describe a Markov Chain Monte Carlo based procedure to compute a specific detection posterior density. We demonstrate via simulation that our approach results in an up to $25$ percent higher detection rate than Hidden Markov Model based solutions, which are generally considered to be the leading techniques for these problems. △ Less

Submitted 10 November, 2022; originally announced November 2022.

Comments: 14 pages, 10 figures

Journal ref: IEEE Open Journal of Signal Processing, vol. 3, pp. 246-260, 2022

NeRF2Real: Sim2real Transfer of Vision-guided Bipedal Motion Skills using Neural Radiance Fields

Authors: Arunkumar Byravan, Jan Humplik, Leonard Hasenclever, Arthur Brussee, Francesco Nori, Tuomas Haarnoja, Ben Moran, Steven Bohez, Fereshteh Sadeghi, Bojan Vujatovic, Nicolas Heess

Abstract: We present a system for applying sim2real approaches to "in the wild" scenes with realistic visuals, and to policies which rely on active perception using RGB cameras. Given a short video of a static scene collected using a generic phone, we learn the scene's contact geometry and a function for novel view synthesis using a Neural Radiance Field (NeRF). We augment the NeRF rendering of the static s… ▽ More We present a system for applying sim2real approaches to "in the wild" scenes with realistic visuals, and to policies which rely on active perception using RGB cameras. Given a short video of a static scene collected using a generic phone, we learn the scene's contact geometry and a function for novel view synthesis using a Neural Radiance Field (NeRF). We augment the NeRF rendering of the static scene by overlaying the rendering of other dynamic objects (e.g. the robot's own body, a ball). A simulation is then created using the rendering engine in a physics simulator which computes contact dynamics from the static scene geometry (estimated from the NeRF volume density) and the dynamic objects' geometry and physical properties (assumed known). We demonstrate that we can use this simulation to learn vision-based whole body navigation and ball pushing policies for a 20 degrees of freedom humanoid robot with an actuated head-mounted RGB camera, and we successfully transfer these policies to a real robot. Project video is available at https://sites.google.com/view/nerf2real/home △ Less

Submitted 10 October, 2022; originally announced October 2022.

Gravity aided navigation using Viterbi map matching algorithm

Authors: Wenchao Li, Christopher Gilliam, Xuezhi Wang, Allison Kealy, Andrew D. Greentree, Bill Moran

Abstract: In GNSS-denied environments, aiding a vehicle's inertial navigation system (INS) is crucial to reducing the accumulated navigation drift caused by sensor errors (e.g. bias and noise). One potential solution is to use measurements of gravity as an aiding source. The measurements are matched to a geo-referenced map of Earth's gravity in order to estimate the vehicle's position. In this paper, we pro… ▽ More In GNSS-denied environments, aiding a vehicle's inertial navigation system (INS) is crucial to reducing the accumulated navigation drift caused by sensor errors (e.g. bias and noise). One potential solution is to use measurements of gravity as an aiding source. The measurements are matched to a geo-referenced map of Earth's gravity in order to estimate the vehicle's position. In this paper, we propose a novel formulation of the map matching problem using a hidden Markov model (HMM). Specifically, we treat the spatial cells of the map as the hidden states of the HMM and present a Viterbi style algorithm to estimate the most likely sequence of states, i.e. most likely sequence of vehicle positions, that results in the sequence of observed gravity measurements. Using a realistic gravity map, we demonstrate the accuracy of our Viterbi map matching algorithm in a navigation scenario and illustrate its robustness compared to existing methods. △ Less

Submitted 22 April, 2022; originally announced April 2022.

The 2020 Census Disclosure Avoidance System TopDown Algorithm

Authors: John M. Abowd, Robert Ashmead, Ryan Cumings-Menon, Simson Garfinkel, Micah Heineck, Christine Heiss, Robert Johns, Daniel Kifer, Philip Leclerc, Ashwin Machanavajjhala, Brett Moran, William Sexton, Matthew Spence, Pavel Zhuravlev

Abstract: The Census TopDown Algorithm (TDA) is a disclosure avoidance system using differential privacy for privacy-loss accounting. The algorithm ingests the final, edited version of the 2020 Census data and the final tabulation geographic definitions. The algorithm then creates noisy versions of key queries on the data, referred to as measurements, using zero-Concentrated Differential Privacy. Another ke… ▽ More The Census TopDown Algorithm (TDA) is a disclosure avoidance system using differential privacy for privacy-loss accounting. The algorithm ingests the final, edited version of the 2020 Census data and the final tabulation geographic definitions. The algorithm then creates noisy versions of key queries on the data, referred to as measurements, using zero-Concentrated Differential Privacy. Another key aspect of the TDA are invariants, statistics that the Census Bureau has determined, as matter of policy, to exclude from the privacy-loss accounting. The TDA post-processes the measurements together with the invariants to produce a Microdata Detail File (MDF) that contains one record for each person and one record for each housing unit enumerated in the 2020 Census. The MDF is passed to the 2020 Census tabulation system to produce the 2020 Census Redistricting Data (P.L. 94-171) Summary File. This paper describes the mathematics and testing of the TDA for this purpose. △ Less

Submitted 19 April, 2022; originally announced April 2022.

Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviors

Authors: Steven Bohez, Saran Tunyasuvunakool, Philemon Brakel, Fereshteh Sadeghi, Leonard Hasenclever, Yuval Tassa, Emilio Parisotto, Jan Humplik, Tuomas Haarnoja, Roland Hafner, Markus Wulfmeier, Michael Neunert, Ben Moran, Noah Siegel, Andrea Huber, Francesco Romano, Nathan Batchelor, Federico Casarini, Josh Merel, Raia Hadsell, Nicolas Heess

Abstract: We investigate the use of prior knowledge of human and animal movement to learn reusable locomotion skills for real legged robots. Our approach builds upon previous work on imitating human or dog Motion Capture (MoCap) data to learn a movement skill module. Once learned, this skill module can be reused for complex downstream tasks. Importantly, due to the prior imposed by the MoCap data, our appro… ▽ More We investigate the use of prior knowledge of human and animal movement to learn reusable locomotion skills for real legged robots. Our approach builds upon previous work on imitating human or dog Motion Capture (MoCap) data to learn a movement skill module. Once learned, this skill module can be reused for complex downstream tasks. Importantly, due to the prior imposed by the MoCap data, our approach does not require extensive reward engineering to produce sensible and natural looking behavior at the time of reuse. This makes it easy to create well-regularized, task-oriented controllers that are suitable for deployment on real robots. We demonstrate how our skill module can be used for imitation, and train controllable walking and ball dribbling policies for both the ANYmal quadruped and OP3 humanoid. These policies are then deployed on hardware via zero-shot simulation-to-reality transfer. Accompanying videos are available at https://bit.ly/robot-npmp. △ Less

Submitted 31 March, 2022; originally announced March 2022.

Comments: 30 pages, 9 figures, 8 tables, 14 videos at https://bit.ly/robot-npmp , submitted to Science Robotics

Probabilistic Map Matching for Robust Inertial Navigation Aiding

Authors: Xuezhi Wang, Christopher Gilliam, Allison Kealy, John Close, Bill Moran

Abstract: Robust aiding of inertial navigation systems in GNSS-denied environments is critical for the removal of accumulated navigation error caused by the drift and bias inherent in inertial sensors. One way to perform such an aiding uses matching of geophysical measurements, such as gravimetry, gravity gradiometry or magnetometry, with a known geo-referenced map. Although simple in concept, this map matc… ▽ More Robust aiding of inertial navigation systems in GNSS-denied environments is critical for the removal of accumulated navigation error caused by the drift and bias inherent in inertial sensors. One way to perform such an aiding uses matching of geophysical measurements, such as gravimetry, gravity gradiometry or magnetometry, with a known geo-referenced map. Although simple in concept, this map matching procedure is challenging: the measurements themselves are noisy; their associated spatial location is uncertain; and the measurements may match multiple points within the map (i.e. non-unique solution). In this paper, we propose a probabilistic multiple hypotheses tracker to solve the map matching problem and allow robust inertial navigation aiding. Our approach addresses the problem both locally, via probabilistic data association, and temporally by incorporating the underlying platform kinematic constraints into the tracker. The map matching output is then integrated into the navigation system using an unscented Kalman filter. Additionally, we present a statistical measure of local map information density -- the map feature variability -- and use it to weight the output covariance of the proposed algorithm. The effectiveness and robustness of the proposed algorithm are demonstrated using a navigation scenario involving gravitational map matching. △ Less

Submitted 31 March, 2022; originally announced March 2022.

Comments: 12 pages. 13 figures

Submarine Cable Network Design for Regional Connectivity

Authors: Tianjiao Wang, Zengfu Wang, Bill Moran, Moshe Zukerman

Abstract: This paper optimizes path planning for a trunkand-branch topology network in an irregular 2-dimensional manifold embedded in 3-dimensional Euclidean space with application to submarine cable network planning. We go beyond our earlier focus on the costs of cable construction (including labor, equipment and materials) together with additional cost to enhance cable resilience, to incorporate the over… ▽ More This paper optimizes path planning for a trunkand-branch topology network in an irregular 2-dimensional manifold embedded in 3-dimensional Euclidean space with application to submarine cable network planning. We go beyond our earlier focus on the costs of cable construction (including labor, equipment and materials) together with additional cost to enhance cable resilience, to incorporate the overall cost of branching units (again including material, construction and laying) and the choice of submarine cable landing stations, where such a station can be anywhere on the coast in a connected region. These are important issues for the economics of cable laying and significantly change the model and the optimization process. We pose the problem as a variant of the Steiner tree problem, but one in which the Steiner nodes can vary in number, while incurring a penalty. We refer to it as the weighted Steiner node problem. It differs from the Euclidean Steiner tree problem, where Steiner points are forced to have degree three; this is no longer the case, in general, when nodes incur a cost. We are able to prove that our algorithm is applicable to Steiner nodes with degree greater than three, enabling optimization of network costs in this context. The optimal solution is achieved in polynomialtime using dynamic programming. △ Less

Submitted 15 January, 2022; originally announced January 2022.

Assessing Data Efficiency in Task-Oriented Semantic Parsing

Authors: Shrey Desai, Akshat Shrivastava, Justin Rill, Brian Moran, Safiyyah Saleem, Alexander Zotov, Ahmed Aly

Abstract: Data efficiency, despite being an attractive characteristic, is often challenging to measure and optimize for in task-oriented semantic parsing; unlike exact match, it can require both model- and domain-specific setups, which have, historically, varied widely across experiments. In our work, as a step towards providing a unified solution to data-efficiency-related questions, we introduce a four-st… ▽ More Data efficiency, despite being an attractive characteristic, is often challenging to measure and optimize for in task-oriented semantic parsing; unlike exact match, it can require both model- and domain-specific setups, which have, historically, varied widely across experiments. In our work, as a step towards providing a unified solution to data-efficiency-related questions, we introduce a four-stage protocol which gives an approximate measure of how much in-domain, "target" data a parser requires to achieve a certain quality bar. Specifically, our protocol consists of (1) sampling target subsets of different cardinalities, (2) fine-tuning parsers on each subset, (3) obtaining a smooth curve relating target subset (%) vs. exact match (%), and (4) referencing the curve to mine ad-hoc (target subset, exact match) points. We apply our protocol in two real-world case studies -- model generalizability and intent complexity -- illustrating its flexibility and applicability to practitioners in task-oriented semantic parsing. △ Less

Submitted 9 July, 2021; originally announced July 2021.

Enhancing Inertial Navigation Performance via Fusion of Classical and Quantum Accelerometers

Authors: Xuezhi Wang, Allison Kealy, Christopher Gilliam, Simon Haine, John Close, Bill Moran, Kyle Talbot, Simon Williams, Kyle Hardman, Chris Freier, Paul Wigley, Angela White, Stuart Szigeti, Sam Legge

Abstract: While quantum accelerometers sense with extremely low drift and low bias, their practical sensing capabilities face two limitations compared with classical accelerometers: a lower sample rate due to cold atom interrogation time, and a reduced dynamic range due to signal phase wrapping. In this paper, we propose a maximum likelihood probabilistic data fusion method, under which the actual phase of… ▽ More While quantum accelerometers sense with extremely low drift and low bias, their practical sensing capabilities face two limitations compared with classical accelerometers: a lower sample rate due to cold atom interrogation time, and a reduced dynamic range due to signal phase wrapping. In this paper, we propose a maximum likelihood probabilistic data fusion method, under which the actual phase of the quantum accelerometer can be unwrapped by fusing it with the output of a classical accelerometer on the platform. Consequently, the proposed method enables quantum accelerometers to be applied in practical inertial navigation scenarios with enhanced performance. The recovered measurement from the quantum accelerometer is also used to re-calibrate the classical accelerometer. We demonstrate the enhanced error performance achieved by the proposed fusion method using a simulated 1D inertial navigation scenario. We conclude with a discussion on fusion error and potential solutions. △ Less

Submitted 16 March, 2021; originally announced March 2021.

The genomic consequences of hybridization

Authors: Benjamin M Moran, Cheyenne Payne, Quinn Langdon, Daniel L Powell, Yaniv Brandvain, Molly Schumer

Abstract: In the past decade, advances in genome sequencing have allowed researchers to uncover the history of hybridization in diverse groups of species, including our own. Although the field has made impressive progress in documenting the extent of natural hybridization, both historical and recent, there are still many unanswered questions about its genetic and evolutionary consequences. Recent work has s… ▽ More In the past decade, advances in genome sequencing have allowed researchers to uncover the history of hybridization in diverse groups of species, including our own. Although the field has made impressive progress in documenting the extent of natural hybridization, both historical and recent, there are still many unanswered questions about its genetic and evolutionary consequences. Recent work has suggested that the outcomes of hybridization in the genome may be in part predictable, but many open questions about the nature of selection on hybrids and the biological variables that shape such selection have hampered progress in this area. We discuss what is known about the mechanisms that drive changes in ancestry in the genome after hybridization, highlight major unresolved questions, and discuss their implications for the predictability of genome evolution after hybridization. △ Less

Submitted 19 March, 2021; v1 submitted 7 December, 2020; originally announced December 2020.

Optimal Tree Topology for a Submarine Cable Network With Constrained Internodal Latency

Authors: Tianjiao Wang, Zengfu Wang, Bill Moran, Moshe Zukerman

Abstract: This paper provides an optimized cable path planning solution for a tree-topology network in an irregular 2D manifold in a 3D Euclidean space, with an application to the planning of submarine cable networks. Our solution method is based on total cost minimization, where the individual cable costs are assumed to be linear to the length of the corresponding submarine cables subject to latency constr… ▽ More This paper provides an optimized cable path planning solution for a tree-topology network in an irregular 2D manifold in a 3D Euclidean space, with an application to the planning of submarine cable networks. Our solution method is based on total cost minimization, where the individual cable costs are assumed to be linear to the length of the corresponding submarine cables subject to latency constraints between pairs of nodes. These latency constraints limit the cable length and number of hops between any pair of nodes. Our method combines the Fast Marching Method (FMM) and a new Integer Linear Programming (ILP) formulation for Minimum Spanning Tree (MST) where there are constraints between pairs of nodes. We note that this problem of MST with constraints is NP-complete. Nevertheless, we demonstrate that ILP running time is adequate for the great majority of existing cable systems. For cable systems for which ILP is not able to find the optimal solution within an acceptable time, we propose an alternative heuristic algorithm based on Prim's algorithm. In addition, we apply our FMM/ILP-based algorithm to a real-world cable path planning example and demonstrate that it can effectively find an MST with latency constraints between pairs of nodes. △ Less

Submitted 28 July, 2020; originally announced July 2020.

Comments: 11 pages, 7 figures

Positive operator-valued measures and densely-defined operator-valued frames

Authors: Benjamin Robinson, Bill Moran, Doug Cochran

Abstract: In the signal-processing literature, a frame is a mechanism for performing analysis and reconstruction in a Hilbert space. By contrast, in quantum theory, a positive operator-valued measure (POVM) decomposes a Hilbert-space vector for the purpose of computing measurement probabilities. Frames and their most common generalizations can be seen to give rise to POVMs, but does every reasonable POVM ar… ▽ More In the signal-processing literature, a frame is a mechanism for performing analysis and reconstruction in a Hilbert space. By contrast, in quantum theory, a positive operator-valued measure (POVM) decomposes a Hilbert-space vector for the purpose of computing measurement probabilities. Frames and their most common generalizations can be seen to give rise to POVMs, but does every reasonable POVM arise from a type of frame? In this paper we answer this question using a Radon-Nikodym-type result. △ Less

Submitted 24 April, 2020; originally announced April 2020.

Comments: 9 pages, submitted to the Rocky Mountain Math Journal

MSC Class: 42C15

Bayesian Recursive Estimation on the Rotation Group

Authors: Sofia Suvorova, Stephen D. Howard, Bill Moran

Abstract: Tracking on the rotation group is a key component of many modern systems for estimation of the motion of rigid bodies. To address this problem, here we describe a Bayesian algorithm that relies on directional measurements for tracking on the special orthogonal (rotation) group. Its novelty lies in the use of maximum entropy distributions on these groups as models for the priors, and justifiable ap… ▽ More Tracking on the rotation group is a key component of many modern systems for estimation of the motion of rigid bodies. To address this problem, here we describe a Bayesian algorithm that relies on directional measurements for tracking on the special orthogonal (rotation) group. Its novelty lies in the use of maximum entropy distributions on these groups as models for the priors, and justifiable approximation algorithms that permit recursive implementation of such a model. We provide the solutions in a recursive closed form. In the two-dimensional case the parameters of the prior and posterior distributions can be computed exactly and the solution has low complexity. Adoption of this approach eliminates the problem of angle wrapping. In higher dimensions the exact solution cannot be computed, and it is necessary to make (very close) approximations, which is done here. We demonstrate in simulations that, in contrast with some other approaches, our algorithm produces very accurate and statistically meaningful outputs. △ Less

Submitted 25 March, 2020; v1 submitted 21 March, 2020; originally announced March 2020.

Comments: I have posted it by mistake

Energy-Efficient Job-Assignment Policy with Asymptotically Guaranteed Performance Deviation

Authors: Jing Fu, Bill Moran

Abstract: We study a job-assignment problem in a large-scale server farm system with geographically deployed servers as abstracted computer components (e.g., storage, network links, and processors) that are potentially diverse. We aim to maximize the energy efficiency of the entire system by effectively controlling carried load on networked servers. A scalable, near-optimal job-assignment policy is proposed… ▽ More We study a job-assignment problem in a large-scale server farm system with geographically deployed servers as abstracted computer components (e.g., storage, network links, and processors) that are potentially diverse. We aim to maximize the energy efficiency of the entire system by effectively controlling carried load on networked servers. A scalable, near-optimal job-assignment policy is proposed. The optimality is gauged as, roughly speaking, energy cost per job. Our key result is an upper bound on the deviation between the proposed policy and the asymptotically optimal energy efficiency, when job sizes are exponentially distributed and blocking probabilities are positive. Relying on Whittle relaxation and the asymptotic optimality theorem of Weber and Weiss, this bound is shown to decrease exponentially as the number of servers and the arrival rates of jobs increase arbitrarily and in proportion. In consequence, the proposed policy is asymptotically optimal and, more importantly, approaches asymptotic optimality quickly (exponentially). This suggests that the proposed policy is close to optimal even for relatively small systems (and indeed any larger systems), and this is consistent with the results of our simulations. Simulations indicate that the policy is effective, and robust to variations in job-size distributions. △ Less

Submitted 26 March, 2020; v1 submitted 15 October, 2019; originally announced October 2019.

Comments: 14 pages, 10 figures

MSC Class: 90B15 (Primary) 90B36; 68M20 (Secondary) ACM Class: G.3; I.m

Information Geometry of Sensor Configuration

Authors: Simon Williams, Arthur George Suvorov, Wang Zeng Fu, Bill Moran

Abstract: In problems of parameter estimation from sensor data, the Fisher Information provides a measure of the performance of the sensor; effectively, in an infinitesimal sense, how much information about the parameters can be obtained from the measurements. From the geometric viewpoint, it is a Riemannian metric on the manifold of parameters of the observed system. In this paper we consider the case of p… ▽ More In problems of parameter estimation from sensor data, the Fisher Information provides a measure of the performance of the sensor; effectively, in an infinitesimal sense, how much information about the parameters can be obtained from the measurements. From the geometric viewpoint, it is a Riemannian metric on the manifold of parameters of the observed system. In this paper we consider the case of parameterized sensors and answer the question, "How best to reconfigure a sensor (vary the parameters of the sensor) to optimize the information collected?" A change in the sensor parameters results in a corresponding change to the metric. We show that the change in information due to reconfiguration exactly corresponds to the natural metric on the infinite dimensional space of Riemannian metrics on the parameter manifold, restricted to finite-dimensional sub-manifold determined by the sensor parameters. The distance measure on this configuration manifold is shown to provide optimal, dynamic sensor reconfiguration based on an information criterion. Geodesics on the configuration manifold are shown to optimize the information gain but only if the change is made at a certain rate. An example of configuring two bearings-only sensors to optimally locate a target is developed in detail to illustrate the mathematical machinery, with Fast-Marching methods employed to efficiently calculate the geodesics and illustrate the practicality of using this approach. △ Less

Submitted 4 November, 2018; v1 submitted 2 November, 2018; originally announced November 2018.

Comments: submitted to Information Geometry 2018-11-02

Controlled Tracking in Urban Terrain: Closing the Loop

Authors: Patricia R. Barbosa, Yugandhar Sarkale, Edwin K. P. Chong, Yun Li, Sofia Suvorova, Bill Moran

Abstract: We investigate the challenging problem of integrating detection, signal processing, target tracking, and adaptive waveform scheduling with lookahead in urban terrain. We propose a closed-loop active sensing system to address this problem by exploiting three distinct levels of diversity: (1) spatial diversity through the use of coordinated multistatic radars; (2) waveform diversity by adaptively sc… ▽ More We investigate the challenging problem of integrating detection, signal processing, target tracking, and adaptive waveform scheduling with lookahead in urban terrain. We propose a closed-loop active sensing system to address this problem by exploiting three distinct levels of diversity: (1) spatial diversity through the use of coordinated multistatic radars; (2) waveform diversity by adaptively scheduling the transmitted waveform; and (3) motion model diversity by using a bank of parallel filters matched to different motion models. Specifically, at every radar scan, the waveform that yields the minimum trace of the one-step-ahead error covariance matrix is transmitted; the received signal goes through a matched-filter, and curve fitting is used to extract range and range-rate measurements that feed the LMIPDA-VSIMM algorithm for data association and filtering. Monte Carlo simulations demonstrate the effectiveness of the proposed system in an urban scenario contaminated by dense and uneven clutter, strong multipath, and limited line-of-sight. △ Less

Submitted 1 May, 2018; originally announced May 2018.

Comments: Submitted to Special Issue: Recent Advances on Data Fusion, Estimation in Navigation and Control in Asian Journal of Control

Journal ref: Asian J Control. 2019; 1-14

A Restless Bandit Model for Resource Allocation, Competition and Reservation

Authors: Jing Fu, Bill Moran, Peter G. Taylor

Abstract: We study a resource allocation problem with varying requests, and with resources of limited capacity shared by multiple requests. It is modeled as a set of heterogeneous Restless Multi-Armed Bandit Problems (RMABPs) connected by constraints imposed by resource capacity. Following Whittle's relaxation idea and Weber and Weiss' asymptotic optimality proof, we propose a simple policy and prove it to… ▽ More We study a resource allocation problem with varying requests, and with resources of limited capacity shared by multiple requests. It is modeled as a set of heterogeneous Restless Multi-Armed Bandit Problems (RMABPs) connected by constraints imposed by resource capacity. Following Whittle's relaxation idea and Weber and Weiss' asymptotic optimality proof, we propose a simple policy and prove it to be asymptotically optimal in a regime where both arrival rates and capacities increase. We provide a simple sufficient condition for asymptotic optimality of the policy, and in complete generality propose a method that generates a set of candidate policies for which asymptotic optimality can be checked. The effectiveness of these results is demonstrated by numerical experiments. To the best of our knowledge, this is the first work providing asymptotic optimality results for such a resource allocation problem and such a combination of multiple RMABPs. △ Less

Submitted 26 March, 2020; v1 submitted 5 April, 2018; originally announced April 2018.

Comments: 78 pages, 8 figures, Latex

MSC Class: 90B36 (Primary) 90B15; 90B22 (Secondary)

A forest building process on simple graphs

Authors: Zhanar Berikkyzy, Steve Butler, Jay Cummings, Kristin Heysse, Paul Horn, Ruth Luo, Brent Moran

Abstract: Consider the following process on a simple graph without isolated vertices: Order the edges randomly and keep an edge if and only if it contains a vertex which is not contained in some preceding edge. The resulting set of edges forms a spanning forest of the graph. The probability of obtaining $k$ components in this process for complete bipartite graphs is determined as well as a formula for the… ▽ More Consider the following process on a simple graph without isolated vertices: Order the edges randomly and keep an edge if and only if it contains a vertex which is not contained in some preceding edge. The resulting set of edges forms a spanning forest of the graph. The probability of obtaining $k$ components in this process for complete bipartite graphs is determined as well as a formula for the expected number of components in any graph. A generic recurrence and some additional basic properties are discussed. △ Less

Submitted 8 September, 2017; v1 submitted 1 August, 2016; originally announced August 2016.

Stochastic Geometry Methods for Modelling Automotive Radar Interference

Authors: Akram Al-Hourani, Robin J. Evans, Sithamparanathan Kandeepan, Bill Moran, Hamid Eltom

Abstract: As the use of automotive radar increases, performance limitations associated with radar-to-radar interference will become more significant. In this paper we employ tools from stochastic geometry to characterize the statistics of radar interference. Specifically, using two different models for vehicle spacial distributions, namely, a Poisson point process and a Bernoulli lattice process, we calcula… ▽ More As the use of automotive radar increases, performance limitations associated with radar-to-radar interference will become more significant. In this paper we employ tools from stochastic geometry to characterize the statistics of radar interference. Specifically, using two different models for vehicle spacial distributions, namely, a Poisson point process and a Bernoulli lattice process, we calculate for each case the interference statistics and obtain analytical expressions for the probability of successful range estimation. Our study shows that the regularity of the geometrical model appears to have limited effect on the interference statistics, and so it is possible to obtain tractable tight bounds for worst case performance. A technique is proposed for designing the duty cycle for random spectrum access which optimizes the total performance. This analytical framework is verified using Monte-Carlo simulations. △ Less

Submitted 20 June, 2016; originally announced July 2016.

Regularizing Random Points: Complementary Matérn Hard-Core Point Process

Authors: Akram Al-Hourani, Bill Moran, Sithamparanathan Kandeepan

Abstract: In this paper we present a tractable approach for regularizing randomly placed points, by splitting them into two subsets: the first is generated by means of the Matérn hard-core point process, while the remaining points constitute the complementary Matérn hard-core point process. We study the characteristics of these processes, deriving its pair-correlation functions, and the distribution of the… ▽ More In this paper we present a tractable approach for regularizing randomly placed points, by splitting them into two subsets: the first is generated by means of the Matérn hard-core point process, while the remaining points constitute the complementary Matérn hard-core point process. We study the characteristics of these processes, deriving its pair-correlation functions, and the distribution of the distance to the nearest neighbour. The results have several applications in wireless communications, including the modeling of wireless sensor networks, where we investigate an example of regularizing such networks and illustrate its advantage in reducing the energy consumption of wireless nodes. △ Less

Submitted 25 May, 2016; v1 submitted 22 May, 2016; originally announced May 2016.

Comments: This paper has been withdrawn by the author due to a possible error in equation (5)

Maximum Likelihood Estimation for Single Linkage Hierarchical Clustering

Authors: Dekang Zhu, Dan P. Guralnik, Xuezhi Wang, Xiang Li, Bill Moran

Abstract: We derive a statistical model for estimation of a dendrogram from single linkage hierarchical clustering (SLHC) that takes account of uncertainty through noise or corruption in the measurements of separation of data. Our focus is on just the estimation of the hierarchy of partitions afforded by the dendrogram, rather than the heights in the latter. The concept of estimating this "dendrogram struct… ▽ More We derive a statistical model for estimation of a dendrogram from single linkage hierarchical clustering (SLHC) that takes account of uncertainty through noise or corruption in the measurements of separation of data. Our focus is on just the estimation of the hierarchy of partitions afforded by the dendrogram, rather than the heights in the latter. The concept of estimating this "dendrogram structure'' is introduced, and an approximate maximum likelihood estimator (MLE) for the dendrogram structure is described. These ideas are illustrated by a simple Monte Carlo simulation that, at least for small data sets, suggests the method outperforms SLHC in the presence of noise. △ Less

Submitted 24 November, 2015; originally announced November 2015.

Comments: 15 pages, 6 figures

Statistical Properties of the Single Linkage Hierarchical Clustering Estimator

Authors: Dekang Zhu, Dan P. Guralnik, Xuezhi Wang, Xiang Li, Bill Moran

Abstract: Distance-based hierarchical clustering (HC) methods are widely used in unsupervised data analysis but few authors take account of uncertainty in the distance data. We incorporate a statistical model of the uncertainty through corruption or noise in the pairwise distances and investigate the problem of estimating the HC as unknown parameters from measurements. Specifically, we focus on single linka… ▽ More Distance-based hierarchical clustering (HC) methods are widely used in unsupervised data analysis but few authors take account of uncertainty in the distance data. We incorporate a statistical model of the uncertainty through corruption or noise in the pairwise distances and investigate the problem of estimating the HC as unknown parameters from measurements. Specifically, we focus on single linkage hierarchical clustering (SLHC) and study its geometry. We prove that under fairly reasonable conditions on the probability distribution governing measurements, SLHC is equivalent to maximum partial profile likelihood estimation (MPPLE) with some of the information contained in the data ignored. At the same time, we show that direct evaluation of SLHC on maximum likelihood estimation (MLE) of pairwise distances yields a consistent estimator. Consequently, a full MLE is expected to perform better than SLHC in getting the correct HC results for the ground truth metric. △ Less

Submitted 31 August, 2016; v1 submitted 24 November, 2015; originally announced November 2015.

Comments: 21 pages, 6 figures

Bounds on Multiple Sensor Fusion

Authors: Bill Moran, Fred Cohen, Zengfu Wang, Sofia Suvorova, Douglas Cochran, Tom Taylor, Peter Farrell, Stephen Howard

Abstract: We consider the problem of fusing measurements from multiple sensors, where the sensing regions overlap and data are non-negative---possibly resulting from a count of indistinguishable discrete entities. Because of overlaps, it is, in general, impossible to fuse this information to arrive at an accurate estimate of the overall amount or count of material present in the union of the sensing regions… ▽ More We consider the problem of fusing measurements from multiple sensors, where the sensing regions overlap and data are non-negative---possibly resulting from a count of indistinguishable discrete entities. Because of overlaps, it is, in general, impossible to fuse this information to arrive at an accurate estimate of the overall amount or count of material present in the union of the sensing regions. Here we study the range of overall values consistent with the data. Posed as a linear programming problem, this leads to interesting questions associated with the geometry of the sensor regions, specifically, the arrangement of their non-empty intersections. We define a computational tool called the fusion polytope and derive a condition for this to be in the positive orthant thus simplifying calculations. We show that, in two dimensions, inflated tiling schemes based on rectangular regions fail to satisfy this condition, whereas inflated tiling schemes based on hexagons do. △ Less

Submitted 12 October, 2014; originally announced October 2014.

Comments: 23 pages

ACM Class: H.3.3; G.1.1; D.6.3

Guaranteed Bounds for General Approximate Dynamic Programming

Authors: Yajing Liu, Edwin K. P. Chong, Ali Pezeshki, Bill Moran

Abstract: In this paper, we will develop a systematic approach to deriving guaranteed bounds for approximate dynamic programming (ADP) schemes in optimal control problems. Our approach is inspired by our recent results on bounding the performance of greedy strategies in optimization of string-submodular functions over a finite horizon. The approach is to derive a string-submodular optimization problem, for… ▽ More In this paper, we will develop a systematic approach to deriving guaranteed bounds for approximate dynamic programming (ADP) schemes in optimal control problems. Our approach is inspired by our recent results on bounding the performance of greedy strategies in optimization of string-submodular functions over a finite horizon. The approach is to derive a string-submodular optimization problem, for which the optimal strategy is the optimal control solution and the greedy strategy is the ADP solution. Using this approach, we show that any ADP solution achieves a performance that is at least a factor of $β$ of the performance of the optimal control solution, which satisfies Bellman's optimality principle. The factor $β$ depends on the specific ADP scheme, as we will explicitly characterize. To illustrate the applicability of our bounding technique, we present examples of ADP schemes, including the popular rollout method. △ Less

Submitted 28 March, 2014; v1 submitted 21 March, 2014; originally announced March 2014.

Polynomial phase estimation by phase unwrapping

Authors: Robby G. McKilliam, Barry G. Quinn, I. Vaughan L. Clarkson, Bill Moran, Badri N. Vellambi

Abstract: Estimating the coefficients of a noisy polynomial phase signal is important in fields including radar, biology and radio communications. One approach attempts to perform polynomial regression on the phase of the signal. This is complicated by the fact that the phase is wrapped modulo 2πand must be unwrapped before regression can be performed. In this paper we consider an estimator that performs ph… ▽ More Estimating the coefficients of a noisy polynomial phase signal is important in fields including radar, biology and radio communications. One approach attempts to perform polynomial regression on the phase of the signal. This is complicated by the fact that the phase is wrapped modulo 2πand must be unwrapped before regression can be performed. In this paper we consider an estimator that performs phase unwrapping in a least squares manner. We describe the asymptotic properties of this estimator, showing that it is strongly consistent and asymptotically normally distributed. △ Less

Submitted 2 November, 2012; originally announced November 2012.

Maximum-entropy Surrogation in Network Signal Detection

Authors: Douglas Cochran, Stephen D. Howard, Bill Moran, Harry A. Schmitt

Abstract: Multiple-channel detection is considered in the context of a sensor network where raw data are shared only by nodes that have a common edge in the network graph. Established multiple-channel detectors, such as those based on generalized coherence or multiple coherence, use pairwise measurements from every pair of sensors in the network and are thus directly applicable only to networks whose graphs… ▽ More Multiple-channel detection is considered in the context of a sensor network where raw data are shared only by nodes that have a common edge in the network graph. Established multiple-channel detectors, such as those based on generalized coherence or multiple coherence, use pairwise measurements from every pair of sensors in the network and are thus directly applicable only to networks whose graphs are completely connected. An approach introduced here uses a maximum-entropy technique to formulate surrogate values for missing measurements corresponding to pairs of nodes that do not share an edge in the network graph. The broader potential merit of maximum-entropy baselines in quantifying the value of information in sensor network applications is also noted. △ Less

Submitted 28 February, 2012; originally announced February 2012.

Comments: 4 pages, submitted to IEEE Statistical Signal Processing Workshop, August 2012

An Information-geometric Approach to Sensor Management

Authors: Bill Moran, Stephen D. Howard, Douglas Cochran

Abstract: An information-geometric approach to sensor management is introduced that is based on following geodesic curves in a manifold of possible sensor configurations. This perspective arises by observing that, given a parameter estimation problem to be addressed through management of sensor assets, any particular sensor configuration corresponds to a Riemannian metric on the parameter manifold. With thi… ▽ More An information-geometric approach to sensor management is introduced that is based on following geodesic curves in a manifold of possible sensor configurations. This perspective arises by observing that, given a parameter estimation problem to be addressed through management of sensor assets, any particular sensor configuration corresponds to a Riemannian metric on the parameter manifold. With this perspective, managing sensors involves navigation on the space of all Riemannian metrics on the parameter manifold, which is itself a Riemannian manifold. Existing work assumes the metric on the parameter manifold is one that, in statistical terms, corresponds to a Jeffreys prior on the parameter to be estimated. It is observed that informative priors, as arise in sensor management, can also be accommodated. Given an initial sensor configuration, the trajectory along which to move in sensor configuration space to gather most information is seen to be locally defined by the geodesic structure of this manifold. Further, divergences based on Fisher and Shannon information lead to the same Riemannian metric and geodesics. △ Less

Submitted 28 February, 2012; originally announced February 2012.

Comments: 4 pages, 3 figures, to appear in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, March 2012

Positive Operator Valued Measures: A General Setting for Frames

Authors: Bill Moran, Stephen Howard, Doug Cochran

Abstract: This paper presents an overview of close parallels that exist between the theory of positive operator-valued measures (POVMs) associated with a separable Hilbert space and the theory of frames on that space, including its most important generalizations. The concept of a framed POVM is introduced, and classical frames, fusion frames, generalized frames, and other variants of frames are all shown to… ▽ More This paper presents an overview of close parallels that exist between the theory of positive operator-valued measures (POVMs) associated with a separable Hilbert space and the theory of frames on that space, including its most important generalizations. The concept of a framed POVM is introduced, and classical frames, fusion frames, generalized frames, and other variants of frames are all shown to to arise as framed POVMs. This observation allows drawing on a rich existing theory of POVMs to provide new perspectives in the study of frames. △ Less

Submitted 6 November, 2011; originally announced November 2011.

Comments: 17 pages; Prepeared for Proceedings of the February Fourier Talks