-
Disentangling Recognition and Decision Regrets in Image-Based Reinforcement Learning
Authors:
Alihan Hüyük,
Arndt Ryo Koblitz,
Atefeh Mohajeri,
Matthew Andrews
Abstract:
In image-based reinforcement learning (RL), policies usually operate in two steps: first extracting lower-dimensional features from raw images (the "recognition" step), and then taking actions based on the extracted features (the "decision" step). Extracting features that are spuriously correlated with performance or irrelevant for decision-making can lead to poor generalization performance, known…
▽ More
In image-based reinforcement learning (RL), policies usually operate in two steps: first extracting lower-dimensional features from raw images (the "recognition" step), and then taking actions based on the extracted features (the "decision" step). Extracting features that are spuriously correlated with performance or irrelevant for decision-making can lead to poor generalization performance, known as observational overfitting in image-based RL. In such cases, it can be hard to quantify how much of the error can be attributed to poor feature extraction vs. poor decision-making. To disentangle the two sources of error, we introduce the notions of recognition regret and decision regret. Using these notions, we characterize and disambiguate the two distinct causes behind observational overfitting: over-specific representations, which include features that are not needed for optimal decision-making (leading to high decision regret), vs. under-specific representations, which only include a limited set of features that were spuriously correlated with performance during training (leading to high recognition regret). Finally, we provide illustrative examples of observational overfitting due to both over-specific and under-specific representations in maze environments and the Atari game Pong.
△ Less
Submitted 2 April, 2025; v1 submitted 19 September, 2024;
originally announced September 2024.
-
Reducing the Environmental Impact of Wireless Communication via Probabilistic Machine Learning
Authors:
A. Ryo Koblitz,
Lorenzo Maggi,
Matthew Andrews
Abstract:
Machine learning methods are increasingly adopted in communications problems, particularly those arising in next generation wireless settings. Though seen as a key climate mitigation and societal adaptation enabler, communications related energy consumption is high and is expected to grow in future networks in spite of anticipated efficiency gains in 6G due to exponential communications traffic gr…
▽ More
Machine learning methods are increasingly adopted in communications problems, particularly those arising in next generation wireless settings. Though seen as a key climate mitigation and societal adaptation enabler, communications related energy consumption is high and is expected to grow in future networks in spite of anticipated efficiency gains in 6G due to exponential communications traffic growth. To make meaningful climate mitigation impact in the communications sector, a mindset shift away from maximizing throughput at all cost and towards prioritizing energy efficiency is needed. Moreover, this must be adopted in both existing (without incurring further embodied carbon costs through equipment replacement) and future network infrastructure, given the long development time of mobile generations. To that end, we present summaries of two such problems, from both current and next generation network specifications, where probabilistic inference methods were used to great effect: using Bayesian parameter tuning we are able to safely reduce the energy consumption of existing hardware on a live communications network by $11\%$ whilst maintaining operator specified performance envelopes; through spatiotemporal Gaussian process surrogate modeling we reduce the overhead in a next generation hybrid beamforming system by over $60\%$, greatly improving the networks' ability to target highly mobile users such as autonomous vehicles. The Bayesian paradigm is itself helpful in terms of energy usage, since training a Bayesian optimization model can require much less computation than, say, training a deep neural network.
△ Less
Submitted 19 September, 2023;
originally announced November 2023.
-
Direct numerical simulation of particle sedimentation in a Bingham fluid
Authors:
A. R. Koblitz,
S. Lovett,
N. Nikiforakis
Abstract:
The settling efficiency, and stability with respect to settling, of a dilute suspension of infinite circular cylinders in a quiescent viscoplastic fluid is examined by means of direct numerical simulations with varying solid volume fraction, $φ$, and yield number, $Y$ . For $Y$ sufficiently large we find higher settling efficiency for increasing $φ$, similar to what is found in shear-thinning flui…
▽ More
The settling efficiency, and stability with respect to settling, of a dilute suspension of infinite circular cylinders in a quiescent viscoplastic fluid is examined by means of direct numerical simulations with varying solid volume fraction, $φ$, and yield number, $Y$ . For $Y$ sufficiently large we find higher settling efficiency for increasing $φ$, similar to what is found in shear-thinning fluids and opposite to what is found in Newtonian fluids. The critical yield number at which the suspension is held stationary in the carrier fluid is found to increase monotonically with $φ$, while the transition to settling is found to be diffuse: in the same suspension, particle clusters may settle while more isolated particles remain arrested. In this regime, complex flow features are observed in the sedimenting suspension, including the mobilization of lone particles by nearby sedimentation clusters. Understanding this regime, and the transition to a fully arrested state, is relevant to many industrial and natural problems involving the sedimentation of viscoplastic suspensions under quiescent flow conditions.
△ Less
Submitted 12 June, 2018;
originally announced June 2018.
-
On the viscoplastic squeeze flow between two identical infinite circular cylinders
Authors:
Arndt Ryo Koblitz,
Sean Lovett,
Nikolaos Nikiforakis
Abstract:
Direct numerical simulations of closely interacting infinite circular cylinders in a Bingham fluid are presented, and results compared to asymptotic solutions based on lubrication theory in the gap. Unlike for a Newtonian fluid, the macroscopic flow outside of the gap between the cylinders is shown to have a large effect on the pressure profile within the gap and the resulting lubrication force on…
▽ More
Direct numerical simulations of closely interacting infinite circular cylinders in a Bingham fluid are presented, and results compared to asymptotic solutions based on lubrication theory in the gap. Unlike for a Newtonian fluid, the macroscopic flow outside of the gap between the cylinders is shown to have a large effect on the pressure profile within the gap and the resulting lubrication force on the cylinders. The presented results indicate that the asymptotic lubrication solution can be used to predict the lubrication pressure only if the surrounding viscoplastic matrix is yielded by a macroscopic flow. This has implications for the use of sub-grid-scale lubrication models in simulations of non-colloidal particulate suspensions in viscoplastic fluids.
△ Less
Submitted 7 February, 2018; v1 submitted 16 December, 2017;
originally announced December 2017.
-
Direct numerical simulation of particulate flows with an overset grid method
Authors:
A. R. Koblitz,
S. Lovett,
N. Nikiforakis,
W. D. Henshaw
Abstract:
We evaluate an efficient overset grid method for two-dimensional and three-dimensional particulate flows for small numbers of particles at finite Reynolds number. The rigid particles are discretised using moving overset grids overlaid on a Cartesian background grid. This allows for strongly-enforced boundary conditions and local grid refinement at particle surfaces, thereby accurately capturing th…
▽ More
We evaluate an efficient overset grid method for two-dimensional and three-dimensional particulate flows for small numbers of particles at finite Reynolds number. The rigid particles are discretised using moving overset grids overlaid on a Cartesian background grid. This allows for strongly-enforced boundary conditions and local grid refinement at particle surfaces, thereby accurately capturing the viscous boundary layer at modest computational cost. The incompressible Navier--Stokes equations are solved with a fractional-step scheme which is second-order-accurate in space and time, while the fluid--solid coupling is achieved with a partitioned approach including multiple sub-iterations to increase stability for light, rigid bodies. Through a series of benchmark studies we demonstrate the accuracy and efficiency of this approach compared to other boundary conformal and static grid methods in the literature. In particular, we find that fully resolving boundary layers at particle surfaces is crucial to obtain accurate solutions to many common test cases. With our approach we are able to compute accurate solutions using as little as one third the number of grid points as uniform grid computations in the literature. A detailed convergence study shows a 13-fold decrease in CPU time over a uniform grid test case whilst maintaining comparable solution accuracy.
△ Less
Submitted 3 February, 2017;
originally announced February 2017.
