-
AgentBreeder: Mitigating the AI Safety Impact of Multi-Agent Scaffolds via Self-Improvement
Authors:
J Rosser,
Jakob Nicolaus Foerster
Abstract:
Scaffolding Large Language Models (LLMs) into multi-agent systems often improves performance on complex tasks, but the safety impact of such scaffolds has not been thoroughly explored. We introduce AgentBreeder, a framework for multi-objective self-improving evolutionary search over scaffolds. We evaluate discovered scaffolds on widely recognized reasoning, mathematics, and safety benchmarks and c…
▽ More
Scaffolding Large Language Models (LLMs) into multi-agent systems often improves performance on complex tasks, but the safety impact of such scaffolds has not been thoroughly explored. We introduce AgentBreeder, a framework for multi-objective self-improving evolutionary search over scaffolds. We evaluate discovered scaffolds on widely recognized reasoning, mathematics, and safety benchmarks and compare them with popular baselines. In 'blue' mode, we see a 79.4% average uplift in safety benchmark performance while maintaining or improving capability scores. In 'red' mode, we find adversarially weak scaffolds emerging concurrently with capability optimization. Our work demonstrates the risks of multi-agent scaffolding and provides a framework for mitigating them. Code is available at https://github.com/J-Rosser-UK/AgentBreeder.
△ Less
Submitted 14 April, 2025; v1 submitted 2 February, 2025;
originally announced February 2025.
-
Resolving Ambiguity via Dialogue to Correct Unsynthesizable Controllers for Free-Flying Robots
Authors:
Joshua Rosser,
Jacob Arkin,
Siddharth Patki,
Thomas M. Howard
Abstract:
In situations such as habitat construction, station inspection, or cooperative exploration, incorrect assumptions about the environment or task across the team could lead to mission failure. Thus it is important to resolve any ambiguity about the mission between teammates before embarking on a commanded task. The safeguards guaranteed by formal methods can be used to synthesize correct-by-construc…
▽ More
In situations such as habitat construction, station inspection, or cooperative exploration, incorrect assumptions about the environment or task across the team could lead to mission failure. Thus it is important to resolve any ambiguity about the mission between teammates before embarking on a commanded task. The safeguards guaranteed by formal methods can be used to synthesize correct-by-construction reactive controllers for a robot using Linear Temporal Logic. If a robot fails to synthesize a controller given an instruction, it is clear that there exists a logical inconsistency in the environmental assumptions and/or described interactions. These specifications however are typically crafted in a language unique to the verification framework, requiring the human collaborator to be fluent in the software tool used to construct it. Furthermore, if the controller fails to synthesize, it may prove difficult to easily repair the specification. Language is a natural medium to generate these specifications using modern symbol grounding techniques. Using language empowers non-expert humans to describe tasks to robot teammates while retaining the benefits of formal verification. Additionally, dialogue could be used to inform robots about the environment and/or resolve any ambiguities before mission execution. This paper introduces an architecture for natural language interaction using a symbolic representation that informs the construction of a specification in Linear Temporal Logic. The novel aspect of this approach is that it provides a mechanism for resolving synthesis failure by hypothesizing corrections to the specification that are verified through human-robot dialogue. Experiments involving the proposed architecture are demonstrated using a simulation of an Astrobee robot navigating in the International Space Station.
△ Less
Submitted 11 April, 2023;
originally announced April 2023.
-
Verification of Readout Electronics in the ATLAS ITk Strips Detector
Authors:
Benjamin John Rosser
Abstract:
Particle physics detectors increasingly make use of custom FPGA firmware and application-specific integrated circuits (ASICs) for data readout and triggering. As these designs become more complex, it is important to ensure that they are simulated under realistic operating conditions before beginning fabrication. One tool to assist with the development of such designs is cocotb, an open source digi…
▽ More
Particle physics detectors increasingly make use of custom FPGA firmware and application-specific integrated circuits (ASICs) for data readout and triggering. As these designs become more complex, it is important to ensure that they are simulated under realistic operating conditions before beginning fabrication. One tool to assist with the development of such designs is cocotb, an open source digital logic verification framework. Using cocotb, verification can be done at high level using the Python programming language, allowing sophisticated data flow simulations to be conducted and issues to be identified early in the design phase. Cocotb was used successfully in the development of a testbench for several custom ASICs for the ATLAS ITk Strips detector, which found and resolved many problems during the development of the chips.
△ Less
Submitted 15 October, 2019;
originally announced October 2019.
-
Behavioral Characteristics and CO+CO2 Production Rates of Halley-Type Comets Observed by NEOWISE
Authors:
Joshua D. Rosser,
James M. Bauer,
Amy K. Mainzer,
Emily Kramer,
Joseph R. Masiero,
Carrie R. Nugent,
Sarah Sonnett,
Yanga R. Fernandez,
Kinjal Ruecker,
Philip Krings,
Edward L. Wright
Abstract:
From the entire dataset of comets observed by NEOWISE, we have analyzed 11 different Halley-Type Comets (HTCs) for dust production rates, CO+CO2 production rates, and nucleus sizes. Incorporating HTCs from previous studies and multiple comet visits we have a total of 21 stacked visits, 13 of which are active and 8 for which we calculated upper limits of production. We determined the nucleus sizes…
▽ More
From the entire dataset of comets observed by NEOWISE, we have analyzed 11 different Halley-Type Comets (HTCs) for dust production rates, CO+CO2 production rates, and nucleus sizes. Incorporating HTCs from previous studies and multiple comet visits we have a total of 21 stacked visits, 13 of which are active and 8 for which we calculated upper limits of production. We determined the nucleus sizes of 27P, P/2006 HR30, P/2012 NJ, and C/2016 S1. Furthermore, we analyzed the relationships between dust production and heliocentric distance, and gas production and heliocentric distance. We concluded that for this population of HTCs, ranging in heliocentric distance from 1.21 AU to 2.66 AU, there was no significant correlation between dust production and heliocentric distance, nor gas production and heliocentric distance.
△ Less
Submitted 19 February, 2018;
originally announced February 2018.
-
Proton irradiation results for long-wave HgCdTe infrared detector arrays for NEOCam
Authors:
M. Dorn,
J. L. Pipher,
C. McMurtry,
S. Hartman,
A. Mainzer,
M. McKelvey,
R. McMurray,
D. Chevara,
J. Rosser
Abstract:
HgCdTe detector arrays with a cutoff wavelength of ~10 $μ$m intended for the NEOCam space mission were subjected to proton beam irradiation at the University of California Davis Crocker Nuclear Laboratory. Three arrays were tested - one with 800 $μ$m substrate intact, one with 30 $μ$m substrate, and one completely substrate-removed. The CdZnTe substrate, on which the HgCdTe detector is grown, has…
▽ More
HgCdTe detector arrays with a cutoff wavelength of ~10 $μ$m intended for the NEOCam space mission were subjected to proton beam irradiation at the University of California Davis Crocker Nuclear Laboratory. Three arrays were tested - one with 800 $μ$m substrate intact, one with 30 $μ$m substrate, and one completely substrate-removed. The CdZnTe substrate, on which the HgCdTe detector is grown, has been shown to produce luminescence in shorter wave HgCdTe arrays that causes elevated signal in non-hit pixels when subjected to proton irradiation. This testing was conducted to ascertain whether or not full substrate removal is necessary. At the dark level of the dewar, we detect no luminescence in non-hit pixels during proton testing for both the substrate-removed detector array and the array with 30 $μ$m substrate. The detector array with full 800 $μ$m substrate exhibited substantial photocurrent for a flux of 103 protons/cm$^2$-s at a beam energy of 18.1 MeV (~ 750 e$^-$/s) and 34.4 MeV ($\sim$ 65 e$^-$/s). For the integrated space-like ambient proton flux level measured by the Spitzer Space Telescope, the luminescence would be well below the NEOCam dark current requirement of <200 e$^-$/s, but the pattern of luminescence could be problematic, possibly complicating calibration.
△ Less
Submitted 15 August, 2016;
originally announced August 2016.
-
Colloquium: Statistical mechanics of money, wealth, and income
Authors:
Victor M. Yakovenko,
J. Barkley Rosser
Abstract:
This Colloquium reviews statistical models for money, wealth, and income distributions developed in the econophysics literature since the late 1990s. By analogy with the Boltzmann-Gibbs distribution of energy in physics, it is shown that the probability distribution of money is exponential for certain classes of models with interacting economic agents. Alternative scenarios are also reviewed. Da…
▽ More
This Colloquium reviews statistical models for money, wealth, and income distributions developed in the econophysics literature since the late 1990s. By analogy with the Boltzmann-Gibbs distribution of energy in physics, it is shown that the probability distribution of money is exponential for certain classes of models with interacting economic agents. Alternative scenarios are also reviewed. Data analysis of the empirical distributions of wealth and income reveals a two-class distribution. The majority of the population belongs to the lower class, characterized by the exponential ("thermal") distribution, whereas a small fraction of the population in the upper class is characterized by the power-law ("superthermal") distribution. The lower part is very stable, stationary in time, whereas the upper part is highly dynamical and out of equilibrium.
△ Less
Submitted 24 December, 2009; v1 submitted 10 May, 2009;
originally announced May 2009.
