-
Large Scale Multi-Task Bayesian Optimization with Large Language Models
Authors:
Yimeng Zeng,
Natalie Maus,
Haydn Thomas Jones,
Jeffrey Tao,
Fangping Wan,
Marcelo Der Torossian Torres,
Cesar de la Fuente-Nunez,
Ryan Marcus,
Osbert Bastani,
Jacob R. Gardner
Abstract:
In multi-task Bayesian optimization, the goal is to leverage experience from optimizing existing tasks to improve the efficiency of optimizing new ones. While approaches using multi-task Gaussian processes or deep kernel transfer exist, the performance improvement is marginal when scaling to more than a moderate number of tasks. We introduce a novel approach leveraging large language models (LLMs)…
▽ More
In multi-task Bayesian optimization, the goal is to leverage experience from optimizing existing tasks to improve the efficiency of optimizing new ones. While approaches using multi-task Gaussian processes or deep kernel transfer exist, the performance improvement is marginal when scaling to more than a moderate number of tasks. We introduce a novel approach leveraging large language models (LLMs) to learn from, and improve upon, previous optimization trajectories, scaling to approximately 2000 distinct tasks. Specifically, we propose an iterative framework in which an LLM is fine-tuned using the high quality solutions produced by BayesOpt to generate improved initializations that accelerate convergence for future optimization tasks based on previous search trajectories. We evaluate our method on two distinct domains: database query optimization and antimicrobial peptide design. Results demonstrate that our approach creates a positive feedback loop, where the LLM's generated initializations gradually improve, leading to better optimization performance. As this feedback loop continues, we find that the LLM is eventually able to generate solutions to new tasks in just a few shots that are better than the solutions produced by "from scratch" by Bayesian optimization while simultaneously requiring significantly fewer oracle calls.
△ Less
Submitted 11 March, 2025;
originally announced March 2025.
-
Covering Multiple Objectives with a Small Set of Solutions Using Bayesian Optimization
Authors:
Natalie Maus,
Kyurae Kim,
Yimeng Zeng,
Haydn Thomas Jones,
Fangping Wan,
Marcelo Der Torossian Torres,
Cesar de la Fuente-Nunez,
Jacob R. Gardner
Abstract:
In multi-objective black-box optimization, the goal is typically to find solutions that optimize a set of $T$ black-box objective functions, $f_1$, ..., $f_T$, simultaneously. Traditional approaches often seek a single Pareto-optimal set that balances trade-offs among all objectives. In this work, we consider a problem setting that departs from this paradigm: finding a small set of K < T solutions…
▽ More
In multi-objective black-box optimization, the goal is typically to find solutions that optimize a set of $T$ black-box objective functions, $f_1$, ..., $f_T$, simultaneously. Traditional approaches often seek a single Pareto-optimal set that balances trade-offs among all objectives. In this work, we consider a problem setting that departs from this paradigm: finding a small set of K < T solutions, that collectively "covers" the T objectives. A set of solutions is defined as "covering" if, for each objective $f_1$, ..., $f_T$, there is at least one good solution. A motivating example for this problem setting occurs in drug design. For example, we may have T pathogens and aim to identify a set of K < T antibiotics such that at least one antibiotic can be used to treat each pathogen. To address this problem, we propose Multi-Objective Coverage Bayesian Optimization (MOCOBO), a principled algorithm designed to efficiently find a covering set. We validate our approach through experiments on challenging high-dimensional tasks, including applications in peptide and molecular design, where MOCOBO is shown to find high-performing covering sets of solutions. The results show that the coverage of the K < T solutions found by MOCOBO matches or nearly matches the coverage of T solutions obtained by optimizing each objective individually. Furthermore, in in vitro experiments, the peptides found by MOCOBO exhibited high potency against drug-resistant pathogens, further demonstrating the potential of MOCOBO for drug discovery.
△ Less
Submitted 16 May, 2025; v1 submitted 31 January, 2025;
originally announced January 2025.
-
Interfacial Rheology of Lanthanide Binding Peptide Surfactants at the Air-Water Interface
Authors:
Stephen A. Crane,
Felipe Jimenez-Angeles,
Yiming Wang,
Luis E. Ortuno Macias,
Jason G. Marmorstein,
Jiayi Deng,
Mehdi Molaei,
E. James Petersson,
Ravi Radhakrishnan,
Cesar de la Fuente-Nunez,
Monica Olvera de la Cruz,
Raymond S. Tu,
Charles Maldarelli,
Ivan J. Dmochowski,
Kathleen J. Stebe
Abstract:
Peptide surfactants (PEPS) are studied to capture and retain rare earth elements (REEs) at air-water interfaces to enable REE separations. Peptide sequences, designed to selectively bind REEs, depend crucially on the position of ligands within their binding loop domain. These ligands form a coordination sphere that wraps and retains the cation. We study variants of lanthanide binding tags (LBTs) d…
▽ More
Peptide surfactants (PEPS) are studied to capture and retain rare earth elements (REEs) at air-water interfaces to enable REE separations. Peptide sequences, designed to selectively bind REEs, depend crucially on the position of ligands within their binding loop domain. These ligands form a coordination sphere that wraps and retains the cation. We study variants of lanthanide binding tags (LBTs) designed to complex strongly with Tb$^{3+}$. The peptide LBT$^{5-}$ (with net charge -5) is known to bind Tb$^{3+}$ and adsorb with more REE cations than peptide molecules, suggesting that undesired non-specific Coulombic interactions occur. Rheological characterization of interfaces of LBT$^{5-}$ and Tb$^{3+}$ solutions reveal the formation of an interfacial gel. To probe whether this gelation reflects chelation among intact adsorbed LBT$^{5-}$:Tb$^{3+}$ complexes or destruction of the binding loop, we study a variant, LBT$^{3-}$, designed to form net neutral LBT$^{3-}$:Tb$^{3+}$ complexes. Solutions of LBT$^{3-}$ and Tb$^{3+}$ form purely viscous layers in the presence of excess Tb$^{3+}$, indicating that each peptide binds a single REE in an intact coordination sphere. We introduce the variant RR-LBT$^{3-}$ with net charge -3 and anionic ligands outside of the coordination sphere. We find that such exposed ligands promote interfacial gelation. Thus, a nuanced requirement for interfacial selectivity of PEPS is proposed: that anionic ligands outside of the coordination sphere must be avoided to prevent the non-selective recruitment of REE cations. This view is supported by simulation, including interfacial molecular dynamics simulations, and interfacial metadynamics simulations of the free energy landscape of the binding loop conformational space.
△ Less
Submitted 28 April, 2024;
originally announced April 2024.
