Distilling and exploiting quantitative insights from Large Language Models for enhanced Bayesian optimization of chemical reactions
Authors:
Roshan Patel,
Saeed Moayedpour,
Louis De Lescure,
Lorenzo Kogler-Anele,
Alan Cherney,
Sven Jager,
Yasser Jangjou
Abstract:
Machine learning and Bayesian optimization (BO) algorithms can significantly accelerate the optimization of chemical reactions. Transfer learning can bolster the effectiveness of BO algorithms in low-data regimes by leveraging pre-existing chemical information or data outside the direct optimization task (i.e., source data). Large language models (LLMs) have demonstrated that chemical information…
▽ More
Machine learning and Bayesian optimization (BO) algorithms can significantly accelerate the optimization of chemical reactions. Transfer learning can bolster the effectiveness of BO algorithms in low-data regimes by leveraging pre-existing chemical information or data outside the direct optimization task (i.e., source data). Large language models (LLMs) have demonstrated that chemical information present in foundation training data can give them utility for processing chemical data. Furthermore, they can be augmented with and help synthesize potentially multiple modalities of source chemical data germane to the optimization task. In this work, we examine how chemical information from LLMs can be elicited and used for transfer learning to accelerate the BO of reaction conditions to maximize yield. Specifically, we show that a survey-like prompting scheme and preference learning can be used to infer a utility function which models prior chemical information embedded in LLMs over a chemical parameter space; we find that the utility function shows modest correlation to true experimental measurements (yield) over the parameter space despite operating in a zero-shot setting. Furthermore, we show that the utility function can be leveraged to focus BO efforts in promising regions of the parameter space, improving the yield of the initial BO query and enhancing optimization in 4 of the 6 datasets studied. Overall, we view this work as a step towards bridging the gap between the chemistry knowledge embedded in LLMs and the capabilities of principled BO methods to accelerate reaction optimization.
△ Less
Submitted 11 April, 2025;
originally announced April 2025.
Many-Shot In-Context Learning for Molecular Inverse Design
Authors:
Saeed Moayedpour,
Alejandro Corrochano-Navarro,
Faryad Sahneh,
Shahriar Noroozizadeh,
Alexander Koetter,
Jiri Vymetal,
Lorenzo Kogler-Anele,
Pablo Mas,
Yasser Jangjou,
Sizhen Li,
Michael Bailey,
Marc Bianciotto,
Hans Matter,
Christoph Grebner,
Gerhard Hessler,
Ziv Bar-Joseph,
Sven Jager
Abstract:
Large Language Models (LLMs) have demonstrated great performance in few-shot In-Context Learning (ICL) for a variety of generative and discriminative chemical design tasks. The newly expanded context windows of LLMs can further improve ICL capabilities for molecular inverse design and lead optimization. To take full advantage of these capabilities we developed a new semi-supervised learning method…
▽ More
Large Language Models (LLMs) have demonstrated great performance in few-shot In-Context Learning (ICL) for a variety of generative and discriminative chemical design tasks. The newly expanded context windows of LLMs can further improve ICL capabilities for molecular inverse design and lead optimization. To take full advantage of these capabilities we developed a new semi-supervised learning method that overcomes the lack of experimental data available for many-shot ICL. Our approach involves iterative inclusion of LLM generated molecules with high predicted performance, along with experimental data. We further integrated our method in a multi-modal LLM which allows for the interactive modification of generated molecular structures using text instructions. As we show, the new method greatly improves upon existing ICL methods for molecular design while being accessible and easy to use for scientists.
△ Less
Submitted 26 July, 2024;
originally announced July 2024.
