Large language models surpass human experts in predicting neuroscience results
Authors:
Xiaoliang Luo,
Akilles Rechardt,
Guangzhi Sun,
Kevin K. Nejad,
Felipe Yáñez,
Bati Yilmaz,
Kangjoo Lee,
Alexandra O. Cohen,
Valentina Borghesani,
Anton Pashkov,
Daniele Marinazzo,
Jonathan Nicholas,
Alessandro Salatiello,
Ilia Sucholutsky,
Pasquale Minervini,
Sepehr Razavi,
Roberta Rocca,
Elkhan Yusifov,
Tereza Okalova,
Nianlong Gu,
Martin Ferianc,
Mikail Khona,
Kaustubh R. Patil,
Pui-Shee Lee,
Rui Mata
, et al. (14 additional authors not shown)
Abstract:
Scientific discoveries often hinge on synthesizing decades of research, a task that potentially outstrips human information processing capacities. Large language models (LLMs) offer a solution. LLMs trained on the vast scientific literature could potentially integrate noisy yet interrelated findings to forecast novel results better than human experts. To evaluate this possibility, we created Brain…
▽ More
Scientific discoveries often hinge on synthesizing decades of research, a task that potentially outstrips human information processing capacities. Large language models (LLMs) offer a solution. LLMs trained on the vast scientific literature could potentially integrate noisy yet interrelated findings to forecast novel results better than human experts. To evaluate this possibility, we created BrainBench, a forward-looking benchmark for predicting neuroscience results. We find that LLMs surpass experts in predicting experimental outcomes. BrainGPT, an LLM we tuned on the neuroscience literature, performed better yet. Like human experts, when LLMs were confident in their predictions, they were more likely to be correct, which presages a future where humans and LLMs team together to make discoveries. Our approach is not neuroscience-specific and is transferable to other knowledge-intensive endeavors.
△ Less
Submitted 28 November, 2024; v1 submitted 4 March, 2024;
originally announced March 2024.
A roadmap to reverse engineering real-world generalization by combining naturalistic paradigms, deep sampling, and predictive computational models
Authors:
Peer Herholz,
Eddy Fortier,
Mariya Toneva,
Nicolas Farrugia,
Leila Wehbe,
Valentina Borghesani
Abstract:
Real-world generalization, e.g., deciding to approach a never-seen-before animal, relies on contextual information as well as previous experiences. Such a seemingly easy behavioral choice requires the interplay of multiple neural mechanisms, from integrative encoding to category-based inference, weighted differently according to the circumstances. Here, we argue that a comprehensive theory of the…
▽ More
Real-world generalization, e.g., deciding to approach a never-seen-before animal, relies on contextual information as well as previous experiences. Such a seemingly easy behavioral choice requires the interplay of multiple neural mechanisms, from integrative encoding to category-based inference, weighted differently according to the circumstances. Here, we argue that a comprehensive theory of the neuro-cognitive substrates of real-world generalization will greatly benefit from empirical research with three key elements. First, the ecological validity provided by multimodal, naturalistic paradigms. Second, the model stability afforded by deep sampling. Finally, the statistical rigor granted by predictive modeling and computational controls.
△ Less
Submitted 14 January, 2022; v1 submitted 23 August, 2021;
originally announced August 2021.
