MAMMAL -- Molecular Aligned Multi-Modal Architecture and Language
Authors:
Yoel Shoshan,
Moshiko Raboh,
Michal Ozery-Flato,
Vadim Ratner,
Alex Golts,
Jeffrey K. Weber,
Ella Barkan,
Simona Rabinovici-Cohen,
Sagi Polaczek,
Ido Amos,
Ben Shapira,
Liam Hazan,
Matan Ninio,
Sivan Ravid,
Michael M. Danziger,
Yosi Shamay,
Sharon Kurant,
Joseph A. Morrone,
Parthasarathy Suryanarayanan,
Michal Rosen-Zvi,
Efrat Hexter
Abstract:
Large language models applied to vast biological datasets have the potential to transform biology by uncovering disease mechanisms and accelerating drug development. However, current models are often siloed, trained separately on small-molecules, proteins, or transcriptomic data, limiting their ability to capture complex, multi-modal interactions. Effective drug discovery requires computational to…
▽ More
Large language models applied to vast biological datasets have the potential to transform biology by uncovering disease mechanisms and accelerating drug development. However, current models are often siloed, trained separately on small-molecules, proteins, or transcriptomic data, limiting their ability to capture complex, multi-modal interactions. Effective drug discovery requires computational tools that integrate multiple biological entities while supporting prediction and generation, a challenge existing models struggle to address. For this purpose, we present MAMMAL - Molecular Aligned Multi-Modal Architecture and Language - a versatile method applied to create a multi-task foundation model that learns from large-scale biological datasets across diverse modalities, including proteins, small-molecules, and omics. MAMMAL's structured prompt syntax supports classification, regression, and generation tasks while handling token and scalar inputs and outputs. Evaluated on eleven diverse downstream tasks, it reaches a new state of the art (SOTA) in nine tasks and is comparable to SOTA in two tasks, all within a unified architecture, unlike prior task-specific models. Additionally, we explored Alphafold 3 binding prediction capabilities on antibody-antigen and nanobody-antigen complexes showing significantly better classification performance of MAMMAL in 3 out of 4 targets. The model code and pretrained weights are publicly available at https://github.com/BiomedSciAI/biomed-multi-alignment and https://huggingface.co/ibm/biomed.omics.bl.sm.ma-ted-458m
△ Less
Submitted 6 May, 2025; v1 submitted 28 October, 2024;
originally announced October 2024.
A Dataset for N-ary Relation Extraction of Drug Combinations
Authors:
Aryeh Tiktinsky,
Vijay Viswanathan,
Danna Niezni,
Dana Meron Azagury,
Yosi Shamay,
Hillel Taub-Tabib,
Tom Hope,
Yoav Goldberg
Abstract:
Combination therapies have become the standard of care for diseases such as cancer, tuberculosis, malaria and HIV. However, the combinatorial set of available multi-drug treatments creates a challenge in identifying effective combination therapies available in a situation. To assist medical professionals in identifying beneficial drug-combinations, we construct an expert-annotated dataset for extr…
▽ More
Combination therapies have become the standard of care for diseases such as cancer, tuberculosis, malaria and HIV. However, the combinatorial set of available multi-drug treatments creates a challenge in identifying effective combination therapies available in a situation. To assist medical professionals in identifying beneficial drug-combinations, we construct an expert-annotated dataset for extracting information about the efficacy of drug combinations from the scientific literature. Beyond its practical utility, the dataset also presents a unique NLP challenge, as the first relation extraction dataset consisting of variable-length relations. Furthermore, the relations in this dataset predominantly require language understanding beyond the sentence level, adding to the challenge of this task. We provide a promising baseline model and identify clear areas for further improvement. We release our dataset, code, and baseline models publicly to encourage the NLP community to participate in this task.
△ Less
Submitted 4 May, 2022;
originally announced May 2022.
