Neural CRNs: A Natural Implementation of Learning in Chemical Reaction Networks
Authors:
Rajiv Teja Nagipogu,
John H. Reif
Abstract:
We present Neural CRNs, an efficient, autonomous, and general-purpose implementation of learning within mass action chemical reaction systems. Unlike prior works, which transliterate discrete neural networks into chemical systems, Neural CRNs are a purely analog chemical system, which encodes neural computations in the concentration dynamics of its chemical species. Consequently, the chemical reac…
▽ More
We present Neural CRNs, an efficient, autonomous, and general-purpose implementation of learning within mass action chemical reaction systems. Unlike prior works, which transliterate discrete neural networks into chemical systems, Neural CRNs are a purely analog chemical system, which encodes neural computations in the concentration dynamics of its chemical species. Consequently, the chemical reactions in this system stay true to their nature, behaving as atomic end-to-end computational units, resulting in concise and efficient reaction network implementations. We demonstrate this efficiency by assembling a highly streamlined supervised learning procedure that requires only two clock phases. We further validate the robustness of our framework by constructing Neural CRN circuits for several linear and nonlinear regression and classification tasks. Furthermore, a minimal linear regression circuit is assembled using only 13 reactions and 15 species. Our nonlinear modeling circuits significantly advance the state-of-the-art through compact and simple implementations. The synergistic nature of our framework with the analog chemical computing hardware leaves ample room for optimizations and approximations in the computational model, several of which are discussed in this work. Our work introduces a novel paradigm for chemical computing and learning, providing a foundational platform for future adaptive biochemical circuits with applications in fields such as synthetic biology, bioengineering, and adaptive biomedicine.
△ Less
Submitted 27 February, 2025; v1 submitted 17 August, 2024;
originally announced September 2024.
MuRIL: Multilingual Representations for Indian Languages
Authors:
Simran Khanuja,
Diksha Bansal,
Sarvesh Mehtani,
Savya Khosla,
Atreyee Dey,
Balaji Gopalan,
Dilip Kumar Margam,
Pooja Aggarwal,
Rajiv Teja Nagipogu,
Shachi Dave,
Shruti Gupta,
Subhash Chandra Bose Gali,
Vish Subramanian,
Partha Talukdar
Abstract:
India is a multilingual society with 1369 rationalized languages and dialects being spoken across the country (INDIA, 2011). Of these, the 22 scheduled languages have a staggering total of 1.17 billion speakers and 121 languages have more than 10,000 speakers (INDIA, 2011). India also has the second largest (and an ever growing) digital footprint (Statista, 2020). Despite this, today's state-of-th…
▽ More
India is a multilingual society with 1369 rationalized languages and dialects being spoken across the country (INDIA, 2011). Of these, the 22 scheduled languages have a staggering total of 1.17 billion speakers and 121 languages have more than 10,000 speakers (INDIA, 2011). India also has the second largest (and an ever growing) digital footprint (Statista, 2020). Despite this, today's state-of-the-art multilingual systems perform suboptimally on Indian (IN) languages. This can be explained by the fact that multilingual language models (LMs) are often trained on 100+ languages together, leading to a small representation of IN languages in their vocabulary and training data. Multilingual LMs are substantially less effective in resource-lean scenarios (Wu and Dredze, 2020; Lauscher et al., 2020), as limited data doesn't help capture the various nuances of a language. One also commonly observes IN language text transliterated to Latin or code-mixed with English, especially in informal settings (for example, on social media platforms) (Rijhwani et al., 2017). This phenomenon is not adequately handled by current state-of-the-art multilingual LMs. To address the aforementioned gaps, we propose MuRIL, a multilingual LM specifically built for IN languages. MuRIL is trained on significantly large amounts of IN text corpora only. We explicitly augment monolingual text corpora with both translated and transliterated document pairs, that serve as supervised cross-lingual signals in training. MuRIL significantly outperforms multilingual BERT (mBERT) on all tasks in the challenging cross-lingual XTREME benchmark (Hu et al., 2020). We also present results on transliterated (native to Latin script) test sets of the chosen datasets and demonstrate the efficacy of MuRIL in handling transliterated data.
△ Less
Submitted 2 April, 2021; v1 submitted 19 March, 2021;
originally announced March 2021.
