NNsight and NDIF: Democratizing Access to Open-Weight Foundation Model Internals
Authors:
Jaden Fiotto-Kaufman,
Alexander R. Loftus,
Eric Todd,
Jannik Brinkmann,
Koyena Pal,
Dmitrii Troitskii,
Michael Ripa,
Adam Belfki,
Can Rager,
Caden Juang,
Aaron Mueller,
Samuel Marks,
Arnab Sen Sharma,
Francesca Lucchetti,
Nikhil Prakash,
Carla Brodley,
Arjun Guha,
Jonathan Bell,
Byron C. Wallace,
David Bau
Abstract:
We introduce NNsight and NDIF, technologies that work in tandem to enable scientific study of the representations and computations learned by very large neural networks. NNsight is an open-source system that extends PyTorch to introduce deferred remote execution. The National Deep Inference Fabric (NDIF) is a scalable inference service that executes NNsight requests, allowing users to share GPU re…
▽ More
We introduce NNsight and NDIF, technologies that work in tandem to enable scientific study of the representations and computations learned by very large neural networks. NNsight is an open-source system that extends PyTorch to introduce deferred remote execution. The National Deep Inference Fabric (NDIF) is a scalable inference service that executes NNsight requests, allowing users to share GPU resources and pretrained models. These technologies are enabled by the Intervention Graph, an architecture developed to decouple experimental design from model runtime. Together, this framework provides transparent and efficient access to the internals of deep neural networks such as very large language models (LLMs) without imposing the cost or complexity of hosting customized models individually. We conduct a quantitative survey of the machine learning literature that reveals a growing gap in the study of the internals of large-scale AI. We demonstrate the design and use of our framework to address this gap by enabling a range of research methods on huge models. Finally, we conduct benchmarks to compare performance with previous approaches.
Code, documentation, and tutorials are available at https://nnsight.net/.
△ Less
Submitted 1 April, 2025; v1 submitted 18 July, 2024;
originally announced July 2024.
A Saliency-based Clustering Framework for Identifying Aberrant Predictions
Authors:
Aina Tersol Montserrat,
Alexander R. Loftus,
Yael Daihes
Abstract:
In machine learning, classification tasks serve as the cornerstone of a wide range of real-world applications. Reliable, trustworthy classification is particularly intricate in biomedical settings, where the ground truth is often inherently uncertain and relies on high degrees of human expertise for labeling. Traditional metrics such as precision and recall, while valuable, are insufficient for ca…
▽ More
In machine learning, classification tasks serve as the cornerstone of a wide range of real-world applications. Reliable, trustworthy classification is particularly intricate in biomedical settings, where the ground truth is often inherently uncertain and relies on high degrees of human expertise for labeling. Traditional metrics such as precision and recall, while valuable, are insufficient for capturing the nuances of these ambiguous scenarios. Here we introduce the concept of aberrant predictions, emphasizing that the nature of classification errors is as critical as their frequency. We propose a novel, efficient training methodology aimed at both reducing the misclassification rate and discerning aberrant predictions. Our framework demonstrates a substantial improvement in model performance, achieving a 20\% increase in precision. We apply this methodology to the less-explored domain of veterinary radiology, where the stakes are high but have not been as extensively studied compared to human medicine. By focusing on the identification and mitigation of aberrant predictions, we enhance the utility and trustworthiness of machine learning classifiers in high-stakes, real-world scenarios, including new applications in the veterinary world.
△ Less
Submitted 10 November, 2023;
originally announced November 2023.
