-
Fourier-Invertible Neural Encoder (FINE) for Homogeneous Flows
Authors:
Anqiao Ouyang,
Hongyi Ke,
Qi Wang
Abstract:
Invertible neural architectures have recently attracted attention for their compactness, interpretability, and information-preserving properties. In this work, we propose the Fourier-Invertible Neural Encoder (FINE), which combines invertible monotonic activation functions with reversible filter structures, and could be extended using Invertible ResNets. This architecture is examined in learning l…
▽ More
Invertible neural architectures have recently attracted attention for their compactness, interpretability, and information-preserving properties. In this work, we propose the Fourier-Invertible Neural Encoder (FINE), which combines invertible monotonic activation functions with reversible filter structures, and could be extended using Invertible ResNets. This architecture is examined in learning low-dimensional representations of one-dimensional nonlinear wave interactions and exact circular translation symmetry. Dimensionality is preserved across layers, except for a Fourier truncation step in the latent space, which enables dimensionality reduction while maintaining shift equivariance and interpretability. Our results demonstrate that FINE significantly outperforms classical linear methods such as Discrete Fourier Transformation (DFT) and Proper Orthogonal Decomposition (POD), and achieves reconstruction accuracy better than conventional deep autoencoders with convolutional layers (CNN) - while using substantially smaller models and offering superior physical interpretability. These findings suggest that invertible single-neuron networks, when combined with spectral truncation, offer a promising framework for learning compact and interpretable representations of physics datasets, and symmetry-aware representation learning in physics-informed machine learning.
△ Less
Submitted 21 May, 2025;
originally announced May 2025.
-
KernelBench: Can LLMs Write Efficient GPU Kernels?
Authors:
Anne Ouyang,
Simon Guo,
Simran Arora,
Alex L. Zhang,
William Hu,
Christopher Ré,
Azalia Mirhoseini
Abstract:
Efficient GPU kernels are crucial for building performant machine learning architectures, but writing them is a time-consuming challenge that requires significant expertise; therefore, we explore using language models (LMs) to automate kernel generation. We introduce KernelBench, an open-source framework for evaluating LMs' ability to write fast and correct kernels on a suite of 250 carefully sele…
▽ More
Efficient GPU kernels are crucial for building performant machine learning architectures, but writing them is a time-consuming challenge that requires significant expertise; therefore, we explore using language models (LMs) to automate kernel generation. We introduce KernelBench, an open-source framework for evaluating LMs' ability to write fast and correct kernels on a suite of 250 carefully selected PyTorch ML workloads. KernelBench represents a real-world engineering environment and making progress on the introduced benchmark directly translates to faster practical kernels. We introduce a new evaluation metric fast_p, which measures the percentage of generated kernels that are functionally correct and offer a speedup greater than an adjustable threshold p over baseline. Our experiments across various state-of-the-art models and test-time methods show that frontier reasoning models perform the best out of the box but still fall short overall, matching the PyTorch baseline in less than 20% of the cases. While we show that results can improve by leveraging execution and profiling feedback during iterative refinement, KernelBench remains a challenging benchmark, with its difficulty increasing as we raise speedup threshold p.
△ Less
Submitted 14 February, 2025;
originally announced February 2025.
-
Let's Think Frame by Frame with VIP: A Video Infilling and Prediction Dataset for Evaluating Video Chain-of-Thought
Authors:
Vaishnavi Himakunthala,
Andy Ouyang,
Daniel Rose,
Ryan He,
Alex Mei,
Yujie Lu,
Chinmay Sonar,
Michael Saxon,
William Yang Wang
Abstract:
Despite exciting recent results showing vision-language systems' capacity to reason about images using natural language, their capacity for video reasoning remains under-explored. We motivate framing video reasoning as the sequential understanding of a small number of keyframes, thereby leveraging the power and robustness of vision-language while alleviating the computational complexities of proce…
▽ More
Despite exciting recent results showing vision-language systems' capacity to reason about images using natural language, their capacity for video reasoning remains under-explored. We motivate framing video reasoning as the sequential understanding of a small number of keyframes, thereby leveraging the power and robustness of vision-language while alleviating the computational complexities of processing videos. To evaluate this novel application, we introduce VIP, an inference-time challenge dataset designed to explore models' reasoning capabilities through video chain-of-thought. Inspired by visually descriptive scene plays, we propose two formats for keyframe description: unstructured dense captions and structured scene descriptions that identify the focus, action, mood, objects, and setting (FAMOuS) of the keyframe. To evaluate video reasoning, we propose two tasks: Video Infilling and Video Prediction, which test abilities to generate multiple intermediate keyframes and predict future keyframes, respectively. We benchmark GPT-4, GPT-3, and VICUNA on VIP, demonstrate the performance gap in these complex video reasoning tasks, and encourage future work to prioritize language models for efficient and generalized video reasoning.
△ Less
Submitted 9 November, 2023; v1 submitted 23 May, 2023;
originally announced May 2023.
-
Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillings
Authors:
Daniel Rose,
Vaishnavi Himakunthala,
Andy Ouyang,
Ryan He,
Alex Mei,
Yujie Lu,
Michael Saxon,
Chinmay Sonar,
Diba Mirza,
William Yang Wang
Abstract:
Recent advances in large language models elicit reasoning in a chain-of-thought that allows models to decompose problems in a human-like fashion. Though this paradigm improves multi-step reasoning ability in language models, it is limited by being unimodal and applied mainly to question-answering tasks. We claim that incorporating visual augmentation into reasoning is essential, especially for com…
▽ More
Recent advances in large language models elicit reasoning in a chain-of-thought that allows models to decompose problems in a human-like fashion. Though this paradigm improves multi-step reasoning ability in language models, it is limited by being unimodal and applied mainly to question-answering tasks. We claim that incorporating visual augmentation into reasoning is essential, especially for complex, imaginative tasks. Consequently, we introduce VCoT, a novel method that leverages chain-of-thought prompting with vision-language grounding to recursively bridge the logical gaps within sequential data. Our method uses visual guidance to generate synthetic multimodal infillings that add consistent and novel information to reduce the logical gaps for downstream tasks that can benefit from temporal reasoning, as well as provide interpretability into models' multi-step reasoning. We apply VCoT to the Visual Storytelling and WikiHow summarization datasets and demonstrate through human evaluation that VCoT offers novel and consistent synthetic data augmentation beating chain-of-thought baselines, which can be used to enhance downstream performance.
△ Less
Submitted 22 January, 2024; v1 submitted 3 May, 2023;
originally announced May 2023.
-
Accelerating Training and Inference of Graph Neural Networks with Fast Sampling and Pipelining
Authors:
Tim Kaler,
Nickolas Stathas,
Anne Ouyang,
Alexandros-Stavros Iliopoulos,
Tao B. Schardl,
Charles E. Leiserson,
Jie Chen
Abstract:
Improving the training and inference performance of graph neural networks (GNNs) is faced with a challenge uncommon in general neural networks: creating mini-batches requires a lot of computation and data movement due to the exponential growth of multi-hop graph neighborhoods along network layers. Such a unique challenge gives rise to a diverse set of system design choices. We argue in favor of pe…
▽ More
Improving the training and inference performance of graph neural networks (GNNs) is faced with a challenge uncommon in general neural networks: creating mini-batches requires a lot of computation and data movement due to the exponential growth of multi-hop graph neighborhoods along network layers. Such a unique challenge gives rise to a diverse set of system design choices. We argue in favor of performing mini-batch training with neighborhood sampling in a distributed multi-GPU environment, under which we identify major performance bottlenecks hitherto under-explored by developers: mini-batch preparation and transfer. We present a sequence of improvements to mitigate these bottlenecks, including a performance-engineered neighborhood sampler, a shared-memory parallelization strategy, and the pipelining of batch transfer with GPU computation. We also conduct an empirical analysis that supports the use of sampling for inference, showing that test accuracies are not materially compromised. Such an observation unifies training and inference, simplifying model implementation. We report comprehensive experimental results with several benchmark data sets and GNN architectures, including a demonstration that, for the ogbn-papers100M data set, our system SALIENT achieves a speedup of 3x over a standard PyTorch-Geometric implementation with a single GPU and a further 8x parallel speedup with 16 GPUs. Therein, training a 3-layer GraphSAGE model with sampling fanout (15, 10, 5) takes 2.0 seconds per epoch and inference with fanout (20, 20, 20) takes 2.4 seconds, attaining test accuracy 64.58%.
△ Less
Submitted 16 March, 2022; v1 submitted 15 October, 2021;
originally announced October 2021.
-
Impact of COVID-19 on Public Transit Accessibility and Ridership
Authors:
Michael Wilbur,
Afiya Ayman,
Anna Ouyang,
Vincent Poon,
Riyan Kabir,
Abhiram Vadali,
Philip Pugliese,
Daniel Freudberg,
Aron Laszka,
Abhishek Dubey
Abstract:
Public transit is central to cultivating equitable communities. Meanwhile, the novel coronavirus disease COVID-19 and associated social restrictions has radically transformed ridership behavior in urban areas. Perhaps the most concerning aspect of the COVID-19 pandemic is that low-income and historically marginalized groups are not only the most susceptible to economic shifts but are also most rel…
▽ More
Public transit is central to cultivating equitable communities. Meanwhile, the novel coronavirus disease COVID-19 and associated social restrictions has radically transformed ridership behavior in urban areas. Perhaps the most concerning aspect of the COVID-19 pandemic is that low-income and historically marginalized groups are not only the most susceptible to economic shifts but are also most reliant on public transportation. As revenue decreases, transit agencies are tasked with providing adequate public transportation services in an increasingly hostile economic environment. Transit agencies therefore have two primary concerns. First, how has COVID-19 impacted ridership and what is the new post-COVID normal? Second, how has ridership varied spatio-temporally and between socio-economic groups? In this work we provide a data-driven analysis of COVID-19's affect on public transit operations and identify temporal variation in ridership change. We then combine spatial distributions of ridership decline with local economic data to identify variation between socio-economic groups. We find that in Nashville and Chattanooga, TN, fixed-line bus ridership dropped by 66.9% and 65.1% from 2019 baselines before stabilizing at 48.4% and 42.8% declines respectively. The largest declines were during morning and evening commute time. Additionally, there was a significant difference in ridership decline between the highest-income areas and lowest-income areas (77% vs 58%) in Nashville.
△ Less
Submitted 5 August, 2020;
originally announced August 2020.
