-
Neuro-Symbolic Generative Diffusion Models for Physically Grounded, Robust, and Safe Generation
Authors:
Jacob K. Christopher,
Michael Cardei,
Jinhao Liang,
Ferdinando Fioretto
Abstract:
Despite the remarkable generative capabilities of diffusion models, their integration into safety-critical or scientifically rigorous applications remains hindered by the need to ensure compliance with stringent physical, structural, and operational constraints. To address this challenge, this paper introduces Neuro-Symbolic Diffusion (NSD), a novel framework that interleaves diffusion steps with…
▽ More
Despite the remarkable generative capabilities of diffusion models, their integration into safety-critical or scientifically rigorous applications remains hindered by the need to ensure compliance with stringent physical, structural, and operational constraints. To address this challenge, this paper introduces Neuro-Symbolic Diffusion (NSD), a novel framework that interleaves diffusion steps with symbolic optimization, enabling the generation of certifiably consistent samples under user-defined functional and logic constraints. This key feature is provided for both standard and discrete diffusion models, enabling, for the first time, the generation of both continuous (e.g., images and trajectories) and discrete (e.g., molecular structures and natural language) outputs that comply with constraints. This ability is demonstrated on tasks spanning three key challenges: (1) Safety, in the context of non-toxic molecular generation and collision-free trajectory optimization; (2) Data scarcity, in domains such as drug discovery and materials engineering; and (3) Out-of-domain generalization, where enforcing symbolic constraints allows adaptation beyond the training distribution.
△ Less
Submitted 1 June, 2025;
originally announced June 2025.
-
Constrained Discrete Diffusion
Authors:
Michael Cardei,
Jacob K Christopher,
Thomas Hartvigsen,
Brian R. Bartoldson,
Bhavya Kailkhura,
Ferdinando Fioretto
Abstract:
Discrete diffusion models are a class of generative models that construct sequences by progressively denoising samples from a categorical noise distribution. Beyond their rapidly growing ability to generate coherent natural language, these models present a new and important opportunity to enforce sequence-level constraints, a capability that current autoregressive models cannot natively provide. T…
▽ More
Discrete diffusion models are a class of generative models that construct sequences by progressively denoising samples from a categorical noise distribution. Beyond their rapidly growing ability to generate coherent natural language, these models present a new and important opportunity to enforce sequence-level constraints, a capability that current autoregressive models cannot natively provide. This paper capitalizes on this opportunity by introducing Constrained Discrete Diffusion (CDD), a novel integration of differentiable constraint optimization within the diffusion process to ensure adherence to constraints, logic rules, or safety requirements for generated sequences. Unlike conventional text generators that often rely on post-hoc filtering or model retraining for controllable generation, CDD directly imposes constraints into the discrete diffusion sampling process, resulting in a training-free and effective approach. Experiments in toxicity-controlled text generation, property-constrained molecule design, and instruction-constrained text completion demonstrate that CDD achieves zero constraint violations in a diverse array of tasks while preserving fluency, novelty, and coherence while outperforming autoregressive and existing discrete diffusion approaches.
△ Less
Submitted 27 May, 2025; v1 submitted 12 March, 2025;
originally announced March 2025.
-
Factorized Deep Q-Network for Cooperative Multi-Agent Reinforcement Learning in Victim Tagging
Authors:
Maria Ana Cardei,
Afsaneh Doryab
Abstract:
Mass casualty incidents (MCIs) are a growing concern, characterized by complexity and uncertainty that demand adaptive decision-making strategies. The victim tagging step in the emergency medical response must be completed quickly and is crucial for providing information to guide subsequent time-constrained response actions. In this paper, we present a mathematical formulation of multi-agent victi…
▽ More
Mass casualty incidents (MCIs) are a growing concern, characterized by complexity and uncertainty that demand adaptive decision-making strategies. The victim tagging step in the emergency medical response must be completed quickly and is crucial for providing information to guide subsequent time-constrained response actions. In this paper, we present a mathematical formulation of multi-agent victim tagging to minimize the time it takes for responders to tag all victims. Five distributed heuristics are formulated and evaluated with simulation experiments. The heuristics considered are on-the go, practical solutions that represent varying levels of situational uncertainty in the form of global or local communication capabilities, showcasing practical constraints. We further investigate the performance of a multi-agent reinforcement learning (MARL) strategy, factorized deep Q-network (FDQN), to minimize victim tagging time as compared to baseline heuristics. Extensive simulations demonstrate that between the heuristics, methods with local communication are more efficient for adaptive victim tagging, specifically choosing the nearest victim with the option to replan. Analyzing all experiments, we find that our FDQN approach outperforms heuristics in smaller-scale scenarios, while heuristics excel in more complex scenarios. Our experiments contain diverse complexities that explore the upper limits of MARL capabilities for real-world applications and reveal key insights.
△ Less
Submitted 1 March, 2025;
originally announced March 2025.
-
Pairwise Spatiotemporal Partial Trajectory Matching for Co-movement Analysis
Authors:
Maria Cardei,
Sabit Ahmed,
Gretchen Chapman,
Afsaneh Doryab
Abstract:
Spatiotemporal pairwise movement analysis involves identifying shared geographic-based behaviors between individuals within specific time frames. Traditionally, this task relies on sequence modeling and behavior analysis techniques applied to tabular or video-based data, but these methods often lack interpretability and struggle to capture partial matching. In this paper, we propose a novel method…
▽ More
Spatiotemporal pairwise movement analysis involves identifying shared geographic-based behaviors between individuals within specific time frames. Traditionally, this task relies on sequence modeling and behavior analysis techniques applied to tabular or video-based data, but these methods often lack interpretability and struggle to capture partial matching. In this paper, we propose a novel method for pairwise spatiotemporal partial trajectory matching that transforms tabular spatiotemporal data into interpretable trajectory images based on specified time windows, allowing for partial trajectory analysis. This approach includes localization of trajectories, checking for spatial overlap, and pairwise matching using a Siamese Neural Network. We evaluate our method on a co-walking classification task, demonstrating its effectiveness in a novel co-behavior identification application. Our model surpasses established methods, achieving an F1-score up to 0.73. Additionally, we explore the method's utility for pair routine pattern analysis in real-world scenarios, providing insights into the frequency, timing, and duration of shared behaviors. This approach offers a powerful, interpretable framework for spatiotemporal behavior analysis, with potential applications in social behavior research, urban planning, and healthcare.
△ Less
Submitted 3 December, 2024;
originally announced December 2024.
-
Exploring Smartphone-based Spectrophotometry for Nutrient Identification and Quantification
Authors:
Andrew Balch,
Maria A. Cardei,
Afsaneh Doryab
Abstract:
Imbalanced nutrition is a global health issue with significant downstream effects. Current methods of assessing nutrient levels face several limitations, with accessibility being a major concern. In this paper, we take a step towards accessibly measuring nutrient status within the body. We explore the potential of smartphone-based spectrophotometry for identifying and quantifying nutrients in a so…
▽ More
Imbalanced nutrition is a global health issue with significant downstream effects. Current methods of assessing nutrient levels face several limitations, with accessibility being a major concern. In this paper, we take a step towards accessibly measuring nutrient status within the body. We explore the potential of smartphone-based spectrophotometry for identifying and quantifying nutrients in a solution by building and testing two prototype devices. We compared the prototypes and found that the limitations posed by the initial, simpler prototype were well addressed in the more portable and reliable second-generation device. With the second-generation prototype, we created and implemented a semi-automatic signal processing and analysis pipeline for analyzing absorption spectra. We thoroughly evaluated the prototypes by analyzing the effect of four different light sources and three reference spectra strategies. Results demonstrate that an LED bulb light source performed best, and all reference spectra strategies performed similarly. We then compared the second-generation prototype to a benchtop laboratory spectrophotometer to further validate the device. We applied the Beer-Lambert Law to demonstrate that our prototype is able to quantify the amount of vitamin B12 in a solution with an accuracy of up to 91.3%. Our in-depth analyses, discussions, and results demonstrate the potential use of smartphone-based spectrophotometry as an accessible method to identify and quantify nutrients and pave the way for future developments that can apply this approach to the human body.
△ Less
Submitted 15 October, 2024; v1 submitted 14 October, 2024;
originally announced October 2024.
-
Towards an Accessible, Noninvasive Micronutrient Status Assessment Method: A Comprehensive Review of Existing Techniques
Authors:
Andrew Balch,
Maria A. Cardei,
Sibylle Kranz,
Afsaneh Doryab
Abstract:
Nutrients are critical to the functioning of the human body and their imbalance can result in detrimental health concerns. The majority of nutritional literature focuses on macronutrients, often ignoring the more critical nuances of micronutrient balance, which require more precise regulation. Currently, micronutrient status is routinely assessed via complex methods that are arduous for both the p…
▽ More
Nutrients are critical to the functioning of the human body and their imbalance can result in detrimental health concerns. The majority of nutritional literature focuses on macronutrients, often ignoring the more critical nuances of micronutrient balance, which require more precise regulation. Currently, micronutrient status is routinely assessed via complex methods that are arduous for both the patient and the clinician. To address the global burden of micronutrient malnutrition, innovations in assessment must be accessible and noninvasive. In support of this task, this article synthesizes useful background information on micronutrients themselves, reviews the state of biofluid and physiological analyses for their assessment, and presents actionable opportunities to push the field forward. By taking a unique, clinical perspective that is absent from technological research on the topic, we find that the state of the art suffers from limited clinical relevance, a lack of overlap between biofluid and physiological approaches, and highly invasive and inaccessible solutions. Future work has the opportunity to maximize the impact of a novel assessment method by incorporating clinical relevance, the holistic nature of micronutrition, and prioritizing accessible and noninvasive systems.
△ Less
Submitted 7 June, 2025; v1 submitted 20 August, 2024;
originally announced August 2024.
-
Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion
Authors:
Jacob K Christopher,
Brian R Bartoldson,
Tal Ben-Nun,
Michael Cardei,
Bhavya Kailkhura,
Ferdinando Fioretto
Abstract:
Speculative decoding has emerged as a widely adopted method to accelerate large language model inference without sacrificing the quality of the model outputs. While this technique has facilitated notable speed improvements by enabling parallel sequence verification, its efficiency remains inherently limited by the reliance on incremental token generation in existing draft models. To overcome this…
▽ More
Speculative decoding has emerged as a widely adopted method to accelerate large language model inference without sacrificing the quality of the model outputs. While this technique has facilitated notable speed improvements by enabling parallel sequence verification, its efficiency remains inherently limited by the reliance on incremental token generation in existing draft models. To overcome this limitation, this paper proposes an adaptation of speculative decoding which uses discrete diffusion models to generate draft sequences. This allows parallelization of both the drafting and verification steps, providing significant speedups to the inference process. Our proposed approach, $\textit{Speculative Diffusion Decoding (SpecDiff)}$, is validated on standard language generation benchmarks and empirically demonstrated to provide up to 7.2x speedups over standard generation processes and up to 1.75x speedups over existing speculative decoding approaches.
△ Less
Submitted 10 February, 2025; v1 submitted 10 August, 2024;
originally announced August 2024.
-
ROADWork Dataset: Learning to Recognize, Observe, Analyze and Drive Through Work Zones
Authors:
Anurag Ghosh,
Robert Tamburo,
Shen Zheng,
Juan R. Alvarez-Padilla,
Hailiang Zhu,
Michael Cardei,
Nicholas Dunn,
Christoph Mertz,
Srinivasa G. Narasimhan
Abstract:
Perceiving and navigating through work zones is challenging and under-explored, even with major strides in self-driving research. An important reason is the lack of open datasets for developing new algorithms to address this long-tailed scenario. We propose the ROADWork dataset to learn how to recognize, observe and analyze and drive through work zones. We find that state-of-the-art foundation mod…
▽ More
Perceiving and navigating through work zones is challenging and under-explored, even with major strides in self-driving research. An important reason is the lack of open datasets for developing new algorithms to address this long-tailed scenario. We propose the ROADWork dataset to learn how to recognize, observe and analyze and drive through work zones. We find that state-of-the-art foundation models perform poorly on work zones. With our dataset, we improve upon detecting work zone objects (+26.2 AP), while discovering work zones with higher precision (+32.5%) at a much higher discovery rate (12.8 times), significantly improve detecting (+23.9 AP) and reading (+14.2% 1-NED) work zone signs and describing work zones (+36.7 SPICE). We also compute drivable paths from work zone navigation videos and show that it is possible to predict navigational goals and pathways such that 53.6% goals have angular error (AE) < 0.5 degrees (+9.9 %) and 75.3% pathways have AE < 0.5 degrees (+8.1 %).
△ Less
Submitted 11 June, 2024;
originally announced June 2024.
