-
Text2CT: Towards 3D CT Volume Generation from Free-text Descriptions Using Diffusion Model
Authors:
Pengfei Guo,
Can Zhao,
Dong Yang,
Yufan He,
Vishwesh Nath,
Ziyue Xu,
Pedro R. A. S. Bassi,
Zongwei Zhou,
Benjamin D. Simon,
Stephanie Anne Harmon,
Baris Turkbey,
Daguang Xu
Abstract:
Generating 3D CT volumes from descriptive free-text inputs presents a transformative opportunity in diagnostics and research. In this paper, we introduce Text2CT, a novel approach for synthesizing 3D CT volumes from textual descriptions using the diffusion model. Unlike previous methods that rely on fixed-format text input, Text2CT employs a novel prompt formulation that enables generation from di…
▽ More
Generating 3D CT volumes from descriptive free-text inputs presents a transformative opportunity in diagnostics and research. In this paper, we introduce Text2CT, a novel approach for synthesizing 3D CT volumes from textual descriptions using the diffusion model. Unlike previous methods that rely on fixed-format text input, Text2CT employs a novel prompt formulation that enables generation from diverse, free-text descriptions. The proposed framework encodes medical text into latent representations and decodes them into high-resolution 3D CT scans, effectively bridging the gap between semantic text inputs and detailed volumetric representations in a unified 3D framework. Our method demonstrates superior performance in preserving anatomical fidelity and capturing intricate structures as described in the input text. Extensive evaluations show that our approach achieves state-of-the-art results, offering promising potential applications in diagnostics, and data augmentation.
△ Less
Submitted 7 May, 2025;
originally announced May 2025.
-
Hyperspectral Image Restoration and Super-resolution with Physics-Aware Deep Learning for Biomedical Applications
Authors:
Yuchen Xiang,
Zhaolu Liu,
Monica Emili Garcia-Segura,
Daniel Simon,
Boxuan Cao,
Vincen Wu,
Kenneth Robinson,
Yu Wang,
Ronan Battle,
Robert T. Murray,
Xavier Altafaj,
Luca Peruzzotti-Jametti,
Zoltan Takats
Abstract:
Hyperspectral imaging is a powerful bioimaging tool which can uncover novel insights, thanks to its sensitivity to the intrinsic properties of materials. However, this enhanced contrast comes at the cost of system complexity, constrained by an inherent trade-off between spatial resolution, spectral resolution, and imaging speed. To overcome this limitation, we present a deep learning-based approac…
▽ More
Hyperspectral imaging is a powerful bioimaging tool which can uncover novel insights, thanks to its sensitivity to the intrinsic properties of materials. However, this enhanced contrast comes at the cost of system complexity, constrained by an inherent trade-off between spatial resolution, spectral resolution, and imaging speed. To overcome this limitation, we present a deep learning-based approach that restores and enhances pixel resolution post-acquisition without any a priori knowledge. Fine-tuned using metrics aligned with the imaging model, our physics-aware method achieves a 16X pixel super-resolution enhancement and a 12X imaging speedup without the need of additional training data for transfer learning. Applied to both synthetic and experimental data from five different sample types, we demonstrate that the model preserves biological integrity, ensuring no features are lost or hallucinated. We also concretely demonstrate the model's ability to reveal disease-associated metabolic changes in Downs syndrome that would otherwise remain undetectable. Furthermore, we provide physical insights into the inner workings of the model, paving the way for future refinements that could potentially surpass instrumental limits in an explainable manner. All methods are available as open-source software on GitHub.
△ Less
Submitted 3 March, 2025;
originally announced March 2025.
-
Towards Super-Resolution CEST MRI for Visualization of Small Structures
Authors:
Lukas Folle,
Katharian Tkotz,
Fasil Gadjimuradov,
Lorenz Kapsner,
Moritz Fabian,
Sebastian Bickelhaupt,
David Simon,
Arnd Kleyer,
Gerhard Krönke,
Moritz Zaiß,
Armin Nagel,
Andreas Maier
Abstract:
The onset of rheumatic diseases such as rheumatoid arthritis is typically subclinical, which results in challenging early detection of the disease. However, characteristic changes in the anatomy can be detected using imaging techniques such as MRI or CT. Modern imaging techniques such as chemical exchange saturation transfer (CEST) MRI drive the hope to improve early detection even further through…
▽ More
The onset of rheumatic diseases such as rheumatoid arthritis is typically subclinical, which results in challenging early detection of the disease. However, characteristic changes in the anatomy can be detected using imaging techniques such as MRI or CT. Modern imaging techniques such as chemical exchange saturation transfer (CEST) MRI drive the hope to improve early detection even further through the imaging of metabolites in the body. To image small structures in the joints of patients, typically one of the first regions where changes due to the disease occur, a high resolution for the CEST MR imaging is necessary. Currently, however, CEST MR suffers from an inherently low resolution due to the underlying physical constraints of the acquisition. In this work we compared established up-sampling techniques to neural network-based super-resolution approaches. We could show, that neural networks are able to learn the mapping from low-resolution to high-resolution unsaturated CEST images considerably better than present methods. On the test set a PSNR of 32.29dB (+10%), a NRMSE of 0.14 (+28%), and a SSIM of 0.85 (+15%) could be achieved using a ResNet neural network, improving the baseline considerably. This work paves the way for the prospective investigation of neural networks for super-resolution CEST MRI and, followingly, might lead to a earlier detection of the onset of rheumatic diseases.
△ Less
Submitted 3 December, 2021;
originally announced December 2021.
-
Tuning of Drone PD Controller Parameters for Medical Supplies Delivery
Authors:
Azin Shamshirgaran,
Hamed Javidi,
Dan Simon
Abstract:
During the COVID-19 pandemic and similar outbreaks in the future, drones can be set up to reduce human interaction for medical supplies delivery, which is crucial in times of pandemic. In this short paper, we introduce the use of two evolutionary algorithms for multi-objective optimization (MOO) and tuning the parameters of the PD controller of a drone to follow the 3D desired path.
During the COVID-19 pandemic and similar outbreaks in the future, drones can be set up to reduce human interaction for medical supplies delivery, which is crucial in times of pandemic. In this short paper, we introduce the use of two evolutionary algorithms for multi-objective optimization (MOO) and tuning the parameters of the PD controller of a drone to follow the 3D desired path.
△ Less
Submitted 22 May, 2021;
originally announced May 2021.
-
Evolutionary Algorithms for Multi-Objective Optimization of Drone Controller Parameters
Authors:
Azin Shamshirgaran,
Hamed Javidi,
Dan Simon
Abstract:
Drones are effective for reducing human activity and interactions by performing tasks such as exploring and inspecting new environments, monitoring resources and delivering packages. Drones need a controller to maintain stability and to reach their goal. The most well-known drone controllers are proportional-integral-derivative (PID) and proportional-derivative (PD) controllers. However, the contr…
▽ More
Drones are effective for reducing human activity and interactions by performing tasks such as exploring and inspecting new environments, monitoring resources and delivering packages. Drones need a controller to maintain stability and to reach their goal. The most well-known drone controllers are proportional-integral-derivative (PID) and proportional-derivative (PD) controllers. However, the controller parameters need to be tuned and optimized. In this paper, we introduce the use of two evolutionary algorithms, biogeography-based optimization~(BBO) and particle swarm optimization (PSO), for multi-objective optimization (MOO) to tune the parameters of the PD controller of a drone. The combination of MOO, BBO, and PSO results in various methods for optimization: vector evaluated BBO and PSO, denoted as VEBBO and VEPSO; and non-dominated sorting BBO and PSO, denoted as NSBBO and NSPSO. The multi-objective cost function is based on tracking errors for the four states of the system. Two criteria for evaluating the Pareto fronts of the optimization methods, normalized hypervolume and relative coverage, are used to compare performance. Results show that NSBBO generally performs better than the other methods.
△ Less
Submitted 18 May, 2021;
originally announced May 2021.
-
A Generalized Unscented Transformation for Probability Distributions
Authors:
Donald Ebeigbe,
Tyrus Berry,
Michael M. Norton,
Andrew J. Whalen,
Dan Simon,
Timothy Sauer,
Steven J. Schiff
Abstract:
The unscented transform uses a weighted set of samples called sigma points to propagate the means and covariances of nonlinear transformations of random variables. However, unscented transforms developed using either the Gaussian assumption or a minimum set of sigma points typically fall short when the random variable is not Gaussian distributed and the nonlinearities are substantial. In this pape…
▽ More
The unscented transform uses a weighted set of samples called sigma points to propagate the means and covariances of nonlinear transformations of random variables. However, unscented transforms developed using either the Gaussian assumption or a minimum set of sigma points typically fall short when the random variable is not Gaussian distributed and the nonlinearities are substantial. In this paper, we develop the generalized unscented transform (GenUT), which uses 2n+1 sigma points to accurately capture up to the diagonal components of the skewness and kurtosis tensors of most probability distributions. Constraints can be analytically enforced on the sigma points while guaranteeing at least second-order accuracy. The GenUT uses the same number of sigma points as the original unscented transform while also being applicable to non-Gaussian distributions, including the assimilation of observations in the modeling of infectious diseases such as coronavirus (SARS-CoV-2) causing COVID-19.
△ Less
Submitted 15 November, 2021; v1 submitted 5 April, 2021;
originally announced April 2021.
-
A multi-objective optimization framework for on-line ridesharing systems
Authors:
Hamed Javidi,
Dan Simon,
Ling Zhu,
Yan Wang
Abstract:
The ultimate goal of ridesharing systems is to matchtravelers who do not have a vehicle with those travelers whowant to share their vehicle. A good match can be found amongthose who have similar itineraries and time schedules. In thisway each rider can be served without any delay and also eachdriver can earn as much as possible without having too muchdeviation from their original route. We propose…
▽ More
The ultimate goal of ridesharing systems is to matchtravelers who do not have a vehicle with those travelers whowant to share their vehicle. A good match can be found amongthose who have similar itineraries and time schedules. In thisway each rider can be served without any delay and also eachdriver can earn as much as possible without having too muchdeviation from their original route. We propose an algorithmthat leverages biogeography-based optimization to solve a multi-objective optimization problem for online ridesharing. It isnecessary to solve the ridesharing problem as a multi-objectiveproblem since there are some important objectives that must beconsidered simultaneously. We test our algorithm by evaluatingperformance on the Beijing ridesharing dataset. The simulationresults indicate that BBO provides competitive performancerelative to state-of-the-art ridesharing optimization algorithms.
△ Less
Submitted 7 December, 2020;
originally announced December 2020.
-
Learned Greedy Method (LGM): A Novel Neural Architecture for Sparse Coding and Beyond
Authors:
Rajaei Khatib,
Dror Simon,
Michael Elad
Abstract:
The fields of signal and image processing have been deeply influenced by the introduction of deep neural networks. These are successfully deployed in a wide range of real-world applications, obtaining state of the art results and surpassing well-known and well-established classical methods. Despite their impressive success, the architectures used in many of these neural networks come with no clear…
▽ More
The fields of signal and image processing have been deeply influenced by the introduction of deep neural networks. These are successfully deployed in a wide range of real-world applications, obtaining state of the art results and surpassing well-known and well-established classical methods. Despite their impressive success, the architectures used in many of these neural networks come with no clear justification. As such, these are usually treated as "black box" machines that lack any kind of interpretability. A constructive remedy to this drawback is a systematic design of such networks by unfolding well-understood iterative algorithms. A popular representative of this approach is the Iterative Shrinkage-Thresholding Algorithm (ISTA) and its learned version -- LISTA, aiming for the sparse representations of the processed signals. In this paper we revisit this sparse coding task and propose an unfolded version of a greedy pursuit algorithm for the same goal. More specifically, we concentrate on the well-known Orthogonal-Matching-Pursuit (OMP) algorithm, and introduce its unfolded and learned version. Key features of our Learned Greedy Method (LGM) are the ability to accommodate a dynamic number of unfolded layers, and a stopping mechanism based on representation error, both adapted to the input. We develop several variants of the proposed LGM architecture and test some of them in various experiments, demonstrating their flexibility and efficiency.
△ Less
Submitted 20 October, 2020; v1 submitted 14 October, 2020;
originally announced October 2020.
-
Rethinking the CSC Model for Natural Images
Authors:
Dror Simon,
Michael Elad
Abstract:
Sparse representation with respect to an overcomplete dictionary is often used when regularizing inverse problems in signal and image processing. In recent years, the Convolutional Sparse Coding (CSC) model, in which the dictionary consists of shift-invariant filters, has gained renewed interest. While this model has been successfully used in some image processing problems, it still falls behind t…
▽ More
Sparse representation with respect to an overcomplete dictionary is often used when regularizing inverse problems in signal and image processing. In recent years, the Convolutional Sparse Coding (CSC) model, in which the dictionary consists of shift-invariant filters, has gained renewed interest. While this model has been successfully used in some image processing problems, it still falls behind traditional patch-based methods on simple tasks such as denoising.
In this work we provide new insights regarding the CSC model and its capability to represent natural images, and suggest a Bayesian connection between this model and its patch-based ancestor. Armed with these observations, we suggest a novel feed-forward network that follows an MMSE approximation process to the CSC model, using strided convolutions. The performance of this supervised architecture is shown to be on par with state of the art methods while using much fewer parameters.
△ Less
Submitted 12 September, 2019;
originally announced September 2019.
-
MMSE Approximation For Sparse Coding Algorithms Using Stochastic Resonance
Authors:
Dror Simon,
Jeremias Sulam,
Yaniv Romano,
Yue M. Lu,
Michael Elad
Abstract:
Sparse coding refers to the pursuit of the sparsest representation of a signal in a typically overcomplete dictionary. From a Bayesian perspective, sparse coding provides a Maximum a Posteriori (MAP) estimate of the unknown vector under a sparse prior. In this work, we suggest enhancing the performance of sparse coding algorithms by a deliberate and controlled contamination of the input with rando…
▽ More
Sparse coding refers to the pursuit of the sparsest representation of a signal in a typically overcomplete dictionary. From a Bayesian perspective, sparse coding provides a Maximum a Posteriori (MAP) estimate of the unknown vector under a sparse prior. In this work, we suggest enhancing the performance of sparse coding algorithms by a deliberate and controlled contamination of the input with random noise, a phenomenon known as stochastic resonance. The proposed method adds controlled noise to the input and estimates a sparse representation from the perturbed signal. A set of such solutions is then obtained by projecting the original input signal onto the recovered set of supports. We present two variants of the described method, which differ in their final step. The first is a provably convergent approximation to the Minimum Mean Square Error (MMSE) estimator, relying on the generative model and applying a weighted average over the recovered solutions. The second is a relaxed variant of the former that simply applies an empirical mean. We show that both methods provide a computationally efficient approximation to the MMSE estimator, which is typically intractable to compute. We demonstrate our findings empirically and provide a theoretical analysis of our method under several different cases.
△ Less
Submitted 11 April, 2019; v1 submitted 26 June, 2018;
originally announced June 2018.
-
State Estimation For An Agonistic-Antagonistic Muscle System
Authors:
Thang Nguyen,
Holly Warner,
Hung La,
Hanieh Mohammadi,
Dan Simon,
Hanz Richter
Abstract:
Research on assistive technology, rehabilitation, and prosthesis requires the understanding of human machine interaction, in which human muscular properties play a pivotal role. This paper studies a nonlinear agonistic-antagonistic muscle system based on the Hill muscle model. To investigate the characteristics of the muscle model, the problem of estimating the state variables and activation signa…
▽ More
Research on assistive technology, rehabilitation, and prosthesis requires the understanding of human machine interaction, in which human muscular properties play a pivotal role. This paper studies a nonlinear agonistic-antagonistic muscle system based on the Hill muscle model. To investigate the characteristics of the muscle model, the problem of estimating the state variables and activation signals of the dual muscle system is considered. In this work, parameter uncertainty and unknown inputs are taken into account for the estimation problem. Three observers are presented: a high gain observer, a sliding mode observer, and an adaptive sliding mode observer. Theoretical analysis shows the convergence of the three observers. To facilitate numerical simulations, a backstepping controller is employed to drive the muscle system to track a desired trajectory. Numerical simulations reveal that the three observers are comparable and provide reliable estimates in noise free and noisy cases. The proposed schemes may serve as frameworks for estimation of complex multi-muscle systems, which could lead to intelligent exercise machines for adaptive training and rehabilitation, and adaptive prosthetics and exoskeletons.
△ Less
Submitted 1 December, 2017;
originally announced December 2017.
-
CHOPtrey: contextual online polynomial extrapolation for enhanced multi-core co-simulation of complex systems
Authors:
Abir Ben Khaled-El Feki,
Laurent Duval,
Cyril Faure,
Daniel Simon,
Mongi Ben Gaid
Abstract:
The growing complexity of Cyber-Physical Systems (CPS), together with increasingly available parallelism provided by multi-core chips, fosters the parallelization of simulation. Simulation speed-ups are expected from co-simulation and parallelization based on model splitting into weak-coupled sub-models, as for instance in the framework of Functional Mockup Interface (FMI). However, slackened sync…
▽ More
The growing complexity of Cyber-Physical Systems (CPS), together with increasingly available parallelism provided by multi-core chips, fosters the parallelization of simulation. Simulation speed-ups are expected from co-simulation and parallelization based on model splitting into weak-coupled sub-models, as for instance in the framework of Functional Mockup Interface (FMI). However, slackened synchronization between sub-models and their associated solvers running in parallel introduces integration errors, which must be kept inside acceptable bounds.
CHOPtrey denotes a forecasting framework enhancing the performance of complex system co-simulation, with a trivalent articulation. First, we consider the framework of a Computationally Hasty Online Prediction system (CHOPred). It allows to improve the trade-off between integration speed-ups, needing large communication steps, and simulation precision, needing frequent updates for model inputs. Second, smoothed adaptive forward prediction improves co-simulation accuracy. It is obtained by past-weighted extrapolation based on Causal Hopping Oblivious Polynomials (CHOPoly). And third, signal behavior is segmented to handle the discontinuities of the exchanged signals: the segmentation is performed in a Contextual \& Hierarchical Ontology of Patterns (CHOPatt).
Implementation strategies and simulation results demonstrate the framework ability to adaptively relax data communication constraints beyond synchronization points which sensibly accelerate simulation. The CHOPtrey framework extends the range of applications of standard Lagrange-type methods, often deemed unstable. The embedding of predictions in lag-dependent smoothing and discontinuity handling demonstrates its practical efficiency.
△ Less
Submitted 4 February, 2017; v1 submitted 28 October, 2016;
originally announced October 2016.
-
Robust MRAC augmentation of flight control laws for center of gravity adaptation
Authors:
Daniel Simon
Abstract:
When an aircraft is flying and burning fuel the center of gravity (c.g.) of the aircraft shifts slowly. The c.g. can also be shifted abruptly when e.g. a fighter aircraft releases a weapon. The shift in c.g. is difficult to measure or estimate so the flight control systems need to be robustly designed to cope with this variation. However for fighter aircrafts with high manoeuvrability there is roo…
▽ More
When an aircraft is flying and burning fuel the center of gravity (c.g.) of the aircraft shifts slowly. The c.g. can also be shifted abruptly when e.g. a fighter aircraft releases a weapon. The shift in c.g. is difficult to measure or estimate so the flight control systems need to be robustly designed to cope with this variation. However for fighter aircrafts with high manoeuvrability there is room for improvements. In this project we investigate if the use of adaptive control law augmentation can be used to better cope with the change in c.g. We augment a baseline controller with a robust Model Reference Adaptive Control (MRAC) design and analyse its benefits and possible issues.
△ Less
Submitted 7 April, 2016;
originally announced April 2016.
-
Stability analysis of Model Predictive Controllers using Mixed Integer Linear Programming
Authors:
Daniel Simon,
Johan Löfberg
Abstract:
It is a well known fact that finite time optimal controllers, such as MPC does not necessarily result in closed loop stable systems. Within the MPC community it is common practice to add a final state constraint and/or a final state penalty in order to obtain guaranteed stability. However, for more advanced controller structures it can be difficult to show stability using these techniques. Additio…
▽ More
It is a well known fact that finite time optimal controllers, such as MPC does not necessarily result in closed loop stable systems. Within the MPC community it is common practice to add a final state constraint and/or a final state penalty in order to obtain guaranteed stability. However, for more advanced controller structures it can be difficult to show stability using these techniques. Additionally in some cases the final state constraint set consists of so many inequalities that the complexity of the MPC problem is too big for use in certain fast and time critical applications. In this paper we instead focus on deriving a tool for a-postiori analysis of the closed loop stability for linear systems controlled with MPC controllers. We formulate an optimisation problem that gives a sufficient condition for stability of the closed loop system and we show that the problem can be written as a Mixed Integer Linear Programming Problem (MILP)
△ Less
Submitted 4 April, 2016;
originally announced April 2016.
