Showing 1–12 of 12 results for author: Heiß, C

Multi-level Neural Networks for high-dimensional parametric obstacle problems

Authors: Martin Eigel, Cosmas Heiß, Janina E. Schütte

Abstract: A new method to solve computationally challenging (random) parametric obstacle problems is developed and analyzed, where the parameters can influence the related partial differential equation (PDE) and determine the position and surface structure of the obstacle. As governing equation, a stationary elliptic diffusion problem is assumed. The high-dimensional solution of the obstacle problem is appr… ▽ More A new method to solve computationally challenging (random) parametric obstacle problems is developed and analyzed, where the parameters can influence the related partial differential equation (PDE) and determine the position and surface structure of the obstacle. As governing equation, a stationary elliptic diffusion problem is assumed. The high-dimensional solution of the obstacle problem is approximated by a specifically constructed convolutional neural network (CNN). This novel algorithm is inspired by a finite element constrained multigrid algorithm to represent the parameter to solution map. This has two benefits: First, it allows for efficient practical computations since multi-level data is used as an explicit output of the NN thanks to an appropriate data preprocessing. This improves the efficacy of the training process and subsequently leads to small errors in the natural energy norm. Second, the comparison of the CNN to a multigrid algorithm provides means to carry out a complete a priori convergence and complexity analysis of the proposed NN architecture. Numerical experiments illustrate a state-of-the-art performance for this challenging problem. △ Less

Submitted 7 April, 2025; originally announced April 2025.

MSC Class: 68T07; 68T09; 35J85 ACM Class: I.2.0; I.5.2; I.5.4; G.1.8; F.1

Machine-learning for photoplethysmography analysis: Benchmarking feature, image, and signal-based approaches

Authors: Mohammad Moulaeifard, Loic Coquelin, Mantas Rinkevičius, Andrius Sološenko, Oskar Pfeffer, Ciaran Bench, Nando Hegemann, Sara Vardanega, Manasi Nandi, Jordi Alastruey, Christian Heiss, Vaidotas Marozas, Andrew Thompson, Philip J. Aston, Peter H. Charlton, Nils Strodthoff

Abstract: Photoplethysmography (PPG) is a widely used non-invasive physiological sensing technique, suitable for various clinical applications. Such clinical applications are increasingly supported by machine learning methods, raising the question of the most appropriate input representation and model choice. Comprehensive comparisons, in particular across different input representations, are scarce. We add… ▽ More Photoplethysmography (PPG) is a widely used non-invasive physiological sensing technique, suitable for various clinical applications. Such clinical applications are increasingly supported by machine learning methods, raising the question of the most appropriate input representation and model choice. Comprehensive comparisons, in particular across different input representations, are scarce. We address this gap in the research landscape by a comprehensive benchmarking study covering three kinds of input representations, interpretable features, image representations and raw waveforms, across prototypical regression and classification use cases: blood pressure and atrial fibrillation prediction. In both cases, the best results are achieved by deep neural networks operating on raw time series as input representations. Within this model class, best results are achieved by modern convolutional neural networks (CNNs). but depending on the task setup, shallow CNNs are often also very competitive. We envision that these results will be insightful for researchers to guide their choice on machine learning tasks for PPG data, even beyond the use cases presented in this work. △ Less

Submitted 27 February, 2025; originally announced February 2025.

Comments: 39 pages, 9 figures, code available at https://gitlab.com/qumphy/d1-code

TransPeakNet: Solvent-Aware 2D NMR Prediction via Multi-Task Pre-Training and Unsupervised Learning

Authors: Yunrui Li, Hao Xu, Ambrish Kumar, Duosheng Wang, Christian Heiss, Parastoo Azadi, Pengyu Hong

Abstract: Nuclear Magnetic Resonance (NMR) spectroscopy is essential for revealing molecular structure, electronic environment, and dynamics. Accurate NMR shift prediction allows researchers to validate structures by comparing predicted and observed shifts. While Machine Learning (ML) has improved one-dimensional (1D) NMR shift prediction, predicting 2D NMR remains challenging due to limited annotated data.… ▽ More Nuclear Magnetic Resonance (NMR) spectroscopy is essential for revealing molecular structure, electronic environment, and dynamics. Accurate NMR shift prediction allows researchers to validate structures by comparing predicted and observed shifts. While Machine Learning (ML) has improved one-dimensional (1D) NMR shift prediction, predicting 2D NMR remains challenging due to limited annotated data. To address this, we introduce an unsupervised training framework for predicting cross-peaks in 2D NMR, specifically Heteronuclear Single Quantum Coherence (HSQC).Our approach pretrains an ML model on an annotated 1D dataset of 1H and 13C shifts, then finetunes it in an unsupervised manner using unlabeled HSQC data, which simultaneously generates cross-peak annotations. Our model also adjusts for solvent effects. Evaluation on 479 expert-annotated HSQC spectra demonstrates our model's superiority over traditional methods (ChemDraw and Mestrenova), achieving Mean Absolute Errors (MAEs) of 2.05 ppm and 0.165 ppm for 13C shifts and 1H shifts respectively. Our algorithmic annotations show a 95.21% concordance with experts' assignments, underscoring the approach's potential for structural elucidation in fields like organic chemistry, pharmaceuticals, and natural products. △ Less

Submitted 15 December, 2024; v1 submitted 17 March, 2024; originally announced March 2024.

Multilevel CNNs for Parametric PDEs

Authors: Cosmas Heiß, Ingo Gühring, Martin Eigel

Abstract: We combine concepts from multilevel solvers for partial differential equations (PDEs) with neural network based deep learning and propose a new methodology for the efficient numerical solution of high-dimensional parametric PDEs. An in-depth theoretical analysis shows that the proposed architecture is able to approximate multigrid V-cycles to arbitrary precision with the number of weights only dep… ▽ More We combine concepts from multilevel solvers for partial differential equations (PDEs) with neural network based deep learning and propose a new methodology for the efficient numerical solution of high-dimensional parametric PDEs. An in-depth theoretical analysis shows that the proposed architecture is able to approximate multigrid V-cycles to arbitrary precision with the number of weights only depending logarithmically on the resolution of the finest mesh. As a consequence, approximation bounds for the solution of parametric PDEs by neural networks that are independent on the (stochastic) parameter dimension can be derived. The performance of the proposed method is illustrated on high-dimensional parametric linear elliptic PDEs that are common benchmark problems in uncertainty quantification. We find substantial improvements over state-of-the-art deep learning-based solvers. As particularly challenging examples, random conductivity with high-dimensional non-affine Gaussian fields in 100 parameter dimensions and a random cookie problem are examined. Due to the multilevel structure of our method, the amount of training samples can be reduced on finer levels, hence significantly lowering the generation time for training data and the training time of our method. △ Less

Submitted 4 April, 2023; v1 submitted 1 April, 2023; originally announced April 2023.

Comments: 42 pages, 5 figures, 5 tables

On Assignment Problems Related to Gromov-Wasserstein Distances on the Real Line

Authors: Robert Beinert, Cosmas Heiss, Gabriele Steidl

Abstract: Let $x_1 < \dots < x_n$ and $y_1 < \dots < y_n$, $n \in \mathbb N$, be real numbers. We show by an example that the assignment problem $$ \max_{σ\in S_n} F_σ(x,y) := \frac12 \sum_{i,k=1}^n |x_i - x_k|^α\, |y_{σ(i)} - y_{σ(k)}|^α, \quad α>0, $$ is in general neither solved by the identical permutation (id) nor the anti-identical permutation (a-id) if $n > 2 +2^α$. Indeed the above maximum can be, d… ▽ More Let $x_1 < \dots < x_n$ and $y_1 < \dots < y_n$, $n \in \mathbb N$, be real numbers. We show by an example that the assignment problem $$ \max_{σ\in S_n} F_σ(x,y) := \frac12 \sum_{i,k=1}^n |x_i - x_k|^α\, |y_{σ(i)} - y_{σ(k)}|^α, \quad α>0, $$ is in general neither solved by the identical permutation (id) nor the anti-identical permutation (a-id) if $n > 2 +2^α$. Indeed the above maximum can be, depending on the number of points, arbitrary far away from $F_\text{id}(x,y)$ and $F_\text{a-id}(x,y)$. The motivation to deal with such assignment problems came from their relation to Gromov-Wasserstein divergences which have recently attained a lot of attention. △ Less

Submitted 18 May, 2022; originally announced May 2022.

The 2020 Census Disclosure Avoidance System TopDown Algorithm

Authors: John M. Abowd, Robert Ashmead, Ryan Cumings-Menon, Simson Garfinkel, Micah Heineck, Christine Heiss, Robert Johns, Daniel Kifer, Philip Leclerc, Ashwin Machanavajjhala, Brett Moran, William Sexton, Matthew Spence, Pavel Zhuravlev

Abstract: The Census TopDown Algorithm (TDA) is a disclosure avoidance system using differential privacy for privacy-loss accounting. The algorithm ingests the final, edited version of the 2020 Census data and the final tabulation geographic definitions. The algorithm then creates noisy versions of key queries on the data, referred to as measurements, using zero-Concentrated Differential Privacy. Another ke… ▽ More The Census TopDown Algorithm (TDA) is a disclosure avoidance system using differential privacy for privacy-loss accounting. The algorithm ingests the final, edited version of the 2020 Census data and the final tabulation geographic definitions. The algorithm then creates noisy versions of key queries on the data, referred to as measurements, using zero-Concentrated Differential Privacy. Another key aspect of the TDA are invariants, statistics that the Census Bureau has determined, as matter of policy, to exclude from the privacy-loss accounting. The TDA post-processes the measurements together with the invariants to produce a Microdata Detail File (MDF) that contains one record for each person and one record for each housing unit enumerated in the 2020 Census. The MDF is passed to the 2020 Census tabulation system to produce the 2020 Census Redistricting Data (P.L. 94-171) Summary File. This paper describes the mathematics and testing of the TDA for this purpose. △ Less

Submitted 19 April, 2022; originally announced April 2022.

Lagrangian Motion Magnification with Double Sparse Optical Flow Decomposition

Authors: Philipp Flotho, Cosmas Heiss, Gabriele Steidl, Daniel J. Strauss

Abstract: Microexpressions are fast and spatially small facial expressions that are difficult to detect. Therefore motion magnification techniques, which aim at amplifying and hence revealing subtle motion in videos, appear useful for handling such expressions. There are basically two main approaches, namely via Eulerian or Lagrangian techniques. While the first one magnifies motion implicitly by operating… ▽ More Microexpressions are fast and spatially small facial expressions that are difficult to detect. Therefore motion magnification techniques, which aim at amplifying and hence revealing subtle motion in videos, appear useful for handling such expressions. There are basically two main approaches, namely via Eulerian or Lagrangian techniques. While the first one magnifies motion implicitly by operating directly on image pixels, the Lagrangian approach uses optical flow (OF) techniques to extract and magnify pixel trajectories. In this paper, we propose a novel approach for local Lagrangian motion magnification of facial micro-motions. Our contribution is three-fold: first, we fine tune the recurrent all-pairs field transforms (RAFT) for OFs deep learning approach for faces by adding ground truth obtained from the variational dense inverse search (DIS) for OF algorithm applied to the CASME II video set of facial micro expressions. This enables us to produce OFs of facial videos in an efficient and sufficiently accurate way. Second, since facial micro-motions are both local in space and time, we propose to approximate the OF field by sparse components both in space and time leading to a double sparse decomposition. Third, we use this decomposition to magnify micro-motions in specific areas of the face, where we introduce a new forward warping strategy using a triangular splitting of the image grid and barycentric interpolation of the RGB vectors at the corners of the transformed triangles. We demonstrate the feasibility of our approach by various examples. △ Less

Submitted 15 January, 2024; v1 submitted 15 April, 2022; originally announced April 2022.

OCTAVA: an open-source toolbox for quantitative analysis of optical coherence tomography angiography images

Authors: Gavrielle R. Untracht, Rolando Matos, Nikolaos Dikaios, Mariam Bapir, Abdullah K. Durrani, Teemapron Butsabong, Paola Campagnolo, David D. Sampson, Christian Heiss, Danuta M. Sampson

Abstract: Optical coherence tomography angiography (OCTA) performs non-invasive visualization and characterization of microvasculature in research and clinical applications mainly in ophthalmology and dermatology. A wide variety of instruments, imaging protocols, processing methods and metrics have been used to describe the microvasculature, such that comparing different study outcomes is currently not feas… ▽ More Optical coherence tomography angiography (OCTA) performs non-invasive visualization and characterization of microvasculature in research and clinical applications mainly in ophthalmology and dermatology. A wide variety of instruments, imaging protocols, processing methods and metrics have been used to describe the microvasculature, such that comparing different study outcomes is currently not feasible. With the goal of contributing to standardization of OCTA data analysis, we report a user-friendly, open-source toolbox, OCTAVA (OCTA Vascular Analyzer), to automate the pre-processing, segmentation, and quantitative analysis of en face OCTA maximum intensity projection images in a standardized workflow. We present each analysis step, including optimization of filtering and choice of segmentation algorithm, and definition of metrics. We perform quantitative analysis of OCTA images from different commercial and non-commercial instruments and samples and show OCTAVA can accurately and reproducibly determine metrics for characterization of microvasculature. Wide adoption could enable studies and aggregation of data on a scale sufficient to develop reliable microvascular biomarkers for early detection, and to guide treatment, of microvascular disease. △ Less

Submitted 4 September, 2021; originally announced September 2021.

Comments: 37 pages, 9 figures

Self-Supervised Equivariant Scene Synthesis from Video

Authors: Cinjon Resnick, Or Litany, Cosmas Heiß, Hugo Larochelle, Joan Bruna, Kyunghyun Cho

Abstract: We propose a self-supervised framework to learn scene representations from video that are automatically delineated into background, characters, and their animations. Our method capitalizes on moving characters being equivariant with respect to their transformation across frames and the background being constant with respect to that same transformation. After training, we can manipulate image encod… ▽ More We propose a self-supervised framework to learn scene representations from video that are automatically delineated into background, characters, and their animations. Our method capitalizes on moving characters being equivariant with respect to their transformation across frames and the background being constant with respect to that same transformation. After training, we can manipulate image encodings in real time to create unseen combinations of the delineated components. As far as we know, we are the first method to perform unsupervised extraction and synthesis of interpretable background, character, and animation. We demonstrate results on three datasets: Moving MNIST with backgrounds, 2D video game sprites, and Fashion Modeling. △ Less

Submitted 1 February, 2021; originally announced February 2021.

Comments: arXiv admin note: text overlap with arXiv:2011.05787

In-Distribution Interpretability for Challenging Modalities

Authors: Cosmas Heiß, Ron Levie, Cinjon Resnick, Gitta Kutyniok, Joan Bruna

Abstract: It is widely recognized that the predictions of deep neural networks are difficult to parse relative to simpler approaches. However, the development of methods to investigate the mode of operation of such models has advanced rapidly in the past few years. Recent work introduced an intuitive framework which utilizes generative models to improve on the meaningfulness of such explanations. In this wo… ▽ More It is widely recognized that the predictions of deep neural networks are difficult to parse relative to simpler approaches. However, the development of methods to investigate the mode of operation of such models has advanced rapidly in the past few years. Recent work introduced an intuitive framework which utilizes generative models to improve on the meaningfulness of such explanations. In this work, we display the flexibility of this method to interpret diverse and challenging modalities: music and physical simulations of urban environments. △ Less

Submitted 7 July, 2020; v1 submitted 1 July, 2020; originally announced July 2020.

Interval Neural Networks: Uncertainty Scores

Authors: Luis Oala, Cosmas Heiß, Jan Macdonald, Maximilian März, Wojciech Samek, Gitta Kutyniok

Abstract: We propose a fast, non-Bayesian method for producing uncertainty scores in the output of pre-trained deep neural networks (DNNs) using a data-driven interval propagating network. This interval neural network (INN) has interval valued parameters and propagates its input using interval arithmetic. The INN produces sensible lower and upper bounds encompassing the ground truth. We provide theoretical… ▽ More We propose a fast, non-Bayesian method for producing uncertainty scores in the output of pre-trained deep neural networks (DNNs) using a data-driven interval propagating network. This interval neural network (INN) has interval valued parameters and propagates its input using interval arithmetic. The INN produces sensible lower and upper bounds encompassing the ground truth. We provide theoretical justification for the validity of these bounds. Furthermore, its asymmetric uncertainty scores offer additional, directional information beyond what Gaussian-based, symmetric variance estimation can provide. We find that noise in the data is adequately captured by the intervals produced with our method. In numerical experiments on an image reconstruction task, we demonstrate the practical utility of INNs as a proxy for the prediction error in comparison to two state-of-the-art uncertainty quantification methods. In summary, INNs produce fast, theoretically justified uncertainty scores for DNNs that are easy to interpret, come with added information and pose as improved error proxies - features that may prove useful in advancing the usability of DNNs especially in sensitive applications such as health care. △ Less

Submitted 25 March, 2020; originally announced March 2020.

Comments: LO and CH contributed equally

ACM Class: I.5.1; I.4.5; J.3; I.2.m

A phase field approach to trabecular bone remodeling

Authors: S. Aland, F. Stenger, R. Müller, M. Kampschulte, A. C. Langheinrich, T. El Khassawna, C. Heiß, A. Dreutsch, A. Voigt

Abstract: We introduce a continuous modeling approach which combines elastic responds of the trabecular bone structure, the concentration of signaling molecules within the bone and a mechanism how this concentration at the bone surface is used for local bone formation and resorption. In an abstract setting bone can be considered as a shape changing structure. For similar problems in materials science phase… ▽ More We introduce a continuous modeling approach which combines elastic responds of the trabecular bone structure, the concentration of signaling molecules within the bone and a mechanism how this concentration at the bone surface is used for local bone formation and resorption. In an abstract setting bone can be considered as a shape changing structure. For similar problems in materials science phase field approximations have been established as an efficient computational tool. We adapt such an approach for trabecular bone remodeling. It allows for a smooth representation of the trabecular bone structure and drastically reduces computational costs if compared with traditional micro finite element approaches. We demonstrate the advantage of the approach within a minimal model. We quantitatively compare the results with established micro finite element approaches on simple geometries and consider the bone morphology within a bone segment obtained from $μ$CT data of a sheep vertebra with realistic parameters. △ Less

Submitted 20 November, 2017; originally announced November 2017.