-
Limeade: Let integer molecular encoding aid
Authors:
Shiqiang Zhang,
Christian W. Feldmann,
Frederik Sandfort,
Miriam Mathea,
Juan S. Campos,
Ruth Misener
Abstract:
Mixed-integer programming (MIP) is a well-established framework for computer-aided molecular design (CAMD). By precisely encoding the molecular space and score functions, e.g., a graph neural network, the molecular design problem is represented and solved as an optimization problem, the solution of which corresponds to a molecule with optimal score. However, both the extremely large search space a…
▽ More
Mixed-integer programming (MIP) is a well-established framework for computer-aided molecular design (CAMD). By precisely encoding the molecular space and score functions, e.g., a graph neural network, the molecular design problem is represented and solved as an optimization problem, the solution of which corresponds to a molecule with optimal score. However, both the extremely large search space and complicated scoring process limit the use of MIP-based CAMD to specific and tiny problems. Moreover, optimal molecule may not be meaningful in practice if scores are imperfect. Instead of pursuing optimality, this paper exploits the ability of MIP in molecular generation and proposes Limeade as an end-to-end tool from real-world needs to feasible molecules. Beyond the basic constraints for structural feasibility, Limeade supports inclusion and exclusion of SMARTS patterns, automating the process of interpreting and formulating chemical requirements to mathematical constraints.
△ Less
Submitted 25 November, 2024;
originally announced November 2024.
-
NeRFmentation: NeRF-based Augmentation for Monocular Depth Estimation
Authors:
Casimir Feldmann,
Niall Siegenheim,
Nikolas Hars,
Lovro Rabuzin,
Mert Ertugrul,
Luca Wolfart,
Marc Pollefeys,
Zuria Bauer,
Martin R. Oswald
Abstract:
The capabilities of monocular depth estimation (MDE) models are limited by the availability of sufficient and diverse datasets. In the case of MDE models for autonomous driving, this issue is exacerbated by the linearity of the captured data trajectories. We propose a NeRF-based data augmentation pipeline to introduce synthetic data with more diverse viewing directions into training datasets and d…
▽ More
The capabilities of monocular depth estimation (MDE) models are limited by the availability of sufficient and diverse datasets. In the case of MDE models for autonomous driving, this issue is exacerbated by the linearity of the captured data trajectories. We propose a NeRF-based data augmentation pipeline to introduce synthetic data with more diverse viewing directions into training datasets and demonstrate the benefits of our approach to model performance and robustness. Our data augmentation pipeline, which we call \textit{NeRFmentation}, trains NeRFs on each scene in a dataset, filters out subpar NeRFs based on relevant metrics, and uses them to generate synthetic RGB-D images captured from new viewing directions. In this work, we apply our technique in conjunction with three state-of-the-art MDE architectures on the popular autonomous driving dataset, KITTI, augmenting its training set of the Eigen split. We evaluate the resulting performance gain on the original test set, a separate popular driving dataset, and our own synthetic test set.
△ Less
Submitted 15 September, 2024; v1 submitted 8 January, 2024;
originally announced January 2024.
-
Inference on the state process of periodically inhomogeneous hidden Markov models for animal behavior
Authors:
Jan-Ole Koslik,
Carlina C. Feldmann,
Sina Mews,
Rouven Michels,
Roland Langrock
Abstract:
Over the last decade, hidden Markov models (HMMs) have become increasingly popular in statistical ecology, where they constitute natural tools for studying animal behavior based on complex sensor data. Corresponding analyses sometimes explicitly focus on - and in any case need to take into account - periodic variation, for example by quantifying the activity distribution over the daily cycle or se…
▽ More
Over the last decade, hidden Markov models (HMMs) have become increasingly popular in statistical ecology, where they constitute natural tools for studying animal behavior based on complex sensor data. Corresponding analyses sometimes explicitly focus on - and in any case need to take into account - periodic variation, for example by quantifying the activity distribution over the daily cycle or seasonal variation such as migratory behavior. For HMMs including periodic components, we establish important mathematical properties that allow for comprehensive statistical inference related to periodic variation, thereby also providing guidance for model building and model checking. Specifically, we derive the periodically varying unconditional state distribution as well as the time-varying and overall state dwell-time distributions - all of which are of key interest when the inferential focus lies on the dynamics of the state process. We use the associated novel inference and model-checking tools to investigate changes in the diel activity patterns of fruit flies in response to changing light conditions.
△ Less
Submitted 22 December, 2023;
originally announced December 2023.
-
Augmenting optimization-based molecular design with graph neural networks
Authors:
Shiqiang Zhang,
Juan S. Campos,
Christian Feldmann,
Frederik Sandfort,
Miriam Mathea,
Ruth Misener
Abstract:
Computer-aided molecular design (CAMD) studies quantitative structure-property relationships and discovers desired molecules using optimization algorithms. With the emergence of machine learning models, CAMD score functions may be replaced by various surrogates to automatically learn the structure-property relationships. Due to their outstanding performance on graph domains, graph neural networks…
▽ More
Computer-aided molecular design (CAMD) studies quantitative structure-property relationships and discovers desired molecules using optimization algorithms. With the emergence of machine learning models, CAMD score functions may be replaced by various surrogates to automatically learn the structure-property relationships. Due to their outstanding performance on graph domains, graph neural networks (GNNs) have recently appeared frequently in CAMD. But using GNNs introduces new optimization challenges. This paper formulates GNNs using mixed-integer programming and then integrates this GNN formulation into the optimization and machine learning toolkit OMLT. To characterize and formulate molecules, we inherit the well-established mixed-integer optimization formulation for CAMD and propose symmetry-breaking constraints to remove symmetric solutions caused by graph isomorphism. In two case studies, we investigate fragment-based odorant molecular design with more practical requirements to test the compatibility and performance of our approaches.
△ Less
Submitted 6 December, 2023;
originally announced December 2023.
-
Optimizing over trained GNNs via symmetry breaking
Authors:
Shiqiang Zhang,
Juan S. Campos,
Christian Feldmann,
David Walz,
Frederik Sandfort,
Miriam Mathea,
Calvin Tsay,
Ruth Misener
Abstract:
Optimization over trained machine learning models has applications including: verification, minimizing neural acquisition functions, and integrating a trained surrogate into a larger decision-making problem. This paper formulates and solves optimization problems constrained by trained graph neural networks (GNNs). To circumvent the symmetry issue caused by graph isomorphism, we propose two types o…
▽ More
Optimization over trained machine learning models has applications including: verification, minimizing neural acquisition functions, and integrating a trained surrogate into a larger decision-making problem. This paper formulates and solves optimization problems constrained by trained graph neural networks (GNNs). To circumvent the symmetry issue caused by graph isomorphism, we propose two types of symmetry-breaking constraints: one indexing a node 0 and one indexing the remaining nodes by lexicographically ordering their neighbor sets. To guarantee that adding these constraints will not remove all symmetric solutions, we construct a graph indexing algorithm and prove that the resulting graph indexing satisfies the proposed symmetry-breaking constraints. For the classical GNN architectures considered in this paper, optimizing over a GNN with a fixed graph is equivalent to optimizing over a dense neural network. Thus, we study the case where the input graph is not fixed, implying that each edge is a decision variable, and develop two mixed-integer optimization formulations. To test our symmetry-breaking strategies and optimization formulations, we consider an application in molecular design.
△ Less
Submitted 12 October, 2023; v1 submitted 16 May, 2023;
originally announced May 2023.
-
Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming
Authors:
Vignesh V Menon,
Christian Feldmann,
Klaus Schoeffmann,
Mohammad Ghanbari,
Christian Timmerer
Abstract:
For adaptive streaming applications, low-complexity and accurate video complexity features are necessary to analyze the video content in real time, which ensures fast and compression-efficient video streaming without disruptions. State-of-the-art video complexity features are Spatial Information (SI) and Temporal Information (TI) features which do not correlate well with the encoding parameters in…
▽ More
For adaptive streaming applications, low-complexity and accurate video complexity features are necessary to analyze the video content in real time, which ensures fast and compression-efficient video streaming without disruptions. State-of-the-art video complexity features are Spatial Information (SI) and Temporal Information (TI) features which do not correlate well with the encoding parameters in adaptive streaming applications. To this light, Video Complexity Analyzer (VCA) was introduced, determining the features based on Discrete Cosine Transform (DCT)-energy. This paper presents optimizations on VCA for faster and energy-efficient video complexity analysis. Experimental results show that VCA v2.0, using eight CPU threads, Single Instruction Multiple Data (SIMD), and low-pass DCT optimization, determines seven complexity features of Ultra High Definition 8-bit videos with better accuracy at a speed of up to 292.68 fps and an energy consumption of 97.06% lower than the reference SITI implementation.
△ Less
Submitted 24 April, 2023;
originally announced April 2023.
-
Fast multi-encoding to reduce the cost of video streaming
Authors:
Hadi Amirpour,
Vignesh V Menon,
Ekrem Çetinkaya,
Adithyan Ilangovan,
Christian Feldmann,
Martin Smole,
Christian Timmerer
Abstract:
The growth in video Internet traffic and advancements in video attributes such as framerate, resolution, and bit-depth boost the demand to devise a large-scale, highly efficient video encoding environment. This is even more essential for Dynamic Adaptive Streaming over HTTP (DASH)-based content provisioning as it requires encoding numerous representations of the same video content. High Efficiency…
▽ More
The growth in video Internet traffic and advancements in video attributes such as framerate, resolution, and bit-depth boost the demand to devise a large-scale, highly efficient video encoding environment. This is even more essential for Dynamic Adaptive Streaming over HTTP (DASH)-based content provisioning as it requires encoding numerous representations of the same video content. High Efficiency Video Coding (HEVC) is one standard video codec that significantly improves encoding efficiency over its predecessor Advanced Video Coding (AVC). This improvement is achieved at the expense of significantly increased time complexity, which is a challenge for content and service providers. As various representations are the same video content encoded at different bitrates or resolutions, the encoding analysis information from the already encoded representations can be shared to accelerate the encoding of other representations. Several state-of-the-art schemes first encode a single representation, called a reference representation. During this encoding, the encoder creates analysis metadata with information such as the slicetype decisions, CU, PU, TU partitioning, and the HEVC bitstream itself. The remaining representations, called dependent representations, analyze the above metadata and then reuse it to skip searching some partitioning, thus, reducing the computational complexity. With the emergence of cloud-based encoding services, video encoding is accelerated by utilizing an increased number of resources, i.e., with multi-core CPUs, multiple representations can be encoded in parallel. This paper presents an overview of a wide range of multi-encoding schemes with and without the support of machine learning approaches integrated into the HEVC Test Model (HM) and x265, respectively.
△ Less
Submitted 25 October, 2022;
originally announced October 2022.
-
A Testbed for Investigation of Selective Laser Melting at Elevated Atmospheric Pressure
Authors:
David A. Griggs,
Jonathan S. Gibbs,
Stuart P. Baker,
Ryan W. Penny,
Martin C. Feldmann,
A. John Hart
Abstract:
Metal additive manufacturing (AM) by laser powder bed fusion (L-PBF) builds upon fundamentals established in the field of laser welding which include the influence of gas and plume dynamics on weld depth and quality. L-PBF demands a thorough investigation of the complex thermophysical phenomena that occur where the laser interacts with the metal powder bed. In particular, melt pool turbulence and…
▽ More
Metal additive manufacturing (AM) by laser powder bed fusion (L-PBF) builds upon fundamentals established in the field of laser welding which include the influence of gas and plume dynamics on weld depth and quality. L-PBF demands a thorough investigation of the complex thermophysical phenomena that occur where the laser interacts with the metal powder bed. In particular, melt pool turbulence and evaporation are influenced by the ambient gas chemistry and pressure. This paper presents the design and validation of high pressure laser melting (HPLM) testbed; this accommodates bare metal plate samples as well as manually-coated single powder layers, and operates at up to 300 psig. The open architecture of this testbed allows for full control of all relevant laser parameters in addition to ambient gas pressure and gas flow over the build area. Representative melt tracks and rasters on bare plate and powder are examined in order to validate system performance, and preliminary analysis concludes that pressure has a significant impact on melt pool aspect ratio. The HPLM system thus enables careful study pressure effects on processing of common L-PBF materials, and can be applied in the future to materials that are challenging to process under ambient pressure, such as those with high vapor pressures.
△ Less
Submitted 4 July, 2021;
originally announced July 2021.
-
Surgical Data Science: A Consensus Perspective
Authors:
Lena Maier-Hein,
Matthias Eisenmann,
Carolin Feldmann,
Hubertus Feussner,
Germain Forestier,
Stamatia Giannarou,
Bernard Gibaud,
Gregory D. Hager,
Makoto Hashizume,
Darko Katic,
Hannes Kenngott,
Ron Kikinis,
Michael Kranzfelder,
Anand Malpani,
Keno März,
Beat Müuller-Stich,
Nassir Navab,
Thomas Neumuth,
Nicolas Padoy,
Adrian Park,
Carla Pugh,
Nicolai Schoch,
Danail Stoyanov,
Russell Taylor,
Martin Wagner
, et al. (3 additional authors not shown)
Abstract:
Surgical data science is a scientific discipline with the objective of improving the quality of interventional healthcare and its value through capturing, organization, analysis, and modeling of data. The goal of the 1st workshop on Surgical Data Science was to bring together researchers working on diverse topics in surgical data science in order to discuss existing challenges, potential standards…
▽ More
Surgical data science is a scientific discipline with the objective of improving the quality of interventional healthcare and its value through capturing, organization, analysis, and modeling of data. The goal of the 1st workshop on Surgical Data Science was to bring together researchers working on diverse topics in surgical data science in order to discuss existing challenges, potential standards and new research directions in the field. Inspired by current open space and think tank formats, it was organized in June 2016 in Heidelberg. While the first day of the workshop, which was dominated by interactive sessions, was open to the public, the second day was reserved for a board meeting on which the information gathered on the public day was processed by (1) discussing remaining open issues, (2) deriving a joint definition for surgical data science and (3) proposing potential strategies for advancing the field. This document summarizes the key findings.
△ Less
Submitted 8 June, 2018;
originally announced June 2018.
-
Why rankings of biomedical image analysis competitions should be interpreted with care
Authors:
Lena Maier-Hein,
Matthias Eisenmann,
Annika Reinke,
Sinan Onogur,
Marko Stankovic,
Patrick Scholz,
Tal Arbel,
Hrvoje Bogunovic,
Andrew P. Bradley,
Aaron Carass,
Carolin Feldmann,
Alejandro F. Frangi,
Peter M. Full,
Bram van Ginneken,
Allan Hanbury,
Katrin Honauer,
Michal Kozubek,
Bennett A. Landman,
Keno März,
Oskar Maier,
Klaus Maier-Hein,
Bjoern H. Menze,
Henning Müller,
Peter F. Neher,
Wiro Niessen
, et al. (13 additional authors not shown)
Abstract:
International challenges have become the standard for validation of biomedical image analysis methods. Given their scientific impact, it is surprising that a critical analysis of common practices related to the organization of challenges has not yet been performed. In this paper, we present a comprehensive analysis of biomedical image analysis challenges conducted up to now. We demonstrate the imp…
▽ More
International challenges have become the standard for validation of biomedical image analysis methods. Given their scientific impact, it is surprising that a critical analysis of common practices related to the organization of challenges has not yet been performed. In this paper, we present a comprehensive analysis of biomedical image analysis challenges conducted up to now. We demonstrate the importance of challenges and show that the lack of quality control has critical consequences. First, reproducibility and interpretation of the results is often hampered as only a fraction of relevant information is typically provided. Second, the rank of an algorithm is generally not robust to a number of variables such as the test data used for validation, the ranking scheme applied and the observers that make the reference annotations. To overcome these problems, we recommend best practice guidelines and define open research questions to be addressed in the future.
△ Less
Submitted 18 September, 2019; v1 submitted 6 June, 2018;
originally announced June 2018.
-
Multi-Codec DASH Dataset
Authors:
Anatoliy Zabrovskiy,
Christian Feldmann,
Christian Timmerer
Abstract:
The number of bandwidth-hungry applications and services is constantly growing. HTTP adaptive streaming of audio-visual content accounts for the majority of today's internet traffic. Although the internet bandwidth increases also constantly, audio-visual compression technology is inevitable and we are currently facing the challenge to be confronted with multiple video codecs. This paper proposes a…
▽ More
The number of bandwidth-hungry applications and services is constantly growing. HTTP adaptive streaming of audio-visual content accounts for the majority of today's internet traffic. Although the internet bandwidth increases also constantly, audio-visual compression technology is inevitable and we are currently facing the challenge to be confronted with multiple video codecs. This paper proposes a multi-codec DASH dataset comprising AVC, HEVC, VP9, and AV1 in order to enable interoperability testing and streaming experiments for the efficient usage of these codecs under various conditions. We adopt state of the art encoding and packaging options and also provide basic quality metrics along with the DASH segments. Additionally, we briefly introduce a multi-codec DASH scheme and possible usage scenarios. Finally, we provide a preliminary evaluation of the encoding efficiency in the context of HTTP adaptive streaming services and applications.
△ Less
Submitted 19 March, 2018;
originally announced March 2018.
