Showing 1–8 of 8 results for author: Lehmann, M

Securing RAG: A Risk Assessment and Mitigation Framework

Authors: Lukas Ammann, Sara Ott, Christoph R. Landolt, Marco P. Lehmann

Abstract: Retrieval Augmented Generation (RAG) has emerged as the de facto industry standard for user-facing NLP applications, offering the ability to integrate data without re-training or fine-tuning Large Language Models (LLMs). This capability enhances the quality and accuracy of responses but also introduces novel security and privacy challenges, particularly when sensitive data is integrated. With the… ▽ More Retrieval Augmented Generation (RAG) has emerged as the de facto industry standard for user-facing NLP applications, offering the ability to integrate data without re-training or fine-tuning Large Language Models (LLMs). This capability enhances the quality and accuracy of responses but also introduces novel security and privacy challenges, particularly when sensitive data is integrated. With the rapid adoption of RAG, securing data and services has become a critical priority. This paper first reviews the vulnerabilities of RAG pipelines, and outlines the attack surface from data pre-processing and data storage management to integration with LLMs. The identified risks are then paired with corresponding mitigations in a structured overview. In a second step, the paper develops a framework that combines RAG-specific security considerations, with existing general security guidelines, industry standards, and best practices. The proposed framework aims to guide the implementation of robust, compliant, secure, and trustworthy RAG systems. △ Less

Submitted 21 May, 2025; v1 submitted 13 May, 2025; originally announced May 2025.

Comments: 8 pages, 3 figures, Sara Ott and Lukas Ammann contributed equally. This work has been submitted to the IEEE for possible publication

Enabling Asymmetric Knowledge Transfer in Multi-Task Learning with Self-Auxiliaries

Authors: Olivier Graffeuille, Yun Sing Koh, Joerg Wicker, Moritz Lehmann

Abstract: Knowledge transfer in multi-task learning is typically viewed as a dichotomy; positive transfer, which improves the performance of all tasks, or negative transfer, which hinders the performance of all tasks. In this paper, we investigate the understudied problem of asymmetric task relationships, where knowledge transfer aids the learning of certain tasks while hindering the learning of others. We… ▽ More Knowledge transfer in multi-task learning is typically viewed as a dichotomy; positive transfer, which improves the performance of all tasks, or negative transfer, which hinders the performance of all tasks. In this paper, we investigate the understudied problem of asymmetric task relationships, where knowledge transfer aids the learning of certain tasks while hindering the learning of others. We propose an optimisation strategy that includes additional cloned tasks named self-auxiliaries into the learning process to flexibly transfer knowledge between tasks asymmetrically. Our method can exploit asymmetric task relationships, benefiting from the positive transfer component while avoiding the negative transfer component. We demonstrate that asymmetric knowledge transfer provides substantial improvements in performance compared to existing multi-task optimisation strategies on benchmark computer vision problems. △ Less

Submitted 21 October, 2024; originally announced October 2024.

MultiOrg: A Multi-rater Organoid-detection Dataset

Authors: Christina Bukas, Harshavardhan Subramanian, Fenja See, Carina Steinchen, Ivan Ezhov, Gowtham Boosarpu, Sara Asgharpour, Gerald Burgstaller, Mareike Lehmann, Florian Kofler, Marie Piraud

Abstract: High-throughput image analysis in the biomedical domain has gained significant attention in recent years, driving advancements in drug discovery, disease prediction, and personalized medicine. Organoids, specifically, are an active area of research, providing excellent models for human organs and their functions. Automating the quantification of organoids in microscopy images would provide an effe… ▽ More High-throughput image analysis in the biomedical domain has gained significant attention in recent years, driving advancements in drug discovery, disease prediction, and personalized medicine. Organoids, specifically, are an active area of research, providing excellent models for human organs and their functions. Automating the quantification of organoids in microscopy images would provide an effective solution to overcome substantial manual quantification bottlenecks, particularly in high-throughput image analysis. However, there is a notable lack of open biomedical datasets, in contrast to other domains, such as autonomous driving, and, notably, only few of them have attempted to quantify annotation uncertainty. In this work, we present MultiOrg a comprehensive organoid dataset tailored for object detection tasks with uncertainty quantification. This dataset comprises over 400 high-resolution 2d microscopy images and curated annotations of more than 60,000 organoids. Most importantly, it includes three label sets for the test data, independently annotated by two experts at distinct time points. We additionally provide a benchmark for organoid detection, and make the best model available through an easily installable, interactive plugin for the popular image visualization tool Napari, to perform organoid quantification. △ Less

Submitted 11 December, 2024; v1 submitted 18 October, 2024; originally announced October 2024.

AI Meets the Classroom: When Do Large Language Models Harm Learning?

Authors: Matthias Lehmann, Philipp B. Cornelius, Fabian J. Sting

Abstract: The effect of large language models (LLMs) in education is debated: Previous research shows that LLMs can help as well as hurt learning. In two pre-registered and incentivized laboratory experiments, we find no effect of LLMs on overall learning outcomes. In exploratory analyses and a field study, we provide evidence that the effect of LLMs on learning outcomes depends on usage behavior. Students… ▽ More The effect of large language models (LLMs) in education is debated: Previous research shows that LLMs can help as well as hurt learning. In two pre-registered and incentivized laboratory experiments, we find no effect of LLMs on overall learning outcomes. In exploratory analyses and a field study, we provide evidence that the effect of LLMs on learning outcomes depends on usage behavior. Students who substitute some of their learning activities with LLMs (e.g., by generating solutions to exercises) increase the volume of topics they can learn about but decrease their understanding of each topic. Students who complement their learning activities with LLMs (e.g., by asking for explanations) do not increase topic volume but do increase their understanding. We also observe that LLMs widen the gap between students with low and high prior knowledge. While LLMs show great potential to improve learning, their use must be tailored to the educational context and students' needs. △ Less

Submitted 7 March, 2025; v1 submitted 29 August, 2024; originally announced September 2024.

The Definitive Guide to Policy Gradients in Deep Reinforcement Learning: Theory, Algorithms and Implementations

Authors: Matthias Lehmann

Abstract: In recent years, various powerful policy gradient algorithms have been proposed in deep reinforcement learning. While all these algorithms build on the Policy Gradient Theorem, the specific design choices differ significantly across algorithms. We provide a holistic overview of on-policy policy gradient algorithms to facilitate the understanding of both their theoretical foundations and their prac… ▽ More In recent years, various powerful policy gradient algorithms have been proposed in deep reinforcement learning. While all these algorithms build on the Policy Gradient Theorem, the specific design choices differ significantly across algorithms. We provide a holistic overview of on-policy policy gradient algorithms to facilitate the understanding of both their theoretical foundations and their practical implementations. In this overview, we include a detailed proof of the continuous version of the Policy Gradient Theorem, convergence results and a comprehensive discussion of practical algorithms. We compare the most prominent algorithms on continuous control environments and provide insights on the benefits of regularization. All code is available at https://github.com/Matt00n/PolicyGradientsJax. △ Less

Submitted 1 March, 2024; v1 submitted 24 January, 2024; originally announced January 2024.

On the accuracy and performance of the lattice Boltzmann method with 64-bit, 32-bit and novel 16-bit number formats

Authors: Moritz Lehmann, Mathias J. Krause, Giorgio Amati, Marcello Sega, Jens Harting, Stephan Gekle

Abstract: Fluid dynamics simulations with the lattice Boltzmann method (LBM) are very memory-intensive. Alongside reduction in memory footprint, significant performance benefits can be achieved by using FP32 (single) precision compared to FP64 (double) precision, especially on GPUs. Here, we evaluate the possibility to use even FP16 and Posit16 (half) precision for storing fluid populations, while still car… ▽ More Fluid dynamics simulations with the lattice Boltzmann method (LBM) are very memory-intensive. Alongside reduction in memory footprint, significant performance benefits can be achieved by using FP32 (single) precision compared to FP64 (double) precision, especially on GPUs. Here, we evaluate the possibility to use even FP16 and Posit16 (half) precision for storing fluid populations, while still carrying arithmetic operations in FP32. For this, we first show that the commonly occurring number range in the LBM is a lot smaller than the FP16 number range. Based on this observation, we develop novel 16-bit formats - based on a modified IEEE-754 and on a modified Posit standard - that are specifically tailored to the needs of the LBM. We then carry out an in-depth characterization of LBM accuracy for six different test systems with increasing complexity: Poiseuille flow, Taylor-Green vortices, Karman vortex streets, lid-driven cavity, a microcapsule in shear flow (utilizing the immersed-boundary method) and finally the impact of a raindrop (based on a Volume-of-Fluid approach). We find that the difference in accuracy between FP64 and FP32 is negligible in almost all cases, and that for a large number of cases even 16-bit is sufficient. Finally, we provide a detailed performance analysis of all precision levels on a large number of hardware microarchitectures and show that significant speedup is achieved with mixed FP32/16-bit. △ Less

Submitted 31 January, 2022; v1 submitted 16 December, 2021; originally announced December 2021.

Comments: 30 pages, 20 figures, 4 tables, 2 code listings

Beamforming for measurements under disturbed propagation conditions using numerically calculated Green's functions

Authors: Marius Lehmann, Daniel Ernst, Marc Schneider, Carsten Spehr, Markus Lummer

Abstract: Beamforming methods for sound source localization are usually based on free-field Green's functions to model the sound propagation between source and microphone. This assumption is known to be incorrect for many industrial applications and the beamforming results can suffer from this inconsistency regarding both, accuracy of source power estimation, and accuracy of source localisation. The aim of… ▽ More Beamforming methods for sound source localization are usually based on free-field Green's functions to model the sound propagation between source and microphone. This assumption is known to be incorrect for many industrial applications and the beamforming results can suffer from this inconsistency regarding both, accuracy of source power estimation, and accuracy of source localisation. The aim of this paper is to investigate whether the use of numerically calculated Green's functions can improve the results of beamforming measurements. The current test cases of numerical and experimental investigations consists of sources placed in a short rectangular duct. The measurement is performed outside the duct in a semi-anechoic chamber. A typical example for this kind of installation is a fan with a heat exchanger. The Green's functions for this test case are calculated numerically using the boundary element method. These tailored Green's functions are used to calculate the corresponding beamforming steering vectors. The weighting of the Green's functions in the steering vectors has a decisive influence on the beamforming results. A generalization of the common steering vector formulations is given based on two scalars. It is shown that arbitrary differentiable Green's functions can be used to find the correct source position or source power level by using the appropriate steering vector formulations. Beamforming measurements are performed using a loudspeaker as a reference source at different positions in the heat exchanger duct. The measurements are evaluated in the frequency domain and by means of different validation criteria it can be shown that the results with the numerical calculated Green's function are improved compared to free field beamforming especially at low frequencies. △ Less

Submitted 30 October, 2020; originally announced October 2020.

Comments: Preprint subitted to "Journal of Sound and Vibration"

Analytic Solution to the Piecewise Linear Interface Construction Problem and its Application in Curvature Calculation for Volume-of-Fluid Simulation Codes

Authors: Moritz Lehmann, Stephan Gekle

Abstract: The plane-cube intersection problem has been around in literature since 1984 and iterative solutions to it have been used as part of piecewise linear interface construction (PLIC) in computational fluid dynamics simulation codes ever since. In many cases, PLIC is the bottleneck of these simulations regarding compute time, so a faster, analytic solution to the plane-cube intersection would greatly… ▽ More The plane-cube intersection problem has been around in literature since 1984 and iterative solutions to it have been used as part of piecewise linear interface construction (PLIC) in computational fluid dynamics simulation codes ever since. In many cases, PLIC is the bottleneck of these simulations regarding compute time, so a faster, analytic solution to the plane-cube intersection would greatly reduce compute time for such simulations. We derive an analytic solution for all intersection cases and compare it to the one previous solution from Scardovelli and Zaleski (Ruben Scardovelli and Stephane Zaleski. "Analytical relations connecting linear interfaces and volume fractions in rectangular grids". In: Journal of Computational Physics 164.1 (2000), pp. 228-237.), which we further improve to include edge cases and micro-optimize to reduce arithmetic operations and branching. We then extend our comparison regarding compute time and accuracy to include two different iterative solutions as well. We find that the best choice depends on the employed hardware platform: on the CPU, Newton-Raphson is fastest with vectorization while analytic solutions perform better without. The reason for this is that vectorization instruction sets do not include trigonometric functions as used in the analytic solutions. On the GPU, the fastest method is our optimized version of the analytic SZ solution. We finally provide details on one of the applications of PLIC: curvature calculation for the Volume-of-Fluid model used for free surface fluid simulations in combination with the lattice Boltzmann method. △ Less

Submitted 18 February, 2021; v1 submitted 23 June, 2020; originally announced June 2020.

Comments: 18 pages, 6 figures