Showing 1–4 of 4 results for author: Pham, D K

Local Control Networks (LCNs): Optimizing Flexibility in Neural Network Data Pattern Capture

Authors: Hy Nguyen, Duy Khoa Pham, Srikanth Thudumu, Hung Du, Rajesh Vasa, Kon Mouzakis

Abstract: The widespread use of Multi-layer perceptrons (MLPs) often relies on a fixed activation function (e.g., ReLU, Sigmoid, Tanh) for all nodes within the hidden layers. While effective in many scenarios, this uniformity may limit the networks ability to capture complex data patterns. We argue that employing the same activation function at every node is suboptimal and propose leveraging different activ… ▽ More The widespread use of Multi-layer perceptrons (MLPs) often relies on a fixed activation function (e.g., ReLU, Sigmoid, Tanh) for all nodes within the hidden layers. While effective in many scenarios, this uniformity may limit the networks ability to capture complex data patterns. We argue that employing the same activation function at every node is suboptimal and propose leveraging different activation functions at each node to increase flexibility and adaptability. To achieve this, we introduce Local Control Networks (LCNs), which leverage B-spline functions to enable distinct activation curves at each node. Our mathematical analysis demonstrates the properties and benefits of LCNs over conventional MLPs. In addition, we demonstrate that more complex architectures, such as Kolmogorov-Arnold Networks (KANs), are unnecessary in certain scenarios, and LCNs can be a more efficient alternative. Empirical experiments on various benchmarks and datasets validate our theoretical findings. In computer vision tasks, LCNs achieve marginal improvements over MLPs and outperform KANs by approximately 5\%, while also being more computationally efficient than KANs. In basic machine learning tasks, LCNs show a 1\% improvement over MLPs and a 0.6\% improvement over KANs. For symbolic formula representation tasks, LCNs perform on par with KANs, with both architectures outperforming MLPs. Our findings suggest that diverse activations at the node level can lead to improved performance and efficiency. △ Less

Submitted 25 April, 2025; v1 submitted 23 January, 2025; originally announced January 2025.

Towards Reliable Medical Question Answering: Techniques and Challenges in Mitigating Hallucinations in Language Models

Authors: Duy Khoa Pham, Bao Quoc Vo

Abstract: The rapid advancement of large language models (LLMs) has significantly impacted various domains, including healthcare and biomedicine. However, the phenomenon of hallucination, where LLMs generate outputs that deviate from factual accuracy or context, poses a critical challenge, especially in high-stakes domains. This paper conducts a scoping study of existing techniques for mitigating hallucinat… ▽ More The rapid advancement of large language models (LLMs) has significantly impacted various domains, including healthcare and biomedicine. However, the phenomenon of hallucination, where LLMs generate outputs that deviate from factual accuracy or context, poses a critical challenge, especially in high-stakes domains. This paper conducts a scoping study of existing techniques for mitigating hallucinations in knowledge-based task in general and especially for medical domains. Key methods covered in the paper include Retrieval-Augmented Generation (RAG)-based techniques, iterative feedback loops, supervised fine-tuning, and prompt engineering. These techniques, while promising in general contexts, require further adaptation and optimization for the medical domain due to its unique demands for up-to-date, specialized knowledge and strict adherence to medical guidelines. Addressing these challenges is crucial for developing trustworthy AI systems that enhance clinical decision-making and patient safety as well as accuracy of biomedical scientific research. △ Less

Submitted 25 August, 2024; originally announced August 2024.

Comments: 9 pages

Best free knot linear spline approximation and its application to neural networks

Authors: Vinesha Peiris, Duy Khoa Pham, Nadezda Sukhorukova

Abstract: The problem of fixed knot approximation is convex and there are several efficient approaches to solve this problem, yet, when the knots joining the affine parts are also variable, finding conditions for a best Chebyshev approximation remains an open problem. It was noticed before that piecewise linear approximation with free knots is equivalent to neural network approximation with piecewise linear… ▽ More The problem of fixed knot approximation is convex and there are several efficient approaches to solve this problem, yet, when the knots joining the affine parts are also variable, finding conditions for a best Chebyshev approximation remains an open problem. It was noticed before that piecewise linear approximation with free knots is equivalent to neural network approximation with piecewise linear activation functions (for example ReLU). In this paper, we demonstrate that in the case of one internal free knot, the problem of linear spline approximation can be reformulated as a mixed-integer linear programming problem and solved efficiently using, for example, a branch and bound type method. We also present a new sufficient optimality condition for a one free knot piecewise linear approximation. The results of numerical experiments are provided. △ Less

Submitted 7 March, 2024; originally announced April 2024.

Global sensitivity analysis for models described by stochastic differential equations

Authors: Pierre Etoré, Clémentine Prieur, Dang Khoi Pham, Long Li

Abstract: Many mathematical models involve input parameters, which are not precisely known. Global sensitivity analysis aims to identify the parameters whose uncertainty has the largest impact on the variability of a quantity of interest. One of the statistical tools used to quantify the influence of each input variable on the quantity of interest are the Sobol' sensitivity indices. In this paper, we consid… ▽ More Many mathematical models involve input parameters, which are not precisely known. Global sensitivity analysis aims to identify the parameters whose uncertainty has the largest impact on the variability of a quantity of interest. One of the statistical tools used to quantify the influence of each input variable on the quantity of interest are the Sobol' sensitivity indices. In this paper, we consider stochastic models described by stochastic differential equations (SDE). We focus the study on mean quantities, defined as the expectation with respect to the Wiener measure of a quantity of interest related to the solution of the SDE itself. Our approach is based on a Feynman-Kac representation of the quantity of interest, from which we get a parametrized partial differential equation (PDE) representation of our initial problem. We then handle the uncertainty on the parametrized PDE using polynomial chaos expansion and a stochastic Galerkin projection. △ Less

Submitted 20 November, 2018; originally announced November 2018.