Showing 1–17 of 17 results for author: Gagunashvili, N

Comparative Analysis of Richardson-Lucy Deconvolution and Data Unfolding with Mean Integrated Square Error Optimization

Authors: Nikolay D. Gagunashvili

Abstract: Two maximum likelihood-based algorithms for unfolding or deconvolution are considered: the Richardson-Lucy method and the Data Unfolding method with Mean Integrated Square Error (MISE) optimization [10]. Unfolding is viewed as a procedure for estimating an unknown probability density function. Both external and internal quality assessment methods can be applied for this purpose. In some cases, ext… ▽ More Two maximum likelihood-based algorithms for unfolding or deconvolution are considered: the Richardson-Lucy method and the Data Unfolding method with Mean Integrated Square Error (MISE) optimization [10]. Unfolding is viewed as a procedure for estimating an unknown probability density function. Both external and internal quality assessment methods can be applied for this purpose. In some cases, external criteria exist to evaluate deconvolution quality. A typical example is the deconvolution of a blurred image, where the sharpness of the restored image serves as an indicator of quality. However, defining such external criteria can be challenging, particularly when a measurement has not been performed previously. In such instances, internal criteria are necessary to assess the quality of the result independently of external information. The article discusses two internal criteria: MISE for the unfolded distribution and the condition number of the correlation matrix of the unfolded distribution. These internal quality criteria are applied to a comparative analysis of the two methods using identical numerical data. The results of the analysis demonstrate the superiority of the Data Unfolding method with MISE optimization over the Richardson-Lucy method. △ Less

Submitted 15 May, 2025; originally announced May 2025.

Comments: 15 pages, 18 figures

MSC Class: 62-07 (Primary); 62F03; 62F10; 62P35; 62P30 (Secondary)

Data Unfolding with Mean Integrated Square Error Optimization

Authors: Nikolay D. Gagunashvili

Abstract: Experimental data in Particle and Nuclear physics, Particle Astrophysics and Radiation Protection Dosimetry are obtained from experimental facilities comprising a complex array of sensors, electronics and software. Computer simulation is used to study the measurement process. Probability Density Functions (PDFs) of measured physical parameters deviate from true PDFs due to resolution, bias, and ef… ▽ More Experimental data in Particle and Nuclear physics, Particle Astrophysics and Radiation Protection Dosimetry are obtained from experimental facilities comprising a complex array of sensors, electronics and software. Computer simulation is used to study the measurement process. Probability Density Functions (PDFs) of measured physical parameters deviate from true PDFs due to resolution, bias, and efficiency effects. Good estimates of the true PDF are necessary for testing theoretical models, comparing results from different experiments, and combining results from various research endeavors. In the article, the histogram method is employed to estimate both the measured and true PDFs. The binning of histograms is determined using the K-means clustering algorithm. The true PDF is estimated through the maximization of the likelihood function with entropy regularization, utilizing a non-linear optimization algorithm specially designed for this purpose. The accuracy of the results is assessed using the Mean Integrated Square Error. To determine the optimal value for the regularization parameter, a bootstrap method is applied. Additionally, a mathematical model of the measurement system is formulated using system identification methods. This approach enhances the robustness and precision of the estimation process, providing a more reliable analysis of the system's characteristics. △ Less

Submitted 7 December, 2024; v1 submitted 20 February, 2024; originally announced February 2024.

Comments: 17 pages, 12 figures

MSC Class: 62-07 (Primary); 62F03; 62F10; 62P35; 62P30 (Secondary)

Journal ref: Computer Physics Communications 308 (2025) 109473

Parametric unfolding. Method and restrictions

Authors: Nikolay Gagunashvili

Abstract: Parametric unfolding of a true distribution distorted due to finite resolution and limited efficiency for the registration of individual events is discussed. Details of the computational algorithm of the unfolding procedure are presented. Parametric unfolding of a true distribution distorted due to finite resolution and limited efficiency for the registration of individual events is discussed. Details of the computational algorithm of the unfolding procedure are presented. △ Less

Submitted 27 April, 2020; originally announced April 2020.

Comments: 14 pages, 9 figures

MSC Class: 62-07 (Primary); 62F03; 62F10; 62P35; 62P30 (Secondary)

Journal ref: Eur. Phys. J. Plus 135, 540 (2020)

Tests for Comparing Weighted Histograms. Review and Improvements

Authors: Nikolai Gagunashvili

Abstract: Histograms with weighted entries are used to estimate probability density functions. Computer simulation is the main application of this type of histograms. A review on chi-square tests for comparing weighted histograms is presented in this paper. Improvements to these tests that have a size closer to its nominal value are proposed. Numerical examples are presented for evaluation and demonstration… ▽ More Histograms with weighted entries are used to estimate probability density functions. Computer simulation is the main application of this type of histograms. A review on chi-square tests for comparing weighted histograms is presented in this paper. Improvements to these tests that have a size closer to its nominal value are proposed. Numerical examples are presented for evaluation and demonstration of various applications of the tests. △ Less

Submitted 24 March, 2017; v1 submitted 21 June, 2016; originally announced June 2016.

Comments: 25 pages, 5 figures

MSC Class: 62-07 (Primary); 62F03; 62F10; 62P35; 62P30 (Secondary)

Journal ref: Eur. Phys. J. Plus (2017) 132: 196

Unfolding problem clarification and solution validation

Authors: Nikolai Gagunashvili

Abstract: The unfolding problem formulation for correcting experimental data distortions due to finite resolution and limited detector acceptance is discussed. A novel validation of the problem solution is proposed. Attention is drawn to fact that different unfolded distributions may satisfy the validation criteria, in which case a conservative approach using entropy is suggested. The importance of analysis… ▽ More The unfolding problem formulation for correcting experimental data distortions due to finite resolution and limited detector acceptance is discussed. A novel validation of the problem solution is proposed. Attention is drawn to fact that different unfolded distributions may satisfy the validation criteria, in which case a conservative approach using entropy is suggested. The importance of analysis of residuals is demonstrated. △ Less

Submitted 7 April, 2016; v1 submitted 18 February, 2016; originally announced February 2016.

Comments: 9 pages,4 figures

MSC Class: 62-07 (Primary); 62F03; 62F10; 62P35; 62P30 (Secondary)

Densities mixture unfolding for data obtained from detectors with finite resolution and limited acceptance

Authors: Nikolai Gagunashvili

Abstract: A procedure based on a Mixture Density Model for correcting experimental data for distortions due to finite resolution and limited detector acceptance is presented. Addressing the case that the solution is known to be non-negative, in the approach presented here, the true distribution is estimated by a weighted sum of probability density functions with positive weights and with the width of the de… ▽ More A procedure based on a Mixture Density Model for correcting experimental data for distortions due to finite resolution and limited detector acceptance is presented. Addressing the case that the solution is known to be non-negative, in the approach presented here, the true distribution is estimated by a weighted sum of probability density functions with positive weights and with the width of the densities acting as a regularisation parameter responsible for the smoothness of the result. To obtain better smoothing in less populated regions, the width parameter is chosen inversely proportional to the square root of the estimated density. Furthermore, the non-negative garrotte method is used to find the most economic representation of the solution. Cross-validation is employed to determine the optimal values of the resolution and garrotte parameters. The proposed approach is directly applicable to multidimensional problems. Numerical examples in one and two dimensions are presented to illustrate the procedure. △ Less

Submitted 5 January, 2015; v1 submitted 6 October, 2014; originally announced October 2014.

Comments: 26 pages, 15 figures

MSC Class: 62-07 (Primary); 62F03; 62F10; 62P35; 62P30 (Secondary)

Journal ref: Nuclear Instruments and Methods A 778 (2015) 92-101

Chi-square goodness of fit tests for weighted histograms. Review and improvements

Authors: Nikolai Gagunashvili

Abstract: Weighted histograms are used for the estimation of probability density functions. Computer simulation is the main domain of application of this type of histogram. A review of chi-square goodness of fit tests for weighted histograms is presented in this paper. Improvements are proposed to these tests that have size more close to its nominal value. Numerical examples are presented in this paper for… ▽ More Weighted histograms are used for the estimation of probability density functions. Computer simulation is the main domain of application of this type of histogram. A review of chi-square goodness of fit tests for weighted histograms is presented in this paper. Improvements are proposed to these tests that have size more close to its nominal value. Numerical examples are presented in this paper for evaluation of tests and to demonstrate various applications of tests. △ Less

Submitted 19 February, 2015; v1 submitted 5 July, 2013; originally announced July 2013.

Comments: 27 pages, 4 figures

MSC Class: 62-07 (Primary) 62F03; 62F10; 62P35; 62P30 (Secondary)

Journal ref: Journal of Instrumentation 10 (2015) P05004

Kernel based unfolding of data obtained from detectors with finite resolution and limited acceptance

Authors: N. D. Gagunashvili, M. Schmelling

Abstract: A kernel based procedure for correcting experimental data for distortions due to the finite resolution and limited detector acceptance is presented. The unfolding problem is known to be an ill-posed problem that can not be solved without some a priori information about solution such as, for example, smoothness or positivity. In the approach presented here the true distribution is estimated by a we… ▽ More A kernel based procedure for correcting experimental data for distortions due to the finite resolution and limited detector acceptance is presented. The unfolding problem is known to be an ill-posed problem that can not be solved without some a priori information about solution such as, for example, smoothness or positivity. In the approach presented here the true distribution is estimated by a weighted sum of kernels, with the width of the kernels acting as a regularization parameter responsible for the smoothness of the result. Cross-validation is used to determine an optimal value for this parameter. A numerical example with a simulation study of systematical and statistical errors is presented to illustrate the procedure. △ Less

Submitted 17 September, 2012; originally announced September 2012.

Comments: 22 pages, 6 figures

MSC Class: 62-07 (Primary); 62F03; 62F10; 62P35; 62P30 (Secondary)

CHICOM: A code of tests for comparing unweighted and weighted histograms and two weighted histograms

Authors: Nikolai Gagunashvili

Abstract: A self-contained Fortran-77 program for calculating test statistics to compare weighted histogram with unweighted histogram and two histograms with weighted entries is presented. The code calculates test statistics for cases of histograms with normalized weights of events and unnormalized weights of events. A self-contained Fortran-77 program for calculating test statistics to compare weighted histogram with unweighted histogram and two histograms with weighted entries is presented. The code calculates test statistics for cases of histograms with normalized weights of events and unnormalized weights of events. △ Less

Submitted 10 October, 2011; v1 submitted 6 May, 2011; originally announced May 2011.

Comments: 9 pages

MSC Class: 62-07 (Primary); 62F03; 62F10; 62P35; 62P30 (Secondary)

Journal ref: Computer Physics Communications, Volume 183, Issue 1, 2012, Pages 193-196

CHIWEI: A code of goodness of fit tests for weighted and unweighted histograms

Authors: Nikolai Gagunashvili

Abstract: A self-contained Fortran-77 program for goodness of fit tests for histograms with weighted entries as well as with unweighted entries is presented. The code calculates test statistics for case of histogram with normalized weights of events and in case of unnormalized weights of events. A self-contained Fortran-77 program for goodness of fit tests for histograms with weighted entries as well as with unweighted entries is presented. The code calculates test statistics for case of histogram with normalized weights of events and in case of unnormalized weights of events. △ Less

Submitted 10 October, 2011; v1 submitted 19 April, 2011; originally announced April 2011.

Comments: 7 pages

MSC Class: 62-07 (Primary) 62F03; 62F10; 62P35; 62P30 (Secondary)

Journal ref: Computer Physics Communications, Volume 183, Issue 2, 2012, Pages 418-421

Classifying extremely imbalanced data sets

Authors: Markward Britsch, Nikolai Gagunashvili, Michael Schmelling

Abstract: Imbalanced data sets containing much more background than signal instances are very common in particle physics, and will also be characteristic for the upcoming analyses of LHC data. Following up the work presented at ACAT 2008, we use the multivariate technique presented there (a rule growing algorithm with the meta-methods bagging and instance weighting) on much more imbalanced data sets, especi… ▽ More Imbalanced data sets containing much more background than signal instances are very common in particle physics, and will also be characteristic for the upcoming analyses of LHC data. Following up the work presented at ACAT 2008, we use the multivariate technique presented there (a rule growing algorithm with the meta-methods bagging and instance weighting) on much more imbalanced data sets, especially a selection of D0 decays without the use of particle identification. It turns out that the quality of the result strongly depends on the number of background instances used for training. We discuss methods to exploit this in order to improve the results significantly, and how to handle and reduce the size of large training sets without loss of result quality in general. We will also comment on how to take into account statistical fluctuation in receiver operation characteristic curves (ROC) for comparing classifier methods. △ Less

Submitted 29 November, 2010; originally announced November 2010.

Journal ref: PoS ACAT2010:047,2010

Parametric fitting of data obtained from detectors with finite resolution and limited acceptance

Authors: N. D. Gagunashvili

Abstract: A goodness-of-fit test for the fitting of a parametric model to data obtained from a detector with finite resolution and limited acceptance is proposed. The parameters of the model are found by minimization of a statistic that is used for comparing experimental data and simulated reconstructed data. Numerical examples are presented to illustrate and validate the fitting procedure. A goodness-of-fit test for the fitting of a parametric model to data obtained from a detector with finite resolution and limited acceptance is proposed. The parameters of the model are found by minimization of a statistic that is used for comparing experimental data and simulated reconstructed data. Numerical examples are presented to illustrate and validate the fitting procedure. △ Less

Submitted 5 January, 2011; v1 submitted 2 November, 2010; originally announced November 2010.

Comments: 11 pages, two figures

MSC Class: 62-07 (Primary); 62F03; 62F10; 62P35; 62P30 (Secondary)

Journal ref: Nuclear Instruments and Methods A 635 (2011) 86-91

Machine learning approach to inverse problem and unfolding procedure

Authors: Nikolai Gagunashvili

Abstract: A procedure for unfolding the true distribution from experimental data is presented. Machine learning methods are applied for simultaneous identification of an apparatus function and solving of an inverse problem. A priori information about the true distribution from theory or previous experiments is used for Monte-Carlo simulation of the training sample. The training sample can be used to calcula… ▽ More A procedure for unfolding the true distribution from experimental data is presented. Machine learning methods are applied for simultaneous identification of an apparatus function and solving of an inverse problem. A priori information about the true distribution from theory or previous experiments is used for Monte-Carlo simulation of the training sample. The training sample can be used to calculate a transformation from the true distribution to the measured one. This transformation provides a robust solution for an unfolding problem with minimal biases and statistical errors for the set of distributions used to create the training sample. The dimensionality of the solved problem can be arbitrary. A numerical example is presented to illustrate and validate the procedure. △ Less

Submitted 25 May, 2011; v1 submitted 12 April, 2010; originally announced April 2010.

Comments: 19 pages, 7 figures

MSC Class: 62-07 (Primary)

Application of the rule-growing algorithm RIPPER to particle physics analysis

Authors: Markward Britsch, Nikolai Gagunashvili, Michael Schmelling

Abstract: A large hadron machine like the LHC with its high track multiplicities always asks for powerful tools that drastically reduce the large background while selecting signal events efficiently. Actually such tools are widely needed and used in all parts of particle physics. Regarding the huge amount of data that will be produced at the LHC, the process of training as well as the process of applying… ▽ More A large hadron machine like the LHC with its high track multiplicities always asks for powerful tools that drastically reduce the large background while selecting signal events efficiently. Actually such tools are widely needed and used in all parts of particle physics. Regarding the huge amount of data that will be produced at the LHC, the process of training as well as the process of applying these tools to data, must be time efficient. Such tools can be multivariate analysis -- also called data mining -- tools. In this contribution we present the results for the application of the multivariate analysis, rule growing algorithm RIPPER on a problem of particle selection. It turns out that the meta-methods bagging and cost-sensitivity are essential for the quality of the outcome. The results are compared to other multivariate analysis techniques. △ Less

Submitted 9 October, 2009; originally announced October 2009.

Comments: 10 pages, 8 figures, conference proceedings from ACAT08 (XII Advanced Computing and Analysis Techniques in Physics Research, November 3-7, 2008, Erice, Italy)

Journal ref: PoS ACAT08:086,2008

Chi-Square Tests for Comparing Weighted Histograms

Authors: N. D. Gagunashvili

Abstract: Weighted histograms in Monte Carlo simulations are often used for the estimation of probability density functions. They are obtained as a result of random experiments with random events that have weights. In this paper, the bin contents of a weighted histogram are considered as a sum of random variables with a random number of terms. Generalizations of the classical chi-square test for comparing… ▽ More Weighted histograms in Monte Carlo simulations are often used for the estimation of probability density functions. They are obtained as a result of random experiments with random events that have weights. In this paper, the bin contents of a weighted histogram are considered as a sum of random variables with a random number of terms. Generalizations of the classical chi-square test for comparing weighted histograms was proposed. Numerical examples illustrate an application of the tests for the histograms with different statistics of events and different weighted functions. The proposed tests can be used for the comparison of experimental data histograms with simulated data histograms, as well as for the two simulated data histograms. △ Less

Submitted 15 December, 2009; v1 submitted 26 May, 2009; originally announced May 2009.

Comments: 26 pages, 8 figures

Journal ref: Nucl.Instrum.Meth.A614:287-296,2010

Goodness of fit tests for weighted histograms

Authors: N. D. Gagunashvili

Abstract: Weighted histogram in Monte-Carlo simulations is often used for the estimation of a probability density function. It is obtained as a result of random experiment with random events that have weights. In this paper the bin contents of weighted histogram are considered as a sum of random variables with random number of terms. Goodness of fit tests for weighted histograms and for weighted histogram… ▽ More Weighted histogram in Monte-Carlo simulations is often used for the estimation of a probability density function. It is obtained as a result of random experiment with random events that have weights. In this paper the bin contents of weighted histogram are considered as a sum of random variables with random number of terms. Goodness of fit tests for weighted histograms and for weighted histograms with unknown normalization are proposed. Sizes and powers of the tests are investigated numerically. △ Less

Submitted 19 June, 2008; v1 submitted 27 December, 2007; originally announced December 2007.

Comments: 15 pages, 5 figures, changed content

Journal ref: Nuclear Instruments and Methods in Physics Research A 596 (2008) 439-445

Comparison of weighted and unweighted histograms

Authors: N. D. Gagunashvili

Abstract: Two modifications of the chi square test for comparing usual(unweighted) and weighted histograms and two weighted histograms are proposed. Numerical examples illustrate an application of the tests for the histograms with different statistics of events. Proposed tests can be used for the comparison of experimental data histograms against simulated data histograms and two simulated data histograms… ▽ More Two modifications of the chi square test for comparing usual(unweighted) and weighted histograms and two weighted histograms are proposed. Numerical examples illustrate an application of the tests for the histograms with different statistics of events. Proposed tests can be used for the comparison of experimental data histograms against simulated data histograms and two simulated data histograms. △ Less

Submitted 15 May, 2006; originally announced May 2006.

Comments: 12 pages, 6 figures

Journal ref: PoS(ACAT)054, 2007, http://pos.sissa.it//archive/conferences/050/060/ACAT_060.pdf