Functional Renormalization for Signal Detection: Dimensional Analysis and Dimensional Phase Transition for Nearly Continuous Spectra Effective Field Theory
Authors:
Riccardo Finotello,
Vincent Lahoche,
Dine Ousmane Samary
Abstract:
Signal detection is one of the main challenges of data science. According to the nature of the data, the presence of noise may corrupt measurements and hinder the discovery of significant patterns. A wide range of techniques aiming at extracting the relevant degrees of freedom from data has been thus developed over the years. However, signal detection in almost continuous spectra, for small signal…
▽ More
Signal detection is one of the main challenges of data science. According to the nature of the data, the presence of noise may corrupt measurements and hinder the discovery of significant patterns. A wide range of techniques aiming at extracting the relevant degrees of freedom from data has been thus developed over the years. However, signal detection in almost continuous spectra, for small signal-to-noise ratios, remains a known difficult issue. This paper develops over recent advancements proposing to tackle this issue by analysing the properties of the underlying effective field theory arising as a sort of maximal entropy distribution in the vicinity of universal random matrix distributions. Nearly continuous spectra provide an intrinsic and non-conventional scaling law for field and couplings, the scaling dimensions depending on the energy scale. The coarse-graining over small eigenvalues of the empirical spectrum defines a specific renormalization group, whose characteristics change when the collective behaviour of "informational" modes become significant, that is, stronger than the intrinsic fluctuations of noise. This paper pursues three different goals. First, we propose to quantify the real effects of fluctuations relative to what can be called "signal", while improving the robustness of the results obtained in our previous work. Second, we show that quantitative changes in the presence of a signal result in a counterintuitive modification of the distribution of eigenvectors. Finally, we propose a method for estimating the number of noise components and define a limit of detection in a general nearly continuous spectrum using the renormalization group. The main statements of this paper are essentially numeric, and their reproducibility can be checked using the associated code.
△ Less
Submitted 30 June, 2025;
originally announced July 2025.
