-
Correspondence between temporal correlations in time series, inverse problems, and the Spherical Model
Authors:
Riccardo Marcaccioli,
Giacomo Livan
Abstract:
In this paper we employ methods from Statistical Mechanics to model temporal correlations in time series. We put forward a methodology based on the Maximum Entropy principle to generate ensembles of time series constrained to preserve part of the temporal structure of an empirical time series of interest. We show that a constraint on the lag-one autocorrelation can be fully handled analytically, a…
▽ More
In this paper we employ methods from Statistical Mechanics to model temporal correlations in time series. We put forward a methodology based on the Maximum Entropy principle to generate ensembles of time series constrained to preserve part of the temporal structure of an empirical time series of interest. We show that a constraint on the lag-one autocorrelation can be fully handled analytically, and corresponds to the well known Spherical Model of a ferromagnet. We then extend such a model to include constraints on more complex temporal correlations by means of perturbation theory, showing that this leads to substantial improvements in capturing the lag-one autocorrelation in the variance. We apply our approach on synthetic data, and illustrate how it can be used to formulate expectations on the future values of a data generating process.
△ Less
Submitted 6 July, 2020; v1 submitted 28 March, 2020;
originally announced March 2020.
-
Maximum Entropy approach to multivariate time series randomization
Authors:
Riccardo Marcaccioli,
Giacomo Livan
Abstract:
Natural and social multivariate systems are commonly studied through sets of simultaneous and time-spaced measurements of the observables that drive their dynamics, i.e., through sets of time series. Typically, this is done via hypothesis testing: the statistical properties of the empirical time series are tested against those expected under a suitable null hypothesis. This is a very challenging t…
▽ More
Natural and social multivariate systems are commonly studied through sets of simultaneous and time-spaced measurements of the observables that drive their dynamics, i.e., through sets of time series. Typically, this is done via hypothesis testing: the statistical properties of the empirical time series are tested against those expected under a suitable null hypothesis. This is a very challenging task in complex interacting systems, where statistical stability is often poor due to lack of stationarity and ergodicity. Here, we describe an unsupervised, data-driven framework to perform hypothesis testing in such situations. This consists of a statistical mechanical approach - analogous to the configuration model for networked systems - for ensembles of time series designed to preserve, on average, some of the statistical properties observed on an empirical set of time series. We showcase its possible applications with a case study on financial portfolio selection.
△ Less
Submitted 30 June, 2020; v1 submitted 1 July, 2019;
originally announced July 2019.
-
A parametric approach to information filtering in complex networks: The Pólya filter
Authors:
Riccardo Marcaccioli,
Giacomo Livan
Abstract:
The ever increasing availability of data demands for techniques to extract relevant information from complex interacting systems, which can often be represented as weighted networks. In recent years, a number of approaches have been proposed to extract network backbones by assessing the statistical significance of links against null hypotheses of random interaction. Yet, it is well known that the…
▽ More
The ever increasing availability of data demands for techniques to extract relevant information from complex interacting systems, which can often be represented as weighted networks. In recent years, a number of approaches have been proposed to extract network backbones by assessing the statistical significance of links against null hypotheses of random interaction. Yet, it is well known that the growth of most real-world networks is highly non-random, as past interactions between nodes typically increase the likelihood of further interaction. Here, we propose a network filtering methodology based on a family of null hypotheses that can be calibrated to assess which links are statistically significant with respect to a given network's own heterogeneity. We design such family of null hypotheses by adapting the Pólya urn, a simple one-parameter combinatorial model driven by a self-reinforcement mechanism, to a network setting. We provide a full analytical description of our filter, and show that it retains or discards links based on a non-trivial interplay between their own local importance and the importance of the nodes they belong to. We prove that the widely used disparity filter can be recovered as a large-strength approximation for a specific value of the Pólya filter's parameter, and illustrate our findings by applying the filter to two large network datasets.
△ Less
Submitted 18 February, 2019; v1 submitted 26 June, 2018;
originally announced June 2018.
-
Introduction to Random Matrices - Theory and Practice
Authors:
Giacomo Livan,
Marcel Novaes,
Pierpaolo Vivo
Abstract:
This is a book for absolute beginners. If you have heard about random matrix theory, commonly denoted RMT, but you do not know what that is, then welcome!, this is the place for you. Our aim is to provide a truly accessible introductory account of RMT for physicists and mathematicians at the beginning of their research career. We tried to write the sort of text we would have loved to read when we…
▽ More
This is a book for absolute beginners. If you have heard about random matrix theory, commonly denoted RMT, but you do not know what that is, then welcome!, this is the place for you. Our aim is to provide a truly accessible introductory account of RMT for physicists and mathematicians at the beginning of their research career. We tried to write the sort of text we would have loved to read when we were beginning Ph.D. students ourselves. Our book is structured with light and short chapters, and the style is informal. The calculations we found most instructive are spelt out in full. Particular attention is paid to the numerical verification of most analytical results. Our book covers standard material - classical ensembles, orthogonal polynomial techniques, spectral densities and spacings - but also more advanced and modern topics - replica approach and free probability - that are not normally included in elementary accounts on RMT. This book is dedicated to the fond memory of Oriol Bohigas.
△ Less
Submitted 21 December, 2017;
originally announced December 2017.
-
Statistical mechanics of complex economies
Authors:
Marco Bardoscia,
Giacomo Livan,
Matteo Marsili
Abstract:
In the pursuit of ever increasing efficiency and growth, our economies have evolved to remarkable degrees of complexity, with nested production processes feeding each other in order to create products of greater sophistication from less sophisticated ones, down to raw materials. The engine of such an expansion have been competitive markets that, according to General Equilibrium Theory (GET), achie…
▽ More
In the pursuit of ever increasing efficiency and growth, our economies have evolved to remarkable degrees of complexity, with nested production processes feeding each other in order to create products of greater sophistication from less sophisticated ones, down to raw materials. The engine of such an expansion have been competitive markets that, according to General Equilibrium Theory (GET), achieve efficient allocations under specific conditions. We study large random economies within the GET framework, as templates of complex economies, and we find that a non-trivial phase transition occurs: the economy freezes in a state where all production processes collapse when either the number of primary goods or the number of available technologies fall below a critical threshold. As in other examples of phase transitions in large random systems, this is an unintended consequence of the growth in complexity. Our findings suggest that the Industrial Revolution can be regarded as a sharp transition between different phases, but also imply that well developed economies can collapse if too many intermediate goods are introduced.
△ Less
Submitted 4 April, 2017; v1 submitted 30 November, 2015;
originally announced November 2015.
-
Invariant sums of random matrices and the onset of level repulsion
Authors:
Zdzisław Burda,
Giacomo Livan,
Pierpaolo Vivo
Abstract:
We compute analytically the joint probability density of eigenvalues and the level spacing statistics for an ensemble of random matrices with interesting features. It is invariant under the standard symmetry groups (orthogonal and unitary) and yet the interaction between eigenvalues is not Vandermondian. The ensemble contains real symmetric or complex hermitian matrices $\mathbf{S}$ of the form…
▽ More
We compute analytically the joint probability density of eigenvalues and the level spacing statistics for an ensemble of random matrices with interesting features. It is invariant under the standard symmetry groups (orthogonal and unitary) and yet the interaction between eigenvalues is not Vandermondian. The ensemble contains real symmetric or complex hermitian matrices $\mathbf{S}$ of the form $\mathbf{S}=\sum_{i=1}^M \langle \mathbf{O}_i \mathbf{D}_i\mathbf{O}_i^{\mathrm{T}}\rangle$ or $\mathbf{S}=\sum_{i=1}^M \langle \mathbf{U}_i \mathbf{D}_i\mathbf{U}_i^\dagger\rangle$ respectively. The diagonal matrices $\mathbf{D}_i=\mathrm{diag}\{λ_1^{(i)},\ldots,λ_N^{(i)}\}$ are constructed from real eigenvalues drawn \emph{independently} from distributions $p^{(i)}(x)$, while the matrices $\mathbf{O}_i$ and $\mathbf{U}_i$ are all orthogonal or unitary. The average $\langle\cdot\rangle$ is simultaneously performed over the symmetry group and the joint distribution of $\{λ_j^{(i)}\}$. We focus on the limits i.) $N\to\infty$ and ii.) $M\to\infty$, with $N=2$. In the limit i.), the resulting sum $\mathbf{S}$ develops level repulsion even though the original matrices do not feature it, and classical RMT universality is restored asymptotically. In the limit ii.) the spacing distribution attains scaling forms that are computed exactly: for the orthogonal case, we recover the $β=1$ Wigner's surmise, while for the unitary case an entirely new universal distribution is obtained. Our results allow to probe analytically the microscopic statistics of the sum of random matrices that become asymptotically free. We also give an interpretation of this model in terms of radial random walks in a matrix space. The analytical results are corroborated by numerical simulations.
△ Less
Submitted 20 July, 2015; v1 submitted 25 August, 2014;
originally announced August 2014.
-
Commutative law for products of infinitely large isotropic random matrices
Authors:
Z. Burda,
G. Livan,
A. Swiech
Abstract:
Ensembles of isotropic random matrices are defined by the invariance of the probability measure under the left (and right) multiplication by an arbitrary unitary matrix. We show that the multiplication of large isotropic random matrices is spectrally commutative and self-averaging in the limit of infinite matrix size $N \rightarrow \infty$. The notion of spectral commutativity means that the eigen…
▽ More
Ensembles of isotropic random matrices are defined by the invariance of the probability measure under the left (and right) multiplication by an arbitrary unitary matrix. We show that the multiplication of large isotropic random matrices is spectrally commutative and self-averaging in the limit of infinite matrix size $N \rightarrow \infty$. The notion of spectral commutativity means that the eigenvalue density of a product ABC... of such matrices is independent of the order of matrix multiplication, for example the matrix ABCD has the same eigenvalue density as ADCB. In turn, the notion of self-averaging means that the product of n independent but identically distributed random matrices, which we symbolically denote by AAA..., has the same eigenvalue density as the corresponding power A^n of a single matrix drawn from the underlying matrix ensemble. For example, the eigenvalue density of ABCCABC is the same as of A^2B^2C^3. We also discuss the singular behavior of the eigenvalue and singular value densities of isotropic matrices and their products for small eigenvalues $λ\rightarrow 0$. We show that the singularities at the origin of the eigenvalue density and of the singular value density are in one-to-one correspondence in the limit $N \rightarrow \infty$: the eigenvalue density of an isotropic random matrix has a power law singularity at the origin $\sim |λ|^{-s}$ with a power $s \in (0,2)$ when and only when the density of its singular values has a power law singularity $\sim λ^{-σ}$ with a power $σ= s/(4-s)$. These results are obtained analytically in the limit $N \rightarrow \infty$. We supplement these results with numerical simulations for large but finite N and discuss finite size effects for the most common ensembles of isotropic random matrices.
△ Less
Submitted 6 May, 2013; v1 submitted 21 March, 2013;
originally announced March 2013.
-
Financial instability from local market measures
Authors:
Marco Bardoscia,
Giacomo Livan,
Matteo Marsili
Abstract:
We study the emergence of instabilities in a stylized model of a financial market, when different market actors calculate prices according to different (local) market measures. We derive typical properties for ensembles of large random markets using techniques borrowed from statistical mechanics of disordered systems. We show that, depending on the number of financial instruments available and on…
▽ More
We study the emergence of instabilities in a stylized model of a financial market, when different market actors calculate prices according to different (local) market measures. We derive typical properties for ensembles of large random markets using techniques borrowed from statistical mechanics of disordered systems. We show that, depending on the number of financial instruments available and on the heterogeneity of local measures, the market moves from an arbitrage-free phase to an unstable one, where the complexity of the market - as measured by the diversity of financial instruments - increases, and arbitrage opportunities arise. A sharp transition separates the two phases. Focusing on two different classes of local measures inspired by real markets strategies, we are able to analytically compute the critical lines, corroborating our findings with numerical simulations.
△ Less
Submitted 1 September, 2012; v1 submitted 2 July, 2012;
originally announced July 2012.
-
On the concentration of large deviations for fat tailed distributions, with application to financial data
Authors:
Mario Filiasi,
Giacomo Livan,
Matteo Marsili,
Maria Peressi,
Erik Vesselli,
Elia Zarinelli
Abstract:
Large deviations for fat tailed distributions, i.e. those that decay slower than exponential, are not only relatively likely, but they also occur in a rather peculiar way where a finite fraction of the whole sample deviation is concentrated on a single variable. The regime of large deviations is separated from the regime of typical fluctuations by a phase transition where the symmetry between the…
▽ More
Large deviations for fat tailed distributions, i.e. those that decay slower than exponential, are not only relatively likely, but they also occur in a rather peculiar way where a finite fraction of the whole sample deviation is concentrated on a single variable. The regime of large deviations is separated from the regime of typical fluctuations by a phase transition where the symmetry between the points in the sample is spontaneously broken. For stochastic processes with a fat tailed microscopic noise, this implies that while typical realizations are well described by a diffusion process with continuous sample paths, large deviation paths are typically discontinuous. For eigenvalues of random matrices with fat tailed distributed elements, a large deviation where the trace of the matrix is anomalously large concentrates on just a single eigenvalue, whereas in the thin tailed world the large deviation affects the whole distribution. These results find a natural application to finance. Since the price dynamics of financial stocks is characterized by fat tailed increments, large fluctuations of stock prices are expected to be realized by discrete jumps. Interestingly, we find that large excursions of prices are more likely realized by continuous drifts rather than by discontinuous jumps. Indeed, auto-correlations suppress the concentration of large deviations. Financial covariance matrices also exhibit an anomalously large eigenvalue, the market mode, as compared to the prediction of random matrix theory. We show that this is explained by a large deviation with excess covariance rather than by one with excess volatility.
△ Less
Submitted 18 June, 2014; v1 submitted 13 January, 2012;
originally announced January 2012.
-
Eigenvalues and Singular Values of Products of Rectangular Gaussian Random Matrices (The Extended Version)
Authors:
Z. Burda,
A. Jarosz,
G. Livan,
M. A. Nowak,
A. Swiech
Abstract:
We consider a product of an arbitrary number of independent rectangular Gaussian random matrices. We derive the mean densities of its eigenvalues and singular values in the thermodynamic limit, eventually verified numerically. These densities are encoded in the form of the so called M-transforms, for which polynomial equations are found. We exploit the methods of planar diagrammatics, enhanced to…
▽ More
We consider a product of an arbitrary number of independent rectangular Gaussian random matrices. We derive the mean densities of its eigenvalues and singular values in the thermodynamic limit, eventually verified numerically. These densities are encoded in the form of the so called M-transforms, for which polynomial equations are found. We exploit the methods of planar diagrammatics, enhanced to the non-Hermitian case, and free random variables, respectively; both are described in the appendices. As particular results of these two main equations, we find the singular behavior of the spectral densities near zero. Moreover, we propose a finite-size form of the spectral density of the product close to the border of its eigenvalues' domain. Also, led by the striking similarity between the two main equations, we put forward a conjecture about a simple relationship between the eigenvalues and singular values of any non-Hermitian random matrix whose spectrum exhibits rotational symmetry around zero.
△ Less
Submitted 21 March, 2011;
originally announced March 2011.
-
Moments of Wishart-Laguerre and Jacobi ensembles of random matrices: application to the quantum transport problem in chaotic cavities
Authors:
Giacomo Livan,
Pierpaolo Vivo
Abstract:
We collect explicit and user-friendly expressions for one-point densities of the real eigenvalues $\{λ_i\}$ of $N\times N$ Wishart-Laguerre and Jacobi random matrices with orthogonal, unitary and symplectic symmetry. Using these formulae, we compute integer moments $τ_n=<\sum_{i=1}^Nλ_i^n>$ for all symmetry classes without any large $N$ approximation. In particular, our results provide exact expre…
▽ More
We collect explicit and user-friendly expressions for one-point densities of the real eigenvalues $\{λ_i\}$ of $N\times N$ Wishart-Laguerre and Jacobi random matrices with orthogonal, unitary and symplectic symmetry. Using these formulae, we compute integer moments $τ_n=<\sum_{i=1}^Nλ_i^n>$ for all symmetry classes without any large $N$ approximation. In particular, our results provide exact expressions for moments of transmission eigenvalues in chaotic cavities with time-reversal or spin-flip symmetry and supporting a finite and arbitrary number of electronic channels in the two incoming leads.
△ Less
Submitted 16 March, 2011; v1 submitted 14 March, 2011;
originally announced March 2011.
-
Eigenvalues and Singular Values of Products of Rectangular Gaussian Random Matrices
Authors:
Z. Burda,
A. Jarosz,
G. Livan,
M. A. Nowak,
A. Swiech
Abstract:
We derive exact analytic expressions for the distributions of eigenvalues and singular values for the product of an arbitrary number of independent rectangular Gaussian random matrices in the limit of large matrix dimensions. We show that they both have power-law behavior at zero and determine the corresponding powers. We also propose a heuristic form of finite size corrections to these expression…
▽ More
We derive exact analytic expressions for the distributions of eigenvalues and singular values for the product of an arbitrary number of independent rectangular Gaussian random matrices in the limit of large matrix dimensions. We show that they both have power-law behavior at zero and determine the corresponding powers. We also propose a heuristic form of finite size corrections to these expressions which very well approximates the distributions for matrices of finite dimensions.
△ Less
Submitted 21 July, 2010;
originally announced July 2010.
