-
Total/dual correlation/coherence, redundancy/synergy, complexity, and O-information for real and complex valued multivariate data
Authors:
Roberto D. Pascual-Marqui,
Kieko Kochi,
Toshihiko Kinoshita
Abstract:
Firstly, assuming Gaussianity, equations for the following information theory measures are presented: total correlation/coherence (TC), dual total correlation/coherence (DTC), O-information, TSE complexity, and the redundancy-synergy index (RSI). Since these measures are functions of the covariance matrix "S" and its inverse "S^-1", the associated Wishart and inverse-Wishart distributions are of n…
▽ More
Firstly, assuming Gaussianity, equations for the following information theory measures are presented: total correlation/coherence (TC), dual total correlation/coherence (DTC), O-information, TSE complexity, and the redundancy-synergy index (RSI). Since these measures are functions of the covariance matrix "S" and its inverse "S^-1", the associated Wishart and inverse-Wishart distributions are of note. The DTC is shown here to be the Kullback-Leibler (KL) divergence for the inverse-Wishart pair "(S^-1)" and its diagonal matrix "diag(S^-1)", shedding light on its interpretation as a measure of "total partial correlation", -lndetP, with test hypothesis H0: P=I, where "P" is the standardized inverse covariance (i.e. P=(D^-1/2)(S^-1)(D^-1/2), with D=diag(S^-1)). The second aim of this paper introduces a generalization of all these measures for structured groups of variables. For instance, consider three or more groups, each consisting of three or more variables, with predominant redundancy within each group, but with synergistic interactions between groups. O-information will miss the between group synergy (since redundancy occurs more often in the system). In contrast, the structured O-information measure presented here will correctly report predominant synergy between groups. This is a relevant generalization towards structured multivariate information measures. A third aim is the presentation of a framework for quantifying the contribution of "connections" between variables, to the system's TC, DTC, O-information, and TSE complexity. A fourth aim is to present a generalization of the redundancy-synergy index for quantifying the contribution of a group of variables to the system's redundancy-synergy balance. Finally, it is shown that the expressions derived here directly apply to data from several other elliptical distributions. All program codes, data files, and executables are available.
△ Less
Submitted 11 July, 2025;
originally announced July 2025.
-
Distance-based Chatterjee correlation: a new generalized robust measure of directed association for multivariate real and complex-valued data
Authors:
Roberto D. Pascual-Marqui,
Kieko Kochi,
Toshihiko Kinoshita
Abstract:
Building upon the Chatterjee correlation (2021: J. Am. Stat. Assoc. 116, p2009) for two real-valued variables, this study introduces a generalized measure of directed association between two vector variables, real or complex-valued, and of possibly different dimensions. The new measure is denoted as the "distance-based Chatterjee correlation", owing to the use here of the "distance transformed dat…
▽ More
Building upon the Chatterjee correlation (2021: J. Am. Stat. Assoc. 116, p2009) for two real-valued variables, this study introduces a generalized measure of directed association between two vector variables, real or complex-valued, and of possibly different dimensions. The new measure is denoted as the "distance-based Chatterjee correlation", owing to the use here of the "distance transformed data" defined in Szekely et al (2007: Ann. Statist. 35, p2769) for the distance correlation. A main property of the new measure, inherited from the original Chatterjee correlation, is its predictive and asymmetric nature: it measures how well one variable can be predicted by the other, asymmetrically. This allows for inferring the causal direction of the association, by using the method of Blobaum et al (2019: PeerJ Comput. Sci. 1, e169). Since the original Chatterjee correlation is based on ranks, it is not available for complex variables, nor for general multivariate data. The novelty of our work is the extension to multivariate real and complex-valued pairs of vectors, offering a robust measure of directed association in a completely non-parametric setting. Informally, the intuitive assumption used here is that distance correlation is mathematically equivalent to Pearson's correlation when applied to "distance transformed" data. The next logical step is to compute Chatterjee's correlation on the same "distance transformed" data, thereby extending the analysis to multivariate vectors of real and complex valued data. As a bonus, the new measure here is robust to outliers, which is not true for the distance correlation of Szekely et al. Additionally, this approach allows for inference regarding the causal direction of the association between the variables.
△ Less
Submitted 25 June, 2024; v1 submitted 24 June, 2024;
originally announced June 2024.
-
Lagged coherence: explicit and testable definition
Authors:
Roberto D. Pascual-Marqui,
Kieko Kochi,
Toshihiko Kinoshita
Abstract:
Measures of association between cortical regions based on activity signals provide useful information for studying brain functional connectivity. Difficulties occur with signals of electric neuronal activity, where an observed signal is a mixture, i.e. an instantaneous weighted average of the true, unobserved signals from all regions, due to volume conduction and low spatial resolution. This is wh…
▽ More
Measures of association between cortical regions based on activity signals provide useful information for studying brain functional connectivity. Difficulties occur with signals of electric neuronal activity, where an observed signal is a mixture, i.e. an instantaneous weighted average of the true, unobserved signals from all regions, due to volume conduction and low spatial resolution. This is why measures of lagged association are of interest, since at least theoretically, "lagged association" is of physiological origin. In contrast, the actual physiological instantaneous zero-lag association is masked and confounded by the mixing artifact. A minimum requirement for a measure of lagged association is that it must not tend to zero with an increase of strength of true instantaneous physiological association. Such biased measures cannot tell apart if a change in its value is due to a change in lagged or a change in instantaneous association. An explicit testable definition for frequency domain lagged connectivity between two multivariate time series is proposed. It is endowed with two important properties: it is invariant to non-singular linear transformations of each vector time series separately, and it is invariant to instantaneous association. As a first sanity check: in the case of two univariate time series, the new definition leads back to the bivariate lagged coherence of 2007 (eqs 25 and 26 in https://doi.org/10.48550/arXiv.0706.1776). As a second stronger sanity check: in the case of a univariate and multivariate vector time series, the new measure presented here leads back to the original multivariate lagged coherence of 2007 (eq 31 in https://doi.org/10.48550/arXiv.0711.1455), which again trivially includes the bivariate case.
△ Less
Submitted 7 January, 2024; v1 submitted 24 November, 2023;
originally announced November 2023.
-
Innovations orthogonalization: a solution to the major pitfalls of EEG/MEG "leakage correction"
Authors:
Roberto D. Pascual-Marqui,
Rolando J. Biscay,
Jorge Bosch-Bayard,
Pascal Faber,
Toshihiko Kinoshita,
Kieko Kochi,
Patricia Milz,
Keiichiro Nishida,
Masafumi Yoshimura
Abstract:
The problem of interest here is the study of brain functional and effective connectivity based on non-invasive EEG-MEG inverse solution time series. These signals generally have low spatial resolution, such that an estimated signal at any one site is an instantaneous linear mixture of the true, actual, unobserved signals across all cortical sites. False connectivity can result from analysis of the…
▽ More
The problem of interest here is the study of brain functional and effective connectivity based on non-invasive EEG-MEG inverse solution time series. These signals generally have low spatial resolution, such that an estimated signal at any one site is an instantaneous linear mixture of the true, actual, unobserved signals across all cortical sites. False connectivity can result from analysis of these low-resolution signals. Recent efforts toward "unmixing" have been developed, under the name of "leakage correction". One recent noteworthy approach is that by Colclough et al (2015 NeuroImage, 117:439-448), which forces the inverse solution signals to have zero cross-correlation at lag zero. One goal is to show that Colclough's method produces false human connectomes under very broad conditions. The second major goal is to develop a new solution, that appropriately "unmixes" the inverse solution signals, based on innovations orthogonalization. The new method first fits a multivariate autoregression to the inverse solution signals, giving the mixed innovations. Second, the mixed innovations are orthogonalized. Third, the mixed and orthogonalized innovations allow the estimation of the "unmixing" matrix, which is then finally used to "unmix" the inverse solution signals. It is shown that under very broad conditions, the new method produces proper human connectomes, even when the signals are not generated by an autoregressive model.
△ Less
Submitted 23 August, 2017; v1 submitted 20 August, 2017;
originally announced August 2017.
-
A measure of association between vectors based on "similarity covariance"
Authors:
Roberto D. Pascual-Marqui,
Dietrich Lehmann,
Kieko Kochi,
Toshihiko Kinoshita,
Naoto Yamada
Abstract:
The "maximum similarity correlation" definition introduced in this study is motivated by the seminal work of Szekely et al on "distance covariance" (Ann. Statist. 2007, 35: 2769-2794; Ann. Appl. Stat. 2009, 3: 1236-1265). Instead of using Euclidean distances "d" as in Szekely et al, we use "similarity", which can be defined as "exp(-d/s)", where the scaling parameter s>0 controls how rapidly the s…
▽ More
The "maximum similarity correlation" definition introduced in this study is motivated by the seminal work of Szekely et al on "distance covariance" (Ann. Statist. 2007, 35: 2769-2794; Ann. Appl. Stat. 2009, 3: 1236-1265). Instead of using Euclidean distances "d" as in Szekely et al, we use "similarity", which can be defined as "exp(-d/s)", where the scaling parameter s>0 controls how rapidly the similarity falls off with distance. Scale parameters are chosen by maximizing the similarity correlation. The motivation for using "similarity" originates in spectral clustering theory (see e.g. Ng et al 2001, Advances in Neural Information Processing Systems 14: 849-856). We show that a particular form of similarity correlation is asymptotically equivalent to distance correlation for large values of the scale parameter. Furthermore, we extend similarity correlation to coherence between complex valued vectors, including its partitioning into real and imaginary contributions. Several toy examples are used for comparing distance and similarity correlations. For instance, points on a noiseless straight line give distance and similarity correlation values equal to 1; but points on a noiseless circle produces near zero distance correlation (dCorr=0.02) while the similarity correlation is distinctly non zero (sCorr=0.36). In distinction to the distance approach, similarity gives more importance to small distances, which emphasizes the local properties of functional relations. This paper represents a preliminary empirical study, showing that the novel similarity association has some distinct practical advantages over distance based association.For the sake of reproducible research, the software code implementing all methods here (using lazarus free-pascal "www.lazarus.freepascal.org"), including all test data, are freely available at: "sites.google.com/site/pascualmarqui/home/similaritycovariance".
△ Less
Submitted 4 February, 2013; v1 submitted 17 January, 2013;
originally announced January 2013.
-
Cortical current source connectivity by means of partial coherence fields
Authors:
Roberto D. Pascual-Marqui,
Rolando J. Biscay,
Pedro A. Valdes-Sosa,
Jorge Bosch-Bayard,
Jorge J. Riera-Diaz
Abstract:
An important field of research in functional neuroimaging is the discovery of integrated, distributed brain systems and networks, whose different regions need to work in unison for normal functioning.
The EEG is a non-invasive technique that can provide information for massive connectivity analyses. Cortical signals of time varying electric neuronal activity can be estimated from the EEG. Althou…
▽ More
An important field of research in functional neuroimaging is the discovery of integrated, distributed brain systems and networks, whose different regions need to work in unison for normal functioning.
The EEG is a non-invasive technique that can provide information for massive connectivity analyses. Cortical signals of time varying electric neuronal activity can be estimated from the EEG. Although such techniques have very high time resolution, two cortical signals even at distant locations will appear to be highly similar due to the low spatial resolution nature of the EEG.
In this study a method for eliminating the effect of common sources due to low spatial resolution is presented. It is based on an efficient estimation of the whole-cortex partial coherence matrix. Using as a starting point any linear EEG tomography that satisfies the EEG forward equation, it is shown that the generalized partial coherences for the cortical grey matter current density time series are invariant to the selected tomography. It is empirically shown with simulation experiments that the generalized partial coherences have higher spatial resolution than the classical coherences. The results demonstrate that with as little as 19 electrodes, lag-connected brain regions can often be missed and misplaced even with lagged coherence measures, while the new method detects and localizes correctly the connected regions using the lagged partial coherences.
△ Less
Submitted 1 August, 2011;
originally announced August 2011.
-
Interaction patterns of brain activity across space, time and frequency. Part I: methods
Authors:
Roberto D. Pascual-Marqui,
Rolando J. Biscay-Lirio
Abstract:
We consider exploratory methods for the discovery of cortical functional connectivity. Typically, data for the i-th subject (i=1...NS) is represented as an NVxNT matrix Xi, corresponding to brain activity sampled at NT moments in time from NV cortical voxels. A widely used method of analysis first concatenates all subjects along the temporal dimension, and then performs an independent component an…
▽ More
We consider exploratory methods for the discovery of cortical functional connectivity. Typically, data for the i-th subject (i=1...NS) is represented as an NVxNT matrix Xi, corresponding to brain activity sampled at NT moments in time from NV cortical voxels. A widely used method of analysis first concatenates all subjects along the temporal dimension, and then performs an independent component analysis (ICA) for estimating the common cortical patterns of functional connectivity. There exist many other interesting variations of this technique, as reviewed in [Calhoun et al. 2009 Neuroimage 45: S163-172]. We present methods for the more general problem of discovering functional connectivity occurring at all possible time lags. For this purpose, brain activity is viewed as a function of space and time, which allows the use of the relatively new techniques of functional data analysis [Ramsay & Silverman 2005: Functional data analysis. New York: Springer]. In essence, our method first vectorizes the data from each subject, which constitutes the natural discrete representation of a function of several variables, followed by concatenation of all subjects. The singular value decomposition (SVD), as well as the ICA of this new matrix of dimension [rows=(NT*NV); columns=NS] will reveal spatio-temporal patterns of connectivity. As a further example, in the case of EEG neuroimaging, Xi of size NVxNW may represent spectral density for electric neuronal activity at NW discrete frequencies from NV cortical voxels, from the i-th EEG epoch. In this case our functional data analysis approach would reveal coupling of brain regions at possibly different frequencies.
△ Less
Submitted 15 March, 2011; v1 submitted 15 March, 2011;
originally announced March 2011.
-
Dynamic interactions in terms of senders, hubs, and receivers (SHR) using the singular value decomposition of time series: Theory and brain connectivity applications
Authors:
Roberto D. Pascual-Marqui,
Rolando J. Biscay-Lirio
Abstract:
Understanding of normal and pathological brain function requires the identification and localization of functional connections between specialized regions. The availability of high time resolution signals of electric neuronal activity at several regions offers information for quantifying the connections in terms of information flow. When the signals cover the whole cortex, the number of connection…
▽ More
Understanding of normal and pathological brain function requires the identification and localization of functional connections between specialized regions. The availability of high time resolution signals of electric neuronal activity at several regions offers information for quantifying the connections in terms of information flow. When the signals cover the whole cortex, the number of connections is very large, making visualization and interpretation very difficult. We introduce here the singular value decomposition of time-lagged multiple signals, which localizes the senders, hubs, and receivers (SHR) of information transmission. Unlike methods that operate on large connectivity matrices, such as correlation thresholding and graph-theoretic analyses, this method operates on the multiple time series directly, providing 3D brain images that assign a score to each location in terms of its sending, relaying, and receiving capacity. The scope of the method is general and encompasses other applications outside the field of brain connectivity.
△ Less
Submitted 6 September, 2010; v1 submitted 3 September, 2010;
originally announced September 2010.
-
Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: frequency decomposition
Authors:
Roberto D. Pascual-Marqui
Abstract:
Measures of linear dependence (coherence) and nonlinear dependence (phase synchronization) between any number of multivariate time series are defined. The measures are expressed as the sum of lagged dependence and instantaneous dependence. The measures are non-negative, and take the value zero only when there is independence of the pertinent type. These measures are defined in the frequency doma…
▽ More
Measures of linear dependence (coherence) and nonlinear dependence (phase synchronization) between any number of multivariate time series are defined. The measures are expressed as the sum of lagged dependence and instantaneous dependence. The measures are non-negative, and take the value zero only when there is independence of the pertinent type. These measures are defined in the frequency domain and are applicable to stationary and non-stationary time series. These new results extend and refine significantly those presented in a previous technical report (Pascual-Marqui 2007, arXiv:0706.1776 [stat.ME], https://arxiv.boxedpaper.com/abs/0706.1776), and have been largely motivated by the seminal paper on linear feedback by Geweke (1982 JASA 77:304-313). One important field of application is neurophysiology, where the time series consist of electric neuronal activity at several brain locations. Coherence and phase synchronization are interpreted as "connectivity" between locations. However, any measure of dependence is highly contaminated with an instantaneous, non-physiological contribution due to volume conduction and low spatial resolution. The new techniques remove this confounding factor considerably. Moreover, the measures of dependence can be applied to any number of brain areas jointly, i.e. distributed cortical networks, whose activity can be estimated with eLORETA (Pascual-Marqui 2007, arXiv:0710.3341 [math-ph]).
△ Less
Submitted 9 November, 2007;
originally announced November 2007.
-
Coherence and phase synchronization: generalization to pairs of multivariate time series, and removal of zero-lag contributions
Authors:
Roberto D. Pascual-Marqui
Abstract:
Coherence and phase synchronization between time series corresponding to different spatial locations are usually interpreted as indicators of the connectivity between locations. In neurophysiology, time series of electric neuronal activity are essential for studying brain interconnectivity. Such signals can either be invasively measured from depth electrodes, or computed from very high time reso…
▽ More
Coherence and phase synchronization between time series corresponding to different spatial locations are usually interpreted as indicators of the connectivity between locations. In neurophysiology, time series of electric neuronal activity are essential for studying brain interconnectivity. Such signals can either be invasively measured from depth electrodes, or computed from very high time resolution, non-invasive, extracranial recordings of scalp electric potential differences (EEG: electroencephalogram) and magnetic fields (MEG: magnetoencephalogram) by means of a tomography such as sLORETA (standardized low resolution brain electromagnetic tomography). There are two problems in this case. First, in the usual situation of unknown cortical geometry, the estimated signal at each brain location is a vector with three components (i.e. a current density vector), which means that coherence and phase synchronization must be generalized to pairs of multivariate time series. Second, the inherent low spatial resolution of the EEG/MEG tomography introduces artificially high zero-lag coherence and phase synchronization. In this report, solutions to both problems are presented. Two additional generalizations are briefly mentioned: (1) conditional coherence and phase synchronization; and (2) non-stationary time-frequency analysis. Finally, a non-parametric randomization method for connectivity significance testing is outlined. The new connectivity measures proposed here can be applied to pairs of univariate EEG/MEG signals, as is traditional in the published literature. However, these calculations cannot be interpreted as connectivity, since it is in general incorrect to associate an extracranial electrode or sensor to the underlying cortex.
△ Less
Submitted 12 July, 2007; v1 submitted 12 June, 2007;
originally announced June 2007.
