-
Conditional multi-step attribution for climate forcings
Authors:
Christopher R. Wentland,
Michael Weylandt,
Laura P. Swiler,
Thomas S. Ehrmann,
Diana Bull
Abstract:
Attribution of climate impacts to a source forcing is critical to understanding, communicating, and addressing the effects of human influence on the climate. While standard attribution methods, such as optimal fingerprinting, have been successfully applied to long-term, widespread effects such as global surface temperature warming, they often struggle in low signal-to-noise regimes, typical of sho…
▽ More
Attribution of climate impacts to a source forcing is critical to understanding, communicating, and addressing the effects of human influence on the climate. While standard attribution methods, such as optimal fingerprinting, have been successfully applied to long-term, widespread effects such as global surface temperature warming, they often struggle in low signal-to-noise regimes, typical of short-term climate forcings or climate variables which are loosely related to the forcing. Single-step approaches, which directly relate a source forcing and final impact, are unable to utilize additional climate information to improve attribution certainty. To address this shortcoming, this paper presents a novel multi-step attribution approach which is capable of analyzing multiple variables conditionally. A connected series of climate effects are treated as dependent, and relationships found in intermediary steps of a causal pathway are leveraged to better characterize the forcing impact. This enables attribution of the forcing level responsible for the observed impacts, while equivalent single-step approaches fail. Utilizing a scalar feature describing the forcing impact, simple forcing response models, and a conditional Bayesian formulation, this method can incorporate several causal pathways to identify the correct forcing magnitude. As an exemplar of a short-term, high-variance forcing, we demonstrate this method for the 1991 eruption of Mt. Pinatubo. Results indicate that including stratospheric and surface temperature and radiative flux measurements increases attribution certainty compared to analyses derived solely from temperature measurements. This framework has potential to improve climate attribution assessments for both geoengineering projects and long-term climate change, for which standard attribution methods may fail.
△ Less
Submitted 2 September, 2024;
originally announced September 2024.
-
Optimal accuracy for linear sets of equations with the graph Laplacian
Authors:
Richard B. Lehoucq,
Michael Weylandt,
Jonathan W. Berry
Abstract:
We show that certain Graph Laplacian linear sets of equations exhibit optimal accuracy, guaranteeing that the relative error is no larger than the norm of the relative residual and that optimality occurs for carefully chosen right-hand sides. Such sets of equations arise in PageRank and Markov chain theory. We establish new relationships among the PageRank teleportation parameter, the Markov chain…
▽ More
We show that certain Graph Laplacian linear sets of equations exhibit optimal accuracy, guaranteeing that the relative error is no larger than the norm of the relative residual and that optimality occurs for carefully chosen right-hand sides. Such sets of equations arise in PageRank and Markov chain theory. We establish new relationships among the PageRank teleportation parameter, the Markov chain discount, and approximations to linear sets of equations. The set of optimally accurate systems can be separated into two groups for an undirected graph -- those that achieve optimality asymptotically with the graph size and those that do not -- determined by the angle between the right-hand side of the linear system and the vector of all ones. We provide supporting numerical experiments.
△ Less
Submitted 13 May, 2024;
originally announced May 2024.
-
To the Fairness Frontier and Beyond: Identifying, Quantifying, and Optimizing the Fairness-Accuracy Pareto Frontier
Authors:
Camille Olivia Little,
Michael Weylandt,
Genevera I Allen
Abstract:
Algorithmic fairness has emerged as an important consideration when using machine learning to make high-stakes societal decisions. Yet, improved fairness often comes at the expense of model accuracy. While aspects of the fairness-accuracy tradeoff have been studied, most work reports the fairness and accuracy of various models separately; this makes model comparisons nearly impossible without a mo…
▽ More
Algorithmic fairness has emerged as an important consideration when using machine learning to make high-stakes societal decisions. Yet, improved fairness often comes at the expense of model accuracy. While aspects of the fairness-accuracy tradeoff have been studied, most work reports the fairness and accuracy of various models separately; this makes model comparisons nearly impossible without a model-agnostic metric that reflects the balance of the two desiderata. We seek to identify, quantify, and optimize the empirical Pareto frontier of the fairness-accuracy tradeoff. Specifically, we identify and outline the empirical Pareto frontier through Tradeoff-between-Fairness-and-Accuracy (TAF) Curves; we then develop a metric to quantify this Pareto frontier through the weighted area under the TAF Curve which we term the Fairness-Area-Under-the-Curve (FAUC). TAF Curves provide the first empirical, model-agnostic characterization of the Pareto frontier, while FAUC provides the first metric to impartially compare model families on both fairness and accuracy. Both TAF Curves and FAUC can be employed with all group fairness definitions and accuracy measures. Next, we ask: Is it possible to expand the empirical Pareto frontier and thus improve the FAUC for a given collection of fitted models? We answer affirmately by developing a novel fair model stacking framework, FairStacks, that solves a convex program to maximize the accuracy of model ensemble subject to a score-bias constraint. We show that optimizing with FairStacks always expands the empirical Pareto frontier and improves the FAUC; we additionally study other theoretical properties of our proposed approach. Finally, we empirically validate TAF, FAUC, and FairStacks through studies on several real benchmark data sets, showing that FairStacks leads to major improvements in FAUC that outperform existing algorithmic fairness approaches.
△ Less
Submitted 31 May, 2022;
originally announced June 2022.
-
Multivariate Analysis for Multiple Network Data via Semi-Symmetric Tensor PCA
Authors:
Michael Weylandt,
George Michailidis
Abstract:
Network data are commonly collected in a variety of applications, representing either directly measured or statistically inferred connections between features of interest. In an increasing number of domains, these networks are collected over time, such as interactions between users of a social media platform on different days, or across multiple subjects, such as in multi-subject studies of brain…
▽ More
Network data are commonly collected in a variety of applications, representing either directly measured or statistically inferred connections between features of interest. In an increasing number of domains, these networks are collected over time, such as interactions between users of a social media platform on different days, or across multiple subjects, such as in multi-subject studies of brain connectivity. When analyzing multiple large networks, dimensionality reduction techniques are often used to embed networks in a more tractable low-dimensional space. To this end, we develop a framework for principal components analysis (PCA) on collections of networks via a specialized tensor decomposition we term Semi-Symmetric Tensor PCA or SS-TPCA. We derive computationally efficient algorithms for computing our proposed SS-TPCA decomposition and establish statistical efficiency of our approach under a standard low-rank signal plus noise model. Remarkably, we show that SS-TPCA achieves the same estimation accuracy as classical matrix PCA, with error proportional to the square root of the number of vertices in the network and not the number of edges as might be expected. Our framework inherits many of the strengths of classical PCA and is suitable for a wide range of unsupervised learning tasks, including identifying principal networks, isolating meaningful changepoints or outlying observations, and for characterizing the "variability network" of the most varying edges. Finally, we demonstrate the effectiveness of our proposal on simulated data and on an example from empirical legal studies. The techniques used to establish our main consistency results are surprisingly straightforward and may find use in a variety of other network analysis problems.
△ Less
Submitted 2 September, 2022; v1 submitted 9 February, 2022;
originally announced February 2022.
-
Sparse Partial Least Squares for Coarse Noisy Graph Alignment
Authors:
Michael Weylandt,
George Michailidis,
T. Mitchell Roddenberry
Abstract:
Graph signal processing (GSP) provides a powerful framework for analyzing signals arising in a variety of domains. In many applications of GSP, multiple network structures are available, each of which captures different aspects of the same underlying phenomenon. To integrate these different data sources, graph alignment techniques attempt to find the best correspondence between vertices of two gra…
▽ More
Graph signal processing (GSP) provides a powerful framework for analyzing signals arising in a variety of domains. In many applications of GSP, multiple network structures are available, each of which captures different aspects of the same underlying phenomenon. To integrate these different data sources, graph alignment techniques attempt to find the best correspondence between vertices of two graphs. We consider a generalization of this problem, where there is no natural one-to-one mapping between vertices, but where there is correspondence between the community structures of each graph. Because we seek to learn structure at this higher community level, we refer to this problem as "coarse" graph alignment. To this end, we propose a novel regularized partial least squares method which both incorporates the observed graph structures and imposes sparsity in order to reflect the underlying block community structure. We provide efficient algorithms for our method and demonstrate its effectiveness in simulations.
△ Less
Submitted 6 April, 2021;
originally announced April 2021.
-
Simultaneous Grouping and Denoising via Sparse Convex Wavelet Clustering
Authors:
Michael Weylandt,
T. Mitchell Roddenberry,
Genevera I. Allen
Abstract:
Clustering is a ubiquitous problem in data science and signal processing. In many applications where we observe noisy signals, it is common practice to first denoise the data, perhaps using wavelet denoising, and then to apply a clustering algorithm. In this paper, we develop a sparse convex wavelet clustering approach that simultaneously denoises and discovers groups. Our approach utilizes convex…
▽ More
Clustering is a ubiquitous problem in data science and signal processing. In many applications where we observe noisy signals, it is common practice to first denoise the data, perhaps using wavelet denoising, and then to apply a clustering algorithm. In this paper, we develop a sparse convex wavelet clustering approach that simultaneously denoises and discovers groups. Our approach utilizes convex fusion penalties to achieve agglomeration and group-sparse penalties to denoise through sparsity in the wavelet domain. In contrast to common practice which denoises then clusters, our method is a unified, convex approach that performs both simultaneously. Our method yields denoised (wavelet-sparse) cluster centroids that both improve interpretability and data compression. We demonstrate our method on synthetic examples and in an application to NMR spectroscopy.
△ Less
Submitted 3 March, 2021; v1 submitted 8 December, 2020;
originally announced December 2020.
-
Automatic Registration and Clustering of Time Series
Authors:
Michael Weylandt,
George Michailidis
Abstract:
Clustering of time series data exhibits a number of challenges not present in other settings, notably the problem of registration (alignment) of observed signals. Typical approaches include pre-registration to a user-specified template or time warping approaches which attempt to optimally align series with a minimum of distortion. For many signals obtained from recording or sensing devices, these…
▽ More
Clustering of time series data exhibits a number of challenges not present in other settings, notably the problem of registration (alignment) of observed signals. Typical approaches include pre-registration to a user-specified template or time warping approaches which attempt to optimally align series with a minimum of distortion. For many signals obtained from recording or sensing devices, these methods may be unsuitable as a template signal is not available for pre-registration, while the distortion of warping approaches may obscure meaningful temporal information. We propose a new method for automatic time series alignment within a clustering problem. Our approach, Temporal Registration using Optimal Unitary Transformations (TROUT), is based on a novel dissimilarity measure between time series that is easy to compute and automatically identifies optimal alignment between pairs of time series. By embedding our new measure in a optimization formulation, we retain well-known advantages of computational and statistical performance. We provide an efficient algorithm for TROUT-based clustering and demonstrate its superior performance over a range of competitors.
△ Less
Submitted 10 February, 2021; v1 submitted 8 December, 2020;
originally announced December 2020.
-
Multi-Rank Sparse and Functional PCA: Manifold Optimization and Iterative Deflation Techniques
Authors:
Michael Weylandt
Abstract:
We consider the problem of estimating multiple principal components using the recently-proposed Sparse and Functional Principal Components Analysis (SFPCA) estimator. We first propose an extension of SFPCA which estimates several principal components simultaneously using manifold optimization techniques to enforce orthogonality constraints. While effective, this approach is computationally burdens…
▽ More
We consider the problem of estimating multiple principal components using the recently-proposed Sparse and Functional Principal Components Analysis (SFPCA) estimator. We first propose an extension of SFPCA which estimates several principal components simultaneously using manifold optimization techniques to enforce orthogonality constraints. While effective, this approach is computationally burdensome so we also consider iterative deflation approaches which take advantage of existing fast algorithms for rank-one SFPCA. We show that alternative deflation schemes can more efficiently extract signal from the data, in turn improving estimation of subsequent components. Finally, we compare the performance of our manifold optimization and deflation techniques in a scenario where orthogonality does not hold and find that they still lead to significantly improved performance.
△ Less
Submitted 10 October, 2019; v1 submitted 28 July, 2019;
originally announced July 2019.
-
Multivariate Modeling of Natural Gas Spot Trading Hubs Incorporating Futures Market Realized Volatility
Authors:
Michael Weylandt,
Yu Han,
Katherine B. Ensor
Abstract:
Financial markets for Liquified Natural Gas (LNG) are an important and rapidly-growing segment of commodities markets. Like other commodities markets, there is an inherent spatial structure to LNG markets, with different price dynamics for different points of delivery hubs. Certain hubs support highly liquid markets, allowing efficient and robust price discovery, while others are highly illiquid,…
▽ More
Financial markets for Liquified Natural Gas (LNG) are an important and rapidly-growing segment of commodities markets. Like other commodities markets, there is an inherent spatial structure to LNG markets, with different price dynamics for different points of delivery hubs. Certain hubs support highly liquid markets, allowing efficient and robust price discovery, while others are highly illiquid, limiting the effectiveness of standard risk management techniques. We propose a joint modeling strategy, which uses high-frequency information from thickly-traded hubs to improve volatility estimation and risk management at thinly traded hubs. The resulting model has superior in- and out-of-sample predictive performance, particularly for several commonly used risk management metrics, demonstrating that joint modeling is indeed possible and useful. To improve estimation, a Bayesian estimation strategy is employed and data-driven weakly informative priors are suggested. Our model is robust to sparse data and can be effectively used in any market with similar irregular patterns of data availability.
△ Less
Submitted 23 July, 2019;
originally announced July 2019.
-
Splitting Methods for Convex Bi-Clustering and Co-Clustering
Authors:
Michael Weylandt
Abstract:
Co-Clustering, the problem of simultaneously identifying clusters across multiple aspects of a data set, is a natural generalization of clustering to higher-order structured data. Recent convex formulations of bi-clustering and tensor co-clustering, which shrink estimated centroids together using a convex fusion penalty, allow for global optimality guarantees and precise theoretical analysis, but…
▽ More
Co-Clustering, the problem of simultaneously identifying clusters across multiple aspects of a data set, is a natural generalization of clustering to higher-order structured data. Recent convex formulations of bi-clustering and tensor co-clustering, which shrink estimated centroids together using a convex fusion penalty, allow for global optimality guarantees and precise theoretical analysis, but their computational properties have been less well studied. In this note, we present three efficient operator-splitting methods for the convex co-clustering problem: a standard two-block ADMM, a Generalized ADMM which avoids an expensive tensor Sylvester equation in the primal update, and a three-block ADMM based on the operator splitting scheme of Davis and Yin. Theoretical complexity analysis suggests, and experimental evidence confirms, that the Generalized ADMM is far more efficient for large problems.
△ Less
Submitted 8 July, 2019; v1 submitted 17 January, 2019;
originally announced January 2019.
-
Dynamic Visualization and Fast Computation for Convex Clustering via Algorithmic Regularization
Authors:
Michael Weylandt,
John Nagorski,
Genevera I. Allen
Abstract:
Convex clustering is a promising new approach to the classical problem of clustering, combining strong performance in empirical studies with rigorous theoretical foundations. Despite these advantages, convex clustering has not been widely adopted, due to its computationally intensive nature and its lack of compelling visualizations. To address these impediments, we introduce Algorithmic Regulariza…
▽ More
Convex clustering is a promising new approach to the classical problem of clustering, combining strong performance in empirical studies with rigorous theoretical foundations. Despite these advantages, convex clustering has not been widely adopted, due to its computationally intensive nature and its lack of compelling visualizations. To address these impediments, we introduce Algorithmic Regularization, an innovative technique for obtaining high-quality estimates of regularization paths using an iterative one-step approximation scheme. We justify our approach with a novel theoretical result, guaranteeing global convergence of the approximate path to the exact solution under easily-checked non-data-dependent assumptions. The application of algorithmic regularization to convex clustering yields the Convex Clustering via Algorithmic Regularization Paths (CARP) algorithm for computing the clustering solution path. On example data sets from genomics and text analysis, CARP delivers over a 100-fold speed-up over existing methods, while attaining a finer approximation grid than standard methods. Furthermore, CARP enables improved visualization of clustering solutions: the fine solution grid returned by CARP can be used to construct a convex clustering-based dendrogram, as well as forming the basis of a dynamic path-wise visualization based on modern web technologies. Our methods are implemented in the open-source R package clustRviz, available at https://github.com/DataSlingers/clustRviz.
△ Less
Submitted 8 July, 2019; v1 submitted 5 January, 2019;
originally announced January 2019.
-
Sparse and Functional Principal Components Analysis
Authors:
Genevera I. Allen,
Michael Weylandt
Abstract:
Regularized variants of Principal Components Analysis, especially Sparse PCA and Functional PCA, are among the most useful tools for the analysis of complex high-dimensional data. Many examples of massive data, have both sparse and functional (smooth) aspects and may benefit from a regularization scheme that can capture both forms of structure. For example, in neuro-imaging data, the brain's respo…
▽ More
Regularized variants of Principal Components Analysis, especially Sparse PCA and Functional PCA, are among the most useful tools for the analysis of complex high-dimensional data. Many examples of massive data, have both sparse and functional (smooth) aspects and may benefit from a regularization scheme that can capture both forms of structure. For example, in neuro-imaging data, the brain's response to a stimulus may be restricted to a discrete region of activation (spatial sparsity), while exhibiting a smooth response within that region. We propose a unified approach to regularized PCA which can induce both sparsity and smoothness in both the row and column principal components. Our framework generalizes much of the previous literature, with sparse, functional, two-way sparse, and two-way functional PCA all being special cases of our approach. Our method permits flexible combinations of sparsity and smoothness that lead to improvements in feature selection and signal recovery, as well as more interpretable PCA factors. We demonstrate the efficacy of our method on simulated data and a neuroimaging example on EEG data.
△ Less
Submitted 19 August, 2019; v1 submitted 11 September, 2013;
originally announced September 2013.
