Showing 1–10 of 10 results for author: Wakayama, T

Clustered Factor Analysis for Multivariate Spatial Data

Authors: Yanxiu Jin, Tomoya Wakayama, Renhe Jiang, Shonosuke Sugasawa

Abstract: Factor analysis has been extensively used to reveal the dependence structures among multivariate variables, offering valuable insight in various fields. However, it cannot incorporate the spatial heterogeneity that is typically present in spatial data. To address this issue, we introduce an effective method specifically designed to discover the potential dependence structures in multivariate spati… ▽ More Factor analysis has been extensively used to reveal the dependence structures among multivariate variables, offering valuable insight in various fields. However, it cannot incorporate the spatial heterogeneity that is typically present in spatial data. To address this issue, we introduce an effective method specifically designed to discover the potential dependence structures in multivariate spatial data. Our approach assumes that spatial locations can be approximately divided into a finite number of clusters, with locations within the same cluster sharing similar dependence structures. By leveraging an iterative algorithm that combines spatial clustering with factor analysis, we simultaneously detect spatial clusters and estimate a unique factor model for each cluster. The proposed method is evaluated through comprehensive simulation studies, demonstrating its flexibility. In addition, we apply the proposed method to a dataset of railway station attributes in the Tokyo metropolitan area, highlighting its practical applicability and effectiveness in uncovering complex spatial dependencies. △ Less

Submitted 13 November, 2024; v1 submitted 11 September, 2024; originally announced September 2024.

Ensemble Prediction via Covariate-dependent Stacking

Authors: Tomoya Wakayama, Shonosuke Sugasawa

Abstract: This study proposes a novel approach to ensemble prediction, called ``covariate-dependent stacking'' (CDST). Unlike traditional stacking methods, CDST allows model weights to vary flexibly as a function of covariates, thereby enhancing predictive performance in complex scenarios. We formulate the covariate-dependent weights through combinations of basis functions, estimate them by optimizing cross… ▽ More This study proposes a novel approach to ensemble prediction, called ``covariate-dependent stacking'' (CDST). Unlike traditional stacking methods, CDST allows model weights to vary flexibly as a function of covariates, thereby enhancing predictive performance in complex scenarios. We formulate the covariate-dependent weights through combinations of basis functions, estimate them by optimizing cross-validation, and develop an expectation-maximization algorithm, ensuring computational efficiency. To analyze the theoretical properties, we establish an oracle inequality regarding the expected loss to be minimized for estimating model weights. Through comprehensive simulation studies and an application to large-scale land price prediction, we demonstrate that the CDST consistently outperforms conventional model averaging methods, particularly on datasets where some models fail to capture the underlying complexity. Our findings suggest that the CDST is especially valuable for, but not limited to, spatio-temporal prediction problems, offering a powerful tool for researchers and practitioners in various data analysis fields. △ Less

Submitted 27 August, 2024; v1 submitted 19 August, 2024; originally announced August 2024.

Comments: 32 pages

Process-based Inference for Spatial Energetics Using Bayesian Predictive Stacking

Authors: Tomoya Wakayama, Sudipto Banerjee

Abstract: Rapid developments in streaming data technologies have enabled real-time monitoring of human activity that can deliver high-resolution data on health variables over trajectories or paths carved out by subjects as they conduct their daily physical activities. Wearable devices, such as wrist-worn sensors that monitor gross motor activity, have become prevalent and have kindled the emerging field of… ▽ More Rapid developments in streaming data technologies have enabled real-time monitoring of human activity that can deliver high-resolution data on health variables over trajectories or paths carved out by subjects as they conduct their daily physical activities. Wearable devices, such as wrist-worn sensors that monitor gross motor activity, have become prevalent and have kindled the emerging field of "spatial energetics" in environmental health sciences. We devise a Bayesian inferential framework for analyzing such data while accounting for information available on specific spatial coordinates comprising a trajectory or path using a Global Positioning System (GPS) device embedded within the wearable device. We offer full probabilistic inference with uncertainty quantification using spatial-temporal process models adapted for data generated from "actigraph" units as the subject traverses a path or trajectory in their daily routine. Anticipating the need for fast inference for mobile health data, we pursue exact inference using conjugate Bayesian models and employ predictive stacking to assimilate inference across these individual models. This circumvents issues with iterative estimation algorithms such as Markov chain Monte Carlo. We devise Bayesian predictive stacking in this context for models that treat time as discrete epochs and that treat time as continuous. We illustrate our methods with simulation experiments and analysis of data from the Physical Activity through Sustainable Transport Approaches (PASTA-LA) study conducted by the Fielding School of Public Health at the University of California, Los Angeles. △ Less

Submitted 10 September, 2024; v1 submitted 16 May, 2024; originally announced May 2024.

Comments: 37 pages, 13 figures

Bayesian Inference for Consistent Predictions in Overparameterized Nonlinear Regression

Authors: Tomoya Wakayama

Abstract: The remarkable generalization performance of large-scale models has been challenging the conventional wisdom of the statistical learning theory. Although recent theoretical studies have shed light on this behavior in linear models and nonlinear classifiers, a comprehensive understanding of overparameterization in nonlinear regression models is still lacking. This study explores the predictive prop… ▽ More The remarkable generalization performance of large-scale models has been challenging the conventional wisdom of the statistical learning theory. Although recent theoretical studies have shed light on this behavior in linear models and nonlinear classifiers, a comprehensive understanding of overparameterization in nonlinear regression models is still lacking. This study explores the predictive properties of overparameterized nonlinear regression within the Bayesian framework, extending the methodology of the adaptive prior considering the intrinsic spectral structure of the data. Posterior contraction is established for generalized linear and single-neuron models with Lipschitz continuous activation functions, demonstrating the consistency in the predictions of the proposed approach. Moreover, the Bayesian framework enables uncertainty estimation of the predictions. The proposed method was validated via numerical simulations and a real data application, showing its ability to achieve accurate predictions and reliable uncertainty estimates. This work provides a theoretical understanding of the advantages of overparameterization and a principled Bayesian approach to large nonlinear models. △ Less

Submitted 15 June, 2024; v1 submitted 6 April, 2024; originally announced April 2024.

Comments: 27 pages

Reconciling Functional Data Regression with Excess Bases

Authors: Tomoya Wakayama, Hidetoshi Matsui

Abstract: As the development of measuring instruments and computers has accelerated the collection of massive amounts of data, functional data analysis (FDA) has experienced a surge of attention. The FDA methodology treats longitudinal data as a set of functions on which inference, including regression, is performed. Functionalizing data typically involves fitting the data with basis functions. In general,… ▽ More As the development of measuring instruments and computers has accelerated the collection of massive amounts of data, functional data analysis (FDA) has experienced a surge of attention. The FDA methodology treats longitudinal data as a set of functions on which inference, including regression, is performed. Functionalizing data typically involves fitting the data with basis functions. In general, the number of basis functions smaller than the sample size is selected. This paper casts doubt on this convention. Recent statistical theory has revealed the so-called double-descent phenomenon in which excess parameters overcome overfitting and lead to precise interpolation. Applying this idea to choosing the number of bases to be used for functional data, we show that choosing an excess number of bases can lead to more accurate predictions. Specifically, we explored this phenomenon in a functional regression context and examined its validity through numerical experiments. In addition, we introduce two real-world datasets to demonstrate that the double-descent phenomenon goes beyond theoretical and numerical experiments, confirming its importance in practical applications. △ Less

Submitted 7 July, 2024; v1 submitted 3 August, 2023; originally announced August 2023.

Similarity-based Random Partition Distribution for Clustering Functional Data

Authors: Tomoya Wakayama, Shonosuke Sugasawa, Genya Kobayashi

Abstract: Random partition distribution is a crucial tool for model-based clustering. This study advances the field of random partition in the context of functional spatial data, focusing on the challenges posed by hourly population data across various regions and dates. We propose an extension of the generalized Dirichlet process, named the similarity-based generalized Dirichlet process (SGDP)-type distrib… ▽ More Random partition distribution is a crucial tool for model-based clustering. This study advances the field of random partition in the context of functional spatial data, focusing on the challenges posed by hourly population data across various regions and dates. We propose an extension of the generalized Dirichlet process, named the similarity-based generalized Dirichlet process (SGDP)-type distribution, to address the limitations of simple random partition distributions (e.g., those induced by the Dirichlet process), such as an overabundance of clusters. This model prevents excess cluster production and incorporates pairwise similarity information to ensure accurate and meaningful clustering. The theoretical properties of the SGDP-type distribution are studied. Then, SGDP-type random partition is applied to a real-world dataset of hourly population flow in $500\text{m}^2$ meshes in the central part of Tokyo. In this empirical context, our method excels at detecting meaningful patterns in the data while accounting for spatial nuances. The results underscore the adaptability and utility of the method, showcasing its prowess in revealing intricate spatiotemporal dynamics. The proposed random partition will significantly contribute to urban planning, transportation, and policy-making and will be a helpful tool for understanding population dynamics and their implications. △ Less

Submitted 13 March, 2025; v1 submitted 3 August, 2023; originally announced August 2023.

Comments: 31 pages

Bayesian Analysis for Over-parameterized Linear Model via Effective Spectra

Authors: Tomoya Wakayama, Masaaki Imaizumi

Abstract: In high-dimensional Bayesian statistics, various methods have been developed, including prior distributions that induce parameter sparsity to handle many parameters. Yet, these approaches often overlook the rich spectral structure of the covariate matrix, which can be crucial when true signals are not sparse. To address this gap, we introduce a data-adaptive Gaussian prior whose covariance is alig… ▽ More In high-dimensional Bayesian statistics, various methods have been developed, including prior distributions that induce parameter sparsity to handle many parameters. Yet, these approaches often overlook the rich spectral structure of the covariate matrix, which can be crucial when true signals are not sparse. To address this gap, we introduce a data-adaptive Gaussian prior whose covariance is aligned with the leading eigenvectors of the sample covariance. This prior design targets the data's intrinsic complexity rather than its ambient dimension by concentrating the parameter search along principal data directions. We establish contraction rates of the corresponding posterior distribution, which reveal how the mass in the spectrum affects the prediction error bounds. Furthermore, we derive a truncated Gaussian approximation to the posterior (i.e., a Bernstein-von Mises-type result), which allows for uncertainty quantification with a reduced computational burden. Our findings demonstrate that Bayesian methods leveraging spectral information of the data are effective for estimation in non-sparse, high-dimensional settings. △ Less

Submitted 5 May, 2025; v1 submitted 25 May, 2023; originally announced May 2023.

Comments: 46 pages

Spatiotemporal factor models for functional data with application to population map forecast

Authors: Tomoya Wakayama, Shonosuke Sugasawa

Abstract: The proliferation of mobile devices has led to the collection of large amounts of population data. This situation has prompted the need to utilize this rich, multidimensional data in practical applications. In response to this trend, we have integrated functional data analysis (FDA) and factor analysis to address the challenge of predicting hourly population changes across various districts in Tok… ▽ More The proliferation of mobile devices has led to the collection of large amounts of population data. This situation has prompted the need to utilize this rich, multidimensional data in practical applications. In response to this trend, we have integrated functional data analysis (FDA) and factor analysis to address the challenge of predicting hourly population changes across various districts in Tokyo. Specifically, by assuming a Gaussian process, we avoided the large covariance matrix parameters of the multivariate normal distribution. In addition, the data were both time and spatially dependent between districts. To capture these characteristics, a Bayesian factor model was introduced, which modeled the time series of a small number of common factors and expressed the spatial structure through factor loading matrices. Furthermore, the factor loading matrices were made identifiable and sparse to ensure the interpretability of the model. We also proposed a Bayesian shrinkage method as a systematic approach for factor selection. Through numerical experiments and data analysis, we investigated the predictive accuracy and interpretability of our proposed method. We concluded that the flexibility of the method allows for the incorporation of additional time series features, thereby improving its accuracy. △ Less

Submitted 17 July, 2024; v1 submitted 8 February, 2023; originally announced February 2023.

Functional Horseshoe Smoothing for Functional Trend Estimation

Authors: Tomoya Wakayama, Shonosuke Sugasawa

Abstract: Due to developments in instruments and computers, functional observations are increasingly popular. However, effective methodologies for flexibly estimating the underlying trends with valid uncertainty quantification for a sequence of functional data (e.g. functional time series) are still scarce. In this work, we develop a locally adaptive smoothing method, called functional horseshoe smoothing,… ▽ More Due to developments in instruments and computers, functional observations are increasingly popular. However, effective methodologies for flexibly estimating the underlying trends with valid uncertainty quantification for a sequence of functional data (e.g. functional time series) are still scarce. In this work, we develop a locally adaptive smoothing method, called functional horseshoe smoothing, by introducing a shrinkage prior to the general order of differences of functional variables. This allows us to capture abrupt changes by making the most of the shrinkage capability and also to assess uncertainty by Bayesian inference. The fully Bayesian framework allows the selection of the number of basis functions via the posterior predictive loss. We provide theoretical properties of the model, which support the shrinkage ability. Also, by taking advantage of the nature of functional data, this method is able to handle heterogeneously observed data without data augmentation. Simulation studies and real data analysis demonstrate that the proposed method has desirable properties. △ Less

Submitted 20 September, 2022; v1 submitted 21 April, 2022; originally announced April 2022.

Trend Filtering for Functional Data

Authors: Tomoya Wakayama, Shonosuke Sugasawa

Abstract: Despite increasing accessibility to function data, effective methods for flexibly estimating underlying functional trend are still scarce. We thereby develop functional version of trend filtering for estimating trend of functional data indexed by time or on general graph by extending the conventional trend filtering, a powerful nonparametric trend estimation technique, for scalar data. We formulat… ▽ More Despite increasing accessibility to function data, effective methods for flexibly estimating underlying functional trend are still scarce. We thereby develop functional version of trend filtering for estimating trend of functional data indexed by time or on general graph by extending the conventional trend filtering, a powerful nonparametric trend estimation technique, for scalar data. We formulate the new trend filtering by introducing penalty terms based on $L_2$-norm of the differences of adjacent trend functions. We develop an efficient iteration algorithm for optimizing the objective function obtained by orthonormal basis expansion. Furthermore, we introduce additional penalty terms to eliminate redundant basis functions, which leads to automatic adaptation of the number of basis functions. The tuning parameter in the proposed method is selected via cross validation. We demonstrate the proposed method through simulation studies and applications to real world datasets. △ Less

Submitted 18 February, 2022; v1 submitted 6 April, 2021; originally announced April 2021.

Comments: 29 pages