Showing 1–50 of 55 results for author: Chong, K

Robust Federated Learning Over the Air: Combating Heavy-Tailed Noise with Median Anchored Clipping

Authors: Jiaxing Li, Zihan Chen, Kai Fong Ernest Chong, Bikramjit Das, Tony Q. S. Quek, Howard H. Yang

Abstract: Leveraging over-the-air computations for model aggregation is an effective approach to cope with the communication bottleneck in federated edge learning. By exploiting the superposition properties of multi-access channels, this approach facilitates an integrated design of communication and computation, thereby enhancing system privacy while reducing implementation costs. However, the inherent elec… ▽ More Leveraging over-the-air computations for model aggregation is an effective approach to cope with the communication bottleneck in federated edge learning. By exploiting the superposition properties of multi-access channels, this approach facilitates an integrated design of communication and computation, thereby enhancing system privacy while reducing implementation costs. However, the inherent electromagnetic interference in radio channels often exhibits heavy-tailed distributions, giving rise to exceptionally strong noise in globally aggregated gradients that can significantly deteriorate the training performance. To address this issue, we propose a novel gradient clipping method, termed Median Anchored Clipping (MAC), to combat the detrimental effects of heavy-tailed noise. We also derive analytical expressions for the convergence rate of model training with analog over-the-air federated learning under MAC, which quantitatively demonstrates the effect of MAC on training performance. Extensive experimental results show that the proposed MAC algorithm effectively mitigates the impact of heavy-tailed noise, hence substantially enhancing system robustness. △ Less

Submitted 6 July, 2025; v1 submitted 23 September, 2024; originally announced September 2024.

Comments: This is the full version of the paper, and the appendix contains a complete convergence analysis under non-convex conditions

A Performance Bound for the Greedy Algorithm in a Generalized Class of String Optimization Problems

Authors: Brandon Van Over, Bowen Li, Edwin K. P. Chong, Ali Pezeshki

Abstract: We present a simple performance bound for the greedy scheme in string optimization problems that obtains strong results. Our approach vastly generalizes the group of previously established greedy curvature bounds by Conforti and Cornuéjols (1984). We consider three constants, $α_G$, $α_G'$, and $α_G''$ introduced by Conforti and Cornuéjols (1984), that are used in performance bounds of greedy sche… ▽ More We present a simple performance bound for the greedy scheme in string optimization problems that obtains strong results. Our approach vastly generalizes the group of previously established greedy curvature bounds by Conforti and Cornuéjols (1984). We consider three constants, $α_G$, $α_G'$, and $α_G''$ introduced by Conforti and Cornuéjols (1984), that are used in performance bounds of greedy schemes in submodular set optimization. We first generalize both of the $α_G$ and $α_G''$ bounds to string optimization problems in a manner that includes maximizing submodular set functions over matroids as a special case. We then derive a much simpler and computable bound that allows for applications to a far more general class of functions with string domains. We prove that our bound is superior to both the $α_G$ and $α_G''$ bounds and provide a counterexample to show that the $α_G'$ bound is incorrect under the assumptions in Conforti and Cornuéjols (1984). We conclude with two applications. The first is an application of our result to sensor coverage problems. We demonstrate our performance bound in cases where the objective function is set submodular and string submodular. The second is an application to a social welfare maximization problem with black-box utility functions. △ Less

Submitted 8 September, 2024; originally announced September 2024.

On Bounds for Greedy Schemes in String Optimization based on Greedy Curvatures

Authors: Bowen Li, Brandon Van Over, Edwin K. P. Chong, Ali Pezeshki

Abstract: We consider the celebrated bound introduced by Conforti and Cornuéjols (1984) for greedy schemes in submodular optimization. The bound assumes a submodular function defined on a collection of sets forming a matroid and is based on greedy curvature. We show that the bound holds for a very general class of string problems that includes maximizing submodular functions over set matroids as a special c… ▽ More We consider the celebrated bound introduced by Conforti and Cornuéjols (1984) for greedy schemes in submodular optimization. The bound assumes a submodular function defined on a collection of sets forming a matroid and is based on greedy curvature. We show that the bound holds for a very general class of string problems that includes maximizing submodular functions over set matroids as a special case. We also derive a bound that is computable in the sense that they depend only on quantities along the greedy trajectory. We prove that our bound is superior to the greedy curvature bound of Conforti and Cornuéjols. In addition, our bound holds under a condition that is weaker than submodularity. △ Less

Submitted 8 September, 2024; v1 submitted 9 April, 2024; originally announced April 2024.

Comments: This version has been accepted as an invited paper in the 63rd IEEE Conference on Decision and Control, Milan, Italy, December 16--19, 2024

Deep Learning Approach to Forecasting COVID-19 Cases in Residential Buildings of Hong Kong Public Housing Estates: The Role of Environment and Sociodemographics

Authors: E. Leung, J. Guan, KO. Kwok, CT. Hung, CC. Ching, KC. Chong, CHK. Yam, T. Sun, WH. Tsang, EK. Yeoh, A. Lee

Abstract: Introduction: The current study investigates the complex association between COVID-19 and the studied districts' socioecology (e.g. internal and external built environment, sociodemographic profiles, etc.) to quantify their contributions to the early outbreaks and epidemic resurgence of COVID-19. Methods: We aligned the analytic model's architecture with the hierarchical structure of the resident'… ▽ More Introduction: The current study investigates the complex association between COVID-19 and the studied districts' socioecology (e.g. internal and external built environment, sociodemographic profiles, etc.) to quantify their contributions to the early outbreaks and epidemic resurgence of COVID-19. Methods: We aligned the analytic model's architecture with the hierarchical structure of the resident's socioecology using a multi-headed hierarchical convolutional neural network to structure the vast array of hierarchically related predictive features representing buildings' internal and external built environments and residents' sociodemographic profiles as model input. COVID-19 cases accumulated in buildings across three adjacent districts in HK, both before and during HK's epidemic resurgence, were modeled. A forward-chaining validation was performed to examine the model's performance in forecasting COVID-19 cases over the 3-, 7-, and 14-day horizons during the two months subsequent to when the model for COVID-19 resurgence was built to align with the forecasting needs in an evolving pandemic. Results: Different sets of factors were found to be linked to the earlier waves of COVID-19 outbreaks compared to the epidemic resurgence of the pandemic. Sociodemographic factors such as work hours, monthly household income, employment types, and the number of non-working adults or children in household populations were of high importance to the studied buildings' COVID-19 case counts during the early waves of COVID-19. Factors constituting one's internal built environment, such as the number of distinct households in the buildings, the number of distinct households per floor, and the number of floors, corridors, and lifts, had the greatest unique contributions to the building-level COVID-19 case counts during epidemic resurgence. △ Less

Submitted 23 March, 2024; originally announced March 2024.

Improving Uncertainty Sampling with Bell Curve Weight Function

Authors: Zan-Kai Chong, Hiroyuki Ohsaki, Bok-Min Goi

Abstract: Typically, a supervised learning model is trained using passive learning by randomly selecting unlabelled instances to annotate. This approach is effective for learning a model, but can be costly in cases where acquiring labelled instances is expensive. For example, it can be time-consuming to manually identify spam mails (labelled instances) from thousands of emails (unlabelled instances) floodin… ▽ More Typically, a supervised learning model is trained using passive learning by randomly selecting unlabelled instances to annotate. This approach is effective for learning a model, but can be costly in cases where acquiring labelled instances is expensive. For example, it can be time-consuming to manually identify spam mails (labelled instances) from thousands of emails (unlabelled instances) flooding an inbox during initial data collection. Generally, we answer the above scenario with uncertainty sampling, an active learning method that improves the efficiency of supervised learning by using fewer labelled instances than passive learning. Given an unlabelled data pool, uncertainty sampling queries the labels of instances where the predicted probabilities, p, fall into the uncertainty region, i.e., $p \approx 0.5$. The newly acquired labels are then added to the existing labelled data pool to learn a new model. Nonetheless, the performance of uncertainty sampling is susceptible to the area of unpredictable responses (AUR) and the nature of the dataset. It is difficult to determine whether to use passive learning or uncertainty sampling without prior knowledge of a new dataset. To address this issue, we propose bell curve sampling, which employs a bell curve weight function to acquire new labels. With the bell curve centred at p=0.5, bell curve sampling selects instances whose predicted values are in the uncertainty area most of the time without neglecting the rest. Simulation results show that, most of the time bell curve sampling outperforms uncertainty sampling and passive learning in datasets of different natures and with AUR. △ Less

Submitted 2 March, 2024; originally announced March 2024.

Comments: 9 pages, 9 figures, journal

Journal ref: International Journal of Applied Physics and Mathematics, Vol. 13, No. 4, pp. 44-52, 2023

Spectral Co-Distillation for Personalized Federated Learning

Authors: Zihan Chen, Howard H. Yang, Tony Q. S. Quek, Kai Fong Ernest Chong

Abstract: Personalized federated learning (PFL) has been widely investigated to address the challenge of data heterogeneity, especially when a single generic model is inadequate in satisfying the diverse performance requirements of local clients simultaneously. Existing PFL methods are inherently based on the idea that the relations between the generic global and personalized local models are captured by th… ▽ More Personalized federated learning (PFL) has been widely investigated to address the challenge of data heterogeneity, especially when a single generic model is inadequate in satisfying the diverse performance requirements of local clients simultaneously. Existing PFL methods are inherently based on the idea that the relations between the generic global and personalized local models are captured by the similarity of model weights. Such a similarity is primarily based on either partitioning the model architecture into generic versus personalized components, or modeling client relationships via model weights. To better capture similar (yet distinct) generic versus personalized model representations, we propose \textit{spectral distillation}, a novel distillation method based on model spectrum information. Building upon spectral distillation, we also introduce a co-distillation framework that establishes a two-way bridge between generic and personalized model training. Moreover, to utilize the local idle time in conventional PFL, we propose a wait-free local training protocol. Through extensive experiments on multiple datasets over diverse heterogeneous data settings, we demonstrate the outperformance and efficacy of our proposed spectral co-distillation method, as well as our wait-free training protocol. △ Less

Submitted 29 January, 2024; originally announced January 2024.

Comments: 13 pages, NeurIPS 2023. Code at https://github.com/jimmyc96/spectral-dis-FL

The Role of Federated Learning in a Wireless World with Foundation Models

Authors: Zihan Chen, Howard H. Yang, Y. C. Tay, Kai Fong Ernest Chong, Tony Q. S. Quek

Abstract: Foundation models (FMs) are general-purpose artificial intelligence (AI) models that have recently enabled multiple brand-new generative AI applications. The rapid advances in FMs serve as an important contextual backdrop for the vision of next-generation wireless networks, where federated learning (FL) is a key enabler of distributed network intelligence. Currently, the exploration of the interpl… ▽ More Foundation models (FMs) are general-purpose artificial intelligence (AI) models that have recently enabled multiple brand-new generative AI applications. The rapid advances in FMs serve as an important contextual backdrop for the vision of next-generation wireless networks, where federated learning (FL) is a key enabler of distributed network intelligence. Currently, the exploration of the interplay between FMs and FL is still in its nascent stage. Naturally, FMs are capable of boosting the performance of FL, and FL could also leverage decentralized data and computing resources to assist in the training of FMs. However, the exceptionally high requirements that FMs have for computing resources, storage, and communication overhead would pose critical challenges to FL-enabled wireless networks. In this article, we explore the extent to which FMs are suitable for FL over wireless networks, including a broad overview of research challenges and opportunities. In particular, we discuss multiple new paradigms for realizing future intelligent networks that integrate FMs and FL. We also consolidate several broad research directions associated with these paradigms. △ Less

Submitted 7 May, 2024; v1 submitted 6 October, 2023; originally announced October 2023.

Comments: 8 pages, 4 figures, 2 tables. This version has been accepted by IEEE Wireless Communications

An Improved Greedy Curvature Bound in Finite-Horizon String Optimization with Application to a Sensor Coverage Problem

Authors: Brandon Van Over, Bowen Li, Edwin K. P. Chong, Ali Pezeshki

Abstract: We study the optimization problem of choosing strings of finite length to maximize string submodular functions on string matroids, which is a broader class of problems than maximizing set submodular functions on set matroids. We provide a lower bound for the performance of the greedy algorithm in our problem, and then prove that our bound is superior to the greedy curvature bound of Conforti and C… ▽ More We study the optimization problem of choosing strings of finite length to maximize string submodular functions on string matroids, which is a broader class of problems than maximizing set submodular functions on set matroids. We provide a lower bound for the performance of the greedy algorithm in our problem, and then prove that our bound is superior to the greedy curvature bound of Conforti and Cornuejols. Our bound has lower computational complexity than most previously proposed curvature bounds. Finally, we demonstrate the strength of our result on a sensor coverage problem. △ Less

Submitted 7 September, 2023; v1 submitted 30 August, 2023; originally announced August 2023.

GenKL: An Iterative Framework for Resolving Label Ambiguity and Label Non-conformity in Web Images Via a New Generalized KL Divergence

Authors: Xia Huang, Kai Fong Ernest Chong

Abstract: Web image datasets curated online inherently contain ambiguous in-distribution (ID) instances and out-of-distribution (OOD) instances, which we collectively call non-conforming (NC) instances. In many recent approaches for mitigating the negative effects of NC instances, the core implicit assumption is that the NC instances can be found via entropy maximization. For "entropy" to be well-defined, w… ▽ More Web image datasets curated online inherently contain ambiguous in-distribution (ID) instances and out-of-distribution (OOD) instances, which we collectively call non-conforming (NC) instances. In many recent approaches for mitigating the negative effects of NC instances, the core implicit assumption is that the NC instances can be found via entropy maximization. For "entropy" to be well-defined, we are interpreting the output prediction vector of an instance as the parameter vector of a multinomial random variable, with respect to some trained model with a softmax output layer. Hence, entropy maximization is based on the idealized assumption that NC instances have predictions that are "almost" uniformly distributed. However, in real-world web image datasets, there are numerous NC instances whose predictions are far from being uniformly distributed. To tackle the limitation of entropy maximization, we propose $(α, β)$-generalized KL divergence, $\mathcal{D}_{\text{KL}}^{α, β}(p\|q)$, which can be used to identify significantly more NC instances. Theoretical properties of $\mathcal{D}_{\text{KL}}^{α, β}(p\|q)$ are proven, and we also show empirically that a simple use of $\mathcal{D}_{\text{KL}}^{α, β}(p\|q)$ outperforms all baselines on the NC instance identification task. Building upon $(α,β)$-generalized KL divergence, we also introduce a new iterative training framework, GenKL, that identifies and relabels NC instances. When evaluated on three web image datasets, Clothing1M, Food101/Food101N, and mini WebVision 1.0, we achieved new state-of-the-art classification accuracies: $81.34\%$, $85.73\%$ and $78.99\%$/$92.54\%$ (top-1/top-5), respectively. △ Less

Submitted 19 July, 2023; originally announced July 2023.

Comments: Published (with open access) at International Journal of Computer Vision (IJCV, 2023). 25 pages, 8 figures. Code is available at: https://github.com/codetopaper/GenKL

ACM Class: I.2.0; I.4.0

Abstract Visual Reasoning: An Algebraic Approach for Solving Raven's Progressive Matrices

Authors: Jingyi Xu, Tushar Vaidya, Yufei Wu, Saket Chandra, Zhangsheng Lai, Kai Fong Ernest Chong

Abstract: We introduce algebraic machine reasoning, a new reasoning framework that is well-suited for abstract reasoning. Effectively, algebraic machine reasoning reduces the difficult process of novel problem-solving to routine algebraic computation. The fundamental algebraic objects of interest are the ideals of some suitably initialized polynomial ring. We shall explain how solving Raven's Progressive Ma… ▽ More We introduce algebraic machine reasoning, a new reasoning framework that is well-suited for abstract reasoning. Effectively, algebraic machine reasoning reduces the difficult process of novel problem-solving to routine algebraic computation. The fundamental algebraic objects of interest are the ideals of some suitably initialized polynomial ring. We shall explain how solving Raven's Progressive Matrices (RPMs) can be realized as computational problems in algebra, which combine various well-known algebraic subroutines that include: Computing the Gröbner basis of an ideal, checking for ideal containment, etc. Crucially, the additional algebraic structure satisfied by ideals allows for more operations on ideals beyond set-theoretic operations. Our algebraic machine reasoning framework is not only able to select the correct answer from a given answer set, but also able to generate the correct answer with only the question matrix given. Experiments on the I-RAVEN dataset yield an overall $93.2\%$ accuracy, which significantly outperforms the current state-of-the-art accuracy of $77.0\%$ and exceeds human performance at $84.4\%$ accuracy. △ Less

Submitted 21 March, 2023; originally announced March 2023.

Comments: Accepted at IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023. 30 pages, 7 figures (including supplementary material). First three authors contributed equally. Code is available at: https://github.com/Xu-Jingyi/AlgebraicMR

MSC Class: 13P25; 68W30 ACM Class: I.1; I.2.4; I.2.6; I.5.1

A deep learning approach for adaptive zoning

Authors: Massimiliano Lupo Pasini, Luka Malenica, Kwitae Chong, Stuart Slattery

Abstract: We propose a supervised deep learning (DL) approach to perform adaptive zoning on time dependent partial differential equations that model the propagation of 1D shock waves in a compressible medium. We train a neural network on a dataset composed of different static shock profiles associated with the corresponding adapted meshes computed with standard adaptive zoning techniques. We show that the t… ▽ More We propose a supervised deep learning (DL) approach to perform adaptive zoning on time dependent partial differential equations that model the propagation of 1D shock waves in a compressible medium. We train a neural network on a dataset composed of different static shock profiles associated with the corresponding adapted meshes computed with standard adaptive zoning techniques. We show that the trained DL model learns how to capture the presence of shocks in the domain and generates at each time step an adapted non-uniform mesh that relocates the grid nodes to improve the accuracy of Lax-Wendroff and fifth order weighted essentially non-oscillatory (WENO5) space discretization schemes. We also show that the surrogate DL model reduces the computational time to perform adaptive zoning by at least a 2x factor with respect to standard techniques without compromising the accuracy of the reconstruction of the physical quantities of interest. △ Less

Submitted 9 December, 2022; originally announced January 2023.

Comments: 30 pages, 26 figures

MSC Class: 68T10; 68U20; 00A72; 00A79 ACM Class: G.1.1; G.1.8; I.2

Clustering and Analysis of GPS Trajectory Data using Distance-based Features

Authors: Zann Koh, Yuren Zhou, Billy Pik Lik Lau, Ran Liu, Keng Hua Chong, Chau Yuen

Abstract: The proliferation of smartphones has accelerated mobility studies by largely increasing the type and volume of mobility data available. One such source of mobility data is from GPS technology, which is becoming increasingly common and helps the research community understand mobility patterns of people. However, there lacks a standardized framework for studying the different mobility patterns creat… ▽ More The proliferation of smartphones has accelerated mobility studies by largely increasing the type and volume of mobility data available. One such source of mobility data is from GPS technology, which is becoming increasingly common and helps the research community understand mobility patterns of people. However, there lacks a standardized framework for studying the different mobility patterns created by the non-Work, non-Home locations of Working and Nonworking users on Workdays and Offdays using machine learning methods. We propose a new mobility metric, Daily Characteristic Distance, and use it to generate features for each user together with Origin-Destination matrix features. We then use those features with an unsupervised machine learning method, $k$-means clustering, and obtain three clusters of users for each type of day (Workday and Offday). Finally, we propose two new metrics for the analysis of the clustering results, namely User Commonality and Average Frequency. By using the proposed metrics, interesting user behaviors can be discerned and it helps us to better understand the mobility patterns of the users. △ Less

Submitted 30 November, 2022; originally announced December 2022.

Comments: 13 pages, 12 figures. To be published in IEEE Access

Single-Pixel Image Reconstruction Based on Block Compressive Sensing and Deep Learning

Authors: Stephen L. H. Lau, Edwin K. P. Chong

Abstract: Single-pixel imaging (SPI) is a novel imaging technique whose working principle is based on the compressive sensing (CS) theory. In SPI, data is obtained through a series of compressive measurements and the corresponding image is reconstructed. Typically, the reconstruction algorithm such as basis pursuit relies on the sparsity assumption in images. However, recent advances in deep learning have f… ▽ More Single-pixel imaging (SPI) is a novel imaging technique whose working principle is based on the compressive sensing (CS) theory. In SPI, data is obtained through a series of compressive measurements and the corresponding image is reconstructed. Typically, the reconstruction algorithm such as basis pursuit relies on the sparsity assumption in images. However, recent advances in deep learning have found its uses in reconstructing CS images. Despite showing a promising result in simulations, it is often unclear how such an algorithm can be implemented in an actual SPI setup. In this paper, we demonstrate the use of deep learning on the reconstruction of SPI images in conjunction with block compressive sensing (BCS). We also proposed a novel reconstruction model based on convolutional neural networks that outperforms other competitive CS reconstruction algorithms. Besides, by incorporating BCS in our deep learning model, we were able to reconstruct images of any size above a certain smallest image size. In addition, we show that our model is capable of reconstructing images obtained from an SPI setup while being priorly trained on natural images, which can be vastly different from the SPI images. This opens up opportunity for the feasibility of pretrained deep learning models for CS reconstructions of images from various domain areas. △ Less

Submitted 14 July, 2022; originally announced July 2022.

FedCorr: Multi-Stage Federated Learning for Label Noise Correction

Authors: Jingyi Xu, Zihan Chen, Tony Q. S. Quek, Kai Fong Ernest Chong

Abstract: Federated learning (FL) is a privacy-preserving distributed learning paradigm that enables clients to jointly train a global model. In real-world FL implementations, client data could have label noise, and different clients could have vastly different label noise levels. Although there exist methods in centralized learning for tackling label noise, such methods do not perform well on heterogeneous… ▽ More Federated learning (FL) is a privacy-preserving distributed learning paradigm that enables clients to jointly train a global model. In real-world FL implementations, client data could have label noise, and different clients could have vastly different label noise levels. Although there exist methods in centralized learning for tackling label noise, such methods do not perform well on heterogeneous label noise in FL settings, due to the typically smaller sizes of client datasets and data privacy requirements in FL. In this paper, we propose $\texttt{FedCorr}$, a general multi-stage framework to tackle heterogeneous label noise in FL, without making any assumptions on the noise models of local clients, while still maintaining client data privacy. In particular, (1) $\texttt{FedCorr}$ dynamically identifies noisy clients by exploiting the dimensionalities of the model prediction subspaces independently measured on all clients, and then identifies incorrect labels on noisy clients based on per-sample losses. To deal with data heterogeneity and to increase training stability, we propose an adaptive local proximal regularization term that is based on estimated local noise levels. (2) We further finetune the global model on identified clean clients and correct the noisy labels for the remaining noisy clients after finetuning. (3) Finally, we apply the usual training on all clients to make full use of all local data. Experiments conducted on CIFAR-10/100 with federated synthetic label noise, and on a real-world noisy dataset, Clothing1M, demonstrate that $\texttt{FedCorr}$ is robust to label noise and substantially outperforms the state-of-the-art methods at multiple noise levels. △ Less

Submitted 10 April, 2022; originally announced April 2022.

Comments: Accepted at IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022. 18 pages, 10 figures (including supplementary material). First two authors contributed equally. Code is available at: https://github.com/Xu-Jingyi/FedCorr

Well-Conditioned Linear Minimum Mean Square Error Estimation

Authors: Edwin K. P. Chong

Abstract: Linear minimum mean square error (LMMSE) estimation is often ill-conditioned, suggesting that unconstrained minimization of the mean square error is an inadequate approach to filter design. To address this, we first develop a unifying framework for studying constrained LMMSE estimation problems. Using this framework, we explore an important structural property of constrained LMMSE filters involvin… ▽ More Linear minimum mean square error (LMMSE) estimation is often ill-conditioned, suggesting that unconstrained minimization of the mean square error is an inadequate approach to filter design. To address this, we first develop a unifying framework for studying constrained LMMSE estimation problems. Using this framework, we explore an important structural property of constrained LMMSE filters involving a certain prefilter. Optimality is invariant under invertible linear transformations of the prefilter. This parameterizes all optimal filters by equivalence classes of prefilters. We then clarify that merely constraining the rank of the filter does not suitably address the problem of ill-conditioning. Instead, we adopt a constraint that explicitly requires solutions to be well-conditioned in a certain specific sense. We introduce two well-conditioned filters and show that they converge to the unconstrained LMMSE filter as their truncation-power loss goes to zero, at the same rate as the low-rank Wiener filter. We also show extensions to the case of weighted trace and determinant of the error covariance as objective functions. Finally, our quantitative results with historical VIX data demonstrate that our two well-conditioned filters have stable performance while the standard LMMSE filter deteriorates with increasing condition number. △ Less

Submitted 21 March, 2022; v1 submitted 6 January, 2022; originally announced January 2022.

Semi-Supervised Few-Shot Intent Classification and Slot Filling

Authors: Samyadeep Basu, Karine lp Kiun Chong, Amr Sharaf, Alex Fischer, Vishal Rohra, Michael Amoake, Hazem El-Hammamy, Ehi Nosakhare, Vijay Ramani, Benjamin Han

Abstract: Intent classification (IC) and slot filling (SF) are two fundamental tasks in modern Natural Language Understanding (NLU) systems. Collecting and annotating large amounts of data to train deep learning models for such systems is not scalable. This problem can be addressed by learning from few examples using fast supervised meta-learning techniques such as prototypical networks. In this work, we sy… ▽ More Intent classification (IC) and slot filling (SF) are two fundamental tasks in modern Natural Language Understanding (NLU) systems. Collecting and annotating large amounts of data to train deep learning models for such systems is not scalable. This problem can be addressed by learning from few examples using fast supervised meta-learning techniques such as prototypical networks. In this work, we systematically investigate how contrastive learning and unsupervised data augmentation methods can benefit these existing supervised meta-learning pipelines for jointly modelled IC/SF tasks. Through extensive experiments across standard IC/SF benchmarks (SNIPS and ATIS), we show that our proposed semi-supervised approaches outperform standard supervised meta-learning methods: contrastive losses in conjunction with prototypical networks consistently outperform the existing state-of-the-art for both IC and SF tasks, while data augmentation strategies primarily improve few-shot IC by a significant margin. △ Less

Submitted 17 September, 2021; originally announced September 2021.

Dynamic Attention-based Communication-Efficient Federated Learning

Authors: Zihan Chen, Kai Fong Ernest Chong, Tony Q. S. Quek

Abstract: Federated learning (FL) offers a solution to train a global machine learning model while still maintaining data privacy, without needing access to data stored locally at the clients. However, FL suffers performance degradation when client data distribution is non-IID, and a longer training duration to combat this degradation may not necessarily be feasible due to communication limitations. To addr… ▽ More Federated learning (FL) offers a solution to train a global machine learning model while still maintaining data privacy, without needing access to data stored locally at the clients. However, FL suffers performance degradation when client data distribution is non-IID, and a longer training duration to combat this degradation may not necessarily be feasible due to communication limitations. To address this challenge, we propose a new adaptive training algorithm $\texttt{AdaFL}$, which comprises two components: (i) an attention-based client selection mechanism for a fairer training scheme among the clients; and (ii) a dynamic fraction method to balance the trade-off between performance stability and communication efficiency. Experimental results show that our $\texttt{AdaFL}$ algorithm outperforms the usual $\texttt{FedAvg}$ algorithm, and can be incorporated to further improve various state-of-the-art FL algorithms, with respect to three aspects: model accuracy, performance stability, and communication efficiency. △ Less

Submitted 12 August, 2021; originally announced August 2021.

Comments: 7 pages, 3 figures, presented at the International Workshop on Federated and Transfer Learning for Data Sparsity and Confidentiality (FTL-IJCAI 2021) in conjunction with the 30th International Joint Conference on Artificial Intelligence (IJCAI), 2021

ACM Class: I.2

Training Classifiers that are Universally Robust to All Label Noise Levels

Authors: Jingyi Xu, Tony Q. S. Quek, Kai Fong Ernest Chong

Abstract: For classification tasks, deep neural networks are prone to overfitting in the presence of label noise. Although existing methods are able to alleviate this problem at low noise levels, they encounter significant performance reduction at high noise levels, or even at medium noise levels when the label noise is asymmetric. To train classifiers that are universally robust to all noise levels, and th… ▽ More For classification tasks, deep neural networks are prone to overfitting in the presence of label noise. Although existing methods are able to alleviate this problem at low noise levels, they encounter significant performance reduction at high noise levels, or even at medium noise levels when the label noise is asymmetric. To train classifiers that are universally robust to all noise levels, and that are not sensitive to any variation in the noise model, we propose a distillation-based framework that incorporates a new subcategory of Positive-Unlabeled learning. In particular, we shall assume that a small subset of any given noisy dataset is known to have correct labels, which we treat as "positive", while the remaining noisy subset is treated as "unlabeled". Our framework consists of the following two components: (1) We shall generate, via iterative updates, an augmented clean subset with additional reliable "positive" samples filtered from "unlabeled" samples; (2) We shall train a teacher model on this larger augmented clean set. With the guidance of the teacher model, we then train a student model on the whole dataset. Experiments were conducted on the CIFAR-10 dataset with synthetic label noise at multiple noise levels for both symmetric and asymmetric noise. The results show that our framework generally outperforms at medium to high noise levels. We also evaluated our framework on Clothing1M, a real-world noisy dataset, and we achieved 2.94% improvement in accuracy over existing state-of-the-art methods. △ Less

Submitted 27 May, 2021; originally announced May 2021.

Comments: IJCNN 2021 paper, 8 pages, 3 figures. Code available: https://github.com/Xu-Jingyi/PUDistill

ACM Class: I.2.0

WiFi Fingerprint Clustering for Urban Mobility Analysis

Authors: Sumudu HasalaMarakkalage, Billy Pik Lik Lau, Yuren Zhou, Ran Liu, Chau Yuen, Wei Quin Yow, Keng Hua Chong

Abstract: In this paper, we present an unsupervised learning approach to identify the user points of interest (POI) by exploiting WiFi measurements from smartphone application data. Due to the lack of GPS positioning accuracy in indoor, sheltered, and high rise building environments, we rely on widely available WiFi access points (AP) in contemporary urban areas to accurately identify POI and mobility patte… ▽ More In this paper, we present an unsupervised learning approach to identify the user points of interest (POI) by exploiting WiFi measurements from smartphone application data. Due to the lack of GPS positioning accuracy in indoor, sheltered, and high rise building environments, we rely on widely available WiFi access points (AP) in contemporary urban areas to accurately identify POI and mobility patterns, by comparing the similarity in the WiFi measurements. We propose a system architecture to scan the surrounding WiFi AP, and perform unsupervised learning to demonstrate that it is possible to identify three major insights, namely the indoor POI within a building, neighbourhood activity, and micro-mobility of the users. Our results show that it is possible to identify the aforementioned insights, with the fusion of WiFi and GPS, which are not possible to identify by only using GPS. △ Less

Submitted 3 May, 2021; originally announced May 2021.

Comments: accepted by IEEE Access

The Study of Urban Residential's Public Space Activeness using Space-centric Approach

Authors: Billy Pik Lik Lau, Benny Kai Kiat Ng, Chau Yuen, Bige Tuncer, Keng Hua Chong

Abstract: With the advancement of the Internet of Things (IoT) and communication platform, large scale sensor deployment can be easily implemented in an urban city to collect various information. To date, there are only a handful of research studies about understanding the usage of urban public spaces. Leveraging IoT, various sensors have been deployed in an urban residential area to monitor and study publi… ▽ More With the advancement of the Internet of Things (IoT) and communication platform, large scale sensor deployment can be easily implemented in an urban city to collect various information. To date, there are only a handful of research studies about understanding the usage of urban public spaces. Leveraging IoT, various sensors have been deployed in an urban residential area to monitor and study public space utilization patterns. In this paper, we propose a data processing system to generate space-centric insights about the utilization of an urban residential region of multiple points of interest (PoIs) that consists of 190,000m$^2$ real estate. We identify the activeness of each PoI based on the spectral clustering, and then study their corresponding static features, which are composed of transportation, commercial facilities, population density, along with other characteristics. Through the heuristic features inferring, the residential density and commercial facilities are the most significant factors affecting public place utilization. △ Less

Submitted 11 January, 2021; v1 submitted 11 January, 2021; originally announced January 2021.

Comments: Accepted at IEEE Internet of Things Journal 2021

Multiple-Perspective Clustering of Passive Wi-Fi Sensing Trajectory Data

Authors: Zann Koh, Yuren Zhou, Billy Pik Lik Lau, Chau Yuen, Bige Tuncer, Keng Hua Chong

Abstract: Information about the spatiotemporal flow of humans within an urban context has a wide plethora of applications. Currently, although there are many different approaches to collect such data, there lacks a standardized framework to analyze it. The focus of this paper is on the analysis of the data collected through passive Wi-Fi sensing, as such passively collected data can have a wide coverage at… ▽ More Information about the spatiotemporal flow of humans within an urban context has a wide plethora of applications. Currently, although there are many different approaches to collect such data, there lacks a standardized framework to analyze it. The focus of this paper is on the analysis of the data collected through passive Wi-Fi sensing, as such passively collected data can have a wide coverage at low cost. We propose a systematic approach by using unsupervised machine learning methods, namely k-means clustering and hierarchical agglomerative clustering (HAC) to analyze data collected through such a passive Wi-Fi sniffing method. We examine three aspects of clustering of the data, namely by time, by person, and by location, and we present the results obtained by applying our proposed approach on a real-world dataset collected over five months. △ Less

Submitted 21 December, 2020; originally announced December 2020.

Comments: 15 pages, 11 figures

A closer look at the approximation capabilities of neural networks

Authors: Kai Fong Ernest Chong

Abstract: The universal approximation theorem, in one of its most general versions, says that if we consider only continuous activation functions $σ$, then a standard feedforward neural network with one hidden layer is able to approximate any continuous multivariate function $f$ to any given approximation threshold $\varepsilon$, if and only if $σ$ is non-polynomial. In this paper, we give a direct algebrai… ▽ More The universal approximation theorem, in one of its most general versions, says that if we consider only continuous activation functions $σ$, then a standard feedforward neural network with one hidden layer is able to approximate any continuous multivariate function $f$ to any given approximation threshold $\varepsilon$, if and only if $σ$ is non-polynomial. In this paper, we give a direct algebraic proof of the theorem. Furthermore we shall explicitly quantify the number of hidden units required for approximation. Specifically, if $X\subseteq \mathbb{R}^n$ is compact, then a neural network with $n$ input units, $m$ output units, and a single hidden layer with $\binom{n+d}{d}$ hidden units (independent of $m$ and $\varepsilon$), can uniformly approximate any polynomial function $f:X \to \mathbb{R}^m$ whose total degree is at most $d$ for each of its $m$ coordinate functions. In the general case that $f$ is any continuous function, we show there exists some $N\in \mathcal{O}(\varepsilon^{-n})$ (independent of $m$), such that $N$ hidden units would suffice to approximate $f$. We also show that this uniform approximation property (UAP) still holds even under seemingly strong conditions imposed on the weights. We highlight several consequences: (i) For any $δ> 0$, the UAP still holds if we restrict all non-bias weights $w$ in the last layer to satisfy $|w| < δ$. (ii) There exists some $λ>0$ (depending only on $f$ and $σ$), such that the UAP still holds if we restrict all non-bias weights $w$ in the first layer to satisfy $|w|>λ$. (iii) If the non-bias weights in the first layer are \emph{fixed} and randomly chosen from a suitable range, then the UAP holds with probability $1$. △ Less

Submitted 15 February, 2020; originally announced February 2020.

Comments: Published as a conference paper at ICLR 2020

Automated Pavement Crack Segmentation Using U-Net-based Convolutional Neural Network

Authors: Stephen L. H. Lau, Edwin K. P. Chong, Xu Yang, Xin Wang

Abstract: Automated pavement crack image segmentation is challenging because of inherent irregular patterns, lighting conditions, and noise in images. Conventional approaches require a substantial amount of feature engineering to differentiate crack regions from non-affected regions. In this paper, we propose a deep learning technique based on a convolutional neural network to perform segmentation tasks on… ▽ More Automated pavement crack image segmentation is challenging because of inherent irregular patterns, lighting conditions, and noise in images. Conventional approaches require a substantial amount of feature engineering to differentiate crack regions from non-affected regions. In this paper, we propose a deep learning technique based on a convolutional neural network to perform segmentation tasks on pavement crack images. Our approach requires minimal feature engineering compared to other machine learning techniques. We propose a U-Net-based network architecture in which we replace the encoder with a pretrained ResNet-34 neural network. We use a "one-cycle" training schedule based on cyclical learning rates to speed up the convergence. Our method achieves an F1 score of 96% on the CFD dataset and 73% on the Crack500 dataset, outperforming other algorithms tested on these datasets. We perform ablation studies on various techniques that helped us get marginal performance boosts, i.e., the addition of spatial and channel squeeze and excitation (SCSE) modules, training with gradually increasing image sizes, and training various neural network layers with different learning rates. △ Less

Submitted 30 June, 2020; v1 submitted 7 January, 2020; originally announced January 2020.

Comments: Accepted for publication in IEEE Access

Cribriform pattern detection in prostate histopathological images using deep learning models

Authors: Malay Singh, Emarene Mationg Kalaw, Wang Jie, Mundher Al-Shabi, Chin Fong Wong, Danilo Medina Giron, Kian-Tai Chong, Maxine Tan, Zeng Zeng, Hwee Kuan Lee

Abstract: Architecture, size, and shape of glands are most important patterns used by pathologists for assessment of cancer malignancy in prostate histopathological tissue slides. Varying structures of glands along with cumbersome manual observations may result in subjective and inconsistent assessment. Cribriform gland with irregular border is an important feature in Gleason pattern 4. We propose using dee… ▽ More Architecture, size, and shape of glands are most important patterns used by pathologists for assessment of cancer malignancy in prostate histopathological tissue slides. Varying structures of glands along with cumbersome manual observations may result in subjective and inconsistent assessment. Cribriform gland with irregular border is an important feature in Gleason pattern 4. We propose using deep neural networks for cribriform pattern classification in prostate histopathological images. $163708$ Hematoxylin and Eosin (H\&E) stained images were extracted from histopathologic tissue slides of $19$ patients with prostate cancer and annotated for cribriform patterns. Our automated image classification system analyses the H\&E images to classify them as either `Cribriform' or `Non-cribriform'. Our system uses various deep learning approaches and hand-crafted image pixel intensity-based features. We present our results for cribriform pattern detection across various parameters and configuration allowed by our system. The combination of fine-tuned deep learning models outperformed the state-of-art nuclei feature based methods. Our image classification system achieved the testing accuracy of $85.93~\pm~7.54$ (cross-validated) and $88.04~\pm~5.63$ ( additional unseen test set) across three folds. In this paper, we present an annotated cribriform dataset along with analysis of deep learning models and hand-crafted features for cribriform pattern detection in prostate histopathological images. △ Less

Submitted 9 October, 2019; originally announced October 2019.

Comments: 21 pages, 4 figures, 6 tables

Decision Automation for Electric Power Network Recovery

Authors: Yugandhar Sarkale, Saeed Nozhati, Edwin K. P. Chong, Bruce R. Ellingwood

Abstract: Critical infrastructure systems such as electric power networks, water networks, and transportation systems play a major role in the welfare of any community. In the aftermath of disasters, their recovery is of paramount importance; orderly and efficient recovery involves the assignment of limited resources (a combination of human repair workers and machines) to repair damaged infrastructure compo… ▽ More Critical infrastructure systems such as electric power networks, water networks, and transportation systems play a major role in the welfare of any community. In the aftermath of disasters, their recovery is of paramount importance; orderly and efficient recovery involves the assignment of limited resources (a combination of human repair workers and machines) to repair damaged infrastructure components. The decision maker must also deal with uncertainty in the outcome of the resource-allocation actions during recovery. The manual assignment of resources seldom is optimal despite the expertise of the decision maker because of the large number of choices and uncertainties in consequences of sequential decisions. This combinatorial assignment problem under uncertainty is known to be \mbox{NP-hard}. We propose a novel decision technique that addresses the massive number of decision choices for large-scale real-world problems; in addition, our method also features an experiential learning component that adaptively determines the utilization of the computational resources based on the performance of a small number of choices. Our framework is closed-loop, and naturally incorporates all the attractive features of such a decision-making system. In contrast to myopic approaches, which do not account for the future effects of the current choices, our methodology has an anticipatory learning component that effectively incorporates \emph{lookahead} into the solutions. To this end, we leverage the theory of regression analysis, Markov decision processes (MDPs), multi-armed bandits, and stochastic models of community damage from natural disasters to develop a method for near-optimal recovery of communities. Our method contributes to the general problem of MDPs with massive action spaces with application to recovery of communities affected by hazards. △ Less

Submitted 18 October, 2019; v1 submitted 1 October, 2019; originally announced October 2019.

Comments: Submitted to IEEE Transactions on Automation Science and Engineering (13 pages and 6 figures)

An approximate dynamic programming approach to food security of communities following hazards

Authors: Saeed Nozhati, Yugandhar Sarkale, Bruce R. Ellingwood, Edwin K. P. Chong, Hussam Mahmoud

Abstract: Food security can be threatened by extreme natural hazard events for households of all social classes within a community. To address food security issues following a natural disaster, the recovery of several elements of the built environment within a community, including its building portfolio, must be considered. Building portfolio restoration is one of the most challenging elements of recovery o… ▽ More Food security can be threatened by extreme natural hazard events for households of all social classes within a community. To address food security issues following a natural disaster, the recovery of several elements of the built environment within a community, including its building portfolio, must be considered. Building portfolio restoration is one of the most challenging elements of recovery owing to the complexity and dimensionality of the problem. This study introduces a stochastic scheduling algorithm for the identification of optimal building portfolio recovery strategies. The proposed approach provides a computationally tractable formulation to manage multi-state, large-scale infrastructure systems. A testbed community modeled after Gilroy, California, is used to illustrate how the proposed approach can be implemented efficiently and accurately to find the near-optimal decisions related to building recovery following a severe earthquake. △ Less

Submitted 15 December, 2018; originally announced December 2018.

Comments: As opposed to the preemptive scheduling problem, which was addressed in multiple works by us, we deal with a non-preemptive stochastic scheduling problem in this work. Submitted to 13th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP13 Seoul, South Korea, May 26-30, 2019

Journal ref: 13th Int. Conf. Appl. Statistics and Prob. Civ. Eng. (ICASP13), Seoul, South Korea, May 26-30, 2019

An Approximate Dynamic Programming Approach to Community Recovery Management (Extended Abstract)

Authors: Saeed Nozhati, Bruce R. Ellingwood, Hussam Mahmoud, Yugandhar Sarkale, Edwin K. P. Chong, Nathanael Rosenheim

Abstract: The functioning of interdependent civil infrastructure systems in the aftermath of a disruptive event is critical to the performance and vitality of any modern urban community. Post-event stressors and chaotic circumstances, time limitations, and complexities in the community recovery process highlight the necessity for a comprehensive decision-making framework at the community-level for post-even… ▽ More The functioning of interdependent civil infrastructure systems in the aftermath of a disruptive event is critical to the performance and vitality of any modern urban community. Post-event stressors and chaotic circumstances, time limitations, and complexities in the community recovery process highlight the necessity for a comprehensive decision-making framework at the community-level for post-event recovery management. Such a framework must be able to handle large-scale scheduling and decision processes, which involve difficult control problems with large combinatorial decision spaces. This study utilizes approximate dynamic programming algorithms along with heuristics for the identification of optimal community recovery actions following the occurrence of an extreme earthquake event. The proposed approach addresses the curse of dimensionality in its analysis and management of multi-state, large-scale infrastructure systems. Furthermore, the proposed approach can consider the cur-rent recovery policies of responsible public and private entities within the community and shows how their performance might be improved. A testbed community coarsely modeled after Gilroy, California, is utilized as an illustrative example. While the illustration provides optimal policies for the Electrical Power Network serving Gilroy following a severe earthquake, preliminary work shows that the methodology is computationally well suited to other infrastructure systems and hazards. △ Less

Submitted 22 June, 2018; originally announced June 2018.

Comments: The material in this work provides a gist of the material in the journal version of our work (arXiv:1803.01451) that has in-detailed description of the methods employed in this article in addition to a more comprehensive treatment. This work will serve as a suitable introduction to the interested reader

Journal ref: Presented at Engineering Mechanics Institute Conference 2018 (EMI 2018 -- MS36 Robustness of Infrastructures), Cambridge -- Boston, MA, May 29--June 1, 2018

A Modified Approximate Dynamic Programming Algorithm for Community-level Food Security Following Disasters

Authors: Saeed Nozhati, Yugandhar Sarkale, Bruce R. Ellingwood, Edwin K. P. Chong, Hussam Mahmoud

Abstract: In the aftermath of an extreme natural hazard, community residents must have access to functioning food retailers to maintain food security. Food security is dependent on supporting critical infrastructure systems, including electricity, potable water, and transportation. An understanding of the response of such interdependent networks and the process of post-disaster recovery is the cornerstone o… ▽ More In the aftermath of an extreme natural hazard, community residents must have access to functioning food retailers to maintain food security. Food security is dependent on supporting critical infrastructure systems, including electricity, potable water, and transportation. An understanding of the response of such interdependent networks and the process of post-disaster recovery is the cornerstone of an efficient emergency management plan. In this study, the interconnectedness among different critical facilities, such as electrical power networks, water networks, highway bridges, and food retailers, is modeled. The study considers various sources of uncertainty and complexity in the recovery process of a community to capture the stochastic behavior of the spatially distributed infrastructure systems. The study utilizes an approximate dynamic programming (ADP) framework to allocate resources to restore infrastructure components efficiently. The proposed ADP scheme enables us to identify near-optimal restoration decisions at the community level. Furthermore, we employ a simulated annealing (SA) algorithm to complement the proposed ADP framework and to identify near-optimal actions accurately. In the sequel, we use the City of Gilroy, California, USA to illustrate the applicability of the proposed methodology following a severe earthquake. The approach can be implemented efficiently to identify practical policy interventions to hasten recovery of food systems and to reduce adverse food-insecurity impacts for other hazards and communities. △ Less

Submitted 15 May, 2018; v1 submitted 31 March, 2018; originally announced April 2018.

Comments: The material presented here might have some overlap with arXiv:1803.04144 and arXiv:1803.01451. Nonetheless, this material contains substantial additions

Journal ref: Proceedings of the 9th International Congress on Environmental Modelling and Software (iEMSs 2018), Fort Collins, CO, June 24--28, 2018

Solving Markov decision processes for network-level post-hazard recovery via simulation optimization and rollout

Authors: Yugandhar Sarkale, Saeed Nozhati, Edwin K. P. Chong, Bruce Ellingwood, Hussam Mahmoud

Abstract: Computation of optimal recovery decisions for community resilience assurance post-hazard is a combinatorial decision-making problem under uncertainty. It involves solving a large-scale optimization problem, which is significantly aggravated by the introduction of uncertainty. In this paper, we draw upon established tools from multiple research communities to provide an effective solution to this c… ▽ More Computation of optimal recovery decisions for community resilience assurance post-hazard is a combinatorial decision-making problem under uncertainty. It involves solving a large-scale optimization problem, which is significantly aggravated by the introduction of uncertainty. In this paper, we draw upon established tools from multiple research communities to provide an effective solution to this challenging problem. We provide a stochastic model of damage to the water network (WN) within a testbed community following a severe earthquake and compute near-optimal recovery actions for restoration of the water network. We formulate this stochastic decision-making problem as a Markov Decision Process (MDP), and solve it using a popular class of heuristic algorithms known as rollout. A simulation-based representation of MDPs is utilized in conjunction with rollout and the Optimal Computing Budget Allocation (OCBA) algorithm to address the resulting stochastic simulation optimization problem. Our method employs non-myopic planning with efficient use of simulation budget. We show, through simulation results, that rollout fused with OCBA performs competitively with respect to rollout with total equal allocation (TEA) at a meagre simulation budget of 5-10% of rollout with TEA, which is a crucial step towards addressing large-scale community recovery problems following natural disasters. △ Less

Submitted 12 April, 2018; v1 submitted 12 March, 2018; originally announced March 2018.

Comments: Submitted to Simulation Optimization for Cyber Physical Energy Systems (Special Session) in 14th IEEE International Conference on Automation Science and Engineering

Journal ref: 2018 IEEE 14th Int. Conf. Autom. Sci. and Eng. (CASE) 906 - 912

Near-optimal planning using approximate dynamic programming to enhance post-hazard community resilience management

Authors: Saeed Nozhati, Yugandhar Sarkale, Bruce Ellingwood, Edwin K. P. Chong, Hussam Mahmoud

Abstract: The lack of a comprehensive decision-making approach at the community level is an important problem that warrants immediate attention. Network-level decision-making algorithms need to solve large-scale optimization problems that pose computational challenges. The complexity of the optimization problems increases when various sources of uncertainty are considered. This research introduces a sequent… ▽ More The lack of a comprehensive decision-making approach at the community level is an important problem that warrants immediate attention. Network-level decision-making algorithms need to solve large-scale optimization problems that pose computational challenges. The complexity of the optimization problems increases when various sources of uncertainty are considered. This research introduces a sequential discrete optimization approach, as a decision-making framework at the community level for recovery management. The proposed mathematical approach leverages approximate dynamic programming along with heuristics for the determination of recovery actions. Our methodology overcomes the curse of dimensionality and manages multi-state, large-scale infrastructure systems following disasters. We also provide computational results showing that our methodology not only incorporates recovery policies of responsible public and private entities within the community but also substantially enhances the performance of their underlying strategies with limited resources. The methodology can be implemented efficiently to identify near-optimal recovery decisions following a severe earthquake based on multiple objectives for an electrical power network of a testbed community coarsely modeled after Gilroy, California, United States. The proposed optimization method supports risk-informed community decision makers within chaotic post-hazard circumstances. △ Less

Submitted 19 April, 2018; v1 submitted 4 March, 2018; originally announced March 2018.

Comments: The first two authors contributed equally and are listed alphabetically

Journal ref: Reliab. Eng. & Syst. Saf. 181 (2019) 116-126

Large-Scale Experiment on the Importance of Social Learning and Unimodality in the Wisdom of the Crowd

Authors: Dhaval Adjodah, Shi Kai Chong, Yan Leng, Peter Krafft, Alex Pentland

Abstract: In this study, we build on previous research to understand the conditions within which the Wisdom of the Crowd (WoC) improves or worsens as a result of showing individuals the predictions of their peers. Our main novel contributions are: 1) a dataset of unprecedented size and detail; 2) we observe the novel effect of the importance of the unimodality of the social information shown to individuals:… ▽ More In this study, we build on previous research to understand the conditions within which the Wisdom of the Crowd (WoC) improves or worsens as a result of showing individuals the predictions of their peers. Our main novel contributions are: 1) a dataset of unprecedented size and detail; 2) we observe the novel effect of the importance of the unimodality of the social information shown to individuals: if one does not see only one clear peak in the distribution of the crowd's predictions, the WoC is worsened after social exposure; and 3) we estimate social learning weights that we use to show that there exists individuals who are much better at learning from the crowd and can be filtered to improve collective accuracy. △ Less

Submitted 29 December, 2017; originally announced December 2017.

Social Bayesian Learning in the Wisdom of the Crowd

Authors: Dhaval Adjodah, Yan Leng, Shi Kai Chong, Peter Krafft, Alex Pentland

Abstract: Being able to correctly aggregate the beliefs of many people into a single belief is a problem fundamental to many important social, economic and political processes such as policy making, market pricing and voting. Although there exist many models and mechanisms for aggregation, there is a lack of methods and literature regarding the aggregation of opinions when influence and learning between ind… ▽ More Being able to correctly aggregate the beliefs of many people into a single belief is a problem fundamental to many important social, economic and political processes such as policy making, market pricing and voting. Although there exist many models and mechanisms for aggregation, there is a lack of methods and literature regarding the aggregation of opinions when influence and learning between individuals exist. This is in part because there are not many models of how people update their belief when exposed to the beliefs of others, and so it is hard to quantify the dependencies between people's mental models which is essential to minimizing redundancies in the aggregation. In this paper, we explore many models of how users influence and learn from each other, and we benchmark our models against the well-known DeGroot model. Our main contributions are: 1) we collect a new dataset of unprecedented size and detail to be posted online; 2) we develop a new Social Bayesian model of how people update their mental models, 3) we compare of our model to other well-known social learning models. Specifically, we show that our new Social Bayesian model is superior to the other models tested. △ Less

Submitted 28 December, 2017; originally announced December 2017.

Ranking and Selection as Stochastic Control

Authors: Yijie Peng, Edwin K. P. Chong, Chun-Hung Chen, Michael C. Fu

Abstract: Under a Bayesian framework, we formulate the fully sequential sampling and selection decision in statistical ranking and selection as a stochastic control problem, and derive the associated Bellman equation. Using value function approximation, we derive an approximately optimal allocation policy. We show that this policy is not only computationally efficient but also possesses both one-step-ahead… ▽ More Under a Bayesian framework, we formulate the fully sequential sampling and selection decision in statistical ranking and selection as a stochastic control problem, and derive the associated Bellman equation. Using value function approximation, we derive an approximately optimal allocation policy. We show that this policy is not only computationally efficient but also possesses both one-step-ahead and asymptotic optimality for independent normal sampling distributions. Moreover, the proposed allocation policy is easily generalizable in the approximate dynamic programming paradigm. △ Less

Submitted 6 October, 2017; originally announced October 2017.

Comments: 15 pages, 8 figures, to appear in IEEE Transactions on Automatic Control

MSC Class: IEEE

Network Topology Inference Using Information Cascades with Limited Statistical Knowledge

Authors: Feng Ji, Wenchang Tang, Wee Peng Tay, Edwin K. P. Chong

Abstract: We study the problem of inferring network topology from information cascades, in which the amount of time taken for information to diffuse across an edge in the network follows an unknown distribution. Unlike previous studies, which assume knowledge of these distributions, we only require that diffusion along different edges in the network be independent together with limited moment information (e… ▽ More We study the problem of inferring network topology from information cascades, in which the amount of time taken for information to diffuse across an edge in the network follows an unknown distribution. Unlike previous studies, which assume knowledge of these distributions, we only require that diffusion along different edges in the network be independent together with limited moment information (e.g., the means). We introduce the concept of a separating vertex set for a graph, which is a set of vertices in which for any two given distinct vertices of the graph, there exists a vertex whose distance to them are different. We show that a necessary condition for reconstructing a tree perfectly using distance information between pairs of vertices is given by the size of an observed separating vertex set. We then propose an algorithm to recover the tree structure using infection times, whose differences have means corresponding to the distance between two vertices. To improve the accuracy of our algorithm, we propose the concept of redundant vertices, which allows us to perform averaging to better estimate the distance between two vertices. Though the theory is developed mainly for tree networks, we demonstrate how the algorithm can be extended heuristically to general graphs. Simulations using synthetic and real networks, and experiments using real-world data suggest that our proposed algorithm performs better than some current state-of-the-art network reconstruction methods. △ Less

Submitted 4 March, 2019; v1 submitted 28 June, 2017; originally announced June 2017.

Polynomial-Time Methods to Solve Unimodular Quadratic Programs With Performance Guarantees

Authors: Shankarachary Ragi, Edwin K. P. Chong, Hans D. Mittelmann

Abstract: We develop polynomial-time heuristic methods to solve unimodular quadratic programs (UQPs) approximately, which are known to be NP-hard. In the UQP framework, we maximize a quadratic function of a vector of complex variables with unit modulus. Several problems in active sensing and wireless communication applications boil down to UQP. With this motivation, we present three new heuristic methods wi… ▽ More We develop polynomial-time heuristic methods to solve unimodular quadratic programs (UQPs) approximately, which are known to be NP-hard. In the UQP framework, we maximize a quadratic function of a vector of complex variables with unit modulus. Several problems in active sensing and wireless communication applications boil down to UQP. With this motivation, we present three new heuristic methods with polynomial-time complexity to solve the UQP approximately. The first method is called dominant-eigenvector-matching; here the solution is picked that matches the complex arguments of the dominant eigenvector of the Hermitian matrix in the UQP formulation. We also provide a performance guarantee for this method. The second method, a greedy strategy, is shown to provide a performance guarantee of (1-1/e) with respect to the optimal objective value given that the objective function possesses a property called string submodularity. The third heuristic method is called row-swap greedy strategy, which is an extension to the greedy strategy and utilizes certain properties of the UQP to provide a better performance than the greedy strategy at the expense of an increase in computational complexity. We present numerical results to demonstrate the performance of these heuristic methods, and also compare the performance of these methods against a standard heuristic method called semidefinite relaxation. △ Less

Submitted 24 March, 2017; originally announced March 2017.

Performance Bounds for Nash Equilibria in Submodular Utility Systems with User Groups

Authors: Yajing Liu, Edwin K. P. Chong, Ali Pezeshki

Abstract: In this paper, we consider variations of the utility system considered by Vetta, in which users are grouped together. Our aim is to establish how grouping and cooperation among users affect performance bounds. We consider two types of grouping. The first type is from \cite{Zhang2014}, where each user belongs to a group of users having social ties with it. For this type of utility system, each user… ▽ More In this paper, we consider variations of the utility system considered by Vetta, in which users are grouped together. Our aim is to establish how grouping and cooperation among users affect performance bounds. We consider two types of grouping. The first type is from \cite{Zhang2014}, where each user belongs to a group of users having social ties with it. For this type of utility system, each user's strategy maximizes its social group utility function, giving rise to the notion of \emph{social-aware Nash equilibrium}. We prove that this social utility system yields to the bounding results of Vetta for non-cooperative system, thus establishing provable performance guarantees for the social-aware Nash equilibrium. For the second type of grouping, the set of users is partitioned into $l$ disjoint groups, where the users within a group cooperate to maximize their group utility function, giving rise to the notion of \emph{group Nash equilibrium}. In this case, each group can be viewed as a new user with vector-valued actions, and a 1/2 bound for the performance of group Nash equilibrium follows from the result of Vetta. But as we show tighter bounds involving curvature can be established. By defining the group curvature $c_{k_i}$ associated with group $i$ with $k_i$ users, we show that if the social utility function is nondecreasing and submodular, then any group Nash equilibrium achieves at least $1/(1+\max_{1\leq i\leq l}c_{k_i})$ of the optimal social utility, which is tighter than that for the case without grouping. As a special case, if each user has the same action space, then we have that any group Nash equilibrium achieves at least $1/(1+c_{k^*})$ of the optimal social utility, where $k^*$ is the least number of users among the $l$ groups. Finally, we present an example of a utility system for database assisted spectrum access to illustrate our results. △ Less

Submitted 11 October, 2017; v1 submitted 15 March, 2016; originally announced March 2016.

Comments: This paper was accepted by Journal of Control and Decision

Subspace selection for projection maximization with matroid constraints

Authors: Zhenliang Zhang, Yuan Wang, Edwin K. P. Chong, Ali Pezeshki, Louis Scharf

Abstract: Suppose that there is a ground set which consists of a large number of vectors in a Hilbert space. Consider the problem of selecting a subset of the ground set such that the projection of a vector of interest onto the subspace spanned by the vectors in the chosen subset reaches the maximum norm. This problem is generally NP-hard, and alternative approximation algorithms such as forward regression… ▽ More Suppose that there is a ground set which consists of a large number of vectors in a Hilbert space. Consider the problem of selecting a subset of the ground set such that the projection of a vector of interest onto the subspace spanned by the vectors in the chosen subset reaches the maximum norm. This problem is generally NP-hard, and alternative approximation algorithms such as forward regression and orthogonal matching pursuit have been proposed as heuristic approaches. In this paper, we investigate bounds on the performance of these algorithms by introducing the notions of elemental curvatures. More specifically, we derive lower bounds, as functions of these elemental curvatures, for performance of the aforementioned algorithms with respect to that of the optimal solution under uniform and non-uniform matroid constraints, respectively. We show that if the elements in the ground set are mutually orthogonal, then these algorithms are optimal when the matroid is uniform and they achieve at least $1/2$-approximations of the optimal solution when the matroid is non-uniform. △ Less

Submitted 16 July, 2015; originally announced July 2015.

Robust Decentralized Detection and Social Learning in Tandem Networks

Authors: Jack Ho, Wee Peng Tay, Tony Q. S. Quek, Edwin K. P. Chong

Abstract: We study a tandem of agents who make decisions about an underlying binary hypothesis, where the distribution of the agent observations under each hypothesis comes from an uncertainty class. We investigate both decentralized detection rules, where agents collaborate to minimize the error probability of the final agent, and social learning rules, where each agent minimizes its own local minimax erro… ▽ More We study a tandem of agents who make decisions about an underlying binary hypothesis, where the distribution of the agent observations under each hypothesis comes from an uncertainty class. We investigate both decentralized detection rules, where agents collaborate to minimize the error probability of the final agent, and social learning rules, where each agent minimizes its own local minimax error probability. We then extend our results to the infinite tandem network, and derive necessary and sufficient conditions on the uncertainty classes for the minimax error probability to converge to zero when agents know their positions in the tandem. On the other hand, when agents do not know their positions in the network, we study the cases where agents collaborate to minimize the asymptotic minimax error probability, and where agents seek to minimize their worst-case minimax error probability (over all possible positions in the tandem). We show that asymptotic learning of the true hypothesis is no longer possible in these cases, and derive characterizations for the minimax error performance. △ Less

Submitted 23 January, 2015; originally announced January 2015.

Intersection Information based on Common Randomness

Authors: Virgil Griffith, Edwin K. P. Chong, Ryan G. James, Christopher J. Ellison, James P. Crutchfield

Abstract: The introduction of the partial information decomposition generated a flurry of proposals for defining an intersection information that quantifies how much of "the same information" two or more random variables specify about a target random variable. As of yet, none is wholly satisfactory. A palatable measure of intersection information would provide a principled way to quantify slippery concepts,… ▽ More The introduction of the partial information decomposition generated a flurry of proposals for defining an intersection information that quantifies how much of "the same information" two or more random variables specify about a target random variable. As of yet, none is wholly satisfactory. A palatable measure of intersection information would provide a principled way to quantify slippery concepts, such as synergy. Here, we introduce an intersection information measure based on the Gács-Körner common random variable that is the first to satisfy the coveted target monotonicity property. Our measure is imperfect, too, and we suggest directions for improvement. △ Less

Submitted 10 June, 2015; v1 submitted 6 October, 2013; originally announced October 2013.

Comments: 19 pages, 5 figures

Journal ref: Entropy (2014) Volume 16, Issue 4

String Submodular Functions with Curvature Constraints

Authors: Zhenliang Zhang, Edwin K. P. Chong, Ali Pezeshki, William Moran

Abstract: The problem of objectively choosing a string of actions to optimize an objective function that is string submodular has been considered in [1]. There it is shown that the greedy strategy, consisting of a string of actions that only locally maximizes the step-wise gain in the objective function achieves at least a (1-e^{-1})-approximation to the optimal strategy. This paper improves this approximat… ▽ More The problem of objectively choosing a string of actions to optimize an objective function that is string submodular has been considered in [1]. There it is shown that the greedy strategy, consisting of a string of actions that only locally maximizes the step-wise gain in the objective function achieves at least a (1-e^{-1})-approximation to the optimal strategy. This paper improves this approximation by introducing additional constraints on curvatures, namely, total backward curvature, total forward curvature, and elemental forward curvature. We show that if the objective function has total backward curvature σ, then the greedy strategy achieves at least a \frac{1}σ(1-e^{-σ})-approximation of the optimal strategy. If the objective function has total forward curvature ε, then the greedy strategy achieves at least a (1-ε)-approximation of the optimal strategy. Moreover, we consider a generalization of the diminishing-return property by defining the elemental forward curvature. We also consider the problem of maximizing the objective function subject to general a string-matroid constraint. We investigate an applications of string submodular functions with curvature constraints. △ Less

Submitted 25 May, 2015; v1 submitted 12 March, 2013; originally announced March 2013.

Comments: to appear in IEEE Transaction on Automatic Control

Hypothesis Testing in Feedforward Networks with Broadcast Failures

Authors: Zhenliang Zhang, Edwin K. P. Chong, Ali Pezeshki, William Moran

Abstract: Consider a countably infinite set of nodes, which sequentially make decisions between two given hypotheses. Each node takes a measurement of the underlying truth, observes the decisions from some immediate predecessors, and makes a decision between the given hypotheses. We consider two classes of broadcast failures: 1) each node broadcasts a decision to the other nodes, subject to random erasure i… ▽ More Consider a countably infinite set of nodes, which sequentially make decisions between two given hypotheses. Each node takes a measurement of the underlying truth, observes the decisions from some immediate predecessors, and makes a decision between the given hypotheses. We consider two classes of broadcast failures: 1) each node broadcasts a decision to the other nodes, subject to random erasure in the form of a binary erasure channel; 2) each node broadcasts a randomly flipped decision to the other nodes in the form of a binary symmetric channel. We are interested in whether there exists a decision strategy consisting of a sequence of likelihood ratio tests such that the node decisions converge in probability to the underlying truth. In both cases, we show that if each node only learns from a bounded number of immediate predecessors, then there does not exist a decision strategy such that the decisions converge in probability to the underlying truth. However, in case 1, we show that if each node learns from an unboundedly growing number of predecessors, then the decisions converge in probability to the underlying truth, even when the erasure probabilities converge to 1. We also derive the convergence rate of the error probability. In case 2, we show that if each node learns from all of its previous predecessors, then the decisions converge in probability to the underlying truth when the flipping probabilities of the binary symmetric channels are bounded away from 1/2. In the case where the flipping probabilities converge to 1/2, we derive a necessary condition on the convergence rate of the flipping probabilities such that the decisions still converge to the underlying truth. We also explicitly characterize the relationship between the convergence rate of the error probability and the convergence rate of the flipping probabilities. △ Less

Submitted 25 March, 2013; v1 submitted 19 November, 2012; originally announced November 2012.

Submodularity and Optimality of Fusion Rules in Balanced Binary Relay Trees

Authors: Zhenliang Zhang, Edwin K. P. Chong, Ali Pezeshki, William Moran, Stephen D. Howard

Abstract: We study the distributed detection problem in a balanced binary relay tree, where the leaves of the tree are sensors generating binary messages. The root of the tree is a fusion center that makes the overall decision. Every other node in the tree is a fusion node that fuses two binary messages from its child nodes into a new binary message and sends it to the parent node at the next level. We assu… ▽ More We study the distributed detection problem in a balanced binary relay tree, where the leaves of the tree are sensors generating binary messages. The root of the tree is a fusion center that makes the overall decision. Every other node in the tree is a fusion node that fuses two binary messages from its child nodes into a new binary message and sends it to the parent node at the next level. We assume that the fusion nodes at the same level use the same fusion rule. We call a string of fusion rules used at different levels a fusion strategy. We consider the problem of finding a fusion strategy that maximizes the reduction in the total error probability between the sensors and the fusion center. We formulate this problem as a deterministic dynamic program and express the solution in terms of Bellman's equations. We introduce the notion of stringsubmodularity and show that the reduction in the total error probability is a stringsubmodular function. Consequentially, we show that the greedy strategy, which only maximizes the level-wise reduction in the total error probability, is within a factor of the optimal strategy in terms of reduction in the total error probability. △ Less

Submitted 16 October, 2012; originally announced October 2012.

Learning in Hierarchical Social Networks

Authors: Zhenliang Zhang, Edwin K. P. Chong, Ali Pezeshki, William Moran, Stephen D. Howard

Abstract: We study a social network consisting of agents organized as a hierarchical M-ary rooted tree, common in enterprise and military organizational structures. The goal is to aggregate information to solve a binary hypothesis testing problem. Each agent at a leaf of the tree, and only such an agent, makes a direct measurement of the underlying true hypothesis. The leaf agent then makes a decision and s… ▽ More We study a social network consisting of agents organized as a hierarchical M-ary rooted tree, common in enterprise and military organizational structures. The goal is to aggregate information to solve a binary hypothesis testing problem. Each agent at a leaf of the tree, and only such an agent, makes a direct measurement of the underlying true hypothesis. The leaf agent then makes a decision and sends it to its supervising agent, at the next level of the tree. Each supervising agent aggregates the decisions from the M members of its group, produces a summary message, and sends it to its supervisor at the next level, and so on. Ultimately, the agent at the root of the tree makes an overall decision. We derive upper and lower bounds for the Type I and II error probabilities associated with this decision with respect to the number of leaf agents, which in turn characterize the converge rates of the Type I, Type II, and total error probabilities. We also provide a message-passing scheme involving non-binary message alphabets and characterize the exponent of the error probability with respect to the message alphabet size. △ Less

Submitted 21 November, 2012; v1 submitted 30 May, 2012; originally announced June 2012.

Detection Performance in Balanced Binary Relay Trees with Node and Link Failures

Authors: Zhenliang Zhang, Edwin K. P. Chong, Ali Pezeshki, William Moran, Stephen D. Howard

Abstract: We study the distributed detection problem in the context of a balanced binary relay tree, where the leaves of the tree correspond to $N$ identical and independent sensors generating binary messages. The root of the tree is a fusion center making an overall decision. Every other node is a relay node that aggregates the messages received from its child nodes into a new message and sends it up towar… ▽ More We study the distributed detection problem in the context of a balanced binary relay tree, where the leaves of the tree correspond to $N$ identical and independent sensors generating binary messages. The root of the tree is a fusion center making an overall decision. Every other node is a relay node that aggregates the messages received from its child nodes into a new message and sends it up toward the fusion center. We derive upper and lower bounds for the total error probability $P_N$ as explicit functions of $N$ in the case where nodes and links fail with certain probabilities. These characterize the asymptotic decay rate of the total error probability as $N$ goes to infinity. Naturally, this decay rate is not larger than that in the non-failure case, which is $\sqrt N$. However, we derive an explicit necessary and sufficient condition on the decay rate of the local failure probabilities $p_k$ (combination of node and link failure probabilities at each level) such that the decay rate of the total error probability in the failure case is the same as that of the non-failure case. More precisely, we show that $\log P_N^{-1}=Θ(\sqrt N)$ if and only if $\log p_k^{-1}=Ω(2^{k/2})$. △ Less

Submitted 19 November, 2012; v1 submitted 1 June, 2012; originally announced June 2012.

Greedy Adaptive Compression in Signal-Plus-Noise Models

Authors: Entao Liu, Edwin K. P. Chong, Louis L. Scharf

Abstract: The purpose of this article is to examine the greedy adaptive measurement policy in the context of a linear Guassian measurement model with an optimization criterion based on information gain. In the special case of sequential scalar measurements, we provide sufficient conditions under which the greedy policy actually is optimal in the sense of maximizing the net information gain. In the general s… ▽ More The purpose of this article is to examine the greedy adaptive measurement policy in the context of a linear Guassian measurement model with an optimization criterion based on information gain. In the special case of sequential scalar measurements, we provide sufficient conditions under which the greedy policy actually is optimal in the sense of maximizing the net information gain. In the general setting, we also discuss cases where the greedy policy is not optimal. △ Less

Submitted 17 August, 2012; v1 submitted 17 February, 2012; originally announced February 2012.

Detection Performance of M-ary Relay Trees with Non-binary Message Alphabets

Authors: Zhenliang Zhang, Edwin K. P. Chong, Ali Pezeshki, William Moran, Stephen D. Howard

Abstract: We study the detection performance of $M$-ary relay trees, where only the leaves of the tree represent sensors making measurements. The root of the tree represents the fusion center which makes an overall detection decision. Each of the other nodes is a relay node which aggregates $M$ messages sent by its child nodes into a new compressed message and sends the message to its parent node. Building… ▽ More We study the detection performance of $M$-ary relay trees, where only the leaves of the tree represent sensors making measurements. The root of the tree represents the fusion center which makes an overall detection decision. Each of the other nodes is a relay node which aggregates $M$ messages sent by its child nodes into a new compressed message and sends the message to its parent node. Building on previous work on the detection performance of $M$-ary relay trees with binary messages, in this paper we study the case of non-binary relay message alphabets. We characterize the exponent of the error probability with respect to the message alphabet size $\mathcal D$, showing how the detection performance increases with $\mathcal D$. Our method involves reducing a tree with non-binary relay messages into an equivalent higher-degree tree with only binary messages. △ Less

Submitted 1 November, 2012; v1 submitted 10 February, 2012; originally announced February 2012.

Comments: Submitted to SSP workshop 2012

Error Probability Bounds for M-ary Relay Trees

Authors: Zhenliang Zhang, Edwin K. P. Chong, Ali Pezeshki, William Moran, Stephen D. Howard

Abstract: We study the detection error probabilities associated with an M-ary relay tree, where the leaves of the tree correspond to identical and independent sensors. Only these leaves are sensors. The root of the tree represents a fusion center that makes the overall detection decision. Each of the other nodes in the tree is a relay node that combines M summarized messages from its immediate child nodes t… ▽ More We study the detection error probabilities associated with an M-ary relay tree, where the leaves of the tree correspond to identical and independent sensors. Only these leaves are sensors. The root of the tree represents a fusion center that makes the overall detection decision. Each of the other nodes in the tree is a relay node that combines M summarized messages from its immediate child nodes to form a single output message using the majority dominance rule. We derive tight upper and lower bounds for the Type I and II error probabilities at the fusion center as explicit functions of the number of sensors in the case of binary message alphabets. These bounds characterize how fast the error probabilities converge to 0 with respect to the number of sensors. △ Less

Submitted 1 November, 2012; v1 submitted 7 February, 2012; originally announced February 2012.

Comments: Submitted to ISIT 2012

Measurement Design for Detecting Sparse Signals

Authors: Ramin Zahedi, Ali Pezeshki, Edwin K. P. Chong

Abstract: We consider the problem of testing for the presence (or detection) of an unknown sparse signal in additive white noise. Given a fixed measurement budget, much smaller than the dimension of the signal, we consider the general problem of designing compressive measurements to maximize the measurement signal-to-noise ratio (SNR), as increasing SNR improves the detection performance in a large class of… ▽ More We consider the problem of testing for the presence (or detection) of an unknown sparse signal in additive white noise. Given a fixed measurement budget, much smaller than the dimension of the signal, we consider the general problem of designing compressive measurements to maximize the measurement signal-to-noise ratio (SNR), as increasing SNR improves the detection performance in a large class of detectors. We use a lexicographic optimization approach, where the optimal measurement design for sparsity level $k$ is sought only among the set of measurement matrices that satisfy the optimality conditions for sparsity level k-1. We consider optimizing two different SNR criteria, namely a worst-case SNR measure, over all possible realizations of a k-sparse signal, and an average SNR measure with respect to a uniform distribution on the locations of the up to k nonzero entries in the signal. We establish connections between these two criteria and certain classes of tight frames. We constrain our measurement matrices to the class of tight frames to avoid coloring the noise covariance matrix. For the worst-case problem, we show that the optimal measurement matrix is a Grassmannian line packing for most---and a uniform tight frame for all---sparse signals. For the average SNR problem, we prove that the optimal measurement matrix is a uniform tight frame with minimum sum-coherence for most---and a tight frame for all---sparse signals. △ Less

Submitted 9 July, 2011; originally announced July 2011.

Continuum Limits of Markov Chains with Application to Network Modeling

Authors: Yang Zhang, Edwin K. P. Chong, Jan Hannig, Donald Estep

Abstract: In this paper we investigate the continuum limits of a class of Markov chains. The investigation of such limits is motivated by the desire to model very large networks. We show that under some conditions, a sequence of Markov chains converges in some sense to the solution of a partial differential equation. Based on such convergence we approximate Markov chains modeling networks with a large numbe… ▽ More In this paper we investigate the continuum limits of a class of Markov chains. The investigation of such limits is motivated by the desire to model very large networks. We show that under some conditions, a sequence of Markov chains converges in some sense to the solution of a partial differential equation. Based on such convergence we approximate Markov chains modeling networks with a large number of components by partial differential equations. While traditional Monte Carlo simulation for very large networks is practically infeasible, partial differential equations can be solved with reasonable computational overhead using well-established mathematical tools. △ Less

Submitted 21 June, 2011; originally announced June 2011.

Comments: 15 pages, 10 figures

Error Probability Bounds for Binary Relay Trees with Crummy Sensors

Authors: Zhenliang Zhang, Ali Pezeshki, William Moran, Stephen D. Howard, Edwin K. P. Chong

Abstract: We study the detection error probability associated with balanced binary relay trees, in which sensor nodes fail with some probability. We consider N identical and independent crummy sensors, represented by leaf nodes of the tree. The root of the tree represents the fusion center, which makes the final decision between two hypotheses. Every other node is a relay node, which fuses at most two binar… ▽ More We study the detection error probability associated with balanced binary relay trees, in which sensor nodes fail with some probability. We consider N identical and independent crummy sensors, represented by leaf nodes of the tree. The root of the tree represents the fusion center, which makes the final decision between two hypotheses. Every other node is a relay node, which fuses at most two binary messages into one binary message and forwards the new message to its parent node. We derive tight upper and lower bounds for the total error probability at the fusion center as functions of N and characterize how fast the total error probability converges to 0 with respect to N. We show that the convergence of the total error probability is sub-linear, with the same decay exponent as that in a balanced binary relay tree without sensor failures. We also show that the total error probability converges to 0, even if the individual sensors have total error probabilities that converge to 1/2 and the failure probabilities that converge to 1, provided that the convergence rates are sufficiently slow. △ Less

Submitted 31 May, 2011; originally announced June 2011.