Showing 1–8 of 8 results for author: Sultan, S

On the ability of Deep Neural Networks to Learn Granger Causality in Multi-Variate Time Series Data

Authors: Malik Shahid Sultan, Hernando Ombao

Abstract: Granger Causality (GC) offers an elegant statistical framework to study the association between multivariate time series data. Linear Vector Autoregressive models (VAR) though have nice interpretation properties but have limited practical application due to underlying assumptions on the kind of associations that can be captured by these models. Numerous attempts have already been made in the liter… ▽ More Granger Causality (GC) offers an elegant statistical framework to study the association between multivariate time series data. Linear Vector Autoregressive models (VAR) though have nice interpretation properties but have limited practical application due to underlying assumptions on the kind of associations that can be captured by these models. Numerous attempts have already been made in the literature that exploit the functional approximation power of Deep Neural Networks (DNNs) for the task of GC estimation. These methods however treat GC as a variable selection problem. We present a novel paradigm for approaching GC. We present this idea that GC is essentially linked with prediction and if a deep learning model is used to model the time series collectively or jointly, a well regularized model may learn the true granger causal structure from the data, given that there is enough training data. We propose to uncover the learned GC structure by comparing the model uncertainty or distribution of the residuals when the past of everything is used as compared to the one where a specific time series component is dropped from the model. We also compare the effect of input layer dropout on the ability of a neural network to learn granger causality from the data. We show that a well regularized model infact can learn the true GC structure from the data without explicitly adding terms in the loss function that guide the model to select variables or perform sparse regression. △ Less

Submitted 25 June, 2025; originally announced June 2025.

Classification of High-dimensional Time Series in Spectral Domain using Explainable Features

Authors: Sarbojit Roy, Malik Shahid Sultan, Hernando Ombao

Abstract: Interpretable classification of time series presents significant challenges in high dimensions. Traditional feature selection methods in the frequency domain often assume sparsity in spectral density matrices (SDMs) or their inverses, which can be restrictive for real-world applications. In this article, we propose a model-based approach for classifying high-dimensional stationary time series by a… ▽ More Interpretable classification of time series presents significant challenges in high dimensions. Traditional feature selection methods in the frequency domain often assume sparsity in spectral density matrices (SDMs) or their inverses, which can be restrictive for real-world applications. In this article, we propose a model-based approach for classifying high-dimensional stationary time series by assuming sparsity in the difference between inverse SDMs. Our approach emphasizes the interpretability of model parameters, making it especially suitable for fields like neuroscience, where understanding differences in brain network connectivity across various states is crucial. The estimators for model parameters demonstrate consistency under appropriate conditions. We further propose using standard deep learning optimizers for parameter estimation, employing techniques such as mini-batching and learning rate scheduling. Additionally, we introduce a method to screen the most discriminatory frequencies for classification, which exhibits the sure screening property under general conditions. The flexibility of the proposed model allows the significance of covariates to vary across frequencies, enabling nuanced inferences and deeper insights into the underlying problem. The novelty of our method lies in the interpretability of the model parameters, addressing critical needs in neuroscience. The proposed approaches have been evaluated on simulated examples and the `Alert-vs-Drowsy' EEG dataset. △ Less

Submitted 15 August, 2024; originally announced August 2024.

Simpson's Paradox and Lagging Progress in Completion Trends of Underrepresented Students in Computer Science

Authors: John Mason Taylor, Rebecca Drucker, Chris Alvin, Syed Fahad Sultan

Abstract: It is imperative for the Computer Science (CS) community to ensure active participation and success of students from diverse backgrounds. This work compares CS to other areas of study with respect to success of students from three underrepresented groups: Women, Black and Hispanic or Latino. Using a data-driven approach, we show that trends of success over the years for underrepresented groups in… ▽ More It is imperative for the Computer Science (CS) community to ensure active participation and success of students from diverse backgrounds. This work compares CS to other areas of study with respect to success of students from three underrepresented groups: Women, Black and Hispanic or Latino. Using a data-driven approach, we show that trends of success over the years for underrepresented groups in CS are lagging behind other disciplines. Completion of CS programs by Black students in particular shows an alarming regression in the years 2011 through 2019. This national level decline is most concentrated in the Southeast of the United States and seems to be driven mostly by a small number of institutes that produce a large number of graduates. We strongly believe that more data-driven studies in this area are necessary to make progress towards a more equitable and inclusive CS community. Without an understanding of underlying dynamics, policy makers and practitioners will be unable to make informed decisions about how and where to allocate resources to address the problem. △ Less

Submitted 24 November, 2023; originally announced November 2023.

Stylized Projected GAN: A Novel Architecture for Fast and Realistic Image Generation

Authors: Md Nurul Muttakin, Malik Shahid Sultan, Robert Hoehndorf, Hernando Ombao

Abstract: Generative Adversarial Networks are used for generating the data using a generator and a discriminator, GANs usually produce high-quality images, but training GANs in an adversarial setting is a difficult task. GANs require high computation power and hyper-parameter regularization for converging. Projected GANs tackle the training difficulty of GANs by using transfer learning to project the genera… ▽ More Generative Adversarial Networks are used for generating the data using a generator and a discriminator, GANs usually produce high-quality images, but training GANs in an adversarial setting is a difficult task. GANs require high computation power and hyper-parameter regularization for converging. Projected GANs tackle the training difficulty of GANs by using transfer learning to project the generated and real samples into a pre-trained feature space. Projected GANs improve the training time and convergence but produce artifacts in the generated images which reduce the quality of the generated samples, we propose an optimized architecture called Stylized Projected GANs which integrates the mapping network of the Style GANs with Skip Layer Excitation of Fast GAN. The integrated modules are incorporated within the generator architecture of the Fast GAN to mitigate the problem of artifacts in the generated images. △ Less

Submitted 30 July, 2023; originally announced July 2023.

Comments: We present a new architecture for generating realistic images by combining mapping network of Style GANs and Projected GANs

Granger Causality using Neural Networks

Authors: Malik Shahid Sultan, Samuel Horvath, Hernando Ombao

Abstract: Dependence between nodes in a network is an important concept that pervades many areas including finance, politics, sociology, genomics and the brain sciences. One way to characterize dependence between components of a multivariate time series data is via Granger Causality (GC). Standard traditional approaches to GC estimation / inference commonly assume linear dynamics, however such simplificatio… ▽ More Dependence between nodes in a network is an important concept that pervades many areas including finance, politics, sociology, genomics and the brain sciences. One way to characterize dependence between components of a multivariate time series data is via Granger Causality (GC). Standard traditional approaches to GC estimation / inference commonly assume linear dynamics, however such simplification does not hold in many real-world applications where signals are inherently non-linear. In such cases, imposing linear models such as vector autoregressive (VAR) models can lead to mis-characterization of true Granger Causal interactions. To overcome this limitation, Tank et al (IEEE Transactions on Pattern Analysis and Machine Learning, 2022) proposed a solution that uses neural networks with sparse regularization penalties. The regularization encourages learnable weights to be sparse, which enables inference on GC. This paper overcomes the limitations of current methods by leveraging advances in machine learning and deep learning which have been demonstrated to learn hidden patterns in the data. We propose novel classes of models that can handle underlying non-linearity in a computationally efficient manner, simultaneously providing GC and lag order selection. Firstly, we present the Learned Kernel VAR (LeKVAR) model that learns kernel parameterized by a shared neural net followed by penalization on learnable weights to discover GC structure. Secondly, we show one can directly decouple lags and individual time series importance via decoupled penalties. This is important as we want to select the lag order during the process of GC estimation. This decoupling acts as a filtering and can be extended to any DL model including Multi-Layer Perceptrons (MLP), Recurrent Neural Networks (RNN), Long Short Term Memory Networks (LSTM), Transformers etc, for simultaneous GC estimation and lag selection. △ Less

Submitted 7 August, 2024; v1 submitted 7 August, 2022; originally announced August 2022.

Comments: To be Submitted to a Journal work Presented at JSM. arXiv admin note: text overlap with arXiv:1802.05842 by other authors

Calcium Vulnerability Scanner (CVS): A Deeper Look

Authors: Sari Sultan, Ayed Salman

Abstract: Traditional vulnerability scanning methods are time-consuming and indecisive, and they negatively affect network performance by generating high network traffic. In this paper, we present a novel vulnerability scanner that is time-efficient, simple, accurate, and safe. We call it a Calcium Vulnerability Scanner (CVS). Our contribution to vulnerability scanning are the following: (i) minimize its re… ▽ More Traditional vulnerability scanning methods are time-consuming and indecisive, and they negatively affect network performance by generating high network traffic. In this paper, we present a novel vulnerability scanner that is time-efficient, simple, accurate, and safe. We call it a Calcium Vulnerability Scanner (CVS). Our contribution to vulnerability scanning are the following: (i) minimize its required time and network traffic: compared to current technologies, we reduced the former by an average of 79% and the latter by 99.9%, (ii) increase its accuracy: compared to current technologies, we improved this by an average of 2600%, and (iii) enable the scanner to learn from previous scans in order to reduce future scanning time and enhance accuracy: compared to current technologies, CVS reduced scanning time by an average of 97%. CVS enables a new frontier in vulnerability scanning and allow for scalable and efficient deployment of such tools in large-scale networks, containers, edge computing, and cloud computing. △ Less

Submitted 3 November, 2019; originally announced November 2019.

Fast and Accurate Network Embeddings via Very Sparse Random Projection

Authors: Haochen Chen, Syed Fahad Sultan, Yingtao Tian, Muhao Chen, Steven Skiena

Abstract: We present FastRP, a scalable and performant algorithm for learning distributed node representations in a graph. FastRP is over 4,000 times faster than state-of-the-art methods such as DeepWalk and node2vec, while achieving comparable or even better performance as evaluated on several real-world networks on various downstream tasks. We observe that most network embedding methods consist of two com… ▽ More We present FastRP, a scalable and performant algorithm for learning distributed node representations in a graph. FastRP is over 4,000 times faster than state-of-the-art methods such as DeepWalk and node2vec, while achieving comparable or even better performance as evaluated on several real-world networks on various downstream tasks. We observe that most network embedding methods consist of two components: construct a node similarity matrix and then apply dimension reduction techniques to this matrix. We show that the success of these methods should be attributed to the proper construction of this similarity matrix, rather than the dimension reduction method employed. FastRP is proposed as a scalable algorithm for network embeddings. Two key features of FastRP are: 1) it explicitly constructs a node similarity matrix that captures transitive relationships in a graph and normalizes matrix entries based on node degrees; 2) it utilizes very sparse random projection, which is a scalable optimization-free method for dimension reduction. An extra benefit from combining these two design choices is that it allows the iterative computation of node embeddings so that the similarity matrix need not be explicitly constructed, which further speeds up FastRP. FastRP is also advantageous for its ease of implementation, parallelization and hyperparameter tuning. The source code is available at https://github.com/GTmac/FastRP. △ Less

Submitted 29 August, 2019; originally announced August 2019.

Comments: CIKM 2019 Long Paper

A Comprehensive Review on Privacy-Preserving Metering in Smart Grid for Billing; Operation; and Rewarding Schemes, with Two Novel Solutions

Authors: Sari Sultan

Abstract: In smart grid, the Utility Provider (UP) collects users power measurements' for two main reasons: billing and operation. Billing needs coarse-grained measurements where there are no, or minimal, privacy concerns. On the other hand, operation needs fine-grained measurements which can highly affect users' privacy. Hence, users might be reluctant to participate in operational metering to protect thei… ▽ More In smart grid, the Utility Provider (UP) collects users power measurements' for two main reasons: billing and operation. Billing needs coarse-grained measurements where there are no, or minimal, privacy concerns. On the other hand, operation needs fine-grained measurements which can highly affect users' privacy. Hence, users might be reluctant to participate in operational metering to protect their privacy. To overcome this issue, UP might offer rewards to allure users to report their measurements which endangers their privacy. In this paper, we survey the literature on privacy-preserving metering in smart grid. We propose a new taxonomy to categorize the literature based on whether measurements need to be attributable to users or not. This categorization is highly effective because attribution is one of the main causes of privacy issues. Our survey shows that rewarding schemes and incentive-based demand response are new research topics with limited studies on it. Hence, we propose two privacy-preserving rewarding schemes for operational metering. First, a light-weight solution that utilizes an already established link of trust i.e. Banks. It is based on the assumption that banks will not leak a client account details. This solution is based on hash chains and blind signatures to assure users anonymity with minimal overhead for the UP and the smart meter. The bank in this approach is state-less in which is not required the bank to keep communications states of users and the UP. Second, we proposed another approach that does not rely on or banks or trusted third parties. It is based on Paillier homomorphic encryption and Pedersen commitments. It assumes that the smart meters are organized in a tree structure; which is widely used implementation in the literature. Security analysis for the proposed approaches is presented including unlinkability proof for measurements to users. △ Less

Submitted 28 October, 2018; v1 submitted 22 February, 2018; originally announced February 2018.

Comments: error in the text