-
Bayesian Semiparametric Joint Modeling of Gap-Time Distribution for Multitype Recurrent Events and a Terminal Event
Authors:
Mithun Kumar Acharjee,
AKM Fazlur Rahman
Abstract:
In biomedical settings, multitype recurrent events such as stroke and heart failure occur frequently, often concluding with a terminal event such as death. Understanding the links between these recurring and terminal events is fundamental to developing interventions that delay detrimental outcomes. Joint modeling is needed to quantify the dependence between event types and between recurrent events…
▽ More
In biomedical settings, multitype recurrent events such as stroke and heart failure occur frequently, often concluding with a terminal event such as death. Understanding the links between these recurring and terminal events is fundamental to developing interventions that delay detrimental outcomes. Joint modeling is needed to quantify the dependence between event types and between recurrent events and mortality. We propose a Bayesian semiparametric joint model on the gap-time scale for multitype recurrent events and a terminal event. The model includes a shared frailty that links all recurrent types and the terminal event. Each baseline hazard is assigned a gamma-process prior, while regression and frailty parameters receive standard parametric priors. This ensures flexible baselines and familiar effect measures. The construction gives closed-form expressions for the cumulative hazard and frailty component and connects to Breslow-Aalen type estimators as a special case of our estimator, linking the Bayesian procedure to the classical approach. Computationally, we develop a simple MCMC sampler that avoids large matrix factorizations and scales nearly linearly in sample size. A comprehensive simulation evaluates four criteria: accuracy, prediction, robustness, and computation. There is no exact frequentist version of our specification; for comparison, we fit the same model with an EM algorithm in a frequentist framework. Our model and MCMC algorithm demonstrate superior performance on each criterion. We illustrate the approach with data from the Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT), jointly analyzing acute and chronic cardiovascular recurrences and death.
△ Less
Submitted 12 September, 2025;
originally announced September 2025.
-
Seeing Isn't Believing: Addressing the Societal Impact of Deepfakes in Low-Tech Environments
Authors:
Azmine Toushik Wasi,
Rahatun Nesa Priti,
Mahir Absar Khan,
Abdur Rahman,
Mst Rafia Islam
Abstract:
Deepfakes, AI-generated multimedia content that mimics real media, are becoming increasingly prevalent, posing significant risks to political stability, social trust, and economic well-being, especially in developing societies with limited media literacy and technological infrastructure. This work aims to understand how these technologies are perceived and impact resource-limited communities. We c…
▽ More
Deepfakes, AI-generated multimedia content that mimics real media, are becoming increasingly prevalent, posing significant risks to political stability, social trust, and economic well-being, especially in developing societies with limited media literacy and technological infrastructure. This work aims to understand how these technologies are perceived and impact resource-limited communities. We conducted a survey to assess public awareness, perceptions, and experiences with deepfakes, leading to the development of a comprehensive framework for prevention, detection, and mitigation in tech-limited environments. Our findings reveal critical knowledge gaps and a lack of effective detection tools, emphasizing the need for targeted education and accessible verification solutions. This work offers actionable insights to support vulnerable populations and calls for further interdisciplinary efforts to tackle deepfake challenges globally, particularly in the Global South.
△ Less
Submitted 13 August, 2025;
originally announced August 2025.
-
To Buy an Electric Vehicle or Not? A Bayesian Analysis of Consumer Intent in the United States
Authors:
Nafisa Lohawala,
Mohammad Arshad Rahman
Abstract:
The adoption of electric vehicles (EVs) is considered critical to achieving climate goals, yet it hinges on consumer interest. This study explores how public intent to purchase EVs relates to four unexamined factors: exposure to EV information, perceptions of EVs' environmental benefits, views on government climate policy, and confidence in future EV infrastructure; while controlling for prior EV…
▽ More
The adoption of electric vehicles (EVs) is considered critical to achieving climate goals, yet it hinges on consumer interest. This study explores how public intent to purchase EVs relates to four unexamined factors: exposure to EV information, perceptions of EVs' environmental benefits, views on government climate policy, and confidence in future EV infrastructure; while controlling for prior EV ownership, political affiliation, and demographic characteristics (e.g., age, gender, education, and geographic location). We utilize data from three nationally representative opinion polls conducted by the Pew Research Center between 2021 and 2023, and employ Bayesian techniques to estimate the ordinal probit and ordinal quantile models. Results from ordinal probit show that respondents who are well-informed about EVs, perceive them as environmentally beneficial, or are confident in development of charging stations are more likely to express strong interest in buying an EV, with covariate effects--a metric rarely reported in EV research--of 10.2, 15.5, and 19.1 percentage points, respectively. In contrast, those skeptical of government climate initiatives are more likely to express no interest, by more than 10 percentage points. Prior EV ownership exhibits the highest covariate effect (ranging from 19.0 to 23.1 percentage points), and the impact of most demographic variables is consistent with existing studies. The ordinal quantile models demonstrate significant variation in covariate effects across the distribution of EV purchase intent, offering insights beyond the ordinal probit model. This article is the first to use quantile modeling to reveal how covariate effects differ significantly throughout the spectrum of EV purchase intent.
△ Less
Submitted 13 April, 2025;
originally announced April 2025.
-
Assessing and Predicting Air Pollution in Asia: A Regional and Temporal Study (2018-2023)
Authors:
Anika Rahman,
Mst. Taskia Khatun
Abstract:
This study analyzes and predicts air pollution in Asia, focusing on PM 2.5 levels from 2018 to 2023 across five regions: Central, East, South, Southeast, and West Asia. South Asia emerged as the most polluted region, with Bangladesh, India, and Pakistan consistently having the highest PM 2.5 levels and death rates, especially in Nepal, Pakistan, and India. East Asia showed the lowest pollution lev…
▽ More
This study analyzes and predicts air pollution in Asia, focusing on PM 2.5 levels from 2018 to 2023 across five regions: Central, East, South, Southeast, and West Asia. South Asia emerged as the most polluted region, with Bangladesh, India, and Pakistan consistently having the highest PM 2.5 levels and death rates, especially in Nepal, Pakistan, and India. East Asia showed the lowest pollution levels. K-means clustering categorized countries into high, moderate, and low pollution groups. The ARIMA model effectively predicted 2023 PM 2.5 levels (MAE: 3.99, MSE: 33.80, RMSE: 5.81, R: 0.86). The findings emphasize the need for targeted interventions to address severe pollution and health risks in South Asia.
△ Less
Submitted 26 January, 2025;
originally announced January 2025.
-
A Shape-Based Functional Index for Objective Assessment of Pediatric Motor Function
Authors:
Shashwat Kumar,
Arafat Rahman,
Robert Gutierrez,
Sarah Livermon,
Allison N. McCrady,
Silvia Blemker,
Rebecca Scharf,
Anuj Srivastava,
Laura E. Barnes
Abstract:
Clinical assessments for neuromuscular disorders, such as Spinal Muscular Atrophy (SMA) and Duchenne Muscular Dystrophy (DMD), continue to rely on subjective measures to monitor treatment response and disease progression. We introduce a novel method using wearable sensors to objectively assess motor function during daily activities in 19 patients with DMD, 9 with SMA, and 13 age-matched controls.…
▽ More
Clinical assessments for neuromuscular disorders, such as Spinal Muscular Atrophy (SMA) and Duchenne Muscular Dystrophy (DMD), continue to rely on subjective measures to monitor treatment response and disease progression. We introduce a novel method using wearable sensors to objectively assess motor function during daily activities in 19 patients with DMD, 9 with SMA, and 13 age-matched controls. Pediatric movement data is complex due to confounding factors such as limb length variations in growing children and variability in movement speed. Our approach uses Shape-based Principal Component Analysis to align movement trajectories and identify distinct kinematic patterns, including variations in motion speed and asymmetry. Both DMD and SMA cohorts have individuals with motor function on par with healthy controls. Notably, patients with SMA showed greater activation of the motion asymmetry pattern. We further combined projections on these principal components with partial least squares (PLS) to identify a covariation mode with a canonical correlation of r = 0.78 (95% CI: [0.34, 0.94]) with muscle fat infiltration, the Brooke score (a motor function score), and age-related degenerative changes, proposing a novel motor function index. This data-driven method can be deployed in home settings, enabling better longitudinal tracking of treatment efficacy for children with neuromuscular disorders.
△ Less
Submitted 2 January, 2025;
originally announced January 2025.
-
Dhoroni: Exploring Bengali Climate Change and Environmental Views with a Multi-Perspective News Dataset and Natural Language Processing
Authors:
Azmine Toushik Wasi,
Wahid Faisal,
Taj Ahmad,
Abdur Rahman,
Mst Rafia Islam
Abstract:
Climate change poses critical challenges globally, disproportionately affecting low-income countries that often lack resources and linguistic representation on the international stage. Despite Bangladesh's status as one of the most vulnerable nations to climate impacts, research gaps persist in Bengali-language studies related to climate change and NLP. To address this disparity, we introduce Dhor…
▽ More
Climate change poses critical challenges globally, disproportionately affecting low-income countries that often lack resources and linguistic representation on the international stage. Despite Bangladesh's status as one of the most vulnerable nations to climate impacts, research gaps persist in Bengali-language studies related to climate change and NLP. To address this disparity, we introduce Dhoroni, a novel Bengali (Bangla) climate change and environmental news dataset, comprising a 2300 annotated Bangla news articles, offering multiple perspectives such as political influence, scientific/statistical data, authenticity, stance detection, and stakeholder involvement. Furthermore, we present an in-depth exploratory analysis of Dhoroni and introduce BanglaBERT-Dhoroni family, a novel baseline model family for climate and environmental opinion detection in Bangla, fine-tuned on our dataset. This research contributes significantly to enhancing accessibility and analysis of climate discourse in Bengali (Bangla), addressing crucial communication and research gaps in climate-impacted regions like Bangladesh with 180 million people.
△ Less
Submitted 1 November, 2024; v1 submitted 22 October, 2024;
originally announced October 2024.
-
Flexible Bayesian Quantile Analysis of Residential Rental Rates
Authors:
Ivan Jeliazkov,
Shubham Karnawat,
Mohammad Arshad Rahman,
Angela Vossmeyer
Abstract:
This article develops a random effects quantile regression model for panel data that allows for increased distributional flexibility, multivariate heterogeneity, and time-invariant covariates in situations where mean regression may be unsuitable. Our approach is Bayesian and builds upon the generalized asymmetric Laplace distribution to decouple the modeling of skewness from the quantile parameter…
▽ More
This article develops a random effects quantile regression model for panel data that allows for increased distributional flexibility, multivariate heterogeneity, and time-invariant covariates in situations where mean regression may be unsuitable. Our approach is Bayesian and builds upon the generalized asymmetric Laplace distribution to decouple the modeling of skewness from the quantile parameter. We derive an efficient simulation-based estimation algorithm, demonstrate its properties and performance in targeted simulation studies, and employ it in the computation of marginal likelihoods to enable formal Bayesian model comparisons. The methodology is applied in a study of U.S. residential rental rates following the Global Financial Crisis. Our empirical results provide interesting insights on the interaction between rents and economic, demographic and policy variables, weigh in on key modeling features, and overwhelmingly support the additional flexibility at nearly all quantiles and across several sub-samples. The practical differences that arise as a result of allowing for flexible modeling can be nontrivial, especially for quantiles away from the median.
△ Less
Submitted 6 September, 2023; v1 submitted 23 May, 2023;
originally announced May 2023.
-
Energy Consumption Modeling for DED-based Hybrid Additive Manufacturing
Authors:
Md Rabiul Hasan,
Zhichao Liu,
Asif Rahman
Abstract:
The awareness of energy consumption is gaining much more attention in manufacturing due to its economic and sustainability benefits. An energy consumption model is needed for quantifying the consumption and predicting the impact of various process parameters in manufacturing. This paper aims to develop an energy consumption model for Direct Energy Deposition (DED) based Hybrid Additive Manufacturi…
▽ More
The awareness of energy consumption is gaining much more attention in manufacturing due to its economic and sustainability benefits. An energy consumption model is needed for quantifying the consumption and predicting the impact of various process parameters in manufacturing. This paper aims to develop an energy consumption model for Direct Energy Deposition (DED) based Hybrid Additive Manufacturing (HAM) for an Inconel 718 part. The Specific Energy Consumption (SEC) is used while developing the energy consumption of the product manufacturing lifecycle. This study focuses on the analysis to investigate three significant factors (scanning speed, laser power, and feed rate), their interactions' effects, and whether they have a significant effect.in energy consumption. The results suggest that all the factors have a strong influence, but their interaction effects have a weak influence on the energy consumption for HAM. Among the three process parameters, it is found that laser power has the most significant effect on energy consumption. Again, based on the regression analysis, this study also recommends high scanning speed while the laser power and feed rate should be low. Also, idle time has significant energy consumption during the whole HAM process.
△ Less
Submitted 15 May, 2023;
originally announced May 2023.
-
An operational framework to automatically evaluate the quality of weather observations from third-party stations
Authors:
Quanxi Shao,
Ming Li,
Joel Janek Dabrowski,
Shuvo Bakar,
Ashfaqur Rahman,
Andrea Powell,
Brent Henderson
Abstract:
With increasing number of crowdsourced private automatic weather stations (called TPAWS) established to fill the gap of official network and obtain local weather information for various purposes, the data quality is a major concern in promoting their usage. Proper quality control and assessment are necessary to reach mutual agreement on the TPAWS observations. To derive near real-time assessment f…
▽ More
With increasing number of crowdsourced private automatic weather stations (called TPAWS) established to fill the gap of official network and obtain local weather information for various purposes, the data quality is a major concern in promoting their usage. Proper quality control and assessment are necessary to reach mutual agreement on the TPAWS observations. To derive near real-time assessment for operational system, we propose a simple, scalable and interpretable framework based on AI/Stats/ML models. The framework constructs separate models for individual data from official sources and then provides the final assessment by fusing the individual models. The performance of our proposed framework is evaluated by synthetic data and demonstrated by applying it to a re-al TPAWS network.
△ Less
Submitted 4 December, 2022;
originally announced December 2022.
-
Quality Control in Weather Monitoring with Dynamic Linear Models
Authors:
Joel Janek Dabrowski,
Ashfaqur Rahman,
Ming Li,
Quanxi Shao,
Shuvo Bakar,
Andrea Powell,
Brent Henderson
Abstract:
Decisions in agriculture are frequently based on weather. With an increase in the availability and affordability of off-the-shelf weather stations, farmers able to acquire localised weather information. However, with uncertainty in the sensor and installation quality, farmers are at risk of making poor decisions based on incorrect data. We present an automated approach to perform quality control o…
▽ More
Decisions in agriculture are frequently based on weather. With an increase in the availability and affordability of off-the-shelf weather stations, farmers able to acquire localised weather information. However, with uncertainty in the sensor and installation quality, farmers are at risk of making poor decisions based on incorrect data. We present an automated approach to perform quality control on weather sensors. Our approach uses time-series modelling and data fusion with Bayesian principles to provide predictions with uncertainty quantification. These predictions and uncertainty are used to estimate the validity of a sensor observation. We test on temperature, wind, and humidity data and achieve error hit rates above 80% and false negative rates below 11%.
△ Less
Submitted 2 March, 2023; v1 submitted 8 November, 2022;
originally announced November 2022.
-
Bayesian Quantile Regression for Ordinal Models
Authors:
Mohammad Arshad Rahman
Abstract:
The paper introduces a Bayesian estimation method for quantile regression in univariate ordinal models. Two algorithms are presented that utilize the latent variable inferential framework of Albert and Chib (1993) and the normal-exponential mixture representation of the asymmetric Laplace distribution. Estimation utilizes Markov chain Monte Carlo simulation - either Gibbs sampling together with th…
▽ More
The paper introduces a Bayesian estimation method for quantile regression in univariate ordinal models. Two algorithms are presented that utilize the latent variable inferential framework of Albert and Chib (1993) and the normal-exponential mixture representation of the asymmetric Laplace distribution. Estimation utilizes Markov chain Monte Carlo simulation - either Gibbs sampling together with the Metropolis-Hastings algorithm or only Gibbs sampling. The algorithms are employed in two simulation studies and implemented in the analysis of problems in economics (educational attainment) and political economy (public opinion on extending "Bush Tax" cuts). Investigations into model comparison exemplify the practical utility of quantile ordinal models.
△ Less
Submitted 29 September, 2022;
originally announced September 2022.
-
Spike-and-Slab LASSO Generalized Additive Models and Scalable Algorithms for High-Dimensional Data Analysis
Authors:
Boyi Guo,
Byron C. Jaeger,
A. K. M. Fazlur Rahman,
D. Leann Long,
Nengjun Yi
Abstract:
There are proposals that extend the classical generalized additive models (GAMs) to accommodate high-dimensional data ($p>>n$) using group sparse regularization. However, the sparse regularization may induce excess shrinkage when estimating smooth functions, damaging predictive performance. Moreover, most of these GAMs consider an "all-in-all-out" approach for functional selection, rendering them…
▽ More
There are proposals that extend the classical generalized additive models (GAMs) to accommodate high-dimensional data ($p>>n$) using group sparse regularization. However, the sparse regularization may induce excess shrinkage when estimating smooth functions, damaging predictive performance. Moreover, most of these GAMs consider an "all-in-all-out" approach for functional selection, rendering them difficult to answer if nonlinear effects are necessary. While some Bayesian models can address these shortcomings, using Markov chain Monte Carlo algorithms for model fitting creates a new challenge, scalability. Hence, we propose Bayesian hierarchical generalized additive models as a solution: we consider the smoothing penalty for proper shrinkage of curve interpolation via reparameterization. A novel two-part spike-and-slab LASSO prior for smooth functions is developed to address the sparsity of signals while providing extra flexibility to select the linear or nonlinear components of smooth functions. A scalable and deterministic algorithm, EM-Coordinate Descent, is implemented in an open-source R package BHAM. Simulation studies and metabolomics data analyses demonstrate improved predictive and computational performance against state-of-the-art models. Functional selection performance suggests trade-offs exist regarding the effect hierarchy assumption.
△ Less
Submitted 16 May, 2022; v1 submitted 27 October, 2021;
originally announced October 2021.
-
bqror: An R package for Bayesian Quantile Regression in Ordinal Models
Authors:
Prajual Maheshwari,
Mohammad Arshad Rahman
Abstract:
This article describes an R package bqror that estimates Bayesian quantile regression for ordinal models introduced in Rahman (2016). The paper classifies ordinal models into two types and offers computationally efficient, yet simple, Markov chain Monte Carlo (MCMC) algorithms for estimating ordinal quantile regression. The generic ordinal model with 3 or more outcomes (labeled ORI model) is estim…
▽ More
This article describes an R package bqror that estimates Bayesian quantile regression for ordinal models introduced in Rahman (2016). The paper classifies ordinal models into two types and offers computationally efficient, yet simple, Markov chain Monte Carlo (MCMC) algorithms for estimating ordinal quantile regression. The generic ordinal model with 3 or more outcomes (labeled ORI model) is estimated by a combination of Gibbs sampling and Metropolis-Hastings algorithm. Whereas an ordinal model with exactly 3 outcomes (labeled ORII model) is estimated using Gibbs sampling only. In line with the Bayesian literature, we suggest using marginal likelihood for comparing alternative quantile regression models and explain how to compute the same. The models and their estimation procedures are illustrated via multiple simulation studies and implemented in two applications. The article also describes several other functions contained within the bqror package, which are necessary for estimation, inference, and assessing model fit.
△ Less
Submitted 27 May, 2023; v1 submitted 28 September, 2021;
originally announced September 2021.
-
Modeling and Analysis of Discrete Response Data: Applications to Public Opinion on Marijuana Legalization in the United States
Authors:
Mohit Batham,
Soudeh Mirghasemi,
Mohammad Arshad Rahman,
Manini Ojha
Abstract:
This chapter presents an overview of a specific form of limited dependent variable models, namely discrete choice models, where the dependent (response or outcome) variable takes values which are discrete, inherently ordered, and characterized by an underlying continuous latent variable. Within this setting, the dependent variable may take only two discrete values (such as 0 and 1) giving rise to…
▽ More
This chapter presents an overview of a specific form of limited dependent variable models, namely discrete choice models, where the dependent (response or outcome) variable takes values which are discrete, inherently ordered, and characterized by an underlying continuous latent variable. Within this setting, the dependent variable may take only two discrete values (such as 0 and 1) giving rise to binary models (e.g., probit and logit models) or more than two values (say $j=1,2, \ldots, J$, where $J$ is some integer, typically small) giving rise to ordinal models (e.g., ordinal probit and ordinal logit models). In these models, the primary goal is to model the probability of responses/outcomes conditional on the covariates. We connect the outcomes of a discrete choice model to the random utility framework in economics, discuss estimation techniques, present the calculation of covariate effects and measures to assess model fitting. Some recent advances in discrete data modeling are also discussed. Following the theoretical review, we utilize the binary and ordinal models to analyze public opinion on marijuana legalization and the extent of legalization -- a socially relevant but controversial topic in the United States. We obtain several interesting results including that past use of marijuana, belief about legalization and political partisanship are important factors that shape the public opinion.
△ Less
Submitted 27 May, 2023; v1 submitted 21 September, 2021;
originally announced September 2021.
-
Kolmogorov-Smirnov Test-Based Actively-Adaptive Thompson Sampling for Non-Stationary Bandits
Authors:
Gourab Ghatak,
Hardhik Mohanty,
Aniq Ur Rahman
Abstract:
We consider the non-stationary multi-armed bandit (MAB) framework and propose a Kolmogorov-Smirnov (KS) test based Thompson Sampling (TS) algorithm named TS-KS, that actively detects change points and resets the TS parameters once a change is detected. In particular, for the two-armed bandit case, we derive bounds on the number of samples of the reward distribution to detect the change once it occ…
▽ More
We consider the non-stationary multi-armed bandit (MAB) framework and propose a Kolmogorov-Smirnov (KS) test based Thompson Sampling (TS) algorithm named TS-KS, that actively detects change points and resets the TS parameters once a change is detected. In particular, for the two-armed bandit case, we derive bounds on the number of samples of the reward distribution to detect the change once it occurs. Consequently, we show that the proposed algorithm has sub-linear regret. Contrary to existing works, our algorithm is able to detect a change when the underlying reward distribution changes even though the mean reward remains the same. Finally, to test the efficacy of the proposed algorithm, we employ it in two case-studies: i) task-offloading scenario in wireless edge-computing, and ii) portfolio optimization. Our results show that the proposed TS-KS algorithm outperforms not only the static TS algorithm but also it performs better than other bandit algorithms designed for non-stationary environments. Moreover, the performance of TS-KS is at par with the state-of-the-art forecasting algorithms such as Facebook-PROPHET and ARIMA.
△ Less
Submitted 21 October, 2021; v1 submitted 30 May, 2021;
originally announced May 2021.
-
Machine Learning in Generation, Detection, and Mitigation of Cyberattacks in Smart Grid: A Survey
Authors:
Nur Imtiazul Haque,
Md Hasan Shahriar,
Md Golam Dastgir,
Anjan Debnath,
Imtiaz Parvez,
Arif Sarwat,
Mohammad Ashiqur Rahman
Abstract:
Smart grid (SG) is a complex cyber-physical system that utilizes modern cyber and physical equipment to run at an optimal operating point. Cyberattacks are the principal threats confronting the usage and advancement of the state-of-the-art systems. The advancement of SG has added a wide range of technologies, equipment, and tools to make the system more reliable, efficient, and cost-effective. Des…
▽ More
Smart grid (SG) is a complex cyber-physical system that utilizes modern cyber and physical equipment to run at an optimal operating point. Cyberattacks are the principal threats confronting the usage and advancement of the state-of-the-art systems. The advancement of SG has added a wide range of technologies, equipment, and tools to make the system more reliable, efficient, and cost-effective. Despite attaining these goals, the threat space for the adversarial attacks has also been expanded because of the extensive implementation of the cyber networks. Due to the promising computational and reasoning capability, machine learning (ML) is being used to exploit and defend the cyberattacks in SG by the attackers and system operators, respectively. In this paper, we perform a comprehensive summary of cyberattacks generation, detection, and mitigation schemes by reviewing state-of-the-art research in the SG domain. Additionally, we have summarized the current research in a structured way using tabular format. We also present the shortcomings of the existing works and possible future research direction based on our investigation.
△ Less
Submitted 1 September, 2020;
originally announced October 2020.
-
Likelihood-based inference for modelling packet transit from thinned flow summaries
Authors:
Prosha A. Rahman,
Boris Beranger,
Matthew Roughan,
Scott A. Sisson
Abstract:
The substantial growth of network traffic speed and volume presents practical challenges to network data analysis. Packet thinning and flow aggregation protocols such as NetFlow reduce the size of datasets by providing structured data summaries, but conversely this impedes statistical inference. Methods which aim to model patterns of traffic propagation typically do not account for the packet thin…
▽ More
The substantial growth of network traffic speed and volume presents practical challenges to network data analysis. Packet thinning and flow aggregation protocols such as NetFlow reduce the size of datasets by providing structured data summaries, but conversely this impedes statistical inference. Methods which aim to model patterns of traffic propagation typically do not account for the packet thinning and summarisation process into the analysis, and are often simplistic, e.g.~method-of-moments. As a result, they can be of limited practical use.
We introduce a likelihood-based analysis which fully incorporates packet thinning and NetFlow summarisation into the analysis. As a result, inferences can be made for models on the level of individual packets while only observing thinned flow summary information. We establish consistency of the resulting maximum likelihood estimator, derive bounds on the volume of traffic which should be observed to achieve required levels of estimator accuracy, and identify an ideal family of models. The robust performance of the estimator is examined through simulated analyses and an application on a publicly available trace dataset containing over 36m packets over a 1 minute period.
△ Less
Submitted 31 August, 2020;
originally announced August 2020.
-
Smart Weather Forecasting Using Machine Learning:A Case Study in Tennessee
Authors:
A H M Jakaria,
Md Mosharaf Hossain,
Mohammad Ashiqur Rahman
Abstract:
Traditionally, weather predictions are performed with the help of large complex models of physics, which utilize different atmospheric conditions over a long period of time. These conditions are often unstable because of perturbations of the weather system, causing the models to provide inaccurate forecasts. The models are generally run on hundreds of nodes in a large High Performance Computing (H…
▽ More
Traditionally, weather predictions are performed with the help of large complex models of physics, which utilize different atmospheric conditions over a long period of time. These conditions are often unstable because of perturbations of the weather system, causing the models to provide inaccurate forecasts. The models are generally run on hundreds of nodes in a large High Performance Computing (HPC) environment which consumes a large amount of energy. In this paper, we present a weather prediction technique that utilizes historical data from multiple weather stations to train simple machine learning models, which can provide usable forecasts about certain weather conditions for the near future within a very short period of time. The models can be run on much less resource intensive environments. The evaluation results show that the accuracy of the models is good enough to be used alongside the current state-of-the-art techniques. Furthermore, we show that it is beneficial to leverage the weather station data from multiple neighboring areas over the data of only the area for which weather forecasting is being performed.
△ Less
Submitted 24 August, 2020;
originally announced August 2020.
-
Performance Evaluation of t-SNE and MDS Dimensionality Reduction Techniques with KNN, ENN and SVM Classifiers
Authors:
Shadman Sakib,
Md. Abu Bakr Siddique,
Md. Abdur Rahman
Abstract:
The central goal of this paper is to establish two commonly available dimensionality reduction (DR) methods i.e. t-distributed Stochastic Neighbor Embedding (t-SNE) and Multidimensional Scaling (MDS) in Matlab and to observe their application in several datasets. These DR techniques are applied to nine different datasets namely CNAE9, Segmentation, Seeds, Pima Indians diabetes, Parkinsons, Movemen…
▽ More
The central goal of this paper is to establish two commonly available dimensionality reduction (DR) methods i.e. t-distributed Stochastic Neighbor Embedding (t-SNE) and Multidimensional Scaling (MDS) in Matlab and to observe their application in several datasets. These DR techniques are applied to nine different datasets namely CNAE9, Segmentation, Seeds, Pima Indians diabetes, Parkinsons, Movement Libras, Mammographic Masses, Knowledge, and Ionosphere acquired from UCI machine learning repository. By applying t-SNE and MDS algorithms, each dataset is transformed to the half of its original dimension by eliminating unnecessary features from the datasets. Subsequently, these datasets with reduced dimensions are fed into three supervised classification algorithms for classification. These classification algorithms are K Nearest Neighbors (KNN), Extended Nearest Neighbors (ENN), and Support Vector Machine (SVM). Again, all these algorithms are implemented in Matlab. The training and test data ratios are maintained as ninety percent: ten percent for each dataset. Upon accuracy observation, the efficiency for every dimensionality technique with availed classification algorithms is analyzed and the performance of each classifier is evaluated.
△ Less
Submitted 20 June, 2020;
originally announced July 2020.
-
A versatile trend test for the evaluation of tumor incidences in long-term carcinogenicity bioassays
Authors:
Ludwig A. Hothorn,
Atiar M. Rahman,
Frank Schaarschmidt
Abstract:
For the evaluation of carcinogenicity bioassays a new trend test is proposed which is based on a maximum of arithmetic, ordinal, and logarithmic regression scores as well as the Williams-type contrasts for either crude proportions or more appropriate poly3-estimates for the tumor-by-time relationships. This test provides an almost appropriate power for most shapes of dose-response relationships (i…
▽ More
For the evaluation of carcinogenicity bioassays a new trend test is proposed which is based on a maximum of arithmetic, ordinal, and logarithmic regression scores as well as the Williams-type contrasts for either crude proportions or more appropriate poly3-estimates for the tumor-by-time relationships. This test provides an almost appropriate power for most shapes of dose-response relationships (including for possible downturn effect at high(er) dose(s)), common signs of significance (p-value, confidence limits) and the information on the probable shape. Related software is easily available within the CRAN-packages tukeytrend, MCPAN, multcomp.
△ Less
Submitted 24 July, 2020;
originally announced July 2020.
-
Lifelong Learning using Eigentasks: Task Separation, Skill Acquisition, and Selective Transfer
Authors:
Aswin Raghavan,
Jesse Hostetler,
Indranil Sur,
Abrar Rahman,
Ajay Divakaran
Abstract:
We introduce the eigentask framework for lifelong learning. An eigentask is a pairing of a skill that solves a set of related tasks, paired with a generative model that can sample from the skill's input space. The framework extends generative replay approaches, which have mainly been used to avoid catastrophic forgetting, to also address other lifelong learning goals such as forward knowledge tran…
▽ More
We introduce the eigentask framework for lifelong learning. An eigentask is a pairing of a skill that solves a set of related tasks, paired with a generative model that can sample from the skill's input space. The framework extends generative replay approaches, which have mainly been used to avoid catastrophic forgetting, to also address other lifelong learning goals such as forward knowledge transfer. We propose a wake-sleep cycle of alternating task learning and knowledge consolidation for learning in our framework, and instantiate it for lifelong supervised learning and lifelong RL. We achieve improved performance over the state-of-the-art in supervised continual learning, and show evidence of forward knowledge transfer in a lifelong RL application in the game Starcraft2.
△ Less
Submitted 14 July, 2020;
originally announced July 2020.
-
Towards Open Ad Hoc Teamwork Using Graph-based Policy Learning
Authors:
Arrasy Rahman,
Niklas Höpner,
Filippos Christianos,
Stefano V. Albrecht
Abstract:
Ad hoc teamwork is the challenging problem of designing an autonomous agent which can adapt quickly to collaborate with teammates without prior coordination mechanisms, including joint training. Prior work in this area has focused on closed teams in which the number of agents is fixed. In this work, we consider open teams by allowing agents with different fixed policies to enter and leave the envi…
▽ More
Ad hoc teamwork is the challenging problem of designing an autonomous agent which can adapt quickly to collaborate with teammates without prior coordination mechanisms, including joint training. Prior work in this area has focused on closed teams in which the number of agents is fixed. In this work, we consider open teams by allowing agents with different fixed policies to enter and leave the environment without prior notification. Our solution builds on graph neural networks to learn agent models and joint-action value models under varying team compositions. We contribute a novel action-value computation that integrates the agent model and joint-action value model to produce action-value estimates. We empirically demonstrate that our approach successfully models the effects other agents have on the learner, leading to policies that robustly adapt to dynamic team compositions and significantly outperform several alternative methods.
△ Less
Submitted 9 June, 2021; v1 submitted 18 June, 2020;
originally announced June 2020.
-
Seemingly Unrelated Regression with Measurement Error: Estimation via Markov chain Monte Carlo and Mean Field Variational Bayes Approximation
Authors:
Georges Bresson,
Anoop Chaturvedi,
Mohammad Arshad Rahman,
Shalabh
Abstract:
Linear regression with measurement error in the covariates is a heavily studied topic, however, the statistics/econometrics literature is almost silent to estimating a multi-equation model with measurement error. This paper considers a seemingly unrelated regression model with measurement error in the covariates and introduces two novel estimation methods: a pure Bayesian algorithm (based on Marko…
▽ More
Linear regression with measurement error in the covariates is a heavily studied topic, however, the statistics/econometrics literature is almost silent to estimating a multi-equation model with measurement error. This paper considers a seemingly unrelated regression model with measurement error in the covariates and introduces two novel estimation methods: a pure Bayesian algorithm (based on Markov chain Monte Carlo techniques) and its mean field variational Bayes (MFVB) approximation. The MFVB method has the added advantage of being computationally fast and can handle big data. An issue pertinent to measurement error models is parameter identification, and this is resolved by employing a prior distribution on the measurement error variance. The methods are shown to perform well in multiple simulation studies, where we analyze the impact on posterior estimates arising due to different values of reliability ratio or variance of the true unobserved quantity used in the data generating process. The paper further implements the proposed algorithms in an application drawn from the health literature and shows that modeling measurement error in the data can improve model fitting.
△ Less
Submitted 12 June, 2020;
originally announced June 2020.
-
A Novel Incremental Clustering Technique with Concept Drift Detection
Authors:
Mitchell D. Woodbright,
Md Anisur Rahman,
Md Zahidul Islam
Abstract:
Data are being collected from various aspects of life. These data can often arrive in chunks/batches. Traditional static clustering algorithms are not suitable for dynamic datasets, i.e., when data arrive in streams of chunks/batches. If we apply a conventional clustering technique over the combined dataset, then every time a new batch of data comes, the process can be slow and wasteful. Moreover,…
▽ More
Data are being collected from various aspects of life. These data can often arrive in chunks/batches. Traditional static clustering algorithms are not suitable for dynamic datasets, i.e., when data arrive in streams of chunks/batches. If we apply a conventional clustering technique over the combined dataset, then every time a new batch of data comes, the process can be slow and wasteful. Moreover, it can be challenging to store the combined dataset in memory due to its ever-increasing size. As a result, various incremental clustering techniques have been proposed. These techniques need to efficiently update the current clustering result whenever a new batch arrives, to adapt the current clustering result/solution with the latest data. These techniques also need the ability to detect concept drifts when the clustering pattern of a new batch is significantly different from older batches. Sometimes, clustering patterns may drift temporarily in a single batch while the next batches do not exhibit the drift. Therefore, incremental clustering techniques need the ability to detect a temporary drift and sustained drift. In this paper, we propose an efficient incremental clustering algorithm called UIClust. It is designed to cluster streams of data chunks, even when there are temporary or sustained concept drifts. We evaluate the performance of UIClust by comparing it with a recently published, high-quality incremental clustering algorithm. We use real and synthetic datasets. We compare the results by using well-known clustering evaluation criteria: entropy, sum of squared errors (SSE), and execution time. Our results show that UIClust outperforms the existing technique in all our experiments.
△ Less
Submitted 30 March, 2020;
originally announced March 2020.
-
Human Activity Recognition from Wearable Sensor Data Using Self-Attention
Authors:
Saif Mahmud,
M Tanjid Hasan Tonmoy,
Kishor Kumar Bhaumik,
A K M Mahbubur Rahman,
M Ashraful Amin,
Mohammad Shoyaib,
Muhammad Asif Hossain Khan,
Amin Ahsan Ali
Abstract:
Human Activity Recognition from body-worn sensor data poses an inherent challenge in capturing spatial and temporal dependencies of time-series signals. In this regard, the existing recurrent or convolutional or their hybrid models for activity recognition struggle to capture spatio-temporal context from the feature space of sensor reading sequence. To address this complex problem, we propose a se…
▽ More
Human Activity Recognition from body-worn sensor data poses an inherent challenge in capturing spatial and temporal dependencies of time-series signals. In this regard, the existing recurrent or convolutional or their hybrid models for activity recognition struggle to capture spatio-temporal context from the feature space of sensor reading sequence. To address this complex problem, we propose a self-attention based neural network model that foregoes recurrent architectures and utilizes different types of attention mechanisms to generate higher dimensional feature representation used for classification. We performed extensive experiments on four popular publicly available HAR datasets: PAMAP2, Opportunity, Skoda and USC-HAD. Our model achieve significant performance improvement over recent state-of-the-art models in both benchmark test subjects and Leave-one-subject-out evaluation. We also observe that the sensor attention maps produced by our model is able capture the importance of the modality and placement of the sensors in predicting the different activity classes.
△ Less
Submitted 17 March, 2020;
originally announced March 2020.
-
Enforcing Mean Reversion in State Space Models for Prawn Pond Water Quality Forecasting
Authors:
Joel Janek Dabrowski,
Ashfaqur Rahman,
Daniel Edward Pagendam,
Andrew George
Abstract:
The contribution of this study is a novel approach to introduce mean reversion in multi-step-ahead forecasts of state-space models. This approach is demonstrated in a prawn pond water quality forecasting application. The mean reversion constrains forecasts by gradually drawing them to an average of previously observed dynamics. This corrects deviations in forecasts caused by irregularities such as…
▽ More
The contribution of this study is a novel approach to introduce mean reversion in multi-step-ahead forecasts of state-space models. This approach is demonstrated in a prawn pond water quality forecasting application. The mean reversion constrains forecasts by gradually drawing them to an average of previously observed dynamics. This corrects deviations in forecasts caused by irregularities such as chaotic, non-linear, and stochastic trends. The key features of the approach include (1) it enforces mean reversion, (2) it provides a means to model both short and long-term dynamics, (3) it is able to apply mean reversion to select structural state-space components, and (4) it is simple to implement. Our mean reversion approach is demonstrated on various state-space models and compared with several time-series models on a prawn pond water quality dataset. Results show that mean reversion reduces long-term forecast errors by over 60% to produce the most accurate models in the comparison.
△ Less
Submitted 25 February, 2020;
originally announced February 2020.
-
Deep Learning and Statistical Models for Time-Critical Pedestrian Behaviour Prediction
Authors:
Joel Janek Dabrowski,
Johan Pieter de Villiers,
Ashfaqur Rahman,
Conrad Beyers
Abstract:
The time it takes for a classifier to make an accurate prediction can be crucial in many behaviour recognition problems. For example, an autonomous vehicle should detect hazardous pedestrian behaviour early enough for it to take appropriate measures. In this context, we compare the switching linear dynamical system (SLDS) and a three-layered bi-directional long short-term memory (LSTM) neural netw…
▽ More
The time it takes for a classifier to make an accurate prediction can be crucial in many behaviour recognition problems. For example, an autonomous vehicle should detect hazardous pedestrian behaviour early enough for it to take appropriate measures. In this context, we compare the switching linear dynamical system (SLDS) and a three-layered bi-directional long short-term memory (LSTM) neural network, which are applied to infer pedestrian behaviour from motion tracks. We show that, though the neural network model achieves an accuracy of 80%, it requires long sequences to achieve this (100 samples or more). The SLDS, has a lower accuracy of 74%, but it achieves this result with short sequences (10 samples). To our knowledge, such a comparison on sequence length has not been considered in the literature before. The results provide a key intuition of the suitability of the models in time-critical problems.
△ Less
Submitted 25 February, 2020;
originally announced February 2020.
-
Sequence-to-Sequence Imputation of Missing Sensor Data
Authors:
Joel Janek Dabrowski,
Ashfaqur Rahman
Abstract:
Although the sequence-to-sequence (encoder-decoder) model is considered the state-of-the-art in deep learning sequence models, there is little research into using this model for recovering missing sensor data. The key challenge is that the missing sensor data problem typically comprises three sequences (a sequence of observed samples, followed by a sequence of missing samples, followed by another…
▽ More
Although the sequence-to-sequence (encoder-decoder) model is considered the state-of-the-art in deep learning sequence models, there is little research into using this model for recovering missing sensor data. The key challenge is that the missing sensor data problem typically comprises three sequences (a sequence of observed samples, followed by a sequence of missing samples, followed by another sequence of observed samples) whereas, the sequence-to-sequence model only considers two sequences (an input sequence and an output sequence). We address this problem by formulating a sequence-to-sequence in a novel way. A forward RNN encodes the data observed before the missing sequence and a backward RNN encodes the data observed after the missing sequence. A decoder decodes the two encoders in a novel way to predict the missing data. We demonstrate that this model produces the lowest errors in 12% more cases than the current state-of-the-art.
△ Less
Submitted 25 February, 2020;
originally announced February 2020.
-
ForecastNet: A Time-Variant Deep Feed-Forward Neural Network Architecture for Multi-Step-Ahead Time-Series Forecasting
Authors:
Joel Janek Dabrowski,
YiFan Zhang,
Ashfaqur Rahman
Abstract:
Recurrent and convolutional neural networks are the most common architectures used for time series forecasting in deep learning literature. These networks use parameter sharing by repeating a set of fixed architectures with fixed parameters over time or space. The result is that the overall architecture is time-invariant (shift-invariant in the spatial domain) or stationary. We argue that time-inv…
▽ More
Recurrent and convolutional neural networks are the most common architectures used for time series forecasting in deep learning literature. These networks use parameter sharing by repeating a set of fixed architectures with fixed parameters over time or space. The result is that the overall architecture is time-invariant (shift-invariant in the spatial domain) or stationary. We argue that time-invariance can reduce the capacity to perform multi-step-ahead forecasting, where modelling the dynamics at a range of scales and resolutions is required. We propose ForecastNet which uses a deep feed-forward architecture to provide a time-variant model. An additional novelty of ForecastNet is interleaved outputs, which we show assist in mitigating vanishing gradients. ForecastNet is demonstrated to outperform statistical and deep learning benchmark models on several datasets.
△ Less
Submitted 27 June, 2020; v1 submitted 10 February, 2020;
originally announced February 2020.
-
Bayesian Panel Quantile Regression for Binary Outcomes with Correlated Random Effects: An Application on Crime Recidivism in Canada
Authors:
Georges Bresson,
Guy Lacroix,
Mohammad Arshad Rahman
Abstract:
This article develops a Bayesian approach for estimating panel quantile regression with binary outcomes in the presence of correlated random effects. We construct a working likelihood using an asymmetric Laplace (AL) error distribution and combine it with suitable prior distributions to obtain the complete joint posterior distribution. For posterior inference, we propose two Markov chain Monte Car…
▽ More
This article develops a Bayesian approach for estimating panel quantile regression with binary outcomes in the presence of correlated random effects. We construct a working likelihood using an asymmetric Laplace (AL) error distribution and combine it with suitable prior distributions to obtain the complete joint posterior distribution. For posterior inference, we propose two Markov chain Monte Carlo (MCMC) algorithms but prefer the algorithm that exploits the blocking procedure to produce lower autocorrelation in the MCMC draws. We also explain how to use the MCMC draws to calculate the marginal effects, relative risk and odds ratio. The performance of our preferred algorithm is demonstrated in multiple simulation studies and shown to perform extremely well. Furthermore, we implement the proposed framework to study crime recidivism in Quebec, a Canadian Province, using a novel data from the administrative correctional files. Our results suggest that the recently implemented "tough-on-crime" policy of the Canadian government has been largely successful in reducing the probability of repeat offenses in the post-policy period. Besides, our results support existing findings on crime recidivism and offer new insights at various quantiles.
△ Less
Submitted 25 January, 2020;
originally announced January 2020.
-
A New Graphical Device and Related Tests for the Shape of Non-parametric Regression Function
Authors:
Subhra Sankar Dhar,
Prashant Jha,
Mohammad Arshad Rahman,
Joydeep Dutta
Abstract:
We consider a non-parametric regression model $y = m(x) + ε$ and propose a novel graphical device to check whether the $r$-th ($r \geqslant 1$) derivative of the regression function $m(x)$ is positive or otherwise. Since the shape of the regression function can be completely characterized by its derivatives, the graphical device can correctly identify the shape of the regression function. The prop…
▽ More
We consider a non-parametric regression model $y = m(x) + ε$ and propose a novel graphical device to check whether the $r$-th ($r \geqslant 1$) derivative of the regression function $m(x)$ is positive or otherwise. Since the shape of the regression function can be completely characterized by its derivatives, the graphical device can correctly identify the shape of the regression function. The proposed device includes the check for monotonicity and convexity of the function as special cases. We also present an example to elucidate the practical utility of the graphical device. In addition, we employ the graphical device to formulate a class of test statistics and derive its asymptotic distribution. The tests are exhibited in various simulated and real data examples.
△ Less
Submitted 23 January, 2021; v1 submitted 6 October, 2019;
originally announced October 2019.
-
Estimation and Applications of Quantile Regression for Binary Longitudinal Data
Authors:
Mohammad Arshad Rahman,
Angela Vossmeyer
Abstract:
This paper develops a framework for quantile regression in binary longitudinal data settings. A novel Markov chain Monte Carlo (MCMC) method is designed to fit the model and its computational efficiency is demonstrated in a simulation study. The proposed approach is flexible in that it can account for common and individual-specific parameters, as well as multivariate heterogeneity associated with…
▽ More
This paper develops a framework for quantile regression in binary longitudinal data settings. A novel Markov chain Monte Carlo (MCMC) method is designed to fit the model and its computational efficiency is demonstrated in a simulation study. The proposed approach is flexible in that it can account for common and individual-specific parameters, as well as multivariate heterogeneity associated with several covariates. The methodology is applied to study female labor force participation and home ownership in the United States. The results offer new insights at the various quantiles, which are of interest to policymakers and researchers alike.
△ Less
Submitted 12 September, 2019;
originally announced September 2019.
-
Improving Malaria Parasite Detection from Red Blood Cell using Deep Convolutional Neural Networks
Authors:
Aimon Rahman,
Hasib Zunair,
M Sohel Rahman,
Jesia Quader Yuki,
Sabyasachi Biswas,
Md Ashraful Alam,
Nabila Binte Alam,
M. R. C. Mahdy
Abstract:
Malaria is a female anopheles mosquito-bite inflicted life-threatening disease which is considered endemic in many parts of the world. This article focuses on improving malaria detection from patches segmented from microscopic images of red blood cell smears by introducing a deep convolutional neural network. Compared to the traditional methods that use tedious hand engineering feature extraction,…
▽ More
Malaria is a female anopheles mosquito-bite inflicted life-threatening disease which is considered endemic in many parts of the world. This article focuses on improving malaria detection from patches segmented from microscopic images of red blood cell smears by introducing a deep convolutional neural network. Compared to the traditional methods that use tedious hand engineering feature extraction, the proposed method uses deep learning in an end-to-end arrangement that performs both feature extraction and classification directly from the raw segmented patches of the red blood smears. The dataset used in this study was taken from National Institute of Health named NIH Malaria Dataset. The evaluation metric accuracy and loss along with 5-fold cross validation was used to compare and select the best performing architecture. To maximize the performance, existing standard pre-processing techniques from the literature has also been experimented. In addition, several other complex architectures have been implemented and tested to pick the best performing model. A holdout test has also been conducted to verify how well the proposed model generalizes on unseen data. Our best model achieves an accuracy of almost 97.77%.
△ Less
Submitted 23 July, 2019;
originally announced July 2019.
-
Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning
Authors:
Georgios Papoudakis,
Filippos Christianos,
Arrasy Rahman,
Stefano V. Albrecht
Abstract:
Recent developments in deep reinforcement learning are concerned with creating decision-making agents which can perform well in various complex domains. A particular approach which has received increasing attention is multi-agent reinforcement learning, in which multiple agents learn concurrently to coordinate their actions. In such multi-agent environments, additional learning problems arise due…
▽ More
Recent developments in deep reinforcement learning are concerned with creating decision-making agents which can perform well in various complex domains. A particular approach which has received increasing attention is multi-agent reinforcement learning, in which multiple agents learn concurrently to coordinate their actions. In such multi-agent environments, additional learning problems arise due to the continually changing decision-making policies of agents. This paper surveys recent works that address the non-stationarity problem in multi-agent deep reinforcement learning. The surveyed methods range from modifications in the training procedure, such as centralized training, to learning representations of the opponent's policy, meta-learning, communication, and decentralized learning. The survey concludes with a list of open problems and possible lines of future research.
△ Less
Submitted 11 June, 2019;
originally announced June 2019.
-
Native Language Identification using i-vector
Authors:
Ahmed Nazim Uddin,
Md Ashequr Rahman,
Md. Rafidul Islam,
Mohammad Ariful Haque
Abstract:
The task of determining a speaker's native language based only on his speeches in a second language is known as Native Language Identification or NLI. Due to its increasing applications in various domains of speech signal processing, this has emerged as an important research area in recent times. In this paper we have proposed an i-vector based approach to develop an automatic NLI system using MFC…
▽ More
The task of determining a speaker's native language based only on his speeches in a second language is known as Native Language Identification or NLI. Due to its increasing applications in various domains of speech signal processing, this has emerged as an important research area in recent times. In this paper we have proposed an i-vector based approach to develop an automatic NLI system using MFCC and GFCC features. For evaluation of our approach, we have tested our framework on the 2016 ComParE Native language sub-challenge dataset which has English language speakers from 11 different native language backgrounds. Our proposed method outperforms the baseline system with an improvement in accuracy by 21.95% for the MFCC feature based i-vector framework and 22.81% for the GFCC feature based i-vector framework.
△ Less
Submitted 9 November, 2018;
originally announced November 2018.
-
Supervised classification for object identification in urban areas using satellite imagery
Authors:
Hazrat Ali,
Adnan Ali Awan,
Sanaullah Khan,
Omer Shafique,
Atiq ur Rahman,
Shahid Khan
Abstract:
This paper presents a useful method to achieve classification in satellite imagery. The approach is based on pixel level study employing various features such as correlation, homogeneity, energy and contrast. In this study gray-scale images are used for training the classification model. For supervised classification, two classification techniques are employed namely the Support Vector Machine (SV…
▽ More
This paper presents a useful method to achieve classification in satellite imagery. The approach is based on pixel level study employing various features such as correlation, homogeneity, energy and contrast. In this study gray-scale images are used for training the classification model. For supervised classification, two classification techniques are employed namely the Support Vector Machine (SVM) and the Naive Bayes. With textural features used for gray-scale images, Naive Bayes performs better with an overall accuracy of 76% compared to 68% achieved by SVM. The computational time is evaluated while performing the experiment with two different window sizes i.e., 50x50 and 70x70. The required computational time on a single image is found to be 27 seconds for a window size of 70x70 and 45 seconds for a window size of 50x50.
△ Less
Submitted 2 August, 2018;
originally announced August 2018.
-
Modeling Data Containing Outliers using ARIMA Additive Outlier (ARIMA-AO)
Authors:
Ansari Saleh Ahmar,
Suryo Guritno,
Abdurakhman,
Abdul Rahman,
Awi,
Alimuddin,
Ilham Minggi,
M. Arif Tiro,
M. Kasim Aidid,
Suwardi Annas,
Dian Utami Sutiksno,
S. Ahmar Dewi,
H. Ahmar Kurniawan,
A. Abqary Ahmar,
Ahmad Zaki,
Dahlan Abdullah,
Robbi Rahim,
Heri Nurdiyanto,
Rahmat Hidayat,
Darmawan Napitupulu,
Janner Simarmata,
Nuning Kurniasih,
Leon Andretti Abdillah,
Andri Pranolo,
Haviluddin
, et al. (2 additional authors not shown)
Abstract:
The aim this study is discussed on the detection and correction of data containing the additive outlier (AO) on the model ARIMA (p, d, q). The process of detection and correction of data using an iterative procedure popularized by Box, Jenkins, and Reinsel (1994). By using this method we obtained an ARIMA models were fit to the data containing AO, this model is added to the original model of ARIMA…
▽ More
The aim this study is discussed on the detection and correction of data containing the additive outlier (AO) on the model ARIMA (p, d, q). The process of detection and correction of data using an iterative procedure popularized by Box, Jenkins, and Reinsel (1994). By using this method we obtained an ARIMA models were fit to the data containing AO, this model is added to the original model of ARIMA coefficients obtained from the iteration process using regression methods. This shows that there is an improvement of forecasting error rate data.
△ Less
Submitted 1 March, 2018;
originally announced March 2018.
-
Densely Connected Convolutional Networks and Signal Quality Analysis to Detect Atrial Fibrillation Using Short Single-Lead ECG Recordings
Authors:
Jonathan Rubin,
Saman Parvaneh,
Asif Rahman,
Bryan Conroy,
Saeed Babaeizadeh
Abstract:
The development of new technology such as wearables that record high-quality single channel ECG, provides an opportunity for ECG screening in a larger population, especially for atrial fibrillation screening. The main goal of this study is to develop an automatic classification algorithm for normal sinus rhythm (NSR), atrial fibrillation (AF), other rhythms (O), and noise from a single channel sho…
▽ More
The development of new technology such as wearables that record high-quality single channel ECG, provides an opportunity for ECG screening in a larger population, especially for atrial fibrillation screening. The main goal of this study is to develop an automatic classification algorithm for normal sinus rhythm (NSR), atrial fibrillation (AF), other rhythms (O), and noise from a single channel short ECG segment (9-60 seconds). For this purpose, signal quality index (SQI) along with dense convolutional neural networks was used. Two convolutional neural network (CNN) models (main model that accepts 15 seconds ECG and secondary model that processes 9 seconds shorter ECG) were trained using the training data set. If the recording is determined to be of low quality by SQI, it is immediately classified as noisy. Otherwise, it is transformed to a time-frequency representation and classified with the CNN as NSR, AF, O, or noise. At the final step, a feature-based post-processing algorithm classifies the rhythm as either NSR or O in case the CNN model's discrimination between the two is indeterminate. The best result achieved at the official phase of the PhysioNet/CinC challenge on the blind test set was 0.80 (F1 for NSR, AF, and O were 0.90, 0.80, and 0.70, respectively).
△ Less
Submitted 10 October, 2017;
originally announced October 2017.
-
An Ensemble Boosting Model for Predicting Transfer to the Pediatric Intensive Care Unit
Authors:
Jonathan Rubin,
Cristhian Potes,
Minnan Xu-Wilson,
Junzi Dong,
Asif Rahman,
Hiep Nguyen,
David Moromisato
Abstract:
Our work focuses on the problem of predicting the transfer of pediatric patients from the general ward of a hospital to the pediatric intensive care unit. Using data collected over 5.5 years from the electronic health records of two medical facilities, we develop classifiers based on adaptive boosting and gradient tree boosting. We further combine these learned classifiers into an ensemble model a…
▽ More
Our work focuses on the problem of predicting the transfer of pediatric patients from the general ward of a hospital to the pediatric intensive care unit. Using data collected over 5.5 years from the electronic health records of two medical facilities, we develop classifiers based on adaptive boosting and gradient tree boosting. We further combine these learned classifiers into an ensemble model and compare its performance to a modified pediatric early warning score (PEWS) baseline that relies on expert defined guidelines. To gauge model generalizability, we perform an inter-facility evaluation where we train our algorithm on data from one facility and perform evaluation on a hidden test dataset from a separate facility. We show that improvements are witnessed over the PEWS baseline in accuracy (0.77 vs. 0.69), sensitivity (0.80 vs. 0.68), specificity (0.74 vs. 0.70) and AUROC (0.85 vs. 0.73).
△ Less
Submitted 16 July, 2017;
originally announced July 2017.
-
Euclidean distance matrix completion and point configurations from the minimal spanning tree
Authors:
Adam Rahman,
Wayne Oldford
Abstract:
The paper introduces a special case of the Euclidean distance matrix completion problem (edmcp) of interest in statistical data analysis where only the minimal spanning tree distances are given and the matrix completion must preserve the minimal spanning tree. Two solutions are proposed, one an adaptation of a more general method based on a dissimilarity parameterized formulation, the other an ent…
▽ More
The paper introduces a special case of the Euclidean distance matrix completion problem (edmcp) of interest in statistical data analysis where only the minimal spanning tree distances are given and the matrix completion must preserve the minimal spanning tree. Two solutions are proposed, one an adaptation of a more general method based on a dissimilarity parameterized formulation, the other an entirely novel method which constructs the point configuration directly through a guided random search. These methods as well as three standard edcmp methods are described and compared experimentally on real and synthetic data. It is found that the constructive method given by the guided random search algorithm clearly outperforms all others considered here. Notably, standard methods including the adaptation force peculiar, and generally unwanted, geometric structure on the point configurations their completions produce.
△ Less
Submitted 20 October, 2016;
originally announced October 2016.
