Showing 1–25 of 25 results for author: Paschalidis, I C

Distributionally Robust Learning in Survival Analysis

Authors: Yeping Jin, Lauren Wise, Ioannis Ch. Paschalidis

Abstract: We introduce an innovative approach that incorporates a Distributionally Robust Learning (DRL) approach into Cox regression to enhance the robustness and accuracy of survival predictions. By formulating a DRL framework with a Wasserstein distance-based ambiguity set, we develop a variant Cox model that is less sensitive to assumptions about the underlying data distribution and more resilient to mo… ▽ More We introduce an innovative approach that incorporates a Distributionally Robust Learning (DRL) approach into Cox regression to enhance the robustness and accuracy of survival predictions. By formulating a DRL framework with a Wasserstein distance-based ambiguity set, we develop a variant Cox model that is less sensitive to assumptions about the underlying data distribution and more resilient to model misspecification and data perturbations. By leveraging Wasserstein duality, we reformulate the original min-max DRL problem into a tractable regularized empirical risk minimization problem, which can be computed by exponential conic programming. We provide guarantees on the finite sample behavior of our DRL-Cox model. Moreover, through extensive simulations and real world case studies, we demonstrate that our regression model achieves superior performance in terms of prediction accuracy and robustness compared with traditional methods. △ Less

Submitted 7 June, 2025; v1 submitted 2 June, 2025; originally announced June 2025.

MDP Geometry, Normalization and Reward Balancing Solvers

Authors: Arsenii Mustafin, Aleksei Pakharev, Alex Olshevsky, Ioannis Ch. Paschalidis

Abstract: We present a new geometric interpretation of Markov Decision Processes (MDPs) with a natural normalization procedure that allows us to adjust the value function at each state without altering the advantage of any action with respect to any policy. This advantage-preserving transformation of the MDP motivates a class of algorithms which we call Reward Balancing, which solve MDPs by iterating throug… ▽ More We present a new geometric interpretation of Markov Decision Processes (MDPs) with a natural normalization procedure that allows us to adjust the value function at each state without altering the advantage of any action with respect to any policy. This advantage-preserving transformation of the MDP motivates a class of algorithms which we call Reward Balancing, which solve MDPs by iterating through these transformations, until an approximately optimal policy can be trivially found. We provide a convergence analysis of several algorithms in this class, in particular showing that for MDPs for unknown transition probabilities we can improve upon state-of-the-art sample complexity results. △ Less

Submitted 5 March, 2025; v1 submitted 9 July, 2024; originally announced July 2024.

Comments: AISTATS 2025 camera-ready version

A Stackelberg Game Approach to Control the Overall Load Consumption of a Residential Neighborhood

Authors: Erhan Can Ozcan, Ioannis Ch. Paschalidis

Abstract: This paper formulates a Stackelberg game between a coordination agent and participating homes to control the overall load consumption of a residential neighborhood. Each home optimizes a comfort-cost trade off to determine a load schedule of its available appliances in response to a price vector set by the coordination agent. The goal of the coordination agent is to find a price vector that will k… ▽ More This paper formulates a Stackelberg game between a coordination agent and participating homes to control the overall load consumption of a residential neighborhood. Each home optimizes a comfort-cost trade off to determine a load schedule of its available appliances in response to a price vector set by the coordination agent. The goal of the coordination agent is to find a price vector that will keep the overall load consumption of the neighborhood around some target value. After transforming the bilevel optimization problem into a single level optimization problem by using Karush-Kuhn-Tucker (KKT) conditions, we model how each home reacts to any change in the price vector by using the implicit function theorem. By using this information, we develop a distributed optimization framework based on gradient descent to attain a better price vector. We verify the load shaping capacity and the computational performance of the proposed optimization framework in a simulated environment establishing significant benefits over solving the centralized problem using commercial solvers. △ Less

Submitted 19 June, 2023; originally announced June 2023.

A Distributed Optimization Framework to Regulate the Electricity Consumption of a Residential Neighborhood with Renewables

Authors: Erhan Can Ozcan, Emiliano Dall'Anese, Ioannis Ch. Paschalidis

Abstract: Demand response services at the distribution level are emerging as enabling strategies for improving grid reliability in the presence of intermittent renewable generation and grid congestion. For residential loads, space heating and cooling, water heating, electric vehicle charging, and routine appliances make up the bulk of the electricity consumption. Controlling these loads is essential to effe… ▽ More Demand response services at the distribution level are emerging as enabling strategies for improving grid reliability in the presence of intermittent renewable generation and grid congestion. For residential loads, space heating and cooling, water heating, electric vehicle charging, and routine appliances make up the bulk of the electricity consumption. Controlling these loads is essential to effectively partake into grid operations and provide services such as peak shaving and demand response. However, maintaining user comfort is important for ensuring user participation to such a program. This paper formulates a novel mixed integer linear programming problem to control the overall electricity consumption of a residential neighborhood by considering the users' comfort and preferences. To efficiently solve the problem for communities involving a large number of homes, a distributed optimization framework based on the Dantzig-Wolfe decomposition technique is developed. We demonstrate the load shaping capacity and the computational performance of the proposed optimization framework in a simulated environment. △ Less

Submitted 21 May, 2025; v1 submitted 16 June, 2023; originally announced June 2023.

Unconstrained Dynamic Regret via Sparse Coding

Authors: Zhiyu Zhang, Ashok Cutkosky, Ioannis Ch. Paschalidis

Abstract: Motivated by the challenge of nonstationarity in sequential decision making, we study Online Convex Optimization (OCO) under the coupling of two problem structures: the domain is unbounded, and the comparator sequence $u_1,\ldots,u_T$ is arbitrarily time-varying. As no algorithm can guarantee low regret simultaneously against all comparator sequences, handling this setting requires moving from min… ▽ More Motivated by the challenge of nonstationarity in sequential decision making, we study Online Convex Optimization (OCO) under the coupling of two problem structures: the domain is unbounded, and the comparator sequence $u_1,\ldots,u_T$ is arbitrarily time-varying. As no algorithm can guarantee low regret simultaneously against all comparator sequences, handling this setting requires moving from minimax optimality to comparator adaptivity. That is, sensible regret bounds should depend on certain complexity measures of the comparator relative to one's prior knowledge. This paper achieves a new type of these adaptive regret bounds via a sparse coding framework. The complexity of the comparator is measured by its energy and its sparsity on a user-specified dictionary, which offers considerable versatility. Equipped with a wavelet dictionary for example, our framework improves the state-of-the-art bound (Jacobsen & Cutkosky, 2022) by adapting to both ($i$) the magnitude of the comparator average $||\bar u||=||\sum_{t=1}^Tu_t/T||$, rather than the maximum $\max_t||u_t||$; and ($ii$) the comparator variability $\sum_{t=1}^T||u_t-\bar u||$, rather than the uncentered sum $\sum_{t=1}^T||u_t||$. Furthermore, our analysis is simpler due to decoupling function approximation from regret minimization. △ Less

Submitted 25 October, 2023; v1 submitted 30 January, 2023; originally announced January 2023.

Comments: NeurIPS 2023

Planning Strategies for Lane Reversals in Transportation Networks

Authors: Salomon Wollenstein-Betech, Ioannis Ch. Paschalidis, Christos G. Cassandras

Abstract: This paper studies strategies to optimize the lane configuration of a transportation network for a given set of Origin-Destination demands using a planning macroscopic network flow model. The lane reversal problem is, in general, NP-hard since the optimization is made over integer variables. To overcome this burden, we reformulate the problem using a piecewise affine approximation of the travel la… ▽ More This paper studies strategies to optimize the lane configuration of a transportation network for a given set of Origin-Destination demands using a planning macroscopic network flow model. The lane reversal problem is, in general, NP-hard since the optimization is made over integer variables. To overcome this burden, we reformulate the problem using a piecewise affine approximation of the travel latency function which allows us to exploit the total unimodularity property of Integer Linear Programming (ILP). Consequently, we transform the ILP problem to a linear program by relaxing the integer variables. In addition, our method is capable of solving the problem for a desired number of lane reversals which serves to perform cost-benefit analysis. We perform a case study using the transportation network of Eastern Massachusetts (EMA) and we test our method against the original lane configuration and a projected lower bound solution. Our empirical results quantify the travel time savings for different levels of demand intensity. We observe reduction in travel times up to 40% for certain links in the network. △ Less

Submitted 14 July, 2021; originally announced July 2021.

Comments: Proc. IEEE Int. Conf. on Intelligent Transportation Systems, Indianapolis, USA, 2021. (In Press)

Communication-efficient SGD: From Local SGD to One-Shot Averaging

Authors: Artin Spiridonoff, Alex Olshevsky, Ioannis Ch. Paschalidis

Abstract: We consider speeding up stochastic gradient descent (SGD) by parallelizing it across multiple workers. We assume the same data set is shared among $N$ workers, who can take SGD steps and coordinate with a central server. While it is possible to obtain a linear reduction in the variance by averaging all the stochastic gradients at every step, this requires a lot of communication between the workers… ▽ More We consider speeding up stochastic gradient descent (SGD) by parallelizing it across multiple workers. We assume the same data set is shared among $N$ workers, who can take SGD steps and coordinate with a central server. While it is possible to obtain a linear reduction in the variance by averaging all the stochastic gradients at every step, this requires a lot of communication between the workers and the server, which can dramatically reduce the gains from parallelism. The Local SGD method, proposed and analyzed in the earlier literature, suggests machines should make many local steps between such communications. While the initial analysis of Local SGD showed it needs $Ω( \sqrt{T} )$ communications for $T$ local gradient steps in order for the error to scale proportionately to $1/(NT)$, this has been successively improved in a string of papers, with the state of the art requiring $Ω\left( N \left( \mbox{ poly} (\log T) \right) \right)$ communications. In this paper, we suggest a Local SGD scheme that communicates less overall by communicating less frequently as the number of iterations grows. Our analysis shows that this can achieve an error that scales as $1/(NT)$ with a number of communications that is completely independent of $T$. In particular, we show that $Ω(N)$ communications are sufficient. Empirical evidence suggests this bound is close to tight as we further show that $\sqrt{N}$ or $N^{3/4}$ communications fail to achieve linear speed-up in simulations. Moreover, we show that under mild assumptions, the main of which is twice differentiability on any neighborhood of the optimal solution, one-shot averaging which only uses a single round of communication can also achieve the optimal convergence rate asymptotically. △ Less

Submitted 27 October, 2021; v1 submitted 8 June, 2021; originally announced June 2021.

Comments: arXiv admin note: substantial text overlap with arXiv:2006.02582

Local SGD With a Communication Overhead Depending Only on the Number of Workers

Authors: Artin Spiridonoff, Alex Olshevsky, Ioannis Ch. Paschalidis

Abstract: We consider speeding up stochastic gradient descent (SGD) by parallelizing it across multiple workers. We assume the same data set is shared among $n$ workers, who can take SGD steps and coordinate with a central server. Unfortunately, this could require a lot of communication between the workers and the server, which can dramatically reduce the gains from parallelism. The Local SGD method, propos… ▽ More We consider speeding up stochastic gradient descent (SGD) by parallelizing it across multiple workers. We assume the same data set is shared among $n$ workers, who can take SGD steps and coordinate with a central server. Unfortunately, this could require a lot of communication between the workers and the server, which can dramatically reduce the gains from parallelism. The Local SGD method, proposed and analyzed in the earlier literature, suggests machines should make many local steps between such communications. While the initial analysis of Local SGD showed it needs $Ω( \sqrt{T} )$ communications for $T$ local gradient steps in order for the error to scale proportionately to $1/(nT)$, this has been successively improved in a string of papers, with the state-of-the-art requiring $Ω\left( n \left( \mbox{ polynomial in log } (T) \right) \right)$ communications. In this paper, we give a new analysis of Local SGD. A consequence of our analysis is that Local SGD can achieve an error that scales as $1/(nT)$ with only a fixed number of communications independent of $T$: specifically, only $Ω(n)$ communications are required. △ Less

Submitted 3 June, 2020; originally announced June 2020.

Joint Pricing and Rebalancing of Autonomous Mobility-on-Demand Systems

Authors: Salomón Wollenstein-Betech, Ioannis Ch. Paschalidis, Christos G. Cassandras

Abstract: This paper studies optimal pricing and rebalancing policies for Autonomous Mobility-on-Demand (AMoD) systems. We take a macroscopic planning perspective to tackle a profit maximization problem while ensuring that the system is load-balanced. We begin by describing the system using a dynamic fluid model to show the existence and stability of an equilibrium (i.e., load balance) through pricing polic… ▽ More This paper studies optimal pricing and rebalancing policies for Autonomous Mobility-on-Demand (AMoD) systems. We take a macroscopic planning perspective to tackle a profit maximization problem while ensuring that the system is load-balanced. We begin by describing the system using a dynamic fluid model to show the existence and stability of an equilibrium (i.e., load balance) through pricing policies. We then develop an optimization framework that allows us to find optimal policies in terms of pricing and rebalancing. We first maximize profit by only using pricing policies, then incorporate rebalancing, and finally we consider whether the solution is found sequentially or jointly. We apply each approach on a data-driven case study using real taxi data from New York City. Depending on which benchmarking solution we use, the joint problem (i.e., pricing and rebalancing) increases profits by 7% to 40% △ Less

Submitted 30 March, 2020; originally announced March 2020.

Comments: Submitted to the 59th IEEE Conference on Decision and Control (CDC 2020)

Congestion-aware Routing and Rebalancing of Autonomous Mobility-on-Demand Systems in Mixed Traffic

Authors: Salomón Wollenstein-Betech, Arian Houshmand, Mauro Salazar, Marco Pavone, Christos G. Cassandras, Ioannis Ch. Paschalidis

Abstract: This paper studies congestion-aware route-planning policies for Autonomous Mobility-on-Demand (AMoD) systems, whereby a fleet of autonomous vehicles provides on-demand mobility under mixed traffic conditions. Specifically, we first devise a network flow model to optimize the AMoD routing and rebalancing strategies in a congestion-aware fashion by accounting for the endogenous impact of AMoD flows… ▽ More This paper studies congestion-aware route-planning policies for Autonomous Mobility-on-Demand (AMoD) systems, whereby a fleet of autonomous vehicles provides on-demand mobility under mixed traffic conditions. Specifically, we first devise a network flow model to optimize the AMoD routing and rebalancing strategies in a congestion-aware fashion by accounting for the endogenous impact of AMoD flows on travel time. Second, we capture reactive exogenous traffic consisting of private vehicles selfishly adapting to the AMoD flows in a user-centric fashion by leveraging an iterative approach. Finally, we showcase the effectiveness of our framework with two case-studies considering the transportation sub-networks in Eastern Massachusetts and New York City. Our results suggest that for high levels of demand, pure AMoD travel can be detrimental due to the additional traffic stemming from its rebalancing flows, while the combination of AMoD with walking or micromobility options can significantly improve the overall system performance. △ Less

Submitted 9 March, 2020; originally announced March 2020.

Comments: Submitted to the 23rd IEEE International Conference on Intelligent Transportation Systems (ITSC'20)

Joint Estimation of OD Demands and Cost Functions in Transportation Networks from Data

Authors: Salomón Wollenstein-Betech, Chuangchuang Sun, Jing Zhang, Ioannis Ch. Paschalidis

Abstract: Existing work has tackled the problem of estimating Origin-Destination (OD) demands and recovering travel latency functions in transportation networks under the Wardropian assumption. The ultimate objective is to derive an accurate predictive model of the network to enable optimization and control. However, these two problems are typically treated separately and estimation is based on parametric m… ▽ More Existing work has tackled the problem of estimating Origin-Destination (OD) demands and recovering travel latency functions in transportation networks under the Wardropian assumption. The ultimate objective is to derive an accurate predictive model of the network to enable optimization and control. However, these two problems are typically treated separately and estimation is based on parametric models. In this paper, we propose a method to jointly recover nonparametric travel latency cost functions and estimate OD demands using traffic flow data. We formulate the problem as a bilevel optimization problem and develop an iterative first-order optimization algorithm to solve it. A numerical example using the Braess Network is presented to demonstrate the effectiveness of our method. △ Less

Submitted 2 September, 2019; originally announced September 2019.

Comments: To appear at the Proceedings of the 58th IEEE Conference on Decision and Control

Journal ref: 2019 IEEE 58th Conference on Decision and Control (CDC)

Asymptotic Network Independence in Distributed Stochastic Optimization for Machine Learning

Authors: Shi Pu, Alex Olshevsky, Ioannis Ch. Paschalidis

Abstract: We provide a discussion of several recent results which, in certain scenarios, are able to overcome a barrier in distributed stochastic optimization for machine learning. Our focus is the so-called asymptotic network independence property, which is achieved whenever a distributed method executed over a network of n nodes asymptotically converges to the optimal solution at a comparable rate to a ce… ▽ More We provide a discussion of several recent results which, in certain scenarios, are able to overcome a barrier in distributed stochastic optimization for machine learning. Our focus is the so-called asymptotic network independence property, which is achieved whenever a distributed method executed over a network of n nodes asymptotically converges to the optimal solution at a comparable rate to a centralized method with the same computational power as the entire network. We explain this property through an example involving the training of ML models and sketch a short mathematical analysis for comparing the performance of distributed stochastic gradient descent (DSGD) with centralized stochastic gradient decent (SGD). △ Less

Submitted 18 February, 2020; v1 submitted 28 June, 2019; originally announced June 2019.

A Sharp Estimate on the Transient Time of Distributed Stochastic Gradient Descent

Authors: Shi Pu, Alex Olshevsky, Ioannis Ch. Paschalidis

Abstract: This paper is concerned with minimizing the average of $n$ cost functions over a network in which agents may communicate and exchange information with each other. We consider the setting where only noisy gradient information is available. To solve the problem, we study the distributed stochastic gradient descent (DSGD) method and perform a non-asymptotic convergence analysis. For strongly convex a… ▽ More This paper is concerned with minimizing the average of $n$ cost functions over a network in which agents may communicate and exchange information with each other. We consider the setting where only noisy gradient information is available. To solve the problem, we study the distributed stochastic gradient descent (DSGD) method and perform a non-asymptotic convergence analysis. For strongly convex and smooth objective functions, DSGD asymptotically achieves the optimal network independent convergence rate compared to centralized stochastic gradient descent (SGD). Our main contribution is to characterize the transient time needed for DSGD to approach the asymptotic convergence rate, which we show behaves as $K_T=\mathcal{O}\left(\frac{n}{(1-ρ_w)^2}\right)$, where $1-ρ_w$ denotes the spectral gap of the mixing matrix. Moreover, we construct a "hard" optimization problem for which we show the transient time needed for DSGD to approach the asymptotic convergence rate is lower bounded by $Ω\left(\frac{n}{(1-ρ_w)^2} \right)$, implying the sharpness of the obtained result. Numerical experiments demonstrate the tightness of the theoretical results. △ Less

Submitted 29 January, 2021; v1 submitted 6 June, 2019; originally announced June 2019.

Convergence of Parameter Estimates for Regularized Mixed Linear Regression Models

Authors: Taiyao Wang, Ioannis Ch. Paschalidis

Abstract: We consider {\em Mixed Linear Regression (MLR)}, where training data have been generated from a mixture of distinct linear models (or clusters) and we seek to identify the corresponding coefficient vectors. We introduce a {\em Mixed Integer Programming (MIP)} formulation for MLR subject to regularization constraints on the coefficient vectors. We establish that as the number of training samples gr… ▽ More We consider {\em Mixed Linear Regression (MLR)}, where training data have been generated from a mixture of distinct linear models (or clusters) and we seek to identify the corresponding coefficient vectors. We introduce a {\em Mixed Integer Programming (MIP)} formulation for MLR subject to regularization constraints on the coefficient vectors. We establish that as the number of training samples grows large, the MIP solution converges to the true coefficient vectors in the absence of noise. Subject to slightly stronger assumptions, we also establish that the MIP identifies the clusters from which the training samples were generated. In the special case where training data come from a single cluster, we establish that the corresponding MIP yields a solution that converges to the true coefficient vector even when training data are perturbed by (martingale difference) noise. We provide a counterexample indicating that in the presence of noise, the MIP may fail to produce the true coefficient vectors for more than one clusters. We also provide numerical results testing the MIP solutions in synthetic examples with noise. △ Less

Submitted 9 September, 2019; v1 submitted 21 March, 2019; originally announced March 2019.

Learning from Past Bids to Participate Strategically in Day-Ahead Electricity Markets

Authors: Ruidi Chen, Ioannis Ch. Paschalidis, Michael C. Caramanis, Panagiotis Andrianesis

Abstract: We consider the process of bidding by electricity suppliers in a day-ahead market context where each supplier bids a linear non-decreasing function of her generating capacity with the goal of maximizing her individual profit given other competing suppliers' bids. Based on the submitted bids, the market operator schedules suppliers to meet demand during each hour and determines hourly market cleari… ▽ More We consider the process of bidding by electricity suppliers in a day-ahead market context where each supplier bids a linear non-decreasing function of her generating capacity with the goal of maximizing her individual profit given other competing suppliers' bids. Based on the submitted bids, the market operator schedules suppliers to meet demand during each hour and determines hourly market clearing prices. Eventually, this game-theoretic process reaches a Nash equilibrium when no supplier is motivated to modify her bid. However, solving the individual profit maximization problem requires information of rivals' bids, which are typically not available. To address this issue, we develop an inverse optimization approach for estimating rivals' production cost functions given historical market clearing prices and production levels. We then use these functions to bid strategically and compute Nash equilibrium bids. We present numerical experiments illustrating our methodology, showing good agreement between bids based on the estimated production cost functions with the bids based on the true cost functions. We discuss an extension of our approach that takes into account network congestion resulting in location-dependent prices. △ Less

Submitted 14 November, 2018; originally announced November 2018.

Robust Asynchronous Stochastic Gradient-Push: Asymptotically Optimal and Network-Independent Performance for Strongly Convex Functions

Authors: Artin Spiridonoff, Alex Olshevsky, Ioannis Ch. Paschalidis

Abstract: We consider the standard model of distributed optimization of a sum of functions $F(\bz) = \sum_{i=1}^n f_i(\bz)$, where node $i$ in a network holds the function $f_i(\bz)$. We allow for a harsh network model characterized by asynchronous updates, message delays, unpredictable message losses, and directed communication among nodes. In this setting, we analyze a modification of the Gradient-Push me… ▽ More We consider the standard model of distributed optimization of a sum of functions $F(\bz) = \sum_{i=1}^n f_i(\bz)$, where node $i$ in a network holds the function $f_i(\bz)$. We allow for a harsh network model characterized by asynchronous updates, message delays, unpredictable message losses, and directed communication among nodes. In this setting, we analyze a modification of the Gradient-Push method for distributed optimization, assuming that \begin{enumerate*}[label=(\roman*)] \item node $i$ is capable of generating gradients of its function $f_i(\bz)$ corrupted by zero-mean bounded-support additive noise at each step, \item $F(\bz)$ is strongly convex, and \item each $f_i(\bz)$ has Lipschitz gradients. We show that our proposed method asymptotically performs as well as the best bounds on centralized gradient descent that takes steps in the direction of the sum of the noisy gradients of all the functions $f_1(\bz), \ldots, f_n(\bz)$ at each step. △ Less

Submitted 29 December, 2019; v1 submitted 9 November, 2018; originally announced November 2018.

Fully Asynchronous Push-Sum With Growing Intercommunication Intervals

Authors: Alex Olshevsky, Ioannis Ch. Paschalidis, Artin Spiridonoff

Abstract: We propose an algorithm for average consensus over a directed graph which is both fully asynchronous and robust to unreliable communications. We show its convergence to the average, while allowing for slowly growing but potentially unbounded communication failures. We propose an algorithm for average consensus over a directed graph which is both fully asynchronous and robust to unreliable communications. We show its convergence to the average, while allowing for slowly growing but potentially unbounded communication failures. △ Less

Submitted 23 February, 2018; originally announced February 2018.

Data-Driven Estimation of Travel Latency Cost Functions via Inverse Optimization in Multi-Class Transportation Networks

Authors: Jing Zhang, Ioannis Ch. Paschalidis

Abstract: We develop a method to estimate from data travel latency cost functions in multi-class transportation networks, which accommodate different types of vehicles with very different characteristics (e.g., cars and trucks). Leveraging our earlier work on inverse variational inequalities, we develop a data-driven approach to estimate the travel latency cost functions. Extensive numerical experiments usi… ▽ More We develop a method to estimate from data travel latency cost functions in multi-class transportation networks, which accommodate different types of vehicles with very different characteristics (e.g., cars and trucks). Leveraging our earlier work on inverse variational inequalities, we develop a data-driven approach to estimate the travel latency cost functions. Extensive numerical experiments using benchmark networks, ranging from moderate-sized to large-sized, demonstrate the effectiveness and efficiency of our approach. △ Less

Submitted 3 April, 2017; v1 submitted 11 March, 2017; originally announced March 2017.

Comments: Preprint submitted to the 56th IEEE Conference on Decision and Control (2017)

MSC Class: 90C33; 90C90; 90C30

Statistical Anomaly Detection via Composite Hypothesis Testing for Markov Models

Authors: Jing Zhang, Ioannis Ch. Paschalidis

Abstract: Under Markovian assumptions, we leverage a Central Limit Theorem (CLT) for the empirical measure in the test statistic of the composite hypothesis Hoeffding test so as to establish weak convergence results for the test statistic, and, thereby, derive a new estimator for the threshold needed by the test. We first show the advantages of our estimator over an existing estimator by conducting extensiv… ▽ More Under Markovian assumptions, we leverage a Central Limit Theorem (CLT) for the empirical measure in the test statistic of the composite hypothesis Hoeffding test so as to establish weak convergence results for the test statistic, and, thereby, derive a new estimator for the threshold needed by the test. We first show the advantages of our estimator over an existing estimator by conducting extensive numerical experiments. We find that our estimator controls better for false alarms while maintaining satisfactory detection probabilities. We then apply the Hoeffding test with our threshold estimator to detecting anomalies in two distinct applications domains: one in communication networks and the other in transportation networks. The former application seeks to enhance cyber security and the latter aims at building smarter transportation systems in cities. △ Less

Submitted 18 August, 2017; v1 submitted 27 February, 2017; originally announced February 2017.

Comments: Preprint submitted to the IEEE Transactions on Signal Processing

Learning Policies for Markov Decision Processes from Data

Authors: Manjesh K. Hanawal, Hao Liu, Henghui Zhu, Ioannis Ch. Paschalidis

Abstract: We consider the problem of learning a policy for a Markov decision process consistent with data captured on the state-actions pairs followed by the policy. We assume that the policy belongs to a class of parameterized policies which are defined using features associated with the state-action pairs. The features are known a priori, however, only an unknown subset of them could be relevant. The poli… ▽ More We consider the problem of learning a policy for a Markov decision process consistent with data captured on the state-actions pairs followed by the policy. We assume that the policy belongs to a class of parameterized policies which are defined using features associated with the state-action pairs. The features are known a priori, however, only an unknown subset of them could be relevant. The policy parameters that correspond to an observed target policy are recovered using $\ell_1$-regularized logistic regression that best fits the observed state-action samples. We establish bounds on the difference between the average reward of the estimated and the original policy (regret) in terms of the generalization error and the ergodic coefficient of the underlying Markov chain. To that end, we combine sample complexity theory and sensitivity analysis of the stationary distribution of Markov chains. Our analysis suggests that to achieve regret within order $O(\sqrtε)$, it suffices to use training sample size on the order of $Ω(\log n \cdot poly(1/ε))$, where $n$ is the number of the features. We demonstrate the effectiveness of our method on a synthetic robot navigation example. △ Less

Submitted 20 January, 2017; originally announced January 2017.

MSC Class: 90C40 (Primary) 90C39; 93E35; 60J10 (Secondary) ACM Class: I.2.8; I.2.9; I.2.6

The Price of Anarchy in Transportation Networks: Data-Driven Evaluation and Reduction Strategies

Authors: Jing Zhang, Sepideh Pourazarm, Christos G. Cassandras, Ioannis Ch. Paschalidis

Abstract: Among the many functions a Smart City must support, transportation dominates in terms of resource consumption, strain on the environment, and frustration of its citizens. We study transportation networks under two different routing policies, the commonly assumed selfish user-centric routing policy and a socially-optimal system-centric one. We consider a performance metric of efficiency - the Price… ▽ More Among the many functions a Smart City must support, transportation dominates in terms of resource consumption, strain on the environment, and frustration of its citizens. We study transportation networks under two different routing policies, the commonly assumed selfish user-centric routing policy and a socially-optimal system-centric one. We consider a performance metric of efficiency - the Price of Anarchy (PoA) - defined as the ratio of the total travel latency cost under selfish routing over the corresponding quantity under socially-optimal routing. We develop a data-driven approach to estimate the PoA, which we subsequently use to conduct a case study using extensive actual traffic data from the Eastern Massachusetts road network. To estimate the PoA, our approach learns from data a complete model of the transportation network, including origin-destination demand and user preferences. We leverage this model to propose possible strategies to reduce the PoA and increase efficiency. △ Less

Submitted 3 January, 2018; v1 submitted 7 June, 2016; originally announced June 2016.

MSC Class: 90C33; 90C90; 90C30

Robust measurement-based buffer overflow probability estimators for QoS provisioning and traffic anomaly prediction applicationm

Authors: Spyridon Vassilaras, Ioannis Ch. Paschalidis

Abstract: Suitable estimators for a class of Large Deviation approximations of rare event probabilities based on sample realizations of random processes have been proposed in our earlier work. These estimators are expressed as non-linear multi-dimensional optimization problems of a special structure. In this paper, we develop an algorithm to solve these optimization problems very efficiently based on their… ▽ More Suitable estimators for a class of Large Deviation approximations of rare event probabilities based on sample realizations of random processes have been proposed in our earlier work. These estimators are expressed as non-linear multi-dimensional optimization problems of a special structure. In this paper, we develop an algorithm to solve these optimization problems very efficiently based on their characteristic structure. After discussing the nature of the objective function and constraint set and their peculiarities, we provide a formal proof that the developed algorithm is guaranteed to always converge. The existence of efficient and provably convergent algorithms for solving these problems is a prerequisite for using the proposed estimators in real time problems such as call admission control, adaptive modulation and coding with QoS constraints, and traffic anomaly detection in high data rate communication networks. △ Less

Submitted 2 May, 2016; originally announced May 2016.

Data-Driven Estimation in Equilibrium Using Inverse Optimization

Authors: Dimitris Bertsimas, Vishal Gupta, Ioannis Ch. Paschalidis

Abstract: Equilibrium modeling is common in a variety of fields such as game theory and transportation science. The inputs for these models, however, are often difficult to estimate, while their outputs, i.e., the equilibria they are meant to describe, are often directly observable. By combining ideas from inverse optimization with the theory of variational inequalities, we develop an efficient, data-driven… ▽ More Equilibrium modeling is common in a variety of fields such as game theory and transportation science. The inputs for these models, however, are often difficult to estimate, while their outputs, i.e., the equilibria they are meant to describe, are often directly observable. By combining ideas from inverse optimization with the theory of variational inequalities, we develop an efficient, data-driven technique for estimating the parameters of these models from observed equilibria. We use this technique to estimate the utility functions of players in a game from their observed actions and to estimate the congestion function on a road network from traffic count data. A distinguishing feature of our approach is that it supports both parametric and \emph{nonparametric} estimation by leveraging ideas from statistical learning (kernel methods and regularization operators). In computational experiments involving Nash and Wardrop equilibria in a nonparametric setting, we find that a) we effectively estimate the unknown demand or congestion function, respectively, and b) our proposed regularization technique substantially improves the out-of-sample performance of our estimators. △ Less

Submitted 17 May, 2014; v1 submitted 15 August, 2013; originally announced August 2013.

Comments: 36 pages, 5 figures Additional theorems for generalization guarantees and statistical analysis added

Temporal Logic Motion Control using Actor-Critic Methods

Authors: Xu Chu Ding, Jing Wang, Morteza Lahijanian, Ioannis Ch. Paschalidis, Calin A. Belta

Abstract: In this paper, we consider the problem of deploying a robot from a specification given as a temporal logic statement about some properties satisfied by the regions of a large, partitioned environment. We assume that the robot has noisy sensors and actuators and model its motion through the regions of the environment as a Markov Decision Process (MDP). The robot control problem becomes finding the… ▽ More In this paper, we consider the problem of deploying a robot from a specification given as a temporal logic statement about some properties satisfied by the regions of a large, partitioned environment. We assume that the robot has noisy sensors and actuators and model its motion through the regions of the environment as a Markov Decision Process (MDP). The robot control problem becomes finding the control policy maximizing the probability of satisfying the temporal logic task on the MDP. For a large environment, obtaining transition probabilities for each state-action pair, as well as solving the necessary optimization problem for the optimal policy are usually not computationally feasible. To address these issues, we propose an approximate dynamic programming framework based on a least-square temporal difference learning method of the actor-critic type. This framework operates on sample paths of the robot and optimizes a randomized control policy with respect to a small set of parameters. The transition probabilities are obtained only when needed. Hardware-in-the-loop simulations confirm that convergence of the parameters translates to an approximately optimal policy. △ Less

Submitted 23 February, 2012; v1 submitted 9 February, 2012; originally announced February 2012.

Comments: Technical Report which accompanies an ICRA2012 paper

Least Squares Temporal Difference Actor-Critic Methods with Applications to Robot Motion Control

Authors: Reza Moazzez Estanjini, Xu Chu Ding, Morteza Lahijanian, Jing Wang, Calin A. Belta, Ioannis Ch. Paschalidis

Abstract: We consider the problem of finding a control policy for a Markov Decision Process (MDP) to maximize the probability of reaching some states while avoiding some other states. This problem is motivated by applications in robotics, where such problems naturally arise when probabilistic models of robot motion are required to satisfy temporal logic task specifications. We transform this problem into a… ▽ More We consider the problem of finding a control policy for a Markov Decision Process (MDP) to maximize the probability of reaching some states while avoiding some other states. This problem is motivated by applications in robotics, where such problems naturally arise when probabilistic models of robot motion are required to satisfy temporal logic task specifications. We transform this problem into a Stochastic Shortest Path (SSP) problem and develop a new approximate dynamic programming algorithm to solve it. This algorithm is of the actor-critic type and uses a least-square temporal difference learning method. It operates on sample paths of the system and optimizes the policy within a pre-specified class parameterized by a parsimonious set of parameters. We show its convergence to a policy corresponding to a stationary point in the parameters' space. Simulation results confirm the effectiveness of the proposed solution. △ Less

Submitted 30 August, 2011; v1 submitted 23 August, 2011; originally announced August 2011.

Comments: Technical report accompanying an accepted paper to CDC 2011