-
On Design of Representative Distributionally Robust Formulations for Evaluation of Tail Risk Measures
Authors:
Anand Deo
Abstract:
Conditional Value-at-Risk (CVaR) is a risk measure widely used to quantify the impact of extreme losses. Owing to the lack of representative samples CVaR is sensitive to the tails of the underlying distribution. In order to combat this sensitivity, Distributionally Robust Optimization (DRO), which evaluates the worst-case CVaR measure over a set of plausible data distributions is often deployed. U…
▽ More
Conditional Value-at-Risk (CVaR) is a risk measure widely used to quantify the impact of extreme losses. Owing to the lack of representative samples CVaR is sensitive to the tails of the underlying distribution. In order to combat this sensitivity, Distributionally Robust Optimization (DRO), which evaluates the worst-case CVaR measure over a set of plausible data distributions is often deployed. Unfortunately, an improper choice of the DRO formulation can lead to a severe underestimation of tail risk. This paper aims at leveraging extreme value theory to arrive at a DRO formulation which leads to representative worst-case CVaR evaluations in that the above pitfall is avoided while simultaneously, the worst case evaluation is not a gross over-estimate of the true CVaR. We demonstrate theoretically that even when there is paucity of samples in the tail of the distribution, our formulation is readily implementable from data, only requiring calibration of a single scalar parameter. We showcase that our formulation can be easily extended to provide robustness to tail risk in multivariate applications as well as in the evaluation of other commonly used risk measures. Numerical illustrations on synthetic and real-world data showcase the practical utility of our approach.
△ Less
Submitted 19 June, 2025;
originally announced June 2025.
-
The Scaling Behaviors in Achieving High Reliability via Chance-Constrained Optimization
Authors:
Anand Deo,
Karthyek Murthy
Abstract:
We study the problem of resource provisioning under stringent reliability or service level requirements, which arise in applications such as power distribution, emergency response, cloud server provisioning, and regulatory risk management. With chance-constrained optimization serving as a natural starting point for modeling this class of problems, our primary contribution is to characterize how th…
▽ More
We study the problem of resource provisioning under stringent reliability or service level requirements, which arise in applications such as power distribution, emergency response, cloud server provisioning, and regulatory risk management. With chance-constrained optimization serving as a natural starting point for modeling this class of problems, our primary contribution is to characterize how the optimal costs and decisions scale for a generic joint chance-constrained model as the target probability of satisfying the service/reliability constraints approaches its maximal level. Beyond providing insights into the behavior of optimal solutions, our scaling framework has three key algorithmic implications. First, in distributionally robust optimization (DRO) modeling of chance constraints, we show that widely used approaches based on KL-divergences, Wasserstein distances, and moments heavily distort the scaling properties of optimal decisions, leading to exponentially higher costs. In contrast, incorporating marginal distributions or using appropriately chosen f-divergence balls preserves the correct scaling, ensuring decisions remain conservative by at most a constant or logarithmic factor. Second, we leverage the scaling framework to quantify the conservativeness of common inner approximations and propose a simple line search to refine their solutions, yielding near-optimal decisions. Finally, given N data samples, we demonstrate how the scaling framework enables the estimation of approximately Pareto-optimal decisions with constraint violation probabilities significantly smaller than the Omega(1/N)-barrier that arises in the absence of parametric assumptions
△ Less
Submitted 10 April, 2025;
originally announced April 2025.
-
Reduced Sample Complexity in Scenario-Based Control System Design via Constraint Scaling
Authors:
Jaeseok Choi,
Anand Deo,
Constantino Lagoa,
Anirudh Subramanyam
Abstract:
The scenario approach is widely used in robust control system design and chance-constrained optimization, maintaining convexity without requiring assumptions about the probability distribution of uncertain parameters. However, the approach can demand large sample sizes, making it intractable for safety-critical applications that require very low levels of constraint violation. To address this chal…
▽ More
The scenario approach is widely used in robust control system design and chance-constrained optimization, maintaining convexity without requiring assumptions about the probability distribution of uncertain parameters. However, the approach can demand large sample sizes, making it intractable for safety-critical applications that require very low levels of constraint violation. To address this challenge, we propose a novel yet simple constraint scaling method, inspired by large deviations theory. Under mild nonparametric conditions on the underlying probability distribution, we show that our method yields an exponential reduction in sample size requirements for bilinear constraints with low violation levels compared to the classical approach, thereby significantly improving computational tractability. Numerical experiments on robust pole assignment problems support our theoretical findings.
△ Less
Submitted 7 December, 2024; v1 submitted 11 November, 2024;
originally announced November 2024.
-
Efficient Black-Box Importance Sampling for VaR and CVaR Estimation
Authors:
Anand Deo,
Karthyek Murthy
Abstract:
This paper considers Importance Sampling (IS) for the estimation of tail risks of a loss defined in terms of a sophisticated object such as a machine learning feature map or a mixed integer linear optimisation formulation. Assuming only black-box access to the loss and the distribution of the underlying random vector, the paper presents an efficient IS algorithm for estimating the Value at Risk an…
▽ More
This paper considers Importance Sampling (IS) for the estimation of tail risks of a loss defined in terms of a sophisticated object such as a machine learning feature map or a mixed integer linear optimisation formulation. Assuming only black-box access to the loss and the distribution of the underlying random vector, the paper presents an efficient IS algorithm for estimating the Value at Risk and Conditional Value at Risk. The key challenge in any IS procedure, namely, identifying an appropriate change-of-measure, is automated with a self-structuring IS transformation that learns and replicates the concentration properties of the conditional excess from less rare samples. The resulting estimators enjoy asymptotically optimal variance reduction when viewed in the logarithmic scale. Simulation experiments highlight the efficacy and practicality of the proposed scheme
△ Less
Submitted 15 June, 2021;
originally announced June 2021.
-
Electrification of Commercial E-buses by Utilizing Stationary Battery Energy Storage Systems for Mass Transportation Network in Los Angeles
Authors:
Amirhossein Ahmadian,
Atharva Sunil Deo,
Paawan Garg,
Omar Elayyan
Abstract:
The 2028 LA Olympics is approaching, and the mass transportation network requires to be expanded. The green electric bus transportation system is gaining increasing attention as an essential step to mitigate emission concerns. However, the introduction of a large electric bus fleet will increase the peak load demand, which will adversely affect the grid. This research addresses the issue of an inc…
▽ More
The 2028 LA Olympics is approaching, and the mass transportation network requires to be expanded. The green electric bus transportation system is gaining increasing attention as an essential step to mitigate emission concerns. However, the introduction of a large electric bus fleet will increase the peak load demand, which will adversely affect the grid. This research addresses the issue of an increase in the peak load demand due to the introduction of the electric bus fleet. Therefore, the potential implementation of stationary battery energy storage systems is investigated to shave the peak load demand by providing energy during peak hours. Moreover, a fleet management software implementation has been demonstrated for a test case scenario in Los Angeles. As a result, based on specific energy demand values, battery hardware options, standard e-bus driving cycles and profiles, and local bus route characteristics, the optimal battery specifications, and power rates have been addressed. Moreover, techno-economic analysis has been carried out to demonstrate the cost-benefit analysis.
△ Less
Submitted 21 April, 2021;
originally announced April 2021.
-
Achieving Efficiency in Black Box Simulation of Distribution Tails with Self-structuring Importance Samplers
Authors:
Anand Deo,
Karthyek Murthy
Abstract:
This paper presents a novel Importance Sampling (IS) scheme for estimating distribution tails of performance measures modeled with a rich set of tools such as linear programs, integer linear programs, piecewise linear/quadratic objectives, feature maps specified with deep neural networks, etc. The conventional approach of explicitly identifying efficient changes of measure suffers from feasibility…
▽ More
This paper presents a novel Importance Sampling (IS) scheme for estimating distribution tails of performance measures modeled with a rich set of tools such as linear programs, integer linear programs, piecewise linear/quadratic objectives, feature maps specified with deep neural networks, etc. The conventional approach of explicitly identifying efficient changes of measure suffers from feasibility and scalability concerns beyond highly stylized models, due to their need to be tailored intricately to the objective and the underlying probability distribution. This bottleneck is overcome in the proposed scheme with an elementary transformation which is capable of implicitly inducing an effective IS distribution in a variety of models by replicating the concentration properties observed in less rare samples. This novel approach is guided by developing a large deviations principle that brings out the phenomenon of self-similarity of optimal IS distributions. The proposed sampler is the first to attain asymptotically optimal variance reduction across a spectrum of multivariate distributions despite being oblivious to the specifics of the underlying model. Its applicability is illustrated with contextual shortest path and portfolio credit risk models informed by neural networks
△ Less
Submitted 8 July, 2023; v1 submitted 13 February, 2021;
originally announced February 2021.
