-
Fragility-aware Classification for Understanding Risk and Improving Generalization
Authors:
Chen Yang,
Zheng Cui,
Daniel Zhuoyu Long,
Jin Qi,
Ruohan Zhan
Abstract:
Classification models play a critical role in data-driven decision-making applications such as medical diagnosis, user profiling, recommendation systems, and default detection. Traditional performance metrics, such as accuracy, focus on overall error rates but fail to account for the confidence of incorrect predictions, thereby overlooking the risk of confident misjudgments. This risk is particula…
▽ More
Classification models play a critical role in data-driven decision-making applications such as medical diagnosis, user profiling, recommendation systems, and default detection. Traditional performance metrics, such as accuracy, focus on overall error rates but fail to account for the confidence of incorrect predictions, thereby overlooking the risk of confident misjudgments. This risk is particularly significant in cost-sensitive and safety-critical domains like medical diagnosis and autonomous driving, where overconfident false predictions may cause severe consequences. To address this issue, we introduce the Fragility Index (FI), a novel metric that evaluates classification performance from a risk-averse perspective by explicitly capturing the tail risk of confident misjudgments. To enhance generalizability, we define FI within the robust satisficing (RS) framework, incorporating data uncertainty. We further develop a model training approach that optimizes FI while maintaining tractability for common loss functions. Specifically, we derive exact reformulations for cross-entropy loss, hinge-type loss, and Lipschitz loss, and extend the approach to deep learning models. Through synthetic experiments and real-world medical diagnosis tasks, we demonstrate that FI effectively identifies misjudgment risk and FI-based training improves model robustness and generalizability. Finally, we extend our framework to deep neural network training, further validating its effectiveness in enhancing deep learning models.
△ Less
Submitted 18 February, 2025;
originally announced February 2025.
-
Asymptotically Optimal Distributionally Robust Solutions through Forecasting and Operations Decentralization
Authors:
Yue Lin,
Daniel Zhuoyu Long,
Viet Anh Nguyen,
Jin Qi
Abstract:
Two-stage risk-averse distributionally robust optimization (DRO) problems are ubiquitous across many engineering and business applications. Despite their promising resilience, two-stage DRO problems are generally computationally intractable. To address this challenge, we propose a simple framework by decentralizing the decision-making process into two specialized teams: forecasting and operations.…
▽ More
Two-stage risk-averse distributionally robust optimization (DRO) problems are ubiquitous across many engineering and business applications. Despite their promising resilience, two-stage DRO problems are generally computationally intractable. To address this challenge, we propose a simple framework by decentralizing the decision-making process into two specialized teams: forecasting and operations. This decentralization aligns with prevalent organizational practices, in which the operations team uses the information communicated from the forecasting team as input to make decisions. We formalize this decentralized procedure as a bilevel problem to design a communicated distribution that can yield asymptotic optimal solutions to original two-stage risk-averse DRO problems. We identify an optimal solution that is surprisingly simple: The forecasting team only needs to communicate a two-point distribution to the operations team. Consequently, the operations team can solve a highly tractable and scalable optimization problem to identify asymptotic optimal solutions. Specifically, as the magnitude of the problem parameters (including the uncertain parameters and the first-stage capacity) increases to infinity at an appropriate rate, the cost ratio between our induced solution and the original optimal solution converges to one, indicating that our decentralized approach yields high-quality solutions. We compare our decentralized approach against the truncated linear decision rule approximation and demonstrate that our approach has broader applicability and superior computational efficiency while maintaining competitive performance. Using real-world sales data, we have demonstrated the practical effectiveness of our strategy. The finely tuned solution significantly outperforms traditional sample-average approximation methods in out-of-sample performance.
△ Less
Submitted 22 December, 2024;
originally announced December 2024.
-
Scenario-decomposition Solution Framework for Nonseparable Stochastic Control Problems
Authors:
Xin Huang,
Duan Li,
Daniel Zhuoyu Long
Abstract:
When stochastic control problems do not possess separability and/or monotonicity, the dynamic programming pioneered by Bellman in 1950s fails to work as a time-decomposition solution method. Such cases have posted a great challenge to the control society in both theoretical foundation and solution methodologies for many years. With the help of the progressive hedging algorithm proposed by Rockafel…
▽ More
When stochastic control problems do not possess separability and/or monotonicity, the dynamic programming pioneered by Bellman in 1950s fails to work as a time-decomposition solution method. Such cases have posted a great challenge to the control society in both theoretical foundation and solution methodologies for many years. With the help of the progressive hedging algorithm proposed by Rockafellar and Wets in 1991, we develop a novel scenario-decomposition solution framework for stochastic control problems which could be nonseparable and/or non-monotonic, thus extending the reach of stochastic optimal control. We discuss then some of its promising applications, including online quadratic programming problems and dynamic portfolio selection problems with smoothing properties.
△ Less
Submitted 18 October, 2020;
originally announced October 2020.
