-
Throughput-Optimal Scheduling Algorithms for LLM Inference and AI Agents
Authors:
Yueying Li,
Jim Dai,
Tianyi Peng
Abstract:
As demand for Large Language Models (LLMs) and AI agents rapidly grows, optimizing systems for efficient LLM inference becomes critical. While significant efforts have focused on system-level engineering, little is explored from a mathematical modeling and queuing perspective.
In this paper, we aim to develop the queuing fundamentals for large language model (LLM) inference, bridging the gap bet…
▽ More
As demand for Large Language Models (LLMs) and AI agents rapidly grows, optimizing systems for efficient LLM inference becomes critical. While significant efforts have focused on system-level engineering, little is explored from a mathematical modeling and queuing perspective.
In this paper, we aim to develop the queuing fundamentals for large language model (LLM) inference, bridging the gap between the queueing theory and LLM system communities. In particular, we study the throughput aspect in LLM inference systems. We prove that a large class of 'work-conserving' scheduling algorithms can achieve maximum throughput for individual inference LLM engine, highlighting 'work-conserving' as a key design principle in practice. In a network of LLM agents, work-conserving scheduling alone is insufficient, particularly when facing specific workload structures and multi-class workflows that require more sophisticated scheduling strategies. Evaluations of real-world systems show that Orca and Sarathi-serve are throughput-optimal, reassuring practitioners, while FasterTransformer and vanilla vLLM are not maximally stable and should be used with caution. Our results highlight the substantial benefits that the queueing community can offer in improving LLM inference systems and call for more interdisciplinary development.
△ Less
Submitted 24 April, 2025; v1 submitted 9 April, 2025;
originally announced April 2025.
-
Global Continuation of Stable Periodic Orbits in Systems of Competing Predators
Authors:
Kevin E. M. Church,
Jia-Yuan Dai,
Olivier Hénot,
Phillipo Lappicy,
Nicola Vassena
Abstract:
We develop a continuation technique to obtain global families of stable periodic orbits, delimited by transcritical bifurcations at both ends. To this end, we formulate a zero-finding problem whose zeros correspond to families of periodic orbits. We then define a Newton-like fixed-point operator and establish its contraction near a numerically computed approximation of the family. To verify the co…
▽ More
We develop a continuation technique to obtain global families of stable periodic orbits, delimited by transcritical bifurcations at both ends. To this end, we formulate a zero-finding problem whose zeros correspond to families of periodic orbits. We then define a Newton-like fixed-point operator and establish its contraction near a numerically computed approximation of the family. To verify the contraction, we derive sufficient conditions expressed as inequalities on the norms of the fixed-point operator, and involving the numerical approximation. These inequalities are then rigorously checked by the computer via interval arithmetic. To show the efficacy of our approach, we prove the existence of global families in an ecosystem with Holling's type II functional response, and thereby solve a stable connection problem proposed by Butler and Waltler in 1981. Our method does not rely on restricting the choice of parameters and is applicable to many other systems that numerically exhibit global families.
△ Less
Submitted 3 April, 2025;
originally announced April 2025.
-
Asymptotic Product-form Steady-state Distribution for Semimartingale Reflecting Brownian Motion in Multi-scaling Regime
Authors:
Jin Guang,
Xinyun Chen,
J. G. Dai,
Peter W. Glynn
Abstract:
Inspired by Dai et al. [2023], we develop a novel multi-scaling asymptotic regime for semimartingale reflecting Brownian motion (SRBM). In this regime, we establish the steady-state convergence of SRBM to a product-form limit with exponentially distributed components by assuming the P-reflection matrix and a uniform moment bound condition. We further demonstrate that the uniform moment bound condi…
▽ More
Inspired by Dai et al. [2023], we develop a novel multi-scaling asymptotic regime for semimartingale reflecting Brownian motion (SRBM). In this regime, we establish the steady-state convergence of SRBM to a product-form limit with exponentially distributed components by assuming the P-reflection matrix and a uniform moment bound condition. We further demonstrate that the uniform moment bound condition holds in several subclasses of P-matrices. Our proof approach is rooted in the basic adjoint relationship (BAR) for SRBM proposed by Harrison and Williams [1987a].
△ Less
Submitted 12 June, 2025; v1 submitted 25 March, 2025;
originally announced March 2025.
-
Asymptotic Product-form Steady-state for Multiclass Queueing Networks: A Reentrant Line Case Study
Authors:
Jim Dai,
Dongyan Huo
Abstract:
This paper serves as a companion to "Asymptotic Product-form Steady-state for Multiclass Queueing Networks with SBP Service Policies in Multi-scale Heavy Traffic." In this short paper, we illustrate the main results of the main paper through a two-station, five-class reentrant line under a specific static buffer priority policy, while avoiding heavy notations. For this example, we prove the asympt…
▽ More
This paper serves as a companion to "Asymptotic Product-form Steady-state for Multiclass Queueing Networks with SBP Service Policies in Multi-scale Heavy Traffic." In this short paper, we illustrate the main results of the main paper through a two-station, five-class reentrant line under a specific static buffer priority policy, while avoiding heavy notations. For this example, we prove the asymptotic steady-state limit and uniform moment bound under general inter-arrival and service time distributions.
△ Less
Submitted 1 November, 2024;
originally announced November 2024.
-
Inpatient Overflow Management with Proximal Policy Optimization
Authors:
Jingjing Sun,
Jim Dai,
Pengyi Shi
Abstract:
Overflowing patients to non-primary wards can effectively alleviate congestion in hospitals, while undesired overflow also leads to issues like mismatched service quality. Therefore, we need to trade off between congestion and undesired overflow. This overflow management problem is modeled as a discrete-time Markov Decision Process with large state and action space. To overcome the curse-of-dimens…
▽ More
Overflowing patients to non-primary wards can effectively alleviate congestion in hospitals, while undesired overflow also leads to issues like mismatched service quality. Therefore, we need to trade off between congestion and undesired overflow. This overflow management problem is modeled as a discrete-time Markov Decision Process with large state and action space. To overcome the curse-of-dimensionality, we decompose the action at each time into a sequence of atomic actions and use an actor-critic algorithm, Proximal Policy Optimization (PPO), to guide the atomic actions. Moreover, we tailor the design of neural network which represents policy to account for the daily periodic pattern of the system flows. Under hospital settings of different scales, the PPO policies consistently outperform commonly used state-of-art policies.
△ Less
Submitted 17 March, 2025; v1 submitted 17 October, 2024;
originally announced October 2024.
-
Consistent complete independence test in high dimensions based on Chatterjee correlation coefficient
Authors:
Liqi Xia,
Ruiyuan Cao,
Jiang Du,
Jun Dai
Abstract:
In this article, we consider the complete independence test of high-dimensional data. Based on Chatterjee coefficient, we pioneer the development of quadratic test and extreme value test which possess good testing performance for oscillatory data, and establish the corresponding large sample properties under both null hypotheses and alternative hypotheses. In order to overcome the shortcomings of…
▽ More
In this article, we consider the complete independence test of high-dimensional data. Based on Chatterjee coefficient, we pioneer the development of quadratic test and extreme value test which possess good testing performance for oscillatory data, and establish the corresponding large sample properties under both null hypotheses and alternative hypotheses. In order to overcome the shortcomings of quadratic statistic and extreme value statistic, we propose a testing method termed as power enhancement test by adding a screening statistic to the quadratic statistic. The proposed method do not reduce the testing power under dense alternative hypotheses, but can enhance the power significantly under sparse alternative hypotheses. Three synthetic data examples and two real data examples are further used to illustrate the performance of our proposed methods.
△ Less
Submitted 16 September, 2024;
originally announced September 2024.
-
Explicit Steady-State Approximations for Parallel Server Systems with Heterogeneous Servers
Authors:
J. G. Dai,
Yaosheng Xu
Abstract:
The weighted-workload-task-allocation (WWTA) load-balancing policy is known to be throughput optimal for parallel server systems with heterogeneous servers. This work concerns the heavy traffic approximation of steady-state performance for parallel server systems operating under WWTA policy. Under a relaxed complete-resource-pooling condition, we prove that WWTA achieves a "strong form" of state-s…
▽ More
The weighted-workload-task-allocation (WWTA) load-balancing policy is known to be throughput optimal for parallel server systems with heterogeneous servers. This work concerns the heavy traffic approximation of steady-state performance for parallel server systems operating under WWTA policy. Under a relaxed complete-resource-pooling condition, we prove that WWTA achieves a "strong form" of state-space collapse in heavy traffic and that the scaled workload for each server converges in distribution to an exponential random variable, whose parameter is explicitly given by system primitives. Various steady-state performance measures are shown to be approximated from this exponential random variable. Instead of proving a stochastic process limit followed by an interchange of limits - a method that dominates the literature, our method works directly with a pre-limit basic adjoint relationship (BAR) that characterizes the stationary distribution of each pre-limit system.
△ Less
Submitted 6 June, 2024;
originally announced June 2024.
-
Steady-State Convergence of the Continuous-Time Routing System with General Distributions in Heavy Traffic
Authors:
Jin Guang,
Yaosheng Xu,
J. G. Dai
Abstract:
This paper examines a continuous-time routing system with general interarrival and service time distributions, operating under the join-the-shortest-queue and power-of-two-choices policies. Under a weaker set of assumptions than those commonly found in the literature, we prove that the scaled steady-state queue length at each station converges weakly to an identical exponential random variable in…
▽ More
This paper examines a continuous-time routing system with general interarrival and service time distributions, operating under the join-the-shortest-queue and power-of-two-choices policies. Under a weaker set of assumptions than those commonly found in the literature, we prove that the scaled steady-state queue length at each station converges weakly to an identical exponential random variable in heavy traffic. Specifically, our results hold under the assumption of the $(2 + δ_0)$th moment for the interarrival and service distributions with some $δ_0 > 0$. The proof leverages the Palm version of the basic adjoint relationship (BAR) as a key technique.
△ Less
Submitted 17 October, 2024; v1 submitted 17 May, 2024;
originally announced May 2024.
-
Tight matrices and heavy traffic steady state convergence in queueing networks
Authors:
J. G. Dai,
Yiquan Ji,
Masakiyo Miyazawa
Abstract:
We are interested to prove that the stationary distribution of a multiclass queueing network converges to the stationary distribution of a semimartingale reflecting Brownian motion (SRBM) in heavy traffic. A key condition for this convergence is that the sequence of the pre-limit stationary distributions under appropriate scaling is tight. In Braverman et al.(2025), a sufficient condition for this…
▽ More
We are interested to prove that the stationary distribution of a multiclass queueing network converges to the stationary distribution of a semimartingale reflecting Brownian motion (SRBM) in heavy traffic. A key condition for this convergence is that the sequence of the pre-limit stationary distributions under appropriate scaling is tight. In Braverman et al.(2025), a sufficient condition for this tightness is introduced in the term of the reflection matrix $R$ of the SRBM, which is coined for $R$ to be ``tight''. In this paper, we study how we can verify this tightness of $R$ of an SRBM. For a $2$-dimensional SRBM, we give necessary and sufficient conditions for $R$ to be tight, while, for a general dimension, we only give sufficient conditions. We then apply these results to the SRBMs arising from the diffusion approximations of multiclass queueing networks with static buffer priority service disciplines that are studied in Braverman et al.(2025). It is shown that $R$ is always tight for this network with two stations if $R$ is completely-$\sr{S}$. For the case of more than two stations, it is shown that $R$ is tight for reentrant lines with last-buffer-first-service (LBFS) discipline, but it is not always tight for reentrant line with first-buffer-first-service (FBFS) discipline.
△ Less
Submitted 6 July, 2025; v1 submitted 21 April, 2024;
originally announced April 2024.
-
Asymptotic Product-form Steady-state for Multiclass Queueing Networks with SBP Service Policies in Multi-scale Heavy Traffic
Authors:
J. G. Dai,
Dongyan Huo
Abstract:
In this work, we study the stationary distribution of the scaled queue length vector process in multiclass queueing networks operating under static buffer priority service policies. We establish that when subjected to a multi-scale heavy traffic condition, the stationary distribution converges to a product-form limit, with each component in the product form following an exponential distribution. A…
▽ More
In this work, we study the stationary distribution of the scaled queue length vector process in multiclass queueing networks operating under static buffer priority service policies. We establish that when subjected to a multi-scale heavy traffic condition, the stationary distribution converges to a product-form limit, with each component in the product form following an exponential distribution. A major assumption in proving the desired product-form limit is the uniform moment bound for scaled queue lengths. We prove this assumption holds if the unscaled high-priority queue lengths have uniform moment bound and a certain reflection matrix is a P-matrix.
△ Less
Submitted 5 November, 2024; v1 submitted 6 March, 2024;
originally announced March 2024.
-
Uniform Moment Bounds for Generalized Jackson Networks in Multi-scale Heavy Traffic
Authors:
Jin Guang,
Xinyun Chen,
J. G. Dai
Abstract:
We establish uniform moment bounds for steady-state queue lengths of generalized Jackson networks (GJNs) in multi-scale heavy traffic as recently proposed by Dai et al. [2023]. Uniform moment bounds lay the foundation for further analysis of the limit stationary distribution. Our result can be used to verify the crucial moment state space collapse (SSC) assumption in Dai et al. [2023] to establish…
▽ More
We establish uniform moment bounds for steady-state queue lengths of generalized Jackson networks (GJNs) in multi-scale heavy traffic as recently proposed by Dai et al. [2023]. Uniform moment bounds lay the foundation for further analysis of the limit stationary distribution. Our result can be used to verify the crucial moment state space collapse (SSC) assumption in Dai et al. [2023] to establish a product-form limit of GJN in the multi-scale heavy traffic regime. Our proof critically utilizes the Palm version of the basic adjoint relationship (BAR) as developed in Braverman et al. [2023].
△ Less
Submitted 29 January, 2024; v1 submitted 25 January, 2024;
originally announced January 2024.
-
Hybrid Bifurcations: Periodicity from Eliminating a Line of Equilibria
Authors:
Alejandro López-Nieto,
Phillipo Lappicy,
Nicola Vassena,
Hannes Stuke,
Jia-Yuan Dai
Abstract:
We describe a new mechanism that triggers periodic orbits in smooth dynamical systems. To this end, we introduce the concept of hybrid bifurcations: Such bifurcations occur when a line of equilibria with an exchange point of normal stability vanishes. Our main result is the existence and stability criteria of periodic orbits that bifurcate from breaking a line of equilibria. As an application, we…
▽ More
We describe a new mechanism that triggers periodic orbits in smooth dynamical systems. To this end, we introduce the concept of hybrid bifurcations: Such bifurcations occur when a line of equilibria with an exchange point of normal stability vanishes. Our main result is the existence and stability criteria of periodic orbits that bifurcate from breaking a line of equilibria. As an application, we obtain stable periodic coexistent solutions in an ecosystem for two competing predators with Holling's type II functional response.
△ Less
Submitted 29 December, 2024; v1 submitted 30 October, 2023;
originally announced October 2023.
-
Asymptotic product-form steady-state for generalized Jackson networks in multi-scale heavy traffic
Authors:
J. G. Dai,
Peter Glynn,
Yaosheng Xu
Abstract:
We prove that under a multi-scale heavy traffic condition, the stationary distribution of the scaled queue length vector process in any generalized Jackson network has a product-form limit. Each component in the product form follows an exponential distribution, corresponding to the Brownian approximation of a single station queue. The ``single station'' can be constructed precisely and its paramet…
▽ More
We prove that under a multi-scale heavy traffic condition, the stationary distribution of the scaled queue length vector process in any generalized Jackson network has a product-form limit. Each component in the product form follows an exponential distribution, corresponding to the Brownian approximation of a single station queue. The ``single station'' can be constructed precisely and its parameters have a good intuitive interpretation.
△ Less
Submitted 26 January, 2025; v1 submitted 3 April, 2023;
originally announced April 2023.
-
Symmetry Groupoids for Pattern-Selective Feedback Stabilization of the Chafee--Infante Equation
Authors:
Isabelle Schneider,
Jia-Yuan Dai
Abstract:
Reaction-diffusion equations are ubiquitous in various scientific domains and their patterns represent a fascinating area of investigation. However, many of these patterns are unstable and therefore challenging to observe. To overcome this limitation, we present new noninvasive feedback controls based on symmetry groupoids. As a concrete example, we employ these controls to selectively stabilize u…
▽ More
Reaction-diffusion equations are ubiquitous in various scientific domains and their patterns represent a fascinating area of investigation. However, many of these patterns are unstable and therefore challenging to observe. To overcome this limitation, we present new noninvasive feedback controls based on symmetry groupoids. As a concrete example, we employ these controls to selectively stabilize unstable equilibria of the Chafee--Infante equation under Dirichlet boundary conditions on the interval. Unlike conventional reflection-based control schemes, our approach incorporates additional symmetries that enable us to design new convolution controls for stabilization. By demonstrating the efficacy of our method, we provide a new tool for investigating and controlling systems with unstable patterns, with potential implications for a wide range of scientific disciplines.
△ Less
Submitted 16 June, 2023; v1 submitted 31 March, 2023;
originally announced March 2023.
-
Accelerating the Convergence Rate of Consensus for Second-Order Multi-Agent Systems by Memory Information
Authors:
Jiahao Dai,
Jing-Wen Yi,
Li Chai
Abstract:
This paper utilizes the agent's memory in accelerated consensus for second-order multi-agent systems (MASs). In the case of one-tap memory, explicit formulas for the optimal consensus convergence rate and control parameters are derived by applying the Jury stability criterion. It is proved that the optimal consensus convergence rate with one-tap memory is faster than that without memory. In the ca…
▽ More
This paper utilizes the agent's memory in accelerated consensus for second-order multi-agent systems (MASs). In the case of one-tap memory, explicit formulas for the optimal consensus convergence rate and control parameters are derived by applying the Jury stability criterion. It is proved that the optimal consensus convergence rate with one-tap memory is faster than that without memory. In the case of M-tap memory, an iterative algorithm is given to derive the control parameters to accelerate the convergence rate. Moreover, the accelerated consensus with one-tap memory is extended to the formation control, and the control parameters to achieve the fastest formation are obtained. Numerical examples further illustrate the theoretical results.
△ Less
Submitted 24 March, 2023;
originally announced March 2023.
-
The BAR approach for multiclass queueing networks with SBP service policies
Authors:
Anton Braverman,
J. G. Dai,
Masakiyo Miyazawa
Abstract:
The basic adjoint relationship (BAR) approach is an analysis technique based on the stationary equation of a Markov process. This approach was introduced to study heavy-traffic, steady-state convergence of generalized Jackson networks in which each service station has a single job class. We extend it to multiclass queueing networks operating under static-buffer-priority (SBP) service disciplines.…
▽ More
The basic adjoint relationship (BAR) approach is an analysis technique based on the stationary equation of a Markov process. This approach was introduced to study heavy-traffic, steady-state convergence of generalized Jackson networks in which each service station has a single job class. We extend it to multiclass queueing networks operating under static-buffer-priority (SBP) service disciplines. Our extension makes a connection with Palm distributions that allows one to attack a difficulty arising from queue-length truncation, which appears to be unavoidable in the multiclass setting.
For multiclass queueing networks operating under SBP service disciplines, our BAR approach provides an alternative to the "interchange of limits" approach that has dominated the literature in the last twenty years. The BAR approach can produce sharp results and allows one to establish steady-state convergence under three additional conditions: stability, state space collapse (SSC) and a certain matrix being "tight." These three conditions do not appear to depend on the interarrival and service-time distributions beyond their means, and their verification can be studied as three separate modules. In particular, they can be studied in a simpler, continuous-time Markov chain setting when all distributions are exponential.
As an example, these three conditions are shown to hold in reentrant lines operating under last-buffer-first-serve discipline. In a two-station, five-class reentrant line, under the heavy-traffic condition, the tight-matrix condition implies both the stability condition and the SSC condition. Whether such a relationship holds generally is an open problem.
△ Less
Submitted 12 January, 2024; v1 submitted 11 February, 2023;
originally announced February 2023.
-
Heavy-Tailed Loss Frequencies from Mixtures of Negative Binomial and Poisson Counts
Authors:
Jiansheng Dai,
Ziheng Huang,
Michael R. Powers,
Jiaxin Xu
Abstract:
Heavy-tailed random variables have been used in insurance research to model both loss frequencies and loss severities, with substantially more emphasis on the latter. In the present work, we take a step toward addressing this imbalance by exploring the class of heavy-tailed frequency models formed by continuous mixtures of Negative Binomial and Poisson random variables. We begin by defining the co…
▽ More
Heavy-tailed random variables have been used in insurance research to model both loss frequencies and loss severities, with substantially more emphasis on the latter. In the present work, we take a step toward addressing this imbalance by exploring the class of heavy-tailed frequency models formed by continuous mixtures of Negative Binomial and Poisson random variables. We begin by defining the concept of a calibrative family of mixing distributions (each member of which is identifiable from its associated Negative Binomial mixture), and show how to construct such families from only a single member. We then introduce a new heavy-tailed frequency model -- the two-parameter ZY distribution -- as a generalization of both the one-parameter Zeta and Yule distributions, and construct calibrative families for both the new distribution and the heavy-tailed two-parameter Waring distribution. Finally, we pursue natural extensions of both the ZY and Waring families to a unifying, four-parameter heavy-tailed model, providing the foundation for a novel loss-frequency modeling approach to complement conventional GLM analyses. This approach is illustrated by application to a classic set of Swedish commercial motor-vehicle insurance loss data.
△ Less
Submitted 10 November, 2022; v1 submitted 7 November, 2022;
originally announced November 2022.
-
Fast consensus of high-order multi-agent systems
Authors:
Jiahao Dai,
Jing-Wen Yi,
Li Chai
Abstract:
In this paper, the fast consensus problem of high-order multi-agent systems under undirected topologies is considered. The direct link between the consensus convergence rate and the control gains is established. An accelerated consensus algorithm based on gradient descent is proposed to optimize the convergence rate. By applying the Routh-Hurwitz stability criterion, the lower bound on the converg…
▽ More
In this paper, the fast consensus problem of high-order multi-agent systems under undirected topologies is considered. The direct link between the consensus convergence rate and the control gains is established. An accelerated consensus algorithm based on gradient descent is proposed to optimize the convergence rate. By applying the Routh-Hurwitz stability criterion, the lower bound on the convergence rate is derived, and explicit control gains are derived as the necessary condition to achieve the optimal convergence rate. Moreover, a protocol with time-varying control gains is designed to achieve the finite-time consensus. Explicit formulas for the time-varying control gains and the final consensus state are given. Numerical examples and simulation results are presented to illustrate the obtained theoretical results.
△ Less
Submitted 16 May, 2022;
originally announced May 2022.
-
Pattern-Selective Feedback Stabilization of Ginzburg--Landau Spiral Waves
Authors:
Isabelle Schneider,
Babette de Wolff,
Jia-Yuan Dai
Abstract:
The complex Ginzburg--Landau equation serves as a paradigm of pattern formation and the existence and stability properties of Ginzburg--Landau $m$-armed spiral waves have been investigated extensively. However, many multi-armed spiral waves are unstable and thereby rarely visible in experiments and numerical simulations. In this article we selectively stabilize certain significant classes of unsta…
▽ More
The complex Ginzburg--Landau equation serves as a paradigm of pattern formation and the existence and stability properties of Ginzburg--Landau $m$-armed spiral waves have been investigated extensively. However, many multi-armed spiral waves are unstable and thereby rarely visible in experiments and numerical simulations. In this article we selectively stabilize certain significant classes of unstable spiral waves within circular and spherical geometries. As a result, stable spiral waves with an arbitrary number of arms are obtained for the first time. Our tool for stabilization is the symmetry-breaking control triple method, which is an equivariant generalization of the widely applied Pyragas control to the setting of PDEs.
△ Less
Submitted 30 September, 2022; v1 submitted 2 March, 2022;
originally announced March 2022.
-
Optimal Memory Scheme for Accelerated Consensus Over Multi-Agent Networks
Authors:
Jiahao Dai,
Jing-Wen Yi,
Li Chai
Abstract:
The consensus over multi-agent networks can be accelerated by introducing agent's memory to the control protocol. In this paper, a more general protocol with the node memory and the state deviation memory is designed. We aim to provide the optimal memory scheme to accelerate consensus. The contributions of this paper are three: (i) For the one-tap memory scheme, we demonstrate that the state devia…
▽ More
The consensus over multi-agent networks can be accelerated by introducing agent's memory to the control protocol. In this paper, a more general protocol with the node memory and the state deviation memory is designed. We aim to provide the optimal memory scheme to accelerate consensus. The contributions of this paper are three: (i) For the one-tap memory scheme, we demonstrate that the state deviation memory is useless for the optimal convergence. (ii) In the worst case, we prove that it is a vain to add any tap of the state deviation memory, and the one-tap node memory is sufficient to achieve the optimal convergence. (iii) We show that the two-tap state deviation memory is effective on some special networks, such as star networks. Numerical examples are listed to illustrate the validity and correctness of the obtained results.
△ Less
Submitted 13 December, 2021;
originally announced December 2021.
-
$ \infty $-category and some applications on orbifolds
Authors:
Jiajun Dai
Abstract:
This paper is mainly about an early result that the orbifold stack is globally representable via some $ \infty $-categorical techniques.
This paper is mainly about an early result that the orbifold stack is globally representable via some $ \infty $-categorical techniques.
△ Less
Submitted 4 September, 2021; v1 submitted 3 August, 2021;
originally announced August 2021.
-
Refined Policy Improvement Bounds for MDPs
Authors:
J. G. Dai,
Mark Gluzman
Abstract:
The policy improvement bound on the difference of the discounted returns plays a crucial role in the theoretical justification of the trust-region policy optimization (TRPO) algorithm. The existing bound leads to a degenerate bound when the discount factor approaches one, making the applicability of TRPO and related algorithms questionable when the discount factor is close to one. We refine the re…
▽ More
The policy improvement bound on the difference of the discounted returns plays a crucial role in the theoretical justification of the trust-region policy optimization (TRPO) algorithm. The existing bound leads to a degenerate bound when the discount factor approaches one, making the applicability of TRPO and related algorithms questionable when the discount factor is close to one. We refine the results in \cite{Schulman2015, Achiam2017} and propose a novel bound that is "continuous" in the discount factor. In particular, our bound is applicable for MDPs with the long-run average rewards as well.
△ Less
Submitted 16 July, 2021;
originally announced July 2021.
-
A High-fidelity, Machine-learning Enhanced Queueing Network Simulation Model for Hospital Ultrasound Operations
Authors:
Yihan Pan,
Zhenghang Xu,
Jin Guang,
Jingjing Sun,
Chengwenjian Wang,
Xuanming Zhang,
Xinyun Chen,
J. G. Dai,
Yichuan Ding,
Pengyi Shi,
Hongxin Pan,
Kai Yang,
Song Wu
Abstract:
We collaborate with a large teaching hospital in Shenzhen, China and build a high-fidelity simulation model for its ultrasound center to predict key performance metrics, including the distributions of queue length, waiting time and sojourn time, with high accuracy. The key challenge to build an accurate simulation model is to understanding the complicated patient routing at the ultrasound center.…
▽ More
We collaborate with a large teaching hospital in Shenzhen, China and build a high-fidelity simulation model for its ultrasound center to predict key performance metrics, including the distributions of queue length, waiting time and sojourn time, with high accuracy. The key challenge to build an accurate simulation model is to understanding the complicated patient routing at the ultrasound center. To address the issue, we propose a novel two-level routing component to the queueing network model. We apply machine learning tools to calibrate the key components of the queueing model from data with enhanced accuracy.
△ Less
Submitted 12 April, 2021;
originally announced April 2021.
-
High order steady-state diffusion approximations
Authors:
Anton Braverman,
J. G. Dai,
Xiao Fang
Abstract:
We derive and analyze new diffusion approximations of stationary distributions of Markov chains that are based on second- and higher-order terms in the expansion of the Markov chain generator. Our approximations achieve a higher degree of accuracy compared to diffusion approximations widely used for the past fifty years, while retaining a similar computational complexity. To support our approximat…
▽ More
We derive and analyze new diffusion approximations of stationary distributions of Markov chains that are based on second- and higher-order terms in the expansion of the Markov chain generator. Our approximations achieve a higher degree of accuracy compared to diffusion approximations widely used for the past fifty years, while retaining a similar computational complexity. To support our approximations, we present a combination of theoretical and numerical results across three different models. Our approximations are derived recursively through Stein/Poisson equations, and the theoretical results are proved using Stein's method.
△ Less
Submitted 9 July, 2022; v1 submitted 4 December, 2020;
originally announced December 2020.
-
Ginzburg-Landau Spiral Waves in Circular and Spherical Geometries
Authors:
Jia-Yuan Dai
Abstract:
We prove the existence of $m$-armed spiral wave solutions for the complex Ginzburg-Landau equation in the circular and spherical geometries. We establish a new global bifurcation approach and generalize the results of existence for rigidly-rotating spiral waves. Moreover, we prove the existence of two new patterns: frozen spirals in the circular and spherical geometries, and 2-tip spirals in the s…
▽ More
We prove the existence of $m$-armed spiral wave solutions for the complex Ginzburg-Landau equation in the circular and spherical geometries. We establish a new global bifurcation approach and generalize the results of existence for rigidly-rotating spiral waves. Moreover, we prove the existence of two new patterns: frozen spirals in the circular and spherical geometries, and 2-tip spirals in the spherical geometry.
△ Less
Submitted 24 October, 2020;
originally announced October 2020.
-
Scalable Deep Reinforcement Learning for Ride-Hailing
Authors:
Jiekun Feng,
Mark Gluzman,
J. G. Dai
Abstract:
Ride-hailing services, such as Didi Chuxing, Lyft, and Uber, arrange thousands of cars to meet ride requests throughout the day. We consider a Markov decision process (MDP) model of a ride-hailing service system, framing it as a reinforcement learning (RL) problem. The simultaneous control of many agents (cars) presents a challenge for the MDP optimization because the action space grows exponentia…
▽ More
Ride-hailing services, such as Didi Chuxing, Lyft, and Uber, arrange thousands of cars to meet ride requests throughout the day. We consider a Markov decision process (MDP) model of a ride-hailing service system, framing it as a reinforcement learning (RL) problem. The simultaneous control of many agents (cars) presents a challenge for the MDP optimization because the action space grows exponentially with the number of cars. We propose a special decomposition for the MDP actions by sequentially assigning tasks to the drivers. The new actions structure resolves the scalability problem and enables the use of deep RL algorithms for control policy optimization. We demonstrate the benefit of our proposed decomposition with a numerical experiment based on real data from Didi Chuxing.
△ Less
Submitted 27 September, 2020;
originally announced September 2020.
-
Characterizing the Zeta Distribution via Continuous Mixtures
Authors:
Jiansheng Dai,
Ziheng Huang,
Michael R. Powers,
Jiaxin Xu
Abstract:
We offer two novel characterizations of the Zeta distribution: first, as tractable continuous mixtures of Negative Binomial distributions (with fixed shape parameter, r > 0), and second, as a tractable continuous mixture of Poisson distributions. In both the Negative Binomial case for r >= 1 and the Poisson case, the resulting Zeta distributions are identifiable because each mixture can be associa…
▽ More
We offer two novel characterizations of the Zeta distribution: first, as tractable continuous mixtures of Negative Binomial distributions (with fixed shape parameter, r > 0), and second, as a tractable continuous mixture of Poisson distributions. In both the Negative Binomial case for r >= 1 and the Poisson case, the resulting Zeta distributions are identifiable because each mixture can be associated with a unique mixing distribution. In the Negative Binomial case for 0 < r < 1, the mixing distributions are quasi-distributions (for which the quasi-probability density function assumes some negative values).
△ Less
Submitted 4 June, 2021; v1 submitted 14 August, 2020;
originally announced August 2020.
-
Queueing Network Controls via Deep Reinforcement Learning
Authors:
J. G. Dai,
Mark Gluzman
Abstract:
Novel advanced policy gradient (APG) methods, such as Trust Region policy optimization and Proximal policy optimization (PPO), have become the dominant reinforcement learning algorithms because of their ease of implementation and good practical performance. A conventional setup for notoriously difficult queueing network control problems is a Markov decision problem (MDP) that has three features: i…
▽ More
Novel advanced policy gradient (APG) methods, such as Trust Region policy optimization and Proximal policy optimization (PPO), have become the dominant reinforcement learning algorithms because of their ease of implementation and good practical performance. A conventional setup for notoriously difficult queueing network control problems is a Markov decision problem (MDP) that has three features: infinite state space, unbounded costs, and long-run average cost objective. We extend the theoretical framework of these APG methods for such MDP problems. The resulting PPO algorithm is tested on a parallel-server system and large-size multiclass queueing networks. The algorithm consistently generates control policies that outperform state-of-art heuristics in literature in a variety of load conditions from light to heavy traffic. These policies are demonstrated to be near-optimal when the optimal policy can be computed.
A key to the successes of our PPO algorithm is the use of three variance reduction techniques in estimating the relative value function via sampling. First, we use a discounted relative value function as an approximation of the relative value function. Second, we propose regenerative simulation to estimate the discounted relative value function. Finally, we incorporate the approximating martingale-process method into the regenerative estimator.
△ Less
Submitted 14 September, 2021; v1 submitted 30 July, 2020;
originally announced August 2020.
-
Model structures and quantum cohomology of higher orbifolds
Authors:
Jiajun Dai
Abstract:
The author explains local and global model structures on higher orbifolds which are truncated étale differentiable higher stacks, and discuss the application of the model structures to quantum cohomology of higher and derived orbifolds.
The author explains local and global model structures on higher orbifolds which are truncated étale differentiable higher stacks, and discuss the application of the model structures to quantum cohomology of higher and derived orbifolds.
△ Less
Submitted 22 July, 2020;
originally announced July 2020.
-
Ginzburg-Landau patterns in circular and spherical geometries: vortices, spirals and attractors
Authors:
Jia-Yuan Dai,
Phillipo Lappicy
Abstract:
This paper consists of three results on pattern formation of Ginzburg-Landau $m$-armed vortex solutions and spiral waves in circular and spherical geometries. First, we completely describe the global bifurcation diagram of vortex equilibria. Second, we prove persistence of all bifurcation curves under perturbations of parameters, which yields the existence of spiral waves for the complex Ginzburg-…
▽ More
This paper consists of three results on pattern formation of Ginzburg-Landau $m$-armed vortex solutions and spiral waves in circular and spherical geometries. First, we completely describe the global bifurcation diagram of vortex equilibria. Second, we prove persistence of all bifurcation curves under perturbations of parameters, which yields the existence of spiral waves for the complex Ginzburg-Landau equation. Third, we explicitly construct the global attractor of $m$-armed vortex solutions. Our main tool is a new shooting method that allows us to prove hyperbolicity of vortex equilibria in the invariant subspace of vortex solutions.
△ Less
Submitted 21 July, 2021; v1 submitted 31 January, 2019;
originally announced January 2019.
-
Lattice Approximations of Semilinear Stochastic Elliptic Equations with Reflection
Authors:
Jun Dai,
Jing Zhang
Abstract:
We study lattice approximations of reflected stochastic elliptic equations driven by white noise on a bounded domain in $\mathbb{R}^d,\ d=1,2,3$. The convergence of the scheme is established.
We study lattice approximations of reflected stochastic elliptic equations driven by white noise on a bounded domain in $\mathbb{R}^d,\ d=1,2,3$. The convergence of the scheme is established.
△ Less
Submitted 31 July, 2018;
originally announced July 2018.
-
Interior gradient and Hessian estimates for the Dirichlet problem of semi-linear degenerate elliptic systems: a probabilistic approach
Authors:
Jun Dai,
Shanjian Tang,
Bingjie Wu
Abstract:
In this paper, we give interior gradient and Hessian estimates for systems of semi-linear degenerate elliptic partial differential equations on bounded domains, using both tools of backward stochastic differential equations and quasi-derivatives.
In this paper, we give interior gradient and Hessian estimates for systems of semi-linear degenerate elliptic partial differential equations on bounded domains, using both tools of backward stochastic differential equations and quasi-derivatives.
△ Less
Submitted 31 July, 2018;
originally announced July 2018.
-
Empty-car routing in ridesharing systems
Authors:
Anton Braverman,
J. G. Dai,
Xin Liu,
Lei Ying
Abstract:
This paper considers a closed queueing network model of ridesharing systems such as Didi Chuxing, Lyft, and Uber. We focus on empty-car routing, a mechanism by which we control car flow in the network to optimize system-wide utility functions, e.g. the availability of empty cars when a passenger arrives. We establish both process-level and steady-state convergence of the queueing network to a flui…
▽ More
This paper considers a closed queueing network model of ridesharing systems such as Didi Chuxing, Lyft, and Uber. We focus on empty-car routing, a mechanism by which we control car flow in the network to optimize system-wide utility functions, e.g. the availability of empty cars when a passenger arrives. We establish both process-level and steady-state convergence of the queueing network to a fluid limit in a large market regime where demand for rides and supply of cars tend to infinity, and use this limit to study a fluid-based optimization problem. We prove that the optimal network utility obtained from the fluid-based optimization is an upper bound on the utility in the finite car system for any routing policy, both static and dynamic, under which the closed queueing network has a stationary distribution. This upper bound is achieved asymptotically under the fluid-based optimal routing policy. Simulation results with real-world data released by Didi Chuxing demonstrate the benefit of using the fluid-based optimal routing policy compared to various other policies.
△ Less
Submitted 14 August, 2018; v1 submitted 22 September, 2016;
originally announced September 2016.
-
On the Roots of Characteristic Equations of Delay Differential Systems
Authors:
Jia-Yuan Dai
Abstract:
We prove that characteristic equations of certain types of delay differential systems, under some mild conditions on their coefficients, can possess infinitely many complex roots.
We prove that characteristic equations of certain types of delay differential systems, under some mild conditions on their coefficients, can possess infinitely many complex roots.
△ Less
Submitted 2 May, 2016;
originally announced May 2016.
-
High order steady-state diffusion approximation of the Erlang-C system
Authors:
Anton Braverman,
J. G. Dai
Abstract:
In this paper we introduce a new diffusion approximation for the steady-state customer count of the Erlang-C system. Unlike previous diffusion approximations, which use the steady-state distribution of a diffusion process with a constant diffusion coefficient, our approximation uses the steady-state distribution of a diffusion process with a \textit{state-dependent} diffusion coefficient. We show,…
▽ More
In this paper we introduce a new diffusion approximation for the steady-state customer count of the Erlang-C system. Unlike previous diffusion approximations, which use the steady-state distribution of a diffusion process with a constant diffusion coefficient, our approximation uses the steady-state distribution of a diffusion process with a \textit{state-dependent} diffusion coefficient. We show, both analytically and numerically, that our new approximation is an order of magnitude better than its counterpart. To obtain the analytical results, we use Stein's to show that a variant of the Wasserstein distance between the normalized customer count distribution and our approximation vanishes at a rate of $1/R$, where $R$ is the offered load to the system. In contrast, the previous approximation only achieved a rate of $1/R$. We hope our results motivate others to consider diffusion approximations with state-dependent diffusion coefficients.
△ Less
Submitted 9 February, 2016;
originally announced February 2016.
-
Stein's method for steady-state diffusion approximations: an introduction through the Erlang-A and Erlang-C models
Authors:
Anton Braverman,
J. G. Dai,
Jiekun Feng
Abstract:
This paper provides an introduction to the Stein method framework in the context of steady-state diffusion approximations. The framework consists of three components: the Poisson equation and gradient bounds, generator coupling, and moment bounds. Working in the setting of the Erlang-A and Erlang-C models, we prove that both Wasserstein and Kolmogorov distances between the stationary distribution…
▽ More
This paper provides an introduction to the Stein method framework in the context of steady-state diffusion approximations. The framework consists of three components: the Poisson equation and gradient bounds, generator coupling, and moment bounds. Working in the setting of the Erlang-A and Erlang-C models, we prove that both Wasserstein and Kolmogorov distances between the stationary distribution of a normalized customer count process, and that of an appropriately defined diffusion process decrease at a rate of $1/\sqrt{R}$, where $R$ is the offered load. Futhermore, these error bounds are \emph{universal}, valid in any load condition from lightly loaded to heavily loaded.
△ Less
Submitted 17 February, 2017; v1 submitted 31 December, 2015;
originally announced December 2015.
-
Heavy traffic approximation for the stationary distribution of a generalized Jackson network: the BAR approach
Authors:
Anton Braverman,
J. G. Dai,
Masakiyo Miyazawa
Abstract:
In the seminal paper of Gamarnik and Zeevi (2006), the authors justify the steady-state diffusion approximation of a generalized Jackson network (GJN) in heavy traffic. Their approach involves the so-called limit interchange argument, which has since become a popular tool employed by many others who study diffusion approximations. In this paper we illustrate a novel approach by using it to justify…
▽ More
In the seminal paper of Gamarnik and Zeevi (2006), the authors justify the steady-state diffusion approximation of a generalized Jackson network (GJN) in heavy traffic. Their approach involves the so-called limit interchange argument, which has since become a popular tool employed by many others who study diffusion approximations. In this paper we illustrate a novel approach by using it to justify the steady-state approximation of a GJN in heavy traffic. Our approach involves working directly with the basic adjoint relationship (BAR), an integral equation that characterizes the stationary distribution of a Markov process. As we will show, the BAR approach is a more natural choice than the limit interchange approach for justifying steady-state approximations, and can potentially be applied to the study of other stochastic processing networks such as multiclass queueing networks.
△ Less
Submitted 27 June, 2017; v1 submitted 5 October, 2015;
originally announced October 2015.
-
Technical Note for Discrete-Time Diffusion Approximations Motivated from Hospital Inpatient Flow Management
Authors:
J. G. Dai,
Pengyi Shi
Abstract:
This note details the development of a discrete-time diffusion process to approximate the midnight customer count process in a $M_\textrm{per}/\textrm{Geo}_\textrm{2timeScale}/N$ system. We prove a limit theorem that supports this diffusion approximation, and discuss two methods to compute the stationary distribution of this discrete-time diffusion process.
This note details the development of a discrete-time diffusion process to approximate the midnight customer count process in a $M_\textrm{per}/\textrm{Geo}_\textrm{2timeScale}/N$ system. We prove a limit theorem that supports this diffusion approximation, and discuss two methods to compute the stationary distribution of this discrete-time diffusion process.
△ Less
Submitted 20 August, 2015;
originally announced August 2015.
-
Stein's method for steady-state diffusion approximations of $M/Ph/n+M$ systems
Authors:
Anton Braverman,
J. G. Dai
Abstract:
We consider $M/Ph/n+M$ queueing systems in steady state. We prove that the Wasserstein distance between the stationary distribution of the normalized system size process and that of a piecewise Ornstein-Uhlenbeck (OU) process is bounded by $C/\sqrtλ$, where the constant $C$ is independent of the arrival rate $λ$ and the number of servers $n$ as long as they are in the Halfin-Whitt parameter regime…
▽ More
We consider $M/Ph/n+M$ queueing systems in steady state. We prove that the Wasserstein distance between the stationary distribution of the normalized system size process and that of a piecewise Ornstein-Uhlenbeck (OU) process is bounded by $C/\sqrtλ$, where the constant $C$ is independent of the arrival rate $λ$ and the number of servers $n$ as long as they are in the Halfin-Whitt parameter regime. For each integer $m>0$, we also establish a similar bound for the difference of the $m$th steady-state moments. For the proofs, we develop a modular framework that is based on Stein's method. The framework has three components: Poisson equation, generator coupling, and state space collapse. The framework, with further refinement, is likely applicable to steady-state diffusion approximations for other stochastic systems.
△ Less
Submitted 30 November, 2015; v1 submitted 2 March, 2015;
originally announced March 2015.
-
A multi-dimensional SRBM: Geometric views of its product form stationary distribution
Authors:
J. G. Dai,
Masakiyo Miyazawa,
Jian Wu
Abstract:
We present a geometric interpretation of a product form stationary distribution for a $d$-dimensional semimartingale reflecting Brownian motion (SRBM) that lives in the nonnegative orthant. The $d$-dimensional SRBM data can be equivalently specified by $d+1$ geometric objects: an ellipse and $d$ rays. Using these geometric objects, we establish necessary and sufficient conditions for characterizin…
▽ More
We present a geometric interpretation of a product form stationary distribution for a $d$-dimensional semimartingale reflecting Brownian motion (SRBM) that lives in the nonnegative orthant. The $d$-dimensional SRBM data can be equivalently specified by $d+1$ geometric objects: an ellipse and $d$ rays. Using these geometric objects, we establish necessary and sufficient conditions for characterizing product form stationary distribution. The key idea in the characterization is that we decompose the $d$-dimensional problem to $\frac{1}{2}d(d-1)$ two-dimensional SRBMs, each of which is determined by an ellipse and two rays. This characterization contrasts with the algebraic condition of [14]. A $d$-station tandem queue example is presented to illustrate how the product form can be obtained using our characterization. Drawing the two-dimensional results in [1,7], we discuss potential optimal paths for a variational problem associated with the three-station tandem queue. Except Appendix D, the rest of this paper is almost identical to the QUESTA paper with the same title.
△ Less
Submitted 8 May, 2014; v1 submitted 5 December, 2013;
originally announced December 2013.
-
Decomposable stationary distribution of a multidimensional SRBM
Authors:
J. G. Dai,
Masakiyo Miyazawa,
Jian Wu
Abstract:
We call a multidimensional distribution to be decomposable with respect to a partition of two sets of coordinates if the original distribution is the product of the marginal distributions associated with these two sets. We focus on the stationary distribution of a multidimensional semimartingale reflecting Brownian motion (SRBM) on a nonnegative orthant. An SRBM is uniquely determined (in distribu…
▽ More
We call a multidimensional distribution to be decomposable with respect to a partition of two sets of coordinates if the original distribution is the product of the marginal distributions associated with these two sets. We focus on the stationary distribution of a multidimensional semimartingale reflecting Brownian motion (SRBM) on a nonnegative orthant. An SRBM is uniquely determined (in distribution) by its data that consists of a covariance matrix, a drift vector, and a reflection matrix. Assume that the stationary distribution of an SRBM exists. We first characterize two marginal distributions under the decomposability assumption. We prove that they are the stationary distributions of some lower dimensional SRBMs. We also identify the data for these lower dimensional SRBMs. Thus, under the decomposability assumption, we can obtain the stationary distribution of the original SRBM by computing those of the lower dimensional ones. However, this characterization of the marginal distributions is not sufficient for the decomposability. So, we next consider necessary and sufficient conditions for the decomposability. We obtain those conditions for several classes of SRBMs. These classes include SRBMs arising from Brownian models of queueing networks that have two sets of stations with feed-forward routing between these two sets. This work is motivated by applications of SRBMs and geometric interpretations of the product form stationary distributions.
△ Less
Submitted 29 November, 2014; v1 submitted 4 December, 2013;
originally announced December 2013.
-
Validity of heavy-traffic steady-state approximations in many-server queues with abandonment
Authors:
J. G. Dai,
A. B. Dieker,
Xuefeng Gao
Abstract:
We consider GI/Ph/n+M parallel-server systems with a renewal arrival process, a phase-type service time distribution, n homogenous servers, and an exponential patience time distribution with positive rate. We show that in the Halfin-Whitt regime, the sequence of stationary distributions corresponding to the normalized state processes is tight. As a consequence, we establish an interchange of heavy…
▽ More
We consider GI/Ph/n+M parallel-server systems with a renewal arrival process, a phase-type service time distribution, n homogenous servers, and an exponential patience time distribution with positive rate. We show that in the Halfin-Whitt regime, the sequence of stationary distributions corresponding to the normalized state processes is tight. As a consequence, we establish an interchange of heavy traffic and steady state limits for GI/Ph/n+M queues.
△ Less
Submitted 12 January, 2014; v1 submitted 22 June, 2013;
originally announced June 2013.
-
Optimal Control of Brownian Inventory Models with Convex Inventory Cost: Discounted Cost Case
Authors:
Jim Dai,
Dacheng Yao
Abstract:
We consider an inventory system in which inventory level fluctuates as a Brownian motion in the absence of control. The inventory continuously accumulates cost at a rate that is a general convex function of the inventory level, which can be negative when there is a backlog. At any time, the inventory level can be adjusted by a positive or negative amount, which incurs a fixed positive cost and a p…
▽ More
We consider an inventory system in which inventory level fluctuates as a Brownian motion in the absence of control. The inventory continuously accumulates cost at a rate that is a general convex function of the inventory level, which can be negative when there is a backlog. At any time, the inventory level can be adjusted by a positive or negative amount, which incurs a fixed positive cost and a proportional cost. The challenge is to find an adjustment policy that balances the inventory cost and adjustment cost to minimize the expected total discounted cost. We provide a tutorial on using a three-step lower-bound approach to solving the optimal control problem under a discounted cost criterion. In addition, we prove that a four-parameter control band policy is optimal among all feasible policies. A key step is the constructive proof of the existence of a unique solution to the free boundary problem. The proof leads naturally to an algorithm to compute the four parameters of the optimal control band policy.
△ Less
Submitted 29 October, 2011;
originally announced October 2011.
-
Optimal Control of Brownian Inventory Models with Convex Holding Cost: Average Cost Case
Authors:
Jim Dai,
Dacheng Yao
Abstract:
We consider an inventory system in which inventory level fluctuates as a Brownian motion in the absence of control. The inventory continuously accumulates cost at a rate that is a general convex function of the inventory level, which can be negative when there is a backlog. At any time, the inventory level can be adjusted by a positive or negative amount, which incurs a fixed cost and a proportion…
▽ More
We consider an inventory system in which inventory level fluctuates as a Brownian motion in the absence of control. The inventory continuously accumulates cost at a rate that is a general convex function of the inventory level, which can be negative when there is a backlog. At any time, the inventory level can be adjusted by a positive or negative amount, which incurs a fixed cost and a proportional cost. The challenge is to find an adjustment policy that balances the holding cost and adjustment cost to minimize the long-run average cost. When both upward and downward fixed costs are positive, our model is an impulse control problem. When both fixed costs are zero, our model is a singular or instantaneous control problem. For the impulse control problem, we prove that a four-parameter control band policy is optimal among all feasible policies. For the singular control problem, we prove that a two-parameter control band policy is optimal.
We use a lower-bound approach, widely known as "the verification theorem", to prove the optimality of a control band policy for both the impulse and singular control problems. Our major contribution is to prove the existence of a "smooth" solution to the free boundary problem under some mild assumptions on the holding cost function. The existence proof leads naturally to numerical algorithms to compute the optimal control band parameters. We demonstrate that the lower-bound approach also works for Brownian inventory model in which no inventory backlog is allowed. In a companion paper, we will show how the lower-bound approach can be adapted to study a Brownian inventory model under a discounted cost criterion.
△ Less
Submitted 15 October, 2011; v1 submitted 12 October, 2011;
originally announced October 2011.
-
Stationary distribution of a two-dimensional SRBM: geometric views and boundary measures
Authors:
Jim G. Dai,
Masakiyo Miyazawa
Abstract:
We present three sets of results for the stationary distribution of a two-dimensional semimartingale reflecting Brownian motion (SRBM) that lives in the nonnegative quadrant. The SRBM data can equivalently be specified by three geometric objects, an ellipse and two lines, in the two-dimensional Euclidean space. First, we revisit the variational problem (VP) associated with the SRBM. Building on Av…
▽ More
We present three sets of results for the stationary distribution of a two-dimensional semimartingale reflecting Brownian motion (SRBM) that lives in the nonnegative quadrant. The SRBM data can equivalently be specified by three geometric objects, an ellipse and two lines, in the two-dimensional Euclidean space. First, we revisit the variational problem (VP) associated with the SRBM. Building on Avram, Dai and Hasenbein (2001), we show that the value of the VP at a point in the quadrant is equal to the optimal value of a linear function over a convex domain. Depending on the location of the point, the convex domain is either D(1) or D(2) or D(1) cap D(2), where each D(i), i = 1, 2, can easily be described by the three geometric objects. Our results provide a geometric interpretation for the value function of the VP and allow one to see geometrically when one edge of the quadrant has influence on the optimal path traveling from the origin to a destination point. Second, we provide a geometric condition that characterizes the existence of a product form stationary distribution. Third, we establish exact tail asymptotics of two boundary measures that are associated with the stationary distribution; a key step in our proof is to sharpen two asymptotic inversion lemmas in Dai and Miyazawa (2011) that allow one to infer the exact tail asymptotic of a boundary measure from the singularity of its moment generating function.
△ Less
Submitted 3 December, 2012; v1 submitted 9 October, 2011;
originally announced October 2011.
-
Many-server queues with customer abandonment: numerical analysis of their diffusion models
Authors:
Shuangchi He,
J. G. Dai
Abstract:
We use multidimensional diffusion processes to approximate the dynamics of a queue served by many parallel servers. The queue is served in the first-in-first-out (FIFO) order and the customers waiting in queue may abandon the system without service. Two diffusion models are proposed in this paper. They differ in how the patience time distribution is built into them. The first diffusion model uses…
▽ More
We use multidimensional diffusion processes to approximate the dynamics of a queue served by many parallel servers. The queue is served in the first-in-first-out (FIFO) order and the customers waiting in queue may abandon the system without service. Two diffusion models are proposed in this paper. They differ in how the patience time distribution is built into them. The first diffusion model uses the patience time density at zero and the second one uses the entire patience time distribution. To analyze these diffusion models, we develop a numerical algorithm for computing the stationary distribution of such a diffusion process. A crucial part of the algorithm is to choose an appropriate reference density. Using a conjecture on the tail behavior of a limit queue length process, we propose a systematic approach to constructing a reference density. With the proposed reference density, the algorithm is shown to converge quickly in numerical experiments. These experiments also show that the diffusion models are good approximations for many-server queues, sometimes for queues with as few as twenty servers.
△ Less
Submitted 7 April, 2011; v1 submitted 2 April, 2011;
originally announced April 2011.
-
Many-server diffusion limits for $G/Ph/n+GI$ queues
Authors:
J. G. Dai,
Shuangchi He,
Tolga Tezcan
Abstract:
This paper studies many-server limits for multi-server queues that have a phase-type service time distribution and allow for customer abandonment. The first set of limit theorems is for critically loaded $G/Ph/n+GI$ queues, where the patience times are independent and identically distributed following a general distribution. The next limit theorem is for overloaded $G/ Ph/n+M$ queues, where the pa…
▽ More
This paper studies many-server limits for multi-server queues that have a phase-type service time distribution and allow for customer abandonment. The first set of limit theorems is for critically loaded $G/Ph/n+GI$ queues, where the patience times are independent and identically distributed following a general distribution. The next limit theorem is for overloaded $G/ Ph/n+M$ queues, where the patience time distribution is restricted to be exponential. We prove that a pair of diffusion-scaled total-customer-count and server-allocation processes, properly centered, converges in distribution to a continuous Markov process as the number of servers $n$ goes to infinity. In the overloaded case, the limit is a multi-dimensional diffusion process, and in the critically loaded case, the limit is a simple transformation of a diffusion process. When the queues are critically loaded, our diffusion limit generalizes the result by Puhalskii and Reiman (2000) for $GI/Ph/n$ queues without customer abandonment. When the queues are overloaded, the diffusion limit provides a refinement to a fluid limit and it generalizes a result by Whitt (2004) for $M/M/n/+M$ queues with an exponential service time distribution. The proof techniques employed in this paper are innovative. First, a perturbed system is shown to be equivalent to the original system. Next, two maps are employed in both fluid and diffusion scalings. These maps allow one to prove the limit theorems by applying the standard continuous-mapping theorem and the standard random-time-change theorem.
△ Less
Submitted 9 November, 2010;
originally announced November 2010.
-
Positive recurrence of reflecting Brownian motion in three dimensions
Authors:
Maury Bramson,
J. G. Dai,
J. M. Harrison
Abstract:
Consider a semimartingale reflecting Brownian motion (SRBM) $Z$ whose state space is the $d$-dimensional nonnegative orthant. The data for such a process are a drift vector $θ$, a nonsingular $d\times d$ covariance matrix $Σ$, and a $d\times d$ reflection matrix $R$ that specifies the boundary behavior of $Z$. We say that $Z$ is positive recurrent, or stable, if the expected time to hit an arbitra…
▽ More
Consider a semimartingale reflecting Brownian motion (SRBM) $Z$ whose state space is the $d$-dimensional nonnegative orthant. The data for such a process are a drift vector $θ$, a nonsingular $d\times d$ covariance matrix $Σ$, and a $d\times d$ reflection matrix $R$ that specifies the boundary behavior of $Z$. We say that $Z$ is positive recurrent, or stable, if the expected time to hit an arbitrary open neighborhood of the origin is finite for every starting state. In dimension $d=2$, necessary and sufficient conditions for stability are known, but fundamentally new phenomena arise in higher dimensions. Building on prior work by El Kharroubi, Ben Tahar and Yaacoubi [Stochastics Stochastics Rep. 68 (2000) 229--253, Math. Methods Oper. Res. 56 (2002) 243--258], we provide necessary and sufficient conditions for stability of SRBMs in three dimensions; to verify or refute these conditions is a simple computational task. As a byproduct, we find that the fluid-based criterion of Dupuis and Williams [Ann. Probab. 22 (1994) 680--702] is not only sufficient but also necessary for stability of SRBMs in three dimensions. That is, an SRBM in three dimensions is positive recurrent if and only if every path of the associated fluid model is attracted to the origin. The problem of recurrence classification for SRBMs in four and higher dimensions remains open.
△ Less
Submitted 28 September, 2010;
originally announced September 2010.
-
Diffusion limits of limited processor sharing queues
Authors:
Jiheng Zhang,
J. G. Dai,
Bert Zwart
Abstract:
We consider a processor sharing queue where the number of jobs served at any time is limited to $K$, with the excess jobs waiting in a buffer. We use random counting measures on the positive axis to model this system. The limit of this measure-valued process is obtained under diffusion scaling and heavy traffic conditions. As a consequence, the limit of the system size process is proved to be a pi…
▽ More
We consider a processor sharing queue where the number of jobs served at any time is limited to $K$, with the excess jobs waiting in a buffer. We use random counting measures on the positive axis to model this system. The limit of this measure-valued process is obtained under diffusion scaling and heavy traffic conditions. As a consequence, the limit of the system size process is proved to be a piece-wise reflected Brownian motion.
△ Less
Submitted 8 April, 2011; v1 submitted 29 December, 2009;
originally announced December 2009.
-
Asymptotic optimality of maximum pressure policies in stochastic processing networks
Authors:
J. G. Dai,
Wuqin Lin
Abstract:
We consider a class of stochastic processing networks. Assume that the networks satisfy a complete resource pooling condition. We prove that each maximum pressure policy asymptotically minimizes the workload process in a stochastic processing network in heavy traffic. We also show that, under each quadratic holding cost structure, there is a maximum pressure policy that asymptotically minimizes…
▽ More
We consider a class of stochastic processing networks. Assume that the networks satisfy a complete resource pooling condition. We prove that each maximum pressure policy asymptotically minimizes the workload process in a stochastic processing network in heavy traffic. We also show that, under each quadratic holding cost structure, there is a maximum pressure policy that asymptotically minimizes the holding cost. A key to the optimality proofs is to prove a state space collapse result and a heavy traffic limit theorem for the network processes under a maximum pressure policy. We extend a framework of Bramson [Queueing Systems Theory Appl. 30 (1998) 89--148] and Williams [Queueing Systems Theory Appl. 30 (1998b) 5--25] from the multiclass queueing network setting to the stochastic processing network setting to prove the state space collapse result and the heavy traffic limit theorem. The extension can be adapted to other studies of stochastic processing networks.
△ Less
Submitted 16 January, 2009;
originally announced January 2009.
