Title:Proactive Serving Reduces User Delay Exponentially

Abstract:In online service systems, delay experienced by a user from the service request to the service completion is one of the most critical performance metrics. To improve user delay experience, in this paper, we investigate a novel aspect of system design: proactive serving, where the system can allocate its capacity proactively and pre-serve future requests based on user arrival predications. We investigate the fundamentals of proactive serving from a queueing theory perspective. We show that proactive serving reduces the average user delay exponentially (as a function of the prediction window size) for a wide range of queueing models, including M/M/1, M/M/k, and MAP/Geo/1. Furthermore, the delay reduction is robust to prediction errors. Our trace driven evaluation results also demonstrate the practical power of proactive serving : for example, under the YouTube data trace of 1000 different videos, user delay can be decreased by $50\%$ when the system can predict 100 seconds ahead. Our study applies to general online service models with prediction, and thus our results can be used to guide the design of proactive serving in practical online service systems.