A controlled discrete-time queueing system as a model for the orders of two competing companies

We consider two companies that are competing for orders. Let \(X_1(n)\) denote the number of orders processed by the first company at time \(n\), and let \(\tau(k)\) be the first time that \(X_1(n) < j\) or \(X_1(n) = r\), given that \(X_1(0) = k\). We assume that \(\{X_1(n), n=0,1,\ldots\}\) is a controlled discrete-time queueing system. Each company is using some control to increase its share of orders. The aim of the first company is to maximize the expected value of \(\tau(k)\), while its competitor tries to minimize this expected value. The optimal solution is obtained by making use of dynamic programming. Particular problems are solved explicitly.

Uncontrolled Keywords

dynamic programming; difference equations; linear equations; first-passage time; homing problem