Fluid Approximations of Markov Decision Chains

A.S. Gajrat, A. Hordijk, V.A. Malyshev, F.M. Spieksma

1997, v.3, Issue 1, 129-150


Euler limits or fluid approximations have proved useful in obtaining ergodicity conditions and more detailed information on random walks and communication networks. Until now the similar question for controlled Markov chains remained open. For a basic example we establish convergence of Markov decision chains to their fluid approximation. This allows to explicitly obtain the asymptotically optimal solution to our control problem.

Keywords: asymptotic optimality,fluid model,controlled Markov chains,random walk


Please log in or register to leave a comment

There are no comments yet