Fluid Approximations of Markov Decision Chains
1997, v.3, Issue 1, 129-150
Euler limits or fluid approximations have proved useful in obtaining ergodicity conditions and more detailed information on random walks and communication networks. Until now the similar question for controlled Markov chains remained open. For a basic example we establish convergence of Markov decision chains to their fluid approximation. This allows to explicitly obtain the asymptotically optimal solution to our control problem.
Keywords: asymptotic optimality,fluid model,controlled Markov chains,random walk