You can manage bookmarks using lists, please log in to your user account for this.
Media type:
E-Article
Title:
Average optimality for Markov decision processes in borel spaces: a new condition and approach
Contributor:
Guo, Xianping;
Zhu, Quanxin
Published:
Cambridge University Press (CUP), 2006
Published in:
Journal of Applied Probability, 43 (2006) 2, Seite 318-334
Language:
English
DOI:
10.1239/jap/1152413725
ISSN:
0021-9002;
1475-6072
Origination:
Footnote:
Description:
<jats:p>In this paper we study discrete-time Markov decision processes with Borel state and action spaces. The criterion is to minimize average expected costs, and the costs may have <jats:italic>neither upper nor lower</jats:italic> bounds. We first provide <jats:italic>two</jats:italic> average optimality inequalities of opposing directions and give conditions for the existence of solutions to them. Then, using the two inequalities, we ensure the existence of an average optimal (deterministic) stationary policy under additional continuity-compactness assumptions. Our conditions are slightly <jats:italic>weaker</jats:italic> than those in the previous literature. Also, some <jats:italic>new</jats:italic> sufficient conditions for the existence of an average optimal stationary policy are imposed on the <jats:italic>primitive data</jats:italic> of the model. Moreover, our approach is slightly different from the well-known ‘optimality inequality approach’ widely used in Markov decision processes. Finally, we illustrate our results in two examples.</jats:p>