Qiying Hu, Jianyong Liu, Wuyi Yue
College of International Business and Management
Shanghai University
Shanghai 201800, P.R. CHINA
e-mail: qyhu@mail.shu.edu.cn
Institute of Applied Mathematics
Academia Sinica
Beijing 100080, P.R. CHINA
Department of Information Science and Systems Engineering
Faculty of Science and Engineering
Konan University
8-9-1 Okamoto, Higashinada-ku, Kobe 658-8501, JAPAN
e-mail: yue@konan-u.ac.jp

Abstract.This paper discusses a set of necessary conditions for continuous time Markov decision processes with criterion of expected discounted total rewards, where the state space is countable, the reward rate function is extended real-valued and the discount rate is any real number. Under necessary conditions that the model is well defined, the state space is partitioned into three subsets, on which the optimal value function is positive infinity, negative infinity, or finite, respectively. Correspondingly, the model is reduced into three submodels, by generalizing policies and eliminating some worst actions. Then for the submodel with finite optimal value, the validity of the optimality equation is shown and some its properties are obtained.

Received: February 5, 2003

AMS Subject Classification: 26A33

Key Words and Phrases: continuous time Markov decision processes, expected total rewards, model decomposition, necessary conditions

Source: International Journal of Pure and Applied Mathematics
ISSN: 1311-8080
Year: 2003
Volume: 7
Issue: 2