IJPAM: Volume 40, No. 3 (2007)

SUBOPTIMALITY OF THE VALUE ITERATION POLICIES
IN DISCOUNTED LINEAR-QUADRATIC MODELS

Raúl Montes-de-Oca$^1$, Daniel Cruz-Suárez$^2$
$^1$Departamento de Matemáticas
Universidad Autónoma Metropolitana-Iztapalapa
186 San Rafael Atlixco Avenue
Vicentina, Mexico, D.F., 09340, MEXICO
e-mail: [email protected]
$^2$División Académica de Ciencias Básicas
Universidad Juárez Autónoma de Tabasco
P.O. Box 5, Cunduacán, Tabasco, 86690, MEXICO
e-mail: [email protected]


Abstract.This paper deals with one-dimensional linear-quadratic (LQ) models which have been established as discounted Markov decision processes. These models have general and known coefficients. For each LQ-model taken into account, the existence of an optimal policy $f^{*}$ is assumed. Conditions which permit to obtain, for each compact set of states $\varsigma$, and for each $n=1,2,\cdots$, the suboptimality on $\varsigma$ of the value iteration policy $f_{n}$ are given. This suboptimality consists in obtaining the range of the values of $\varepsilon$, such that $f_{n}$ is a uniform on $\varsigma$ $\varepsilon$-approximation to $f^{*}$.

Received: August 31, 2007

AMS Subject Classification: 90C40, 93E20

Key Words and Phrases: discounted linear-quadratic model, optimal policy, value iteration policy, suboptimality of the value iteration policy

Source: International Journal of Pure and Applied Mathematics
ISSN: 1311-8080
Year: 2007
Volume: 40
Issue: 3