Report production coordinator: Claudia Chui, [email protected], +852 3411 7079
Report Web Site: http://www.comp.hkbu.edu.hk/tech-report
Solving Large-Scale POMDP Problems Via
Belief State Analysis
Xin Li, William K. Cheung, Jiming Liu
COMP-05-006
Release Date: June 18, 2005
ISSN: 1810-1054 ISBN: 962-85415-1-X
Department of Computer Science, Hong Kong Baptist University
Abstract
While partially observable Markov decision process (POMDP) is a powerful tool for supporting
optimal decision making in stochastic environments, solving large-scale POMDPs is known to be
computationally intractable. Belief compression by reducing the POMDP's belief state dimension has
recently been shown to be effective in tackling the scale-up issue. This paper proposes to enhance
belief compression by performing a dimension reduction oriented clustering prior to the compression.
With the conjecture that the temporally close belief states should possess a low degree of freedom due
to the problem's instrinsic regularity, a spatio-temporal criterion function that measures belief states'
spatial and temporal discrepancies is adopted to control the belief clustering. Any further reduction in
the belief state dimension can then result in a more effcient POMDP solver. We have rigoriously
tested the proposed belief clustering method using a synthesized navigation problem (Hallway2) and
empirically found it to be more effective when compared with the standard belief compression
method.
Report Cover Designer: So Kin Ming