HKUST Library Institutional Repository Banner

HKUST Institutional Repository >
Electronic and Computer Engineering  >
ECE Doctoral Theses >

Please use this identifier to cite or link to this item: http://hdl.handle.net/1783.1/7120
Title: Event-based optimization for the continuous-time Markov systems
Authors: Cao, Fang
Issue Date: 2008
Abstract: The main contribution of this thesis is two-fold. First, a sample-path-based construction of the sensitivity formulas for the continuous-time Markov systems is given in this thesis. The major complexity involved in the optimization of the continuous-time Markov systems is the mixed effect due to change of the transition rates, in addition to the change of the transition probabilities. By using the sample-path-based construction, we decompose this effect into two parts, each due to a type of perturbations which can be measured by the performance potentials. Second, by using the GSMP model, we provide a general framework of an event-driving optimization approach for the continuous-time Markov systems. Each event is defined as a subset of active events evolved in GSMP. It is associated with a lifetime, and may trigger a transition at the end of that. The control on the transition rates of states is decomposed into the control on the lifetime of different events, and the control on the transition probabilities of states is decomposed into the control on the transitions triggered by different events. Independent actions are applied to different events, instead of states. Potentials, aggregated with events, are used to construct two types of performance sensitivity formulas which lead to gradient-based optimizations and, with some conditions, event-based policy iterations. This approach provides a clear physical meaning and structure insights for the optimization of the continuous-time systems. The main results in this thesis can be applied to many problems that do not fit the standard MDP formulation, and therefore opens up new research directions for many practical problems.
Description: Thesis (Ph.D.)--Hong Kong University of Science and Technology, 2008
xv, 122 leaves : ill. ; 30 cm
HKUST Call Number: Thesis ECED 2008 Cao
URI: http://hdl.handle.net/1783.1/7120
Appears in Collections:ECE Doctoral Theses

Files in This Item:

File Description SizeFormat
th_redirect.html0KbHTMLView/Open

All items in this Repository are protected by copyright, with all rights reserved.