Markov Chains:
Models, Algorithms and Applications
THE UNIVERSITY OF HONG KONG
FACULTY OF SCIENCE
香港大學理學院
Professor Ching’s research interests:
 Operations Research & Modeling
 Scientific Computing & Bioinformatics
 Quantitative Finance & Risk Management
Professor Ching Wai‐Ki
Department of Mathematics
Background.
Markov chains model a
sequence of random variables, which
correspond to the states of a certain system in
such a way that the state at one time depends
only on the state in the previous time. The
concept and idea of a Markov chain is simple
and easy to understand, and it has a lot of
applications in many areas such as economics,
finance,
operations
research,
and
bioinformatics etc.
There are many different types of Markov
chains: hidden Markov chains, high‐order
Markov chains, multivariate Markov chains
etc. The mathematical contents and structures
in Markov chain theory are rich for one to
investigate. The main direction is to generalize
the existing results to high‐dimensional space.
The key practical issues are (i) to construct an
appropriate and effective model for the
underlying problem; (ii) to develop efficient
estimation methods for solving the model
parameters; and (iii) to design fast numerical
algorithms for computing the required system
solution of huge size.
Research Findings. An interactive Hidden
Markov Model (HMM) was proposed for
extracting hidden economic state from credit
default data, see Figure 1. The research is
useful for the identification of market risk.
A Markovian regime‐switching model, taking
account of market impacts, was proposed for
trading in a Limit Order Book (LOB) market.
The model is then adopted for evaluating
different trading strategies, see Figure 2.
A multivariate Markov chain model was
developed to capture the correlations of
multiple data sequences (sales). A better
prediction accuracy for product sales volume
can be achieved, see Figure 3.
x1
x2
We are in the era of Big Data,
sophisticated Markov chain models with efficient
computational techniques can be developed for
classification and clustering of huge datasets
(clinical data, financial data, internet data etc.).
The results can then be applied to the prediction
of diseases, market risk, and consumer behavior
etc.
Figure 1. Defaults and Hidden Economic States
The aging population is rising quickly in our society
and therefore the health costs are getting more
expensive. Markov decision process is a useful
mathematical tool to study problems arising from
health care informatics and medical decision‐
making.
Product A
108
Product B
5
5
4
4
3
3
2
2
106
104
1
102
50
100
150
200
250
1
100
Product C
200
300
Product D
5
5
4
4
3
3
100
stock price
ask price
bid price
average of bid and ask price
98
96
0
20
40
60
80
2
Figure 2. Trading in Limit Order Book (LOB)
Various kinds of mathematical formalisms and
models have been proposed for understanding
dynamical behavior of genetic networks. Among
them, Boolean network (BN) is a simple and
popular model. BN is a deterministic model,
biological systems usually contain intrinsic
stochasticity and experimental data include
noise. Therefore, BN was then extended to
Probabilistic Boolean Network (PBN), a stochastic
model. PBNs can be studied by using Markov
chain theory, see Figure 4.
f1(2) x 2 (t+1) = x 1 (t)
f2(2) x 2 (t+1) = x 3 (t)
x 3 (t)
f1(3) x 3 (t+1) = x 1 (t)
x 2 (t)
Prob.
0.8
0.2
0.7
0.3
Figure 4. An Example of Probabilistic Boolean Network
1.0
2
100
1
Boolean Function
f1(1) x 1 (t+1) = x 2 (t)
f2(1) x 1 (t+1) = x 2 (t)
x3
Research Plans.
[insert picture/graph/
table]
20
40
60
80 100 120 140
1
50
100
150
200
250
Figure 3. Sales Volumes of Four Products
Research Links.
Advanced Modeling and Applied Computing
Laboratory
http://hkumath.hku.hk/~wkc/amacl.htm
HKU strategic research theme: Computation and
Information
http://www.rss.hku.hk/strategic‐research/
International Consortium for Optimization and
Modelling in Science and Industry
http://www.icomsi.org/
Further Reading:
W. Ching, X. Huang, M. Ng and T. Siu,
Markov Chains : Models, Algorithms and
Applications,
International
Series
on
Operations Research and Management
Science, (2nd Edition) Springer, New York,
2013 (240 Pages).