Lahore University of Management Sciences
Math-4314/Math-539
Stochastic Process-I/ Advanced Stochastic Process-I
Fall Semester 2017-2018
Instructor
Room No.
Office Hours
Email
Telephone
Secretary/TA
TA Office Hours
Course URL (if any)
Azmat Hussain
9-135
TBA
[email protected]
35608228
Shazia Zafar & Noreen Sohail /
TBA
Course Basics
Credit Hours
Lecture(s)
Recitation/Lab (per week)
Tutorial (per week)
Course Distribution
Core
Elective
Open for Student Category
Close for Student Category
3
Nbr of Lec(s) Per Week
Nbr of Lec(s) Per Week
Nbr of Lec(s) Per Week
2
Duration
Duration
Duration
75
Open for all
COURSE DESCRIPTION
This course is an introduction to stochastic models with a focus on applications in operations research, engineering and finance. This course can
also serve as a sequel to Math 230 (Probability). In an introductory course to probability (such as Math 230) students learn about random
variables (one or two) and their probability distributions. This subsequent course focuses on discrete and continuous time stochastic processes
(collection of random variables).
Stochastic (process) models are useful and have applications in a number of areas including engineering, operations research, finance, physics,
biology and industrial engineering.
COURSE PREREQUISITE(S)



Math 230 (Probability) and Math 120 (Linear Algebra with Differential Equations)
COURSE OBJECTIVES



-Understand theory of discrete and continuous time Markov chains
-Be able to formulate and do analysis of models in engineering, operations research and finance.
Lahore University of Management Sciences
Learning Outcomes

Grading Breakup and Policy
Assignment(s): 25% (5-8) (lowest score will be dropped)
Home Work:
Quiz(s):
Class Participation:
Attendance:
Midterm Examination: 30%
Project: 15%
Students will work on a project or a research problem (individually or by a group of two students). Students are expected to
use/apply stochastic models to a real world problem (e.g. PDC, bank, highway, post office, gym, airport etc.) in their fields of
interest (engineering, operations research, finance, economics, biology, physics etc.) or work on a research project (problem).
Explain why your model is appropriate; propose methods to help improve the efficiency of system in your proposed project; and
conduct some analysis (numerical or analytic). Instructor will give guidelines and help in selection and completion of the project.
Final Examination: 30%
Final grade:
Variables: ASSIGNS ≡ Assignments, P ≡ Project, M ≡ Midterm Exam, F ≡ Final Exam, FG ≡ Final Grade
FG ≡ ASSIGNS× 25% + P × 15% +M × 30% + F × 30%.
Examination Detail
Midterm
Exam
Final Exam
Yes/No:
Combine Separate:
Duration: 3 hours
Preferred Date:
Exam Specifications:
Students will be allowed to bring one page of double-sided formula sheet and another help sheet will be provided by the
instructor.
Yes/No:
Combine Separate:
Duration: 3 hours
Exam Specifications:
Students will be allowed to bring one page of double-sided formula sheet and another help sheet will be provided by the
instructor.
COURSE OVERVIEW
Week/
Lecture/
Module
Topics
Review on Probability Theory
 Probability space
 Conditional probability and
Bayes’ formula
 Random variables: distribution
Recommended
Readings
Instructor will provide reading notes
and suggest relevant chapters from
the recommended texts.
Objectives/
Application
Lahore University of Management Sciences





functions, discrete and continuous
types
Random variables: expectation,
variance, covariance and moment
generating functions
Limit theorems: strong law of
large number (SLLN) and central
limit theorem (CLT)
Discrete-Time Markov Chain (DTMC)

The Markov property

Classification of states: transience
and recurrence

Chapman-Kolmogorov equations

Steady-state distributions

DTMCs with absorbing
states/classes: canonical forms,
fundamental matrices, and mean
times until absorption

Time reversibility, random walk
on a graph

Applications
Poisson Process (PP)

Exponential distribution: the lackof-memory property and its
applications

Poisson processes: definition

Properties of Poisson:
independent thinning and
superposition

Order statistics and conditional
distributions of the arrival times

Applications

Generalization 1:
Compound Poisson process (CPP)

Generalization 2:
Nonhomogeneous Poisson
process (NPP)

Applications
Continuous-Time Markov Chain
(CTMC)
 CTMC: definition, transition
probability and rate matrices
 Kolmogorov-Chapman equation
and Kolmogorov ODE
 Steady state
 Birth-and-death processes and
Markovian queueing networks
 Time reversibility
 Applications
Textbook(s)/Supplementary Readings
Instructor will provide reading notes
and suggest relevant chapters from
the recommended texts.
Instructor will provide reading notes
and suggest relevant chapters from
the recommended texts.
Instructor will provide reading notes
and suggest relevant chapters from
the recommended texts.
Lahore University of Management Sciences
-Ross, S. M. Introduction to Probability Models. 11th (10th) edition, Academic Press, Elsevier.
-Harchol-Balter, M. Performance Modeling and Design of Computer Systems: Queueing Theory in Action Cambridge University
Press, 2013.
-Karlin, Samuel and Taylor, Howard. A first course in stochastic processes. 2nd edition, Elsevier
1981
-Ross, S. A first course in probability. 8th Edition, Pearson. 2009.
-Bertsekas, Dimitri P and Tsitsiklis, John N. Introduction to probability. 2nd edition. Athena Scientific Belmont, MA.