APPLIED STOCHASTIC PROCESSES
T.KOMOROWSKI
Topics
Part I. General Theory
1) Basic notions. Some measurability issues.
2) The law of a process. Modification of a process, version of a process. The existence
theorem of Kolmogorov. Processes on R∞ [0, +∞). Examples - Brownian motion
and Poisson processes. Moment statistics of a process. Gaussian processes existence.
3) Processes with continuous trajectories. The second theorem of Kolmogorov. The
space of continuous processes C[0, +∞). The end of a construction of a Brownian motion. Donsker’s invariance principle. A criterion for the continuity of a
Gaussian process. An Ornstein-Uhlenbeck process.
4) Processes with jumps. The Skorochod space D[0, +∞). Theorem of KolmogorovChentzov. The compound Poisson processes, jump processes. Random measures.
5) Stationary processes. Strictly and weakly stationary processes. Examples. Spectral measure. Shift operators. Ergodicity of a process. The ergodic theorem.
6) Stopping times. Progressive measurability, adaptability and predictability.
7) Martingales. Definition of a sub (super) martingale and martingale. Some intuitive explanations. Examples. Basic properties: Jensen inequalities, up-crossing,
down-crossings, L1 and a.s. convergence, optional sampling theorem. KolmogorovDoob inequalities. Square integrable martingales. Doob representation theorem.
Semi-martingales. Quadratic variation of a martingale. Martingale characterization of a Brownian motion.
Part II. Markov Processes
1) Basic notions, various formulations of Markov property, Chapman-Kolmogorov
equations, transition probability semigroups, path measures. Examples of Markov
processes
2) Markov chains, continuous time Markov chains, generator of a process, Kolmogorov equations, Dynkin formula, martingales associated with a process, DoobMejer decomposition, quadratic field operator
3) Feller property, strongly continuous transition probability semigroup. Processes
with locally compact state space, transfer semigroup, generator of a process. Processes with continuous and cadlag realizations. Backward, forward Kolmogorov
equations. Feynman-Kac formula.
4) Strong Markov property and its applications.
1
2
T.KOMOROWSKI
5) Processes with independent increments on Rd . Levy-Khinchin characterization,
generator of a Levy process.
6) Ergodic theory of processes. Stationary measure, invariant law, detailed balance
condition.
7) Markov chains: irreducibility, transience, recurrence, positive recurrence. The
cirteria of existence of invariant measure. Ergodic invariant measures. Markov
processes with compact and locally compact state spaces. Krylov-Bogolubov theorem. Uniqueness of invariant measure, strong Feller property
8) Stationary Markov processes, ergodic theorem. Ergodic theorem for Markov
chains.
9) Central limit theorem for stationary Markov processes whose generator has spectral gap property. Application to Markov chains.
STOCHASTIC PROCESSES
3
Part 1. General Theory of Stochastic Processes
1. What is a stochastic process?
A stochastic process is a collection of random variables {Xt , t ≥ 0} indexed by a nonnegative parameter. Typical examples:
• Xt - a position of a tagged particle in a heat bath of interacting molecules. When
the gas is in equilibrium and no external force is present it is called a Brownian
motion,
• suppose we are counting the number {Nt , t ≥ 0} of incoming calls arriving at a
telephone switchboard. This process is called a Poisson process,
• a price of a stock {St , t ≥ 0} on a financial market. Also the derivative {Ct , t ≥ 0}
based on this asset. Could be also a price of a commodity, contact etc,
• workload {Wt , t ≥ 0} of a server working for a certain queueing system, the
numbers of clients in a system etc.
• a number of individuals in a population {Nt , t ≥ 0} - a birth and death process.
• an amount of precipitation per square meter {Nt , t ≥ 0}. Important if we wish
to built a retention container.
All these examples show that a stochastic process is a mathematical object that allows us
to model the situation when we have to work with an incomplete information and a degree
of uncertainty.
2. Definition and some basic problems
Suppose that (Ω, F) is a measurable space. By a stochastic family we understand a
family of random variables {Xt , t ∈ T }, i.e. measurable functions Xt ; Ω → R. When
T = N we call it a sequence of random variables, T = [0, +∞) (T = R) a stochastic
process, T = Rd a random field. Also it is possible that Xt are valued in other spaces than
(R, BR ).
Example. Suppose that {Xn , n ≥ 0} is a sequence of random variables then we can
form a process by letting
X
ξt :=
Xn tn a random power series,
n≥0
or
ξt :=
X
Xn eitn
a random trigonometric series.
n≥0
An important and non-trivial issue when dealing with stochastic processes are properties
of their trajectories. It is by no means certain that supt∈T Xt is even a random variable/
The problem does not exist for random sequences but the measurabiility notions do not
play well when we have to perform operations that are defined by an uncountable number
of operations (e.g. supremum).
Example. Suppose that τ : [0, +∞) → [0, +∞) is a non-measurable function on
([0, +∞), B) such taht τ ≥ 1] is not measurable. Let Xt = 0 for t < τ and Xt = 1
4
T.KOMOROWSKI
for t ≥ τ . Note that supt∈[0,1] Xt need not be measurable. Indeed, the event
A := [ sup Xt = 1] = [τ < 1]
t∈[0,1]
is not measurable.
Definition. A process is called measurable if X : [0, +∞) × Ω → R is a measurable
function with respect to the σ-algebra B ⊗ F.
Exercise 1. For a fixed ω the function t 7→ Xt (ω) (the trajectory of the process) is
measurable. This is clearly true for Xt := 1A (t)1B (ω) when A, B are resp. B and F
measurable. Suppose that L is a family of all sets such that this property holds. Note
that 1) Ω ∈ L, 2) if A, B ∈ L then A \ B ∈ L and 3) if An ⊂ An+1 , An ∈ L, P
then, A :=
S
n
A
∈
L.
Hence
L
⊃
B
⊗
F.
The
rest
goes
by
approximation,
i.e.
X
=
n
t
n≥1
k=1 ck 1Ak
has the required property for fixed ck and Ak ∈ B ⊗ F. Any process measurable process
can be obtained by approximation by such finite sums, i.e. any measurable process has the
required property.
We use here Dynkin’s Theorem on π − λ systems. A family P of sets is called a π-system
if: 1) Ω ∈ P, 2) if A, B ∈ P then A ∩ B ∈ P. A family L is called an λ-system if 1) Ω ∈ L,
2) if A, B ∈ L then A \ B ∈ L and 3) if An ⊂ An+1 , An ∈ L
durrett
Dynkin
Theorem 2.1. (Dynkin, see e.g. p. 22 of [4]) Suppose that P is a π system and L is an
λ-system such that L ⊃ P. Then σ(P) ⊂ L.
Exercise 2. The function YT (ω) := supt∈[0,T ] Xt (ω) is measurable for any T > 0.
Suppose that A is jointly B ⊗F measurable. Then, the theorem holds for supt∈[0,T ] 1A (t, ω).
P
One can show it for nk=1 ck 1Ak where ck are constants and Ak are jointly measurable, An
arbitrary bounded Xt can be approximated uniformly by finite sum processes, supremum
is a continuous functional in a uniform topology. For an unbounded Xt consider ((−M ) ∨
Xt ) ∧ M .
The history of a process (or its natural filtration) is the filtration of σ-algebras Ft :=
σ(Xs , s ≤ t).
Suppose that we are given a filtration {Ft , t ≥ 0}. The process is adapted w.r.t. this
filtration if Xt is Ft -measurable.
3. Stochastic processes over a probability space
Suppose that (Ω, F, P) is a probability space and {Xt , t ≥ 0}, {Yt , t ≥ 0} are stochastic
processes. When can we say that these processes are equivalent.
Definition. We say that {Xt , t ≥ 0} is an equivalent modification of {Yt , t ≥ 0} if
P[Xt = Yt , ∀ t ≥ 0] = 1.
Definition. We say that processes {Xt , t ≥ 0} and {Yt , t ≥ 0} are equivalent versions
if P[Xt = Yt ] = 1 for all t ≥ 0.
Exercise 1. A pair of equivalent processes with continuous trajectories are equivalent
versions of each other.
Exercise 2. Give an example of processes that are equivalent versions of each others
but such that they are not equivalent modifications.
STOCHASTIC PROCESSES
5
Definition. Suppose that 0 ≤ t1 ≤ . . . ≤ tn . The measure on (Rn , B) defined by
µt1 ,...,tn (A) := P[(Xt1 , . . . , Xtn ) ∈ A] is called a finite dimensional distribution of the process.
Denote by M({Xt , t ≥ 0}) the family of all finite dimensional distributions.
Definition. Suppose that {Xt , t ≥ 0} and {Yt , t ≥ 0} are processes given over probability spaces (Ω, F, P) and (Ω0 , F 0 , P0 ) respectively. The are called statisticaly equivalent, if
their respective finite dimensional distributions are the same.
Definition. The canonical path space. Let R[0, +∞) be the space of all functions
ω : [0, +∞) → R. Define πt : R[0, +∞) → R by πt (x) := x(t). It is equipped with
the weakest topology such that all πt are continuous. The corresponding Borel σ-algebra
coincides with the smallest σ-algebra σ(C) containing all cylindrical sets C, i.e. the sets of
the form Ct1 ,...,tn (A) := [(πt1 , . . . , πtn ) ∈ A]. This σ-algebra is called a cylindrical σ-algebra.
Exercise 1. Show that all cylindrical sets form an algebra.
Exercise 2. Show that the cylindrical σ-algebra coincides with the Borel σ-algebra.
A canonical map: X : Ω → R[0, +∞) is defined by X(ω)(t) := Xt (ω), t ≥ 0.
Exercise 3. Show that it is F/σ(C) measurable.
Definition. The law of the process. It is a probability measure µ over (R[0, +∞), σ(C))
given by µ(A) := P[ω : X(ω) ∈ A].
Question 1: Suppose that two processes have the same finite dimensional distributions.
Do they have to have the same laws?
Assume that µ is a probability measure on (R[0, +∞), σ(C)). For any 0 ≤ t1 ≤ . . . ≤ tn
define µ(A; t1 , . . . , tn ) := µ((πt1 , . . . , πtn ) ∈ A) a family of its finite dimensional distributions.
Question 2: Suppose that {νt1 ,...,tn } is a family of finite dimensional probability measures. When does there exist a probability measure µ on (R[0, +∞), σ(C)) such that its
finite dimensional distributions agree with {νt1 ,...,tn }?
An easy observation is the following. Suppose that T := {t1 , . . . , tn } ⊂ S and
S : s0,1 ≤ . . . ≤ s0,k0 ≤ t1 ≤ . . . ≤ tn ≤ sn+1,1 ≤ . . . ≤ sn+1,kn+1
Then, for x := (x01 , . . . , x0,k0 , y1 , . . . , yn , xn+1,1 , . . . , xn+1,kn+1 ) define ΠS,T (x) := (y1 , . . . , yn ).
We have the following consistency condition. Denote by F the family of all finite subsets
of [0, +∞). If the family {µ(·; T ), T ∈ F} consists of finite dimensional distributions of µ
then
µ(Π−1
∀ A ∈ Rn .
(3.1)
S,T (A); S) = µT (A),
Remarkably the converse of this holds.
kolmogorov
Theorem 3.1. (Kolmogorov) Suppose that we are given a family of finite dimensional
measures {νT , t ∈ F}, where each νT is a Borel measure on R#T and that it satisfies the
consistency condition (3.1). Then, there exists a unique measure µ on (R[0, +∞), σ(C))
such that µ(·; T ) = νT for all T ∈ F.
Before coming to the proof of this theorem we formulate some auxiliary facts needed.
We wish to consider µ on σ(C). The following theorem is crucial.
extension
Theorem 3.2. Assume that P is a finite additive measure on an algebra A of subsets of
X. A sufficient and necessary condition such that there exists a measure P̃ on σ(A) such
consistent
6
T.KOMOROWSKI
that P̃ (A) T
= P (A) for all A ∈ A is that for any sequence A1 ⊃ A2 ⊃ . . . of sets from A
satisfying n≥1 An = ∅ we have limn→+∞ P (An ) = 0. Moreover if P1 , P2 are two measures
on σ(A) such that P1 (A) = P2 (A) for all A ∈ A then P1 = P2 .
Proof on an exercise class.
asi-regular
Theorem 3.3. Suppose that (X, d) is a metric space and B(X) is the σ-algebra of Borel
sets. Assume also that µ is a Borel probability measure on X. Then for any > 0 and
a Borel set A there exist G ⊃ A ⊃ F such that G is open and F is closed such that
µ(G \ F ) < .
On exercises. Hint: Note that the topology T is a π-system and the family L of sets
satisfying the conclusion of the theorem is an λ-system containing T . Therefore B(X) ⊂ L.
Definition. A Borel probability measure µ on a topological space (X, T ) is called tight
if for any > 0 there exists a compact set K such that µ(K) ≥ 1 − .
tight
Theorem 3.4. Suppose that (X, d) is a complete separable (Polish) metric space. Every
Borel measure is then tight.
S
(k)
k
Proof. Suppose that {pi , i ≥ 1} is a dense subset of X and > 0. Let GN := N
n=1 B̄(pn , 1/2 ).
T
(k)
(k)
For a sufficiently large Nk we have µ(GNk ) ≥ 1 − /2k+1 . We claim that 1) K := k≥1 GNk
is compact and 2) µ(K) ≥ 1 − . To check compactness
Sm note that K is closed and for any
ρ > 0 there exists a finite set x1 , . . . , xm such that k=1 B(xk , ρ) ⊃ K. Part 2) is obvious,
since
! +∞
X (k) X \ (k)
µ [GNk ]c ≤
µ(K c ) = µ
[GNk ]c ≤
< .
2n+1
n=1
n≥1
k≥1
Definition. A Borel measure on a topological space is called Radon if for any set
A ∈ B(X) we have
µ(A) = sup[µ(K) : K ⊂ A, K compact ] = inf[µ(G) : G ⊃ A, G open].
radon
Corollary 3.5. Suppose that (X, d) is a Polish space. Then every Borel measure is Radon.
The proof of Theorem 3.1. Exercise. Prove that C is an algebra.
Define first µ(·) on C. Suppose that Ct1 ,...,tn (A) := 1A (πt1 , . . . , πtn ) ∈ C. Then we let
µ(Ct1 ,...,tn (A)) := νt1 ,...,tn (A).
The consistency condition implies that the definition is correct., i.e. if
Ct1 ,...,tn (A) = Cs1 ,...,sm (B) = C
then νt1 ,...,tn (A) = νs1 ,...,sm (B).
Exercise. We have 1) µ(A) ≥ 0, 2) µ(R[0, +∞)) = 1 and 3) µ(A ∪ B) = µ(A) + µ(B)
if A ∩ B = ∅ for all A, B ∈ C.
We would like to extend the definition of the measure to the entire σ(C). We use Theorem
4.1. Suppose that Cn := Ct1 ,...,tkn (An ), An × Rkn+1 −kn ⊃ An+1 and An ∈ B(Rkn ) for some
STOCHASTIC PROCESSES
7
strictly
increasing sequence of integers {kn , n ≥ 1}. Then Cn ⊃ Cn+1 . We assume that
T
n≥1 Cn = ∅. Suppose that nevertheless there exists ρ > 0 such that µ(Cn ) ≥ ρ for all
n ≥ 1. We wish to replace An by a compact set. From Corollary 3.5 there exists a compact
set Kn ⊂ An such that νt1 ,...,tkn (An \ Kn ) < ρ/2n+2 . Thus µ(Cn \ Dn ) < ρ/2n+2 , where
Dn := Ct1 ,...,tkn (Kn ). Taking
K̃n := [(x1 , . . . , xn ) : x1 ∈ K1 , (x1 , x2 ) ∈ K2 , . . . , (x1 , . . . , xn ) ∈ Kn ] =
n
\
(Kp × Rkn −kp )
p=1
we can assume that D̃n ⊃ D̃n+1 , where D̃n := Ct1 ,...,tkn (K̃n ). In addition,
!
!
n
n
n
[
[
X
µ(Cn \ D̃n ) ≤ µ
(Cn \ Dp ) ≤ µ
(Cp \ Dp ) ≤
µ (Cp \ Dp ) < ρ/2
p=1
p=1
p=1
T
and µ(D̃n ) ≥ ρ/2 for all n ≥ 1. We show that K := n≥1 D̃n 6= ∅. Indeed, let xn ∈ D̃n
for all n. Since D̃n ⊃ D̃m for m ≥ n we have πt1 ,...,tkn (xm ) ∈ K̃n for all m ≥ n. Since K̃1
(1)
(1)
is compact we can choose a subsequence {xn , n ≥ 1} so that πt1 ,...,tk1 (xn ) converges to
(2)
x∗1 in K̃1 . Out of this subsequence choose another one so that πt1 ,...,tk2 (xn ) converges to
(x∗1 , x∗2 ) in K̃2 etc. In this way we define a sequence {x∗n , n ≥ 1}. Note that any function
∗
∗
x ∈ R[0, +∞)
T satisfying x (tj ) := xi , j ∈ [ki−1 , ki ) belongs to K. Hence it is nonempty
and so is n≥1 Cn . The extension then follows from Theorem 4.1.
4. Discussion on the Kolmogorov Theorem
The theorem is very important because gives a sufficient and necessary condition for the
existence of a process, given its finite dimensional distributions. Yet at the same time it is
somewhat unsatisfactory, because the space supporting the law of the process is huge, it is
R[0, +∞), while the σ-algebra of cylindrical sets is somewhat poor.
Exercise. Show that A ∈ σ(C) iff there exists a countable set D = {ti , i ≥ 0} ⊂ [0, +∞)
and a family {Ai , i ≥ 0} of Borel subsets of R such that
A = [f : f (ti ) ∈ Ai ,
for all i ≥ 0].
As a corollary notice that e.g. the set of all continuous functions C[0, +∞) does not belong
to the cylindrical σ-algebra.
Sometime we wish to extend the definition to better spaces than R[0, +∞). A very
helpful device is the following result.
extension
Theorem 4.1. Suppose that µ is a probability measure on (R[0, +∞), σ(C)) and C ⊂
R[0, +∞) is such that
inf[µ(A) : A ⊃ C, A ∈ σ(C)] = 1.
Then, the family of all sets B = A ∩ C, A ∈ σ(C) forms a σ-algebra of subsets of C.
Moreover µ∗ (B) := µ(A) is well defined and is a probability measure on this σ-algebra.
8
T.KOMOROWSKI
Proof. Exercise.
Exercise. Show that in case C = C[0, +∞) the σ-algebra obtained in that way is the
Borel σ-algebra B on C[0, +∞). Note that the σ-algebra in question coincides with the
σ-algebra M generated by the sets of the form
[f ∈ C[0, +∞) : (πt1 (f ), . . . , πtn (f )) ∈ A],
(4.1)
where 0 ≤ t1 ≤ . . . ≤ tn , A ∈ B(Rn ). Hint: Suppose that B is a ball in C[0, T ] show that
B ∈ M (represent the ball as an intersection of a countable number of cylindrical sets).
Hence B ⊂ M. To prove that the converse holds consider the family F of sets of the form
(4.1). Each set is Borel, so F ⊂ B. The family F is a π-system, while B is a λ-system.
Hence M = σ(F) ⊂ B.
Another useful tool to extend properties that hold with probability one on a countable
and dense set of indices t to the entire path of a process is the notion of a separable process.
Before defining it we suppose that the probability space (Ω, F, P) is complete, i.e. if A ⊂ Ω
is such that for some B ∈ F we have P(B) = 0 and A ⊂ B then A ∈ F and P(A) = 0.
Definition. A process {Xt , t ≥ 0} over (Ω, F, P) is separable if there exists 1) a countable set D = {tj , j ≥ 0} and 2) an event N such that P[N ] = 0 such that for any a ≤ b
and an open set G we have
[ω : a ≤ Xt (ω) ≤ b,
∀ t ∈ G ∩ D] ⊂ [ω : a ≤ Xt (ω) ≤ b,
∀ t ∈ G] ∪ N.
Completeness of the probability space implies that the set [a ≤ Xt ≤ b,
event and of course we have
P[a ≤ Xt ≤ b,
∀ t ∈ G] = P[a ≤ Xt ≤ b,
∀ t ∈ G] is an
∀ t ∈ G ∩ D].
The following result holds.
doob
Theorem 4.2. (Doob, Theorem 2.4, p. 57 of [3]) Suppose that {Xt , t ≥ 0} is a stochastic
process over a complete probability space (Ω, F, P). Then there exists a separable process
{X̃t , t ≥ 0} defined over the same space and taking values in R ∪ {−∞, +∞} and such
that P[Xt = X̃t ] = 1 for each t.
Definition. A process is stochastically continuous at point t0 if for any σ > 0
lim P[|Xt − Xt0 | ≥ σ] = 0.
t→t0
As for the measurable version we have the following.
doob
Theorem 4.3. (Doob, Theorem 2.6, p. 61 of [3]) Suppose that {Xt , t ≥ 0} is a stochastic
process over a complete probability space (Ω, F, P) that is stochastically continuous on a
set Z of full Lebesgue measure. Then, there exists a separable and measurable process
{X̃t , t ≥ 0} defined over the same space and taking values in R ∪ {−∞, +∞} and such
that P[Xt = X̃t ] = 1 for each t.
eqt-2
STOCHASTIC PROCESSES
9
5. Gaussian processes
A random variable X is called Gaussian with mean µ and variance σ > 0 if its density
is of the form
1
(x − µ)2
f (x) = √
exp −
.
2σ 2
2πσ 2
In case σ = 0 we assume that X ≡ µ. The characteristic function of the random variable
equals
2 2
ϕ(ξ) = EeiXξ = eiµξ e−(1/2)σ ξ
A random vector X = (X1 , . . . , Xn ) is called Gaussian with the covariance matrix Σ =
[σij ] = E[Xi Xj ] and mean vector µ[µi ] if its characteristic function equals
ϕ(ξ) = EeiX·ξ = eiµ·ξ e−(1/2)ξ
T Σξ
,
∀ξ ∈ Rn .
Exercise. Matrix Σ is symmetric and non-negative definite.
In case Σ is positive definite the vector X has a density that equals
1
1
T −1
f (x) = p
exp − (x − µ) Σ (x − µ) .
2
(2π)n det Σ
Exercise. Suppose that (X, Y ) is Gaussian and Cov(X, Y ) = 0. Then X and Y are
independent. More generally. If (X1 , . . . , Xn ) and (Y1 , . . . , Yk ) are jointly Gaussians and
Cov(Xi , Yj ) = 0 for all i = 1, . . . , n, j = 1, . . . , k then they are independent.
Definition. Suppose that {Xt , t ≥ 0} is a stochastic process such that E|Xt | < +∞ for
all t ≥ 0. The mean function of the process is defined as m(t) := EXt .
Definition. Suppose that {Xt , t ≥ 0} is a stochastic process such that EXt2 < +∞ for
all t ≥ 0. The covariance function of the process is defined as R(t, s) := E[(Xt −m(t))(Xs −
m(s))].
Exercise. Show that the covariance function is:
1) symmetric, i.e. R(t, s) = R(s, t) for all t, s ≥ 0,
2) non-negative definite, i.e. for any t1 , . . . , tn ≥ 0, x1 , . . . , xn ∈ R we have
n
X
R(ti , tj )xi xj ≥ 0.
i,j=1
Definition. A stochastic process is called Gaussian if all its finite dimensional distributions are Gaussian.
-finite-dim
cor-uni
gauss-exist
Proposition 5.1. Suppose that we are give two Gaussian stochastic processes with the
same mean and covariance functions. Then all its finite dimensional distributions are
equal.
Corollary 5.2. Two Gaussian processes as in the above proposition have the same laws.
Proposition 5.3. Suppose that we are given an arbitrary function m : [0, +∞) → R and a
symmetric, non-negative definite function R : [0, +∞)2 → R. Then there exists a Gaussian
process {Xt , t ≥ 0} with m(t) and R(t, s) as its mean and covariance functions. Moreover,
any other such a process has to be statistically equivalent with {Xt , t ≥ 0}.
10
T.KOMOROWSKI
Proof. Exercise. Consider first m(t) ≡ 0. Check the consistency condition from Kolmogorov’s existence theorem. In general case let Yt := Xt + m(t), where Xt is the process
corresponding to m(t) ≡ 0. The uniqueness follows from corollary 5.2
Exercise.
Suppose that {Xt , t ≥ 0} is a Gaussian proces and a1 , . . . , an , b ∈ R. Show
P
that ni=1 ai Xti + b is a Gaussian random variable.
Definition. A standard Brownian motion (pre)-process is a Gaussian process such that
for any s, t ≥ 0 we have 1) B0 = 0, 2) EBt ≡ 0 and 3)
E(Bt − Bs )2 = |t − s| (Fick’s law).
Note that in that case m(t) ≡ 0 and R(t, s) = t ∧ s.
Exercise. Check that this function R(t, s) is positive definite.
Remark. We shall introduce later on the standard Brownian motion process as a standard Brownian motion (pre)-process whose trajectories are continuous a.s. The physical
phenomenon corresponding to this process called Brownian motion has been observed in
1827 by a Scottish botanist R. Brown. An appearance of such processes as the trajectory
of a stock price has been postulated by L. Batchelier (c.a. 1900). Its physical explanation
based on kinetic theory of gases has been discovered by A. Einstein and M. Smoluchowski
(c.a. 1905), while mathematical theory has been discovered by N. Wiener (in 1920-s). We
shall show that the trajectories of a Brownian motion are a.s. nowhere differentiable (don’t
even have a finite variation).
A Brownian motion pre-process with a drift µ and diffusivity σ ≥ 0 is a process Xt =
µ + σBt , where {Bt , t ≥ 0} is a standard Brownian motion.
Definition. A geometric Brownian motion with a trend µ and volatility σ ≥ 0 is a
process given by Yt := eXt , where Xt is a Brownian motion with a drift µ and diffusivity
σ.
Definition. A standard fractional Brownian motion (pre)-process with a Hurst exponent
H ∈ (0, 1] is a Gaussian process such that for any s, t ≥ 0 we have: 1) B0 = 0, 2) EBt ≡ 0
and 3)
E(Bt − Bs )2 = |t − s|2H .
Exercise. Note that in that case m(t) ≡ 0 and
1 2H
R(t, s) =
t + s2H − |t − s|2H .
2
For H = 1/2 we get R(t, s) = t ∧ s and the process coincides with the standard Brownian
motion.
Exercise. Prove that the function R(t, s) is symmetric and non-negative definite for
H ∈ (0, 1].
Hints about the proof. Step 1. Note that
n
X
R(ti , tj )xi xj = −
i,j=1
where t0 = 0 and x0 = −(x1 + . . . + xn ).
n
1X
|ti − tj |2H xi xj
2 i,j=0
STOCHASTIC PROCESSES
11
Step 2. We have
n
X
e
−c|ti −tj |2H
i,j=0
n
X
+
xi xj =
n h
X
i
− 1 |ti − tj |2H xi xj
(5.1)
|ti − tj |2H xi xj + o(c).
(5.2)
−c|ti −tj |2H
e
i,j=0
2H
|ti − tj |
xi xj = (1 − c)
i,j=0
n
X
i,j=0
P
2H
α
Step 3. To prove that ni,j=0 e−c|ti −tj | xi xj ≥ 0 use the fact that e−|ξ| is a characteristic
function of a stable random variable with the stability index α ∈ (0, 2]. Conclude from
that
n
X
|ti − tj |2H xi xj ≥ 0.
i,j=0
α
Remark. For α > 2 function e−|ξ| is not characteristic, so we do not have positive
definiteness of the function R(t, s), therefore we cannot construct a Gaussian process (or
any other for that matter) corresponding to R(t, s) for H > 1.
Remark. We shall show later on that there exists a process statistically equivalent
with the one introduced above and with continuous trajectories. This process shall be
called a fractional Brownian motion with the Hurst exponent H. It was introduced to the
literature by Kolmogorov in the 1940-s but investigated in details by Maldenbrot and Van
Ness in 1960-s. Also for an arbitrary µ and σ ≥ 0 we can introduce Xt = µ + σBt a general
fractional Brownian motion.
Gaussian stationary processes. A Gaussian process is called stationary
R iξxif m(t) =const
and R(t, s) = R(t − s). By Bochner’s theorem then we have R(x) = e R̂(dξ), where
R̂(·) is a measure with R̂(−A) = R̂(A) for all A ∈ B(R) and R̂(R) < +∞.
Example. An even function R(x) that is continuous and concave in [0, +∞) corresponds
to a covariance function
of some stationary Gaussian process. This follows from the Polya
feller2
criterion, see e.g. [5], p. 449.
How strange Gaussians can be? Example. Suppose that there exist h, c > 0 such
that R(0) − R(t) ≥ c/| log t| for |t| < h and some c > 0. We show that a.s. the sets
Na,b := {Xt , t ∈ (a, b)} are dense in R for any a < b. Consider a measurable function f
and define
mf (A) := m1 [t ∈ [a, b] : f (t) ∈ A],
where m1 (dt) := 1[a,b] dt is the Lebesgue measure on [a, b]. Define its Fourier transform
Z
m̂f (ξ) :=
e−iξt mf (dt).
R
If mf (dt) = g(t)dt then (inversion formula)
1
g(t) =
(2π)
Z
R
eiξt m̂f (ξ)dξ.
12
T.KOMOROWSKI
In fact g(z) is analytic for =z < α if
Z
eα|ξ| m̂f (ξ)dξ < +∞.
R
It suffices only to prove that
Z
eα|ξ| |m̂f (ξ)|2 dξ < +∞.
R
Indeed, then
Z
e
(α/4)|ξ|
Z
|m̂f (ξ)|dξ =
R
e(α/2)|ξ| |m̂f (ξ)|e−(α/4)|ξ| dξ
R
Z
≤
α|ξ|
e
1/2 Z
2
|m̂f (ξ)| dξ
R
−(α/2)|ξ|
e
1/2
dξ
< +∞.
R
We have
Z b
2 #
eαξ |m̂X (ξ)|2 dξ = E
eαξ e−iXt ξ dt dξ
(5.3)
E
R
R
a
Z Z bZ b
Z Z bZ b
0
2
αξ
−i(Xt −Xt0 )ξ
0
eαξ e−[R(0)−R(t−t )]ξ dξdtdt0
e Ee
dξdtdt =
=
R a a
R a a
Z Z bZ b
eαξ exp −[R(0) − R(t − t0 )]ξ 2 dξdtdt0
=
R a a
Z Z bZ b
≤
exp αξ − (c/2 log |t − t0 |)ξ 2 dξdtdt0 .
R
"Z
Z
a
a
Using the formula
Z
R
2
eαξ−cξ dξ = √
1 α2 /(2c)
e
2πc
we get that the right hand side of (5.3) equals
Z bZ b
−1/2
(log |t − t0 |)
exp α2 | log |t − t0 ||/c dtdt0
a a
Z bZ b
dtdt0
−1/2
≤C
(log |t − t0 |)
dtdt0 < +∞,
0 |α2 /c
|t
−
t
a a
provided α2 is sufficiently small. This proves that a.s. mX (dt) has an analytic density.
Thus the density has to be strictly positive and in consequence in any interval [a, b] and
set A there is t such that Xt ∈ A a.s. The above means that on any interval the trajectory
is unbounded.
Another words it is a monstrous process! Actually, Belayev has shown
belayev
in [2] that a Gaussian, stationary process has either continuous trajectories a.s., or the
trajectories are unbounded over any interval a.s.
STOCHASTIC PROCESSES
13
6. Other examples of processes
6.1. Poisson process. Definition. A random variable X has an exponential distribution
with parameter λ > 0 (intensity) if its density is given by
−λx
λe , x > 0,
f (x) =
0,
x < 0.
Exercise 1. Suppose that T is an exponential random variable. Prove that
P[T > t + h|T > t] = P[T > h],
∀ t, h > 0.
(6.1)
eqt-6.1
In fact (6.1) can be generalized to show that for any random variables 0 < H1 < H2 such
that (H1 , H2 ) is independent of T we have a Markov property
P[ t + H2 > T ≥ t + H1 |T > t] = P[H2 > T ≥ H1 ],
∀ t > 0.
(6.2)
The converse also holds. A random variable T satisfying (6.1) has to have an exponential
distribution.
Exercise 2. Suppose that X1 , X2 are two independent random variables with densities
f1 , f2 respectively. Prove that RX1 + X2 is a random variable that possesses a density given
by a convolution f1 ∗ f2 (x) = R f1 (x − y)f2 (y)dy.
Exercise 3. Suppose that X1 , . . . , Xn are exponentially distributed i.i.d. random variables with intensity λ. Prove that X1 + . . . + Xn is a random variable whose density is
given by Γ(n, λ)-distribution given by formula

λn


xn−1 e−λx , x > 0,

(n − 1)!
f (x) =


 0,
x < 0.
Definition. A random variable N is called Poisson distributed with intensity λ > 0,
which is denoted by N (λ), if
P[N = k] =
λk −λ
e ,
k!
k = 0, 1, . . . .
Exercise 4. A characteristic function of a Poisson random variable N ∈ N (λ)
φ(ξ) := EeiξN = exp −λ(1 − eiξ ) .
Suppose that T1 , T2 , . . . are i.i.d random variables exponentially distributed with parameter 1. Let τ0 := 0, . . . , τn := T1 + . . . + Tn , . . ..
Exercise 5. Prove that limn→+∞ τn = +∞ a.s.
For a given t ≥ 0 define Nt as the unique integer such that t ∈ [τNt , τNt +1 ).
Note that:
1) N0 = 0 a.s.,
eqt-6.1a
14
T.KOMOROWSKI
2) Since τn = T1 + . . . + Tn and Tn+1 are independent we can write
Z t Z +∞ n−1 −τ
τ e
dτ
P[Nt = n] = P[τn ≤ t < τn + Tn+1 ] =
e−T dT
t−τ (n − 1)!
0
Z t n−1 −τ
Z t
τ e
τ n−1
tn
−(t−τ )
−t
=
e
dτ = e
dτ = e−t
n!
0 (n − 1)!
0 (n − 1)!
3) Suppose that T1 , T2 , . . ., T̃1 , T̃2 , . . . are two mutually independent sequences of i.i.d.
exponentially distributed random variables with intensity parameter 1. Define also the
respective sums {τn , n ≥ 0} and {τ̃n , n ≥ 0}. For t2 ≥ t1 ≥ 0, m, n non-negative integers
let us compute
P[Nt2 − Nt1 = m, Nt1 = n] = P[Nt2 = m + n, Nt1 = n].
From the definition we get that the right hand side equals
P[τn ≤ t1 < τn+1 , τn+m ≤ t2 < τn+m+1 ] = P[τn ≤ t1 < τn + τ̃1 , τn + τ̃m ≤ t2 < τn + τ̃m+1 ]
m
m+1
X
X
= P[τn ≤ t1 , t1 − τn ≤ T̃1 , T̃1 +
T̃i ≤ t2 − τn < T̃1 +
T̃i ]
i=2
t1
Z
fn (τ )P[t1 − τ ≤ T̃1 , T̃1 +
=
0
Z
"
t1
fn (τ )P[t1 − τ ≤ T̃1 ]P T̃1 +
0
m
X
i=2
T̃i ≤ t2 − τ < T̃1 +
m+1
X
i=2
i=2
m
X
m+1
X
T̃i ≤ t2 − τ < T̃1 +
i=2
T̃i ]dτ
#
T̃i | t1 − τ ≤ T̃1 dτ
i=2
where fn is the density of τn . Using property (6.2) we get that the utmost right hand side
equals
#
"
Z t1
m
m+1
X
X
fn (τ )P[t1 − τ ≤ T̃1 ]dτ P T̃1 +
T̃i ≤ t2 − t1 < T̃1 +
T̃i
0
i=2
"
= P[τn ≤ t1 < τn+1 ]P T̃1 +
m
X
i=2
T̃i ≤ t2 − t1 < T̃1 +
i=2
m+1
X
#
T̃i
i=2
Summing over n we conclude that Nt2 −Nt1 has Poisson distribution with parameter t2 −t1 .
Moreover we have shown also that
P[Nt2 − Nt1 = m, Nt1 = n] = P[Nt2 − Nt1 = m]P[Nt1 = n].
Hence Nt2 − Nt1 , Nt1 are independent. We can generalize this property by induction and
obtain.
Exercise. Suppose 0 ≤ t1 ≤ . . . ≤ tn . Then Nt1 , Nt2 − Nt1 , . . . , Ntn − Ntn−1 are
independent and have Poisson distribution with parameters t1 , . . . , tn − tn−1 respectively.
4) The trajectories of Nt are a.s. increasing and limt→s+ Nt = Ns . Obviously, since the
trajectory is increasing we have limt→s− Nt exists a.s. Processes with such properties of
STOCHASTIC PROCESSES
15
trajectories, i.e. they are right continuous with left limits are called cádlág from French
”continu á droite, limite á gauche”.
We can repeat the above construction using, instead of exponential r.v.-s with intensity
one, the random variables with intensity λ
Definition. Any process {Nt , t ≥ 0} satisfying 1)N0 = 0, 2) Nt ∈ N (λt), 3) for any
0 ≤ t1 ≤ . . . ≤ tn the random variables Nt1 , Nt2 − Nt1 , . . . , Ntn − Ntn−1 are independent
Poisson random variables from N (λt1 ), N (λ(t2 − t1 )), . . . , N (λ(tn − tn−1 )), respectively, 4)
the trajectories are cadlag a.s., is called a Poisson process with intensity parameter λ.
Of course any such two processes are statistically equivalent.
Exercise. Any two equivalent versions of Poisson processes have to be equivalent modifications of each other.
6.2. Compound Poisson process. We can further generalize the construction admitting
an additional sequence of i.i.d.
P variables {ξn , n ≥ 0} distributed according to a law ν on
Rd . Let S0 := 0, . . . , Sn := ni=1 ξi , . . .. Define a process Xt := SNt , where Nt is a Poisson
process with intensity λ > 0. The process is called a compound Poisson process of intensity
λ and jump measure ν. The ”regular” Poisson process corresponds to ν = δ1 .
Exercise. Show that the characteristic function of Xt equals
Eeiξ·Xt = exp {−tψ(ξ)} ,
where ψ(ξ) is called a Levy exponent. It is equal
Z
(1 − eiξ·y )ν(dy).
ψ(ξ) = λ
Rd
6.3. Poisson random measures. Suppose that (Ω, F, P) is a probability space and
Mploc (Rd ) denotes the space of all locally finite, Borel point measures on Rd , i.e. such
measures that µ that µ(A) ∈ Z+ ∪ {∞} for all A ∈ B(Rd ) and µ(K) < +∞ for any K
compact. Suppose that ν is a Borel, σ-finite measure on (Rd , B(Rd )). We wish to construct
a random measure N : Ω → Mploc (Rd ) that has the following properties:
1) for any A ∈ B(Rd ) such that ν(A) < +∞ random variable N (A) is Poisson distributed
with intensity ν(A),
2) for any A1 , . . . , An ∈ B(Rd ) that are pairwise disjoint, i.e. Ai ∩ Aj = ∅ for i 6= j we
have N (A1 ), . . . , N (An ) are independent.
A measure having this property is called a Poisson measure with intensity ν.
Construction: suppose that ν∗ := ν(Rd ) < +∞. Let {Xn , n ≥ 0} be i.i.d. random
variables distributed according to ν̃(·) := ν(·)/ν∗ and N be an independent random variable
belonging to N (ν∗ ). Define
N (A) :=
N
X
i=1
1A (Xi ).
16
T.KOMOROWSKI
We claim that this measure has the desired properties. Indeed, for any A we have
E exp{iξN (A)} =
+∞ n
X
ν
∗
n=0
=
+∞ −ν∗ Z
X
e
n=0
n!
Z
= exp
n!
e−ν∗ {E exp{iξ1A (X0 )}}n
n
Z
−ν∗
exp{iξ1A (x)}ν(dx) = e exp
Rd
exp{iξ1A (x)}ν(dx)
Rd
Z
iξ
[exp{iξ1A (x)} − 1]ν(dx) = exp
(e − 1)ν(dx) = exp (eiξ − 1)ν(A) .
Rd
A
Suppose that Ai are as in 2). We get by the same calculation
(
)m
n
n
+∞ m
X
X
X
ν∗ −ν∗
E exp{i
ξj 1Aj (X0 )}
e
E exp{i
ξj N (Aj )} =
m!
n=0
j=1
j=1
)
( n Z
n
Y
X
[exp{iξj 1Aj (x)} − 1]ν(dx) =
E exp{iξj N (Aj )}
= exp
j=1
Rd
j=1
and the property follows.
Suppose that ν∗ = +∞. Since the measure is σ finite there exist Borel sets Rj , j =
1, 2, . . . such that νj := ν(Rj ) < +∞. Construct the corresponding independent measures
N (j) and let
+∞
X
N (A) :=
N (j) (A)
j=1
d
for any A ∈ B(R ), whether the sum is finite, or not. This measure also satisfies
1) and 2).
R
d
Exercise. Suppose
that
f
:
R
→
R
is
a
measurable
function
such
that
|f
|dν
< +∞.
Rd
R
Define X(ω) := Rd f (y)N (dy; ω). Prove that
Z
if (x)ξ
[e
− 1]ν(dx) .
E exp {iξX} = exp
Rd
d
R Exercise. Poisson shots. Suppose
R that f : R → R is a measurable function such that
|f |dν < +∞. Define f˜(x; ω) := Rd f (x − y)N (dy; ω). Prove that
Rd
)
(Z
)
( n
n
X
X
ξj f˜(xj ) = exp
[exp{i
ξj f (xj − y)} − 1]ν(dy)
E exp i
j=1
Rd
j=1
The random field {f˜(x), x ∈ Rd } is called Poisson shots.
7. Processes with continuous trajectories
We shall formulate sufficient conditions for the existence of continuous trajectory version
of a given process {Xt , t ≥ 0}, defined over (Ω, F, P). There are two possible questions:
first is whether the law µ of the process µ on (R[0, +∞), σ(C)) is such that there exists
a subset Z of C := C[0, +∞), consisting of possibly more regular functions such that its
STOCHASTIC PROCESSES
17
outer measure equals 1. This guarantees that {πt , t ≥ 0} restricted over (Z, σZ (C), µZ ),
where σZ (C), µZ are the σ-algebra and the restricted measure as described in Theorem 4.1,
is statistically equivalent with {Xt , t ≥ 0}. The second question is whether on (Ω, F, P)
we can find {X̃t , t ≥ 0}, that is a version of {Xt , t ≥ 0} that has continuous trajectories
a.s.
Typically, we could be interested in characterizing modulus of continuity of a realization
of the process.
7.1. Kolmogorov regularity theorem. Let Hγ , Hγ,T denote the sets of locally Hölder
functions with the exponent γ and Hölder functions with the exponent γ on [0, T ], respectively.
thm003
Theorem 7.1. Suppose that {Xt , t ≥ 0} is a stochastic process over a probability space
(Ω, F, P) such that: for any T > 0 there exist CT , αT , βT > 0 for which
E|Xt2 − Xt1 |αT ≤ CT |t2 − t1 |1+βT ,
∀ t1 , t2 ∈ [0, T ].
(7.1)
004b
Then, there exists {X̃t , t ≥ 0}, a continuous trajectory process over (Ω, F, P) such that
i) it is a modification of {Xt , t ≥ 0}, i.e. P[Xt = X̃t ] = 1 for any t ≥ 0,
ii) for any T > 0 we have
P[X̃(ω) ∈ Hκ,T ] = 1
for any κ ∈ (0, βT /αT ). Here X̃ : Ω → C[0, +∞) is given by formula X̃(ω)(t) :=
X̃t (ω),
iii) if αT = α, βt = β for all T > 0 we have
P[X̃(ω) ∈ Hκ ] = 1
for any κ ∈ (0, β/α).
Proof. Denote by TnT the set of dyadic points of the n-th generation of the interval
[0, T ), S
i.e the set of points of the form j/2n , j = 0, 1, 2, . . . that belong to [0, T ) and
T
T∞ := n≥0 TnT . We do not write the superscript T if T = +∞.
Exercise. Prove that the process {Xt , t ≥ 0} is stochastically continuous.
(n)
Define Xt = Xt for t ∈ Tn and extend it linearly for other t. We show that there exist
γ, δ > 0 such that
P (Kn ) ≤ 2−δn , ∀ n ≥ 1,
(7.2)
where
#
"
Kn :=
(n+1)
sup |Xt
(n)
− Xt | ≥ 2−γn .
t∈[0,T ]
Hence
X
n≥1
P [Kn ] < +∞
010212
18
T.KOMOROWSKI
and, by Borel-Cantelli lemma, we have
P [L1 ] = 1,
where
L1 := [ω : ∃ n0 (ω)∀ n ≥ n0 (ω) : ω 6∈ Kn ].
For any ω ∈ L1 the series
+∞
X
(n+1)
[Xt
(n)
(1)
− Xt ] + Xt
n=1
(n)
converges uniformly on [0, T ] by Weierstrass criterion. Since its partial sums equal Xt the
(n)
same holds for the latter sequence. Let X̃t := limn→+∞ Xt on L1 and equals 0 otherwise.
This process has all trajectories continuous. On the other hand Xt = X̃t on T∞T . By
stochastic continuity we have
P[Xt = X̃t ] = 1,
∀ t ∈ [0, T ].
Thus part i) of the theorem holds.
We prove (7.2). From Chebyshev inequality we have
Z
−γn
αγn
P |Xk/2n +1/2n+1 − Xk/2n | ≥ 2
≤2
|X(2k+1)/2n+1 − X2k/2n+1 |α ≤ 2[αγ−(1+β)](n+1) .
(n)
Since Xt
(n+1)
and Xt
are piecewise linear and agree on Tn1 we get
[
Kn ⊂
[|Xk/2n +1/2n+1 − Xk/2n | ≥ 2−γn ]
0≤k/2n <T
and
P(Kn ) ≤ ([T ] + 1)2n 2[αγ−(1+β)](n+1) < 2−nδ
if δ = β − αγ > 0.
We show that for any ω ∈ L1 we have X(ω) ∈ Hκ,T . The key for us is the following
lemma, due to Garsia, Rodemich, Rumsey.
lm001
Lemma 7.2. Let Ψ, p : [0, +∞) → [0, +∞) be strictly increasing, continuous functions
satisfying p(0) = Ψ(0) = 0 and limx→+∞ Ψ(x) = +∞. Assume also that f ∈ C[T1 , T2 ] and
Z T2 Z T2 |f (t) − f (s)|
B :=
Ψ
dtds.
p(|t − s|)
T1
T1
Then,
Z
|f (T2 ) − f (T1 )| ≤ 8
T2 −T1
−1
Ψ
0
4B
u2
p(du).
(7.3)
014
STOCHASTIC PROCESSES
19
Before proving the lemma we show how to use it to finish the proof of the theorem. By
(7.1) the process {Xt , t ≥ 0} is stochastically continuous. Let Ψ(x) := xα and p(u) := uρ/α .
These functions satisfy the assumptions of Lemma 7.2. We have
!
Z 1Z 1
|X̃t − X̃s |
EΨ
dtds
p(|t − s|)
0
0
Z 1Z 1
≤C
|t − s|1+β−ρ < +∞
0
0
for some constant C > 0, provided that ρ := γ + 2 and γ ∈ (0, β).
Let
!
Z 1Z 1
|X̃t (ω) − X̃s (ω)|
Ψ
B(ω) :=
dtds.
p(|t − s|)
0
0
There exists a set N ∈ F such that P(N ) = 0 and for any ω ∈ N c we have B(ω) < +∞.
Since X̃t (ω) is continuous, by virtue of Lemma 7.2, we get that for any ω ∈ N c there exists
C(ω) ∈ (0, +∞) such that for 0 ≤ t1 < t2 ≤ T
Z t2 −t1
B(ω)
−1
|X̃t2 − X̃t1 | ≤ 8
p(du)
Ψ
u2
0
Z t2 −t1
= C(ω)
u(ρ−2)/α−1 du = C 0 (ω)(t2 − t1 )γ/α .
0
c
Thus for ω ∈ N we have X̃(ω) ∈ Hκ,T for κ = γ/α. To finish the proof of the theorem it
suffices only to observe that
\
Hκ =
Hκ,n
n≥1
and P[X̃ ∈ Hκ ] = 1 because P[X̃ ∈ Hκ,n ] = 1 for any n ≥ 1.
Proof of Lemma 7.2. Suppose first that T1 = 0 and T2 = 1. Let
|f (t) − f (s)|
jt (s) := Ψ
p(|t − s|)
and
Z
I(t) :=
(7.4)
013011
(7.5)
013
(7.6)
010
(7.7)
012
1
jt (s)ds.
0
Then B =
R1
0
I(t)dt. One can find t0 such that I(t0 ) ≤ B. We claim that
Z 1
4B
−1
|f (t0 ) − f (0)| ≤ 4
Ψ
p(du).
u2
0
Indeed, let t0 > u1 be chosen in such a way that p(u1 ) := 1/2p(t0 ). Then,
Z u1
I(t)dt ≤ B
0
20
T.KOMOROWSKI
and
u1
Z
jt0 (s)ds ≤ I(t0 ).
(7.8)
0
One can choose t1 < u1 in such a way that
I(t1 ) ≤
2B
u1
and
2I(t0 )
.
u1
This can be seen as follows: let A1 := [s ∈ [0, u1 ] : I(s) > 2B/u1 ], A2 := [s ∈ [0, u1 ] :
jt0 (s) > 2B/u1 ]. Note that in view of (7.7) and (7.8) we have m1 [Ai ] < u1 /2, i = 1, 2, where
m1 [·] is the one dimensional Lebesgue measure. But this implies m1 [[0, u1 ] \ (A1 ∪ A2 )] > 0.
Choose then t1 from the above set.
Suppose that t0 > u1 > t1 > . . . > un−1 > tn−1 are chosen. We can find then tn−1 >
un > tn such that
jt0 (t1 ) ≤
p(un ) := 1/2p(tn−1 ),
2B
I(tn ) ≤
un
2I(tn−1 )
jtn−1 (tn ) ≤
.
un
The above however implies first of all that
lim tn = lim un = g
n→+∞
n→+∞
and, since p(g) = 1/2p(g), we have g = 0. In addition,
jtn−1 (tn ) ≤
4B
4B
2I(tn−1 )
≤
≤ 2.
un
un−1 un
un
Hence, from (7.4)
−1
4B
u2n
p(tn−1 − tn )
|f (tn ) − f (tn−1 )| ≤ Ψ
4B
4B
−1
−1
≤Ψ
p(tn−1 ) = 2Ψ
p(un )
2
un
u2n
1
4B
−1
p(un ) − p(un )
≤ 4Ψ
u2n
2
4B
1
−1
≤ 4Ψ
p(un ) − p(tn )
u2n
2
4B
= 4Ψ−1
[p(un ) − p(un+1 )]
u2n
(7.9)
013b
STOCHASTIC PROCESSES
21
In conclusion, summing both sides over n we obtain (7.6). Observe that function g(t) :=
f (1 − t) also satisfies the assumption of the lemma. Let I˜1 (t) be given by (7.5) with f
replaced by g. Note that I˜1 (1 − t0 ) = I1 (t0 ) ≤ B. Hence, by the above argument
Z 1
4B
−1
p(du).
(7.10)
Ψ
|f (t0 ) − f (1)| = |g(1 − t0 ) − g(0)| ≤ 4
u2
0
010b
Therefore, we obtain (7.3) for T1 = 0, T2 = 1. To generalize to an arbitrary interval [T1 , T2 ]
consider the linear change of variables.
Exercise. Generalize the result for any T .
Example. Gaussian processes. Suppose that m(t) is continuous and R(t, s) is positive
definite and locally Hölder, i.e. for any T > 0 there exists CT , γT > 0 such that
|R(t + h1 , s + h2 ) − R(t, s)| ≤ CT (|h1 | + |h2 |)γT
(7.11)
005
for all s, t, s + h1 , t + h2 ∈ [0, T ]. Let {Xt , t ≥ 0} be a Gaussian process with mean m(t)
and covariance function R(t, s). We prove the existence of a version that satisfies: for any
T > 0, γ < γT we have P[ω : X(ω) ∈ Hγ,T ] = 1.
Indeed, we know that there exists a Gaussian process {Xt , t ≥ 0} that corresponds to
the mean 0 and covariance R(t, s). Note that
E|Xt2 − Xt1 |2 = R(t2 , t2 ) + R(t1 , t1 ) − 2R(t1 , t2 ) ≤ C|t2 − t1 |γT .
Exercise. If X is a zero mean Gaussian random variable then EX 2n = (2n − 1)!!(EX 2 )n .
Let n be so large that nγT = 1 + αT for some αT > 0. We have then
E|Xt2 − Xt1 |2n = R(t2 , t2 ) + R(t1 , t1 ) − 2R(t1 , t2 ) ≤ C|t2 − t1 |1+αT .
The result is then a consequence of Theorem 7.4.
Corollary 7.3. A (pre) Brownian motion {Bt , t ≥ 0} with Hurst exponent H ∈ (0, 1] has
a continuous modification.
7.2. Some other consequences of Kolmogorov regularity theorem and GarsiaRodemich-Rumsey lemma.
7.2.1. Outer measure of the set of Hölder functions.
thm003
Theorem 7.4. Suppose that µ is a probability measure on (R[0, +∞), σ(C)). Assume that
for any T > 0 there exist CT , α, β > 0 such that
Z
|πt2 − πt1 |α dµ ≤ CT |t2 − t1 |1+β , ∀ t1 , t2 ∈ [0, T ].
(7.12)
with uniform α, β > 0 but with CT depending possibly on T > 0. Let 0 < κ < β/α. Then,
for any A ⊃ Hκ and A ∈ σ(C) we have µ(A) = 1, i.e. the set Hκ is of outer measure 1.
Proof. Denote by TnT the set of dyadic points of the n-th generation of the interval
[0, T ], i.e the set of points of the form j/2n , j = 0, 1, 2, . . . that belong to [0, T ] and
004
22
T.KOMOROWSKI
S
T∞T := n≥0 TnT . Let {π̃t , t ≥ 0} be a process constructed using dyadic points as in the
proof of Theorem 7.4. We know that
µ(f : π̃· (f ) ∈ Hκ ) = 1
thus, because π̃t is a version of πt we have
µ (FT ) = lim µ (FTm ) = 1,
m→+∞
∀T > 0
(7.13)
010112
for 0 < κ < β/α,
"
#
|πt − πs |
FT :=
sup
< +∞
κ
T ,s6=t |s − t|
s,t∈T∞
and
"
FTm :=
#
|πt − πs |
≤m .
κ
T ,s6=t |s − t|
s,t∈T∞
sup
Suppose that A ⊃ Hκ and A ∈ σ(C). Then, there exists a set {ti , i ≥ 0} ⊃ Q and Borel
measurable sets {Ai , i ≥ 1} such that A = [πti ∈ Ai , i ≥ 1].
T
:= {t1 , t2 , . . .} ∩ [0, T ]. Define
Fix any T > 0. Let SnT := {t1 , . . . , tn } ∩ [0, T ] and S∞
"
#
|πt − πs |
GT :=
sup
< +∞
κ
T ,s6=t |s − t|
s,t∈S∞
and
"
Gm
T
#
|πt − πs |
:=
sup
≤m .
κ
T ,s6=t |s − t|
s,t∈S∞
From (7.13) and stochastic continuity of the process we conclude that
lm-010712
Lemma 7.5. We have
µ(GT ) = lim µ(Gm
T) = 1
(7.14)
020712
µ(FTm ) ≤ µ (Zn,m ) ,
(7.15)
020112
n, m = 1, 2, . . ..
(q)
(q)
Indeed, choosing sequences {tk , q ≥ 1} ⊂ T∞T , k = 1, . . . , n such that tk → tk as
m → +∞ and using stochastic continuity we conclude that
(q)
µ (Zn,m ) = lim µ Zn,m
,
(7.16)
010712
m→+∞
Proof. We claim that
where
"
Zn,m
#
|πt − πs |
sup
:=
≤m
κ
T ,s6=t |s − t|
s,t∈Sn
q→+∞
where
|πt − πs |
(q)
(q)
:=
≤ m, s, t ∈ {t1 , . . . , tn }, s 6= t .
|s − t|κ
Exercise. Prove (7.16).
(q)
Zn,m
STOCHASTIC PROCESSES
23
(q)
On the other hand we have FTm ⊂ Zn,m for all q, n. Thus, by (7.16), we get
(q)
) = µ (Zn,m )
µ(FTm ) ≤ lim µ(Zn,m
q→+∞
and (7.15) follows.
Letting n → +∞ we get that
µ(FTm ) ≤ µ GTm .
Passing with m → +∞ we conclude (7.14).
T
˜
˜
Suppose that f ∈ G and f is its unique continuous extension. Then f ∈ Hκ and thus
˜
f ∈ A which implies that f ∈ A (recall that A is determined by the values of f˜ at tj -s).
Hence we have shown that GT ⊂ A and this implies that µ(A) = 1.
7.2.2. Further generalization of the Garsia-Rodemich-Rumsey lemma. We shall denote by
C := C[0, +∞) and CT := C[0, T ]. The respective Borel σ-algebras shall be denoted by M
and MT . For a function f ∈ CT denote
Z TZ T |f (t) − f (s)|
BT (f ) :=
Ψ
dtds
p(|t − s|)
0
0
and
ω̄f (δ, T ) := sup[|f (t) − f (s)|, 0 ≤ s, t ≤ T, |t − s| < δ].
cor002
Corollary 7.6. We have
Z
ω̄f (δ, T ) ≤ 8
δ
−1
Ψ
0
4BT (f )
u2
p(du)
∀ δ > 0.
Suppose that Ψ, p are such that for any C > 0 we have a(C) such that
Z a(C)
C
−1
Ψ
p(du) < +∞.
u2
0
(7.17)
cont-cond-a
(7.18)
cont-cond
Let H(Ψ, p) be the set consisting of those f ∈ R[0, +∞) such that for any T > 0 there
exists C := Cf,T > 0, for which
Z δ
C
−1
p(du) ∀ δ > 0.
ω̄f (δ, T ) ≤ 8
Ψ
u2
0
Of course H := H(Ψ, p) ⊂ C.
The lemma can be generalized to cover also the case when the function f (·) is only
measurable.
cor005
Corollary 7.7. We use the notation of Lemma 7.2. In addition to assumptions made
about p(·) and Ψ(·) in Lemma 7.2 suppose that Ψ(·) is convex. We shall also assume that
the function f : [T1 , T2 ] → R is measurable and B < +∞. Then there exists a function
f˜ ∈ C[T1 , T2 ] such that 1) f˜ = f a.e. and 2) (7.3) holds.
24
T.KOMOROWSKI
Proof. Let M > 0 and ψM (x) = x for |x| ≤ M , ψ(x) =
R M , x ≥ M and ψ(x) = −M ,
∞
x ≤ −M . Suppose that φ ∈ Cc (R) is non-negative and R φ(x)dx = 1. Let > 0 and
x
.
φ (x) := −1 φ
For a given measurable function f (·) extended to R in any fashion define
Z
M
φ (x − y)ψM ◦ f (y)dy.
f (x) :=
R
We have
fM
∈
Cc∞ (R). Let also f M
Z TZ T M
|f (t)
BT,M :=
Ψ
0
Z
≤
0
Z
T
Z
T
Z
T
φ (y)dy
0
R
Z
≤
Z
0
φ (y)dy
R
0
0
T
:= ψM ◦ f . Note that
− fM (s)|
dtds
p(|t − s|)
M
|f (t − y) − f M (s − y)|
Ψ
dtds
p(|t − s|)
Z
|f (t − y) − f (s − y)|
Ψ
dtds ≤ BT
φ (y)dy = BT .
p(|t − s|)
R
Using estimate (7.17) we conclude that
Z δ
4BT (f )
−1
ω̄fM (δ, T ) ≤ 8
Ψ
p(du).
u2
0
and kfM k∞ ≤ M . The family {fM , ∈ (0, 1]} is equicontinuous and uniformly bounded,
thus is compact in uniform topology on compacts. On the other hand lim→0+ fM = f M in
L1loc [0, +∞). Thus lim→0+ fM = f˜M uniformly on compact sets, where f˜M are continuous
functions that are equal to f M a.e. Note that
Z δ
4BT (f )
−1
Ψ
ω̄f˜M (δ, T ) ≤ 8
p(du).
(7.19)
u2
0
It is easy to observe that f˜(x) = limM →+∞ f M (x) exists for any x. As a consequence of
(7.19) we have
Z δ
4BT (f )
−1
ω̄f˜(δ, T ) ≤ 8
Ψ
p(du).
u2
0
and f˜ is continuos. Since also f M → f , as M → +∞, a.e. we get f˜ = f a.e.
060112
7.2.3. Application to random processes. Recall that H = H(Ψ, p) ⊂ C.
cor10
Theorem 7.8. Assume that functions p(·) and Ψ(·) satisfy the assumptions of Corollary
7.7. Suppose that the canonical process {πt , t ≥ 0} is stochastically continuous and
Z T Z T Z
|πt (f ) − πs (f )|
βT :=
Ψ
µ(df ) dtds < +∞, ∀ T > 0.
(7.20)
p(|t − s|)
0
0
Then, for any A ⊃ H and A ∈ σ(C) we have µ(A) = 1 (H is outer measure 1). In
particular, we can define µH the probability measure that is the restriction of µ to H,
defined on the measurable space (H, σH (C)).
023011
STOCHASTIC PROCESSES
25
Exercise. Prove that stochastic continuity implies that
Z
|πt (f ) − πs (f )|
(t, s) 7→ Ψ
µ(df )
p(|t − s|)
is continuous outside t = s.
Proof. With no loss generality we can assume that µ is complete, as otherwise we
consider its completion. According to Doob’s theorem there exists a process {π̂t , t ≥ 0},
an equivalent version of the process {πt , t ≥ 0}, that is measurable. Since
Z T Z T Z
|πt (f ) − πs (f )|
Ψ
µ(df ) dtds < +∞
p(|t − s|)
0
0
we have a set N such that µ(N ) = 0 and for any f ∈ N c we have
Z TZ T |π̂t (f ) − π̂s (f )|
BT (f ) :=
Ψ
dtds < +∞, ∀ T > 0.
p(|t − s|)
0
0
(7.21)
033011
Here we have adopted convention that |a − b| = ∞ if any of a, b ∈ {−∞, +∞}. Let
π̃t (f ) ≡ 0 for t ≤ 0. For any , M > 0 we let
Z
M,
φ (t − s)ψM ◦ π̂s (f )ds.
π̃t (f ) :=
R
Then, from Corollary (7.7) we conclude that
π̃t (f ) := lim
lim π̃tM, (f )
M →+∞ →0+
exists, is continuous and
Z
ω̄π̃(f ) (δ, T ) ≤ 8
δ
−1
Ψ
0
4BT (f )
u2
p(du),
∀ f ∈ N c.
Moreover π̃t (f ) = π̂t (f ) for a.e. t ≥ 0. Since, by Fubini theorem
Z T
Z TZ
Z
min [|π̃t (f ) − π̂t (f )|, 1] dtµ(df ) = µ(df )
min [|π̃t (f ) − π̂t (f )|, 1] dt = 0
0
0
we get
Z
min [|π̃t (f ) − π̂t (f )|, 1] µ(df ) = 0
(7.22)
for a.e. t ≥ 0. Thanks to stochastic continuity we get that (7.22) holds for all t ≥ 0 thus
µ [π̃t (f ) = πt (f )] = 1,
∀ t ≥ 0.
Suppose that A ⊃ H and A ∈ σ(C). Then, there exists a set T := {ti , i ≥ 0} ⊃ Q and
Borel measurable sets {Ai , i ≥ 1} such that A = [πti ∈ Ai , i ≥ 1]. There exists C such
that 1) µ(C) = 1 and 2) it symmetrically differs from N c by set of outer measure 0 and
3) C ∈ σ(C). We can increase if necessary the set {ti , i ≥ 1} in order to be able to find
Borel sets {Ci , i ≥ 1} such that C = [πti ∈ Ci , i ≥ 1]. Suppose that N1 ⊃ N is such that
µ(N1 ) = 0 and for all f ∈ N1c we have π̃t (f ) = πt (f ), t ∈ T. Note that µ[N1c 4C] = 0.
070112
26
T.KOMOROWSKI
Suppose that f ∈ C ∩ N1c . Lemma 7.2 together with (7.21) imply that f˜ - the continuous
extension of f˜(ti ) = f (ti ) - belongs to H. But this means that f ∈ A (since it is determined
only by values of f at ti ). Thus C ∩ N1c ⊂ A. However µ(C) = 1 and since µ(N1c 4C) = 0
we have C ∩ N1c has to be of outer measure 1.
GRR
Theorem 7.9. Assume that Ψ(·) and p(·) are functions satisfying the hypotheses of Lemma
7.2 such that (7.18) holds. Suppose that {Xt , t ≥ 0} is a stochastically continuous process
over a complete probability space (Ω, F, P) that satisfies
Z TZ T
|Xt − Xs |
dtds < +∞, ∀ T > 0.
EΨ
p(|t − s|)
0
0
Then, there exists a process {X̃t , t ≥ 0} defined over the same space and taking values in
R whose trajectories belong to H a.s and such that P[Xt = X̃t ] = 1 for each t. In particular
the trajectories of {X̃t , t ≥ 0} are continuous a.s.
Proof. Fix any T, δ > 0 and f ∈ R[0, +∞). Define
ω̄fQ (δ, T ) := sup[|f (t) − f (s)|, s, t ∈ [0, T ] ∩ Q, |t − s| < δ].
Let HQ (Ψ, p) be the set consisting of those f such that for any T > 0 there exists C :=
Cf,T > 0, for which
Z δ
C
Q
−1
Ψ
ω̄f (δ, T ) ≤ 8
p(du) ∀ δ > 0.
u2
0
Recall that X : Ω → R[0, +∞) is defined by X(ω)(t) := Xt (ω). Let
H := [ω : X(ω) ∈ HQ (Ψ, p)] .
From Theorem 7.8 and completeness of measure P we conclude that P[H] = 1. Define
X̃t (ω) := lims→t,s∈Q Xs (ω) for ω ∈ H and X̃t ≡ 0, if otherwise. Observe that clearly
P[X̃t = Xt ] = 1 for t ∈ Q. Stochastic continuity implies that this equality holds for all
t ≥ 0.
Example. Assume that R(t) is the covariance function of a stationary Gaussian process
{Xt , t ≥ 0}. Suppose that for some c, a, h > 0 we have
R(0) − R(t) ≤
c
,
| log t|1+a
∀ t ∈ (0, h).
Then the process possesses a continuous modification. Indeed, take Ψ(u) := eu − 1, p(u) =
RT
[| log u|1+β + 1]−1 for β < a. We have 0 Ψ−1 (Cu−2 )p(du) < +∞ and
Z TZ T
|Xt − Xs |
EΨ
dtds < +∞.
p(|t − s|)
0
0
STOCHASTIC PROCESSES
27
8. Brownian motion
Suppose that (Ω, F, P) is a probability space. Process {Bt , t ≥ 0} is called a standard
Brownian motion if
i) it is Gaussian of zero mean and with the covariance function
R(t, s) = t ∧ s,
ii) t 7→ Bt (ω) is continuous for P a.s. ω.
The law W of the process over C is called the Wiener measure.
Problem 8. Show that for any γ < 1/2: W[Hγ ] = 1.
Problem 9. Show that P[B0 = 0] = 1.
Let q(t, x) = qt (x) = (2πt)−1/2 exp{−x2 /(2t)} be the density of a normal variable N (0, t).
Problem 10. Independence of increments. For any 0 ≤ t0 < t1 < . . . < tn the variables
Bt0 , Bt1 − Bt0 , . . . , Btn − Btn−1 are independent. The law of Btp − Btp−1 is N (0, tp − tp−1 ).
Let σ ≥ 0 and µ ∈ R. The process {Xt , t ≥ 0} is called a Brownian motion with variance
2
σ and the average µ if there exists a standard Brownian motion {Bt , t ≥ 0} such that
Xt = σBt + µ.
(1)
(d)
A d dimensional standard Brownian motion. An Rd valued process Bt = (Bt , . . . , Bt )
(1)
(d)
is called a d-dimensional, standard Brownian motion if Bt , . . . , Bt are independent standard Brownian motions.
A d dimensional Brownian motion. Suppose that Σ is a non-negative definite symmetric
matrix and µ ∈ Rd . An Rd valued process Xt is called a d-dimensional Brownian motion
with variance Σ if there exists a standard d-dimensional Brownian motion {Bt , t ≥ 0} such
that Xt = Σ1/2 Bt + µ.
Some properties of a Brownian motion. Let Bt := σ(Bs , 0 ≤ s ≤ t).
1. Independence and stationarity of increments. {Bt+s − Bs , t ≥ 0} is a standard
Brownian motion independent of Bs for each s ≥ 0. Also for any h ≥ 0, k ≥ 1 and
0 = t0 ≤ t1 ≤ . . . ≤ tk we have the laws of (Btk +h − Btk−1 +h , . . . , Bt1 +h − Bh ) and that of
(Btk − Btk−1 , . . . , Bt1 ) are indentical.
Prove it!!!
2. Markov property. Suppose that f ∈ Bb (R), t, h ≥ 0, then
E [f (Bt+h ) | Bt ] = Ph f (Bt ),
where
1
Ph f (x) := f ∗ qt (x) = √
2πt
Z
2 /(2t)
e−(x−y)
f (y)dy.
R
Prove it!!!
3. The martingale property.
Definiton. Suppose that {Ft , t ≥ 0} is a filtration. The process {Xt , t ≥ 0} is called a
martingale with respect to the given filtration if
i) it is {Ft , t ≥ 0} adapted, i.e. Xt is Ft -measurable for each t ≥ 0,
ii) E|Xt | < +∞ for all t ≥ 0,
28
T.KOMOROWSKI
iii) for any t ≥ s ≥ 0 we have
E[Xt |Fs ] = Xs .
A standard Brownian motion {Bt , t ≥ 0} is a martingale. Moreover, the process {Bt2 −
t, t ≥ 0} is also a martingale.
Prove it !!!
3. The total variation of a Brownian path. Let Π : 0 = t0 < t1 < . . . < tn = T be the
partition of [0, T ]. Denote by
X
V(f ; [0, T ]) := sup{
|f (ti+1 ) − f (ti )| : Π}
i
the total variation of f . The supremum is taken over all partitions Π. We show that for a
standard Brownian motion {Bt , t ≥ 0} we have
V(B. ; [0, T ]) = +∞, ∀ T > 0
(8.1)
2.003
for P a.s. realization of the path.
Consider Πn the dyadic partition of [0, 1] by points j/2n , j = 0, . . . , 2n . Let
Vn :=
n −1
2X
|B(j+1)2−n − Bj2−n |.
j=0
The sequence is increasing. Let V∞ = limn→+∞ Vn . We prove that for any M > 0
P[V∞ > M ] = 1.
This, of course, shows that V∞ = +∞, P a.s. Note that the law of Vn is identical with
Sn := 2−n/2
n −1
2X
|Xj |,
j=0
where X0 , X1 , . . . is a sequence of i.i.d. N (0, 1) random variables. By the law of large
numbers 2−n/2 Sn → E|X0 |, hence limn→+∞ Sn = +∞, a.s. We can write then
P[V∞ > M ] ≥ lim P[Vn > M ] = lim P[Sn > M ] = 1.
n→+∞
n→+∞
Remark. The control of total variation allows us to control the regularity of path in terms
of the number of times it crosses a given value. Namely, let
n[a,b] (y; f ) := #[t ∈ [a, b] : f (t) = y].
One can show the following formula of Banach: for any f ∈ C[0, T ]
Z
V(f ; [0, T ]) =
n[0,T ] (z; f )dz,
R
natanson
the proof can be found in [7] p. 225, Theorem 3.
4. The quadratic variation of a Brownian path.
(8.2)
2.005
STOCHASTIC PROCESSES
29
Let Π : 0 = t0 < t1 < . . . < tn = T be the partition of [0, T ]. Let |Π| = max |ti+1 − ti |
and
X
S2 (B. ; Π) :=
(Bti+1 − Bti )2 .
i
We show that
V2 (B. ; [0, T ]) := lim S2 (B. ; Π) = T
|Π|→0+
(8.3)
2.007
exists in probability. V2 (B. ; [0, T ]) is called the quadratic variation of a Brownian path. We
have
Czebyshev E[S (B ; Π) − T ]2
2
.
P [|S2 (B. ; Π) − T | ≥ δ]
≤
δ2
"
#2
X
X 2
= δ −2 E
[(Bti+1 − Bti )2 − ∆ti ] = δ −2
E (Bti+1 − Bti )2 − ∆ti
i
= 2δ
−2
X
i
2
−2
(∆ti ) ≤ 2δ T |Π|
i
and (8.3) follows.
Remark. We have an analogue of (8.2): there exists C > 0 such that
Z
1/2
V2 (f ; [0, T ]) ≤ C n[0,T ] (z; f )dz, ∀ f ∈ C[0, T ].
(8.4)
2.002
R
asatiani
This result has been shown in [1]. The converse inequality does not hold.
5. Regularity of a Brownian path. We know that W[Hγ ] = 1 for any γ ∈ (0, 1/2). We
show that for γ > 1/2 W[Hγ ] = 0. In fact a stronger statement is available. We say that
f is Hölder continuous with exponent γ ∈ (0, 1] at x0 if there is C ∈ (0, +∞) such that
|f (x0 + h) − f (x0 )| ≤ C|h|γ for all h ∈ R. Denote by Hγ the set of all functions that are
Hölder continuous with exponent γ at at least one point. Of course Hγ ⊂ Hγ . However,
for any γ ∈ (1/2, 1) we have
W[Hγ ] = 0.
(8.5)
In particular, the above implies that a.s. path of a Brownian motions {Bt , t ≥ 0} is
nowhere differentiable. To prove (8.5) it suffices only to restrict oneself to [0, 1]. Note that
Hγ (1) := [f : f is γ Hölder continuous at some point in [0, 1]]
satisfies Hγ (1) ⊂ D, where
M
D := f : ∃ M > 0 N ≥ 1∀ n ≥ N : min max[|∆k+r,n |, . . . , |∆k−r,n |] ≤ γ
k
n
−r
[ \ N[
M
=
f : max[|∆k+r,n |, . . . , |∆k−r,n |] ≤ γ .
n
M,N n≥N k=r
Here
∆k,n := f
k+1
n
k
−f
,
n
2.008
30
T.KOMOROWSKI
and r is to be determined later on. Note that
M
M
2r
W max[|∆k+r,n |, . . . , |∆k−r,n |] ≤ γ = W |∆k,n | ≤ γ
n
n
!
2r
Z M/nγ−1/2
C
−x2 /2
=C
e
dx
≤ 2r(γ−1/2) .
n
−M/nγ−1/2
Hence,
M
W f : n ≥ N : min max[|∆k+r,n |, . . . , |∆k−r,n |] ≤ γ ≤ CN 1−2r(γ−1/2) → 0,
k
n
as N → +∞, provided that r(γ − 1/2) > 1. Thus W[D] = 0 and (8.5) follows.
6. Exercise: Scaling properties of a Brownian motion. Each of the processes below are
standard Brownian motions:
−Bt , t ≥ 0,
(a)
Bt
:= aBa−2 t , t ≥ 0 for any a > 0,
B̃t := tB1/t , t ≥ 0.
B̃0 := 0,
7. The law of iterated logarithm. We have
Bt
lim sup p
=1
t→+0
2t log log |t|−1
(8.6)
2.011b
(8.7)
2.012b
(8.8)
2.009
(8.9)
2.010
and
lim inf p
t→+0
Bt
2t log log |t|−1
= −1.
Exercise: Using point 6) prove that
lim sup √
t→+∞
Bt
=1
2t log log t
and
lim inf √
t→+∞
Bt
= −1,
2t log log t
a.s.
Proof: Denote log2 t := log log t for t > 1. Let a ∈ (0, 1) and xk := (2ak log2 a−1 )1/2 ,
> 0 and
Ak := [ sup Bu ≥ (1 + )xk ].
u∈[0,ak ]
We use the reflection principle proved in Proposition 8.3 below, which gives us
P[Ak ] = 2P[Bak ≥ (1 + )xk ].
STOCHASTIC PROCESSES
31
Note however that
1
Z
+∞
2
k
e−u /(2a ) du
P[Bak ≥ (1 + )xk ] = √
k
2πa (1+)xk
Z +∞
1
2
=√
e−u /2 du.
2π (1+)xk /ak/2
Exercise: Prove the asymptotics
Z +∞
1
2
2
e−u /2 du ∼ e−x /2 , x 1.
x
x
Hence,
1
C
2
−k
p
P[Bak ≥ (1 + )xk ] ∼ √
e−(1+) log2 a /2 ∼ a(1+2 )
k
log(k + 1)
2π(1 + ) log2 a−k
and as a result we have
X
P[Ak ] < +∞
k≥1
when a(1 + ) > 1. By Borel-Cantelli lemma we conclude that for a.s. ω we can find k0 (ω)
so that for k ≥ k0 (ω) we have ω 6∈ Ak and therefore
Bt
lim sup p
≤1+
t→0+
2t log2 |t|−1
thus (8.8) holds.
Note that {Bak − Bak+1 , k ≥ 1} are independent. Let
Ck := [Bak − Bak+1 ≥ (1 − /2)xk ].
Note that
C
2
P[Ck ] = P[Bak(1−a) ≥ (1 − /2)xk ] ∼ √
k −(1−/2) /(1−a)
log k
2
Choosing a in such a way that (1 − /2) /(1 − a) < 1 we get
X
P[Ck ] = +∞.
k≥1
Since {Ck , k ≥ 1} are independent by the second Borel-Cantelli lemma we get that for P
a.s. ω and any k there exists k1 > k such that ω ∈ Ck1 . On the other hand by the previous
argument for any δ > 0 there exists k0 (ω) such that But
Bak (ω) ≥ −(1 + δ)xk ,
∀ k ≥ k0 (ω).
Yet there is k1 ≥ k0 (ω) such that
Bak1 (ω) − Bak1 +1 (ω) ≥ (1 − /2)xk1 .
and
Bak1 (ω) = Bak1 (ω) − Bak1 +1 (ω) + Bak1 +1 (ω) ≥ (1 − /2)xk1 − (1 + δ)xk1 +1
> (1 − )xk1
32
T.KOMOROWSKI
if only
which is equivalent to
xk > (1 + δ)xk1 +1 .
2 1
s
log a−k1 −1
(1 + δ) a 2 −k1 < .
log2 a
2
This can be ensured choosing a sufficiently small.
8. The strong Markov property. A random variable τ : Ω → [0, +∞] is called a stopping
time with respect to a filtration {Ft , t ≥ 0} if for any t ≥ 0 we have [τ ≤ t] ∈ Ft . Let F∞
be the σ-algebra generated by all Ft , t ≥ 0.
We define a σ-algebra Fτ of events measurable w.r.t. τ by condition A ∈ Fτ iff 1)
A ∈ F∞ and 2) A ∩ [τ ≤ t] ∈ Ft for all t ≥ 0.
Problem 11. If τ, σ are stopping times such that τ ≤ σ then Fτ ⊂ Fσ .
Definition. Suppose that {Ft , t ≥ 0} is a filtration and {Xt , t ≥ 0} is a standard
Brownian motion. We call it non-anticipative w.r.t. the filtration if it is adapted and for
each t ≥ 0 we have {Bt+s − BtT
, s ≥ 0} is independent of Ft .
Definition. Define Ft+ := s>t Fs . Note that Ft+ ⊃ Ft . Likewise for any τ that is a
stopping time w.r.t. {Ft+ , t ≥ 0} we let Fτ + be the σ-algebra consisting of all A ∈ F such
that A ∩ [τ ≤ t] ∈ Ft+ .
thm2.001
Theorem 8.1. Suppose that a standard Brownian motion {Bt , t ≥ 0} non-anticipative
w.r.t. the filtration {Ft , t ≥ 0}. Then,
i) suppose that τ < +∞ is a stopping time with respect to {Ft+ , t ≥ 0}. Then, {Bt+τ −
Bτ , t ≥ 0} is a standard Brownian motion independent of Fτ ,
ii) the process is non-anticipative w.r.t. the filtration {Ft+ , t ≥ 0}.
Proof. Part i). First assume that: 1) τ is a stopping time with respect to {F
P t+ , t ≥ 0}, 2)
there exists a set {tk , k ≥ 1} ⊂ [0, +∞) such that P[τ = tk ] = pk > 0 and +∞
k=1 pk = 1.
Suppose that s1 ≤ s2 ≤ . . . ≤ sn and B ∈ Fτ . We have
P[Bs1 +τ − Bτ ∈ A1 , . . . , Bsn +τ − Bsn−1 +τ ∈ An , B]
X
=
P[Bs1 +tk − Btk ∈ A1 , . . . , Bsn +tk − Bsn−1 +tk ∈ An , B, τ = tk ].
k
But B ∩ [τ = tk ] ∈ Ftk so by the independence of increments we get that the right hand
side equals
X
P[Bs1 +tk − Btk ∈ A1 , . . . , Bsn +tk − Bsn−1 +tk ∈ An ]P[B, τ = tk ]
k
=
X
P[Bs1 ∈ A1 , . . . , Bsn − Bsn−1 ∈ An ]P[B, τ = tk ]
k
= P[Bs1 ∈ A1 , . . . , Bsn − Bsn−1 ∈ An ]P[B].
Let τ be an arbitrary Markov time w.r.t. the filtration {Ft+ , t ≥ 0}. Let τN := [τ N ]+ /N .
Here [x]+ := [x] + 1. Of course τN → τ +, as N → +∞.
STOCHASTIC PROCESSES
33
Exercise. Prove that τN is a stopping time w.r.t. the filtration {Ft , t ≥ 0}.
Suppose that φ1 , . . . , φn are continuous and bounded functions on R. According to the
proved part of the theorem for any B ∈ Fτ we have:
" n
#
" n
#
Y
Y
E
φi Bsi +τN − Bsi−1 +τN , B = E
φi Bsi +τN − Bsi−1 +τN P[B].
i=1
i=1
Let N → +∞. By the Lebesgue dominated convergence theorem we get that
" n
#
" n
#
Y
Y
E
φi Bsi +τ − Bsi−1 +τ , B = E
φi Bsi +τ − Bsi−1 +τ P[B].
i=1
i=1
and we have shown part i). Part ii) follows easily from part i).
Note that an obvious application follows for the natural filtration {Bt , t ≥ 0} of the
Brownian motion. Let B be the smallest σ-algebra containing of Bt , t ≥ 0. Denote by N
the σ-ring of sets of P zero measure contained in B.
cor2.01
Corollary 8.2. (Blumenthal’s 0-1 law). For any t ≥ 0 there exists Nt ⊂ N such that
Bt+ = σ(Bt , Nt ).
(8.10)
B-01a
B0+ = σ(N0 ),
(8.11)
B-01b
In particular,
i.e. in B0+ are only trivial events.
Proof. (8.11) is a direct consequence of the fact that τ ≡ 0 is a stopping time w.r.t. the
filtration {Bt+ , t ≥ 0} and the strong Markov property ii), which proves that any event
from B0+ has to be independent of itself. As for (8.10) let Bs,t be the smallest σ-algebra
generated by Bt − Bs , t ≥ s and Bt,∞ be the σ-algebra generated by Bt,s for s ≥ t. Note
that B∞ = σ (Bt , Bt,∞ ). Let X be Bt+ measurable. Then X = ξ + η, where ξ = E[X|Bt ]
and η = X − ξ. Note however that η is independent of Bt,∞ (since it is Bt+ -measurable).
For any A ∈ Bt and B ∈ Bt,∞ we can write therefore
E[η, A ∩ B] = E[η, A]P[B] = E[X − E[X|Bt ], A]P[B] = 0.
But A ∩ B, A ∈ Bt , B ∈ Bt,∞ is π-system for B∞ and thus η ≡ 0 a.s. This ends the proof
of the corollary.
The above fact motivates us to the following definition.
Definition. We say that a filtration {Ft , t ≥ 0} satisfies the usual conditions if:
i) F0 contains all sets of P measure 0,
ii) Ft+ = Ft for all t ≥ 0.
An application: the reflection principle. We prove the following result, called the
reflection principle:
lm010812
Proposition 8.3. For any T > 0, x ≥ 0 we have
P[ sup Bt ≥ x] = 2P[BT ≥ x].
t∈[0,T ]
(8.12)
2.010b
34
T.KOMOROWSKI
Proof: Let τx := min[t : Bt = x], or τx = +∞ if the set is empty. This stopping time is
called a hitting time at x. Let σ := τx ∧ (T + 1). We have
P[ sup Bt ≥ x] = P[ sup Bt ≥ x, BT > x] + P[ sup Bt ≥ x, BT ≤ x]
t∈[0,T ]
t∈[0,T ]
(8.13)
t∈[0,T ]
= P[BT ≥ x] + P[σ ≤ T, BT ≤ x].
However, since Bt+σ − Bσ , t ≥ 0 is a Brownian motion independent of Fσ we can write
P[σ ≤ T, BT ≤ x] = P[σ ≤ T, BT − Bσ ≤ 0] = P[σ ≤ T ]P[B(T −σ)∨0 ≤ 0].
(8.14)
Since {−Bt , t ≥ 0} and {Bt , t ≥ 0} have the same laws we obtain that
1
P[B(T −σ)∨0 ≤ 0] = P[−B(T −σ)∨0 ≤ 0] =
2
and therefore, from (8.13) and (8.14) we get (8.12).
Exercise 1. i) Find the density of the distribution of the hitting time τx .
ii) Prove that P[τx < +∞] = 1 and Eτx = +∞.
Exercise 2. (Ornstein-Uhlenbeck process) Prove that for any x and a standard Brownian
motion {Bt , t ≥ 0} equation
Z t
Xt = x −
Xs ds + Bt , t ≥ 0
0
has a unique solution. It is called an Ornstein-Uhlenbeck process. Find formula for the solution and show thatRthe process is Gaussian. Compute its mean and covariance functions.
t
Hint: Denote Nt := 0 Xs ds and solve the resulting linear equation.
9. Martingales
Definiton. Suppose that {Ft , t ≥ 0} is a filtration. The process {Xt , t ≥ 0} is called a
(sub, resp. super)-martingale with respect to the given filtration if
i) it is {Ft , t ≥ 0} adapted, i.e. Xt is Ft -measurable for each t ≥ 0,
ii) E|Xt | < +∞ for all t ≥ 0,
iii) for any t ≥ s ≥ 0 we have
E[Xt |Fs ](≥, resp. ≤) = Xs .
Exercise. Suppose that {Xt , t ≥ 0} is a submartingale (resp. supermartingale). Then,
{−Xt , t ≥ 0} is a supermartingale (resp. submartingale).
9.1. Jensen inequalities.
jensen
Proposition 9.1. Suppose that {Xt , t ≥ 0} is {Ft , t ≥ 0}-martingale and f : R → R is
convex (resp. concave) such that E|f (Xt )| < +∞ for all t ≥ 0. Then, {f (Xt ), t ≥ 0} is
{Ft , t ≥ 0}-submartingale (resp. supermartingale).
Proof. Since f is convex for each x there exists a measurable function
f (y) − f (x)
C(x) = lim
y→x+
y−x
2.012
STOCHASTIC PROCESSES
35
such that we have
f (y) ≥ f (x) + C(x)(y − x), ∀ y ∈ R.
Let t > s and A ∈ Fs . For a given N we let AN := [A ∩ |C(Xs )| ≤ N ] ∈ Fs . Then
f (Xt ) ≥ f (Xs ) + C(Xs )(Xt − Xs )
and
E[f (Xt ), AN ] ≥ E[f (Xs ), AN ] + E[C(Xs )(Xt − Xs ), AN ].
(9.1)
012912
Note that
E[C(Xs )(Xt − Xs ), AN ] = E[C(Xs )(E[Xt |Fs ] − Xs ), AN ] = 0.
thus E[f (Xt ), AN ] ≥ E[f (Xs ), AN ] and letting N → +∞ we get
E[f (Xt ), A] ≥ E[f (Xs ), A],
∀ A ∈ Fs .
jensen-1
Proposition 9.2. Suppose that {Xt , t ≥ 0} is a non-negative {Ft , t ≥ 0}-submartingale
and f : [0, +∞) → R is convex, increasing such that E|f (Xt )| < +∞ for all t ≥ 0. Then,
{f (Xt ), t ≥ 0} is {Ft , t ≥ 0}-submartingale.
Proof. Note that since f is increasing we have C(x) ≥ 0. Picking up from (9.1) we can
write that
E[C(Xs )(Xt − Xs ), AN ] = E[C(Xs )(E[Xt |Fs ] − Xs ), AN ] ≥ 0
so the remaining part of the argument from the previous proposition holds as well.
9.2. Optional stopping theorem.
stop
Theorem 9.3. Suppose that {Xn , n ≥ 0} is an {Fn , n ≥ 0}-submartingale and τ ≥ σ are
two stopping times. Then for any N ≥ 0 we have
h
i
E Xτ ∧N Fσ∧N ≥ Xσ∧N .
(9.2)
Proof. Suppose that A ∈ Fσ∧N then
E [Xτ ∧N , A]
+∞ N
−1
+∞
X
X
X
E [Xk , A, σ = j, τ = k] +
=
E [XN , A, σ = j, τ ≥ N ]
j=1 k=j
=
+∞ N
−1
X
X
(9.3)
j=1
{E [Xk , A, σ = j, τ ≥ k] − E [Xk , A, σ = j, τ ≥ k + 1]}
j=1 k=j
+
+∞
X
E [XN , A, σ = j, τ ≥ N ] .
j=1
We adopt the convention that if the upper limit of summation is smaller than the lower
one the resulting sum equals 0. But A ∩ [σ = j] ∩ [τ ≥ k] belongs to Fk−1 for j < k. When
021801
t-samp-mart
36
T.KOMOROWSKI
j = k the event equals A ∩ [σ = j]. Thus the utmost right hand side of (9.3) is greater
than or equal to
N
−1
X
{E [XN −2 , A, σ = j, τ ≥ N − 1] − E [XN −1 , A, σ = j, τ ≥ N ] + E [XN −3 , A, σ = j, τ ≥ N − 2]
j=1
−E [XN −2 , A, σ = j, τ ≥ N − 1] + E [XN −4 , A, σ = j, τ ≥ N − 3] − . . .
+∞
X
+E [Xj , A, σ = j] − E [Xj , A, σ = j, τ ≥ j + 1]} +
E [XN , A, σ = j, τ ≥ N ]
j=1
=
N
−1
X
E [Xj , A, σ = j] −
j=1
N
−1
X
E [XN −1 , A, σ = j, τ ≥ N ] +
j=1
+∞
X
E [XN , A, σ = j, τ ≥ N ] .
j=1
The second sum on the utmost right hand side equals
N
−1
X
E [XN −1 , A, σ = j, τ ≥ N − 1] ,
j=1
while the third one equals
N
−1
X
E [XN , A, σ = j, τ ≥ N ] +
j=1
+∞
X
E [XN , A, σ = j, τ ≥ N ] .
j=N
Using again the submartingale property for the first term (recall that [σ = j, τ ≥ N ] is
FN −1 -measurable for j ≤ N − 1) and the fact that [σ = j, τ ≥ N ] = [σ = j] for j ≥ N we
get that this expression is greater, or equal than
N
−1
X
E [XN −1 , A, σ = j, τ ≥ N ] +
j=1
+∞
X
E [XN , A, σ = j] .
j=N
Thus the utmost left hand side of (9.3) is estimated from below by
E[Xσ∧N , A]
and (9.2) follows. The proof for a supermaringale {Xn , n ≥ 0} is a consequence of the
proof for the submartingales as {−Xn , n ≥ 0} is a submartingale.
Corollary 9.4. Suppose that {Xn , n ≥ 0} is an {Fn , n ≥ 0}-martingale and τ ≥ σ are
two stopping times. Then for any N ≥ 0 we have
h
i
E Xτ ∧N Fσ∧N = Xσ∧N .
Example. Suppose that {ξk , k ≥ 1} is a sequence of i.i.d. Bernoulli distrtibuted random
variables
Pnwith P[ξ1 = −1] = P[ξ1 = 1] = 1/2. Consider the random sequence X0 = 0 and
Xn = k=1 ξk (random walk).
i) Prove that {Xn , n ≥ 0} is a martingale.
(9.4)
STOCHASTIC PROCESSES
37
ii) Fix an arbitrary x ∈ Z. Let Hx be the first passage time through x, i.e. Hx :=
inf[n ≥ 0 : Xn = x]. We make a convention that the infimum of an empty set
equals ∞. Prove that Hx is a stopping time and P[Hx < +∞] = 1.
Hint: Let f (x) := P[Hx < +∞]. First observe that f (0) = 1. It is easy to see
that
1
1
P[Hx < +∞] = P[Hx < +∞|X1 = −1] + P[Hx < +∞|X1 = 1]
2
2
1
1
= P[Hx+1 < +∞] + P[Hx−1 < +∞]
2
2
and in consequence
1
f (x) = [f (x − 1) + f (x + 1)].
2
Observe that f (x) ∈ [0, 1] thus f (x) ≡ 1. Prove it by induction. Suppose we have
shown already that f (x) ≡ 1 for |x| ≤ N . Then
1
1
1 = f (N ) = [f (N − 1) + f (N + 1)] = [1 + f (N + 1)]
2
2
and f (N + 1) = 1. Likewise we get f (−N − 1) = 1.
iii) Note that Hx ≥ 0 and E[XHx ] = x and E[X0 ] = 0. They are not equal if x 6= 0,
so the optional stopping theorem does not hold without taking minimum with a
deterministic time.
stop-cor
Corollary 9.5. Suppose that {Xn , n ≥ 0} is an {Fn , n ≥ 0}-submartingale (resp. martingale) and τ is a stopping time. Then {Xn∧τ , n ≥ 0} is a submartingale (resp. martingale).
Proof. Exercise.
stop-thm
Theorem 9.6. Suppose that {Xt , t ≥ 0} is a nonnegative {Ft , n ≥ 0}-submartingale
(resp. supermartingale) whose realizations are right continuous a.s. and τ ≥ σ are two
stopping times. Then for any T ≥ 0 we have
h
i
E Xτ ∧T Fσ∧T ≥ ( resp. ≤)Xσ∧T .
(9.5)
Proof. Define
[nτ ] + 1
[nσ] + 1
, σn :=
n
n
Exercise. Show that τn , σn are stopping times.
Of course τn → τ +, σn → σ+. We also have τn ≥ σn ≥ σ. For any A ∈ Fσ we have
A ∈ Fσn , hence from Theorem 9.3 we conclude that
τn :=
E[Xτn ∧T , A] ≥ E[Xσn ∧T , A].
Since the submartingale is right continuous we have
lim Xτn ∧T = Xτ ∧T ,
n→+∞
lim Xσn ∧T = Xσ∧T .
n→+∞
023012
38
T.KOMOROWSKI
It remains to substantiate the passage with the limit under the integral sign. We have
Xτn ∧T ≤ E[XT Fτn ∧T ].
We prove that for any > 0 we can find δ > 0 such that for P[B] < δ we have
E [Xτn ∧T , B] < ,
∀ n ≥ 1.
(9.6)
UI
This condition is called uniform integrability and it suffices to claim the possibility of
passing with the limit under the integral, thus we obtain
E[Xτ ∧T , A] = lim E[Xτn ∧T , A] ≥ lim E[Xσn ∧T , A] = E[Xσ∧T , A]
n→+∞
n→+∞
and (9.8) follows.
To prove (9.7) observe that for any > 0 one can find δ > 0 such that for P[B] < δ we
have
E [XT , B] < , .
(9.7)
2
Suppose that M > 0. Then
1
P [Xτn ∧T ≥ M ] ≤
E [Xτn ∧T , Xτn ∧T ≥ M ]
M
1
δ
1
E [XT , Xτn ∧T ≥ M ] ≤
E [XT ] < , ∀ n ≥ 1,
≤
M
M
2
provided M > 1 is sufficiently large. Thus, for any B such that P[B] < min[δ, /(2M )] we
get
E [Xτn ∧T , B] = E [Xτn ∧T , Xτn ∧T ≥ M ] + E [Xτn ∧T , B, Xτn ∧T ≤ M ]
≤ E [XT , Xτn ∧T ≥ M ] + E [Xτn ∧T , B, Xτn ∧T ≤ M ] < + M P[B] < ,
2
and (9.7) follows.
We also have the following.
op-thm-mart
UI
∀n ≥ 1
Theorem 9.7. Suppose that {Xt , t ≥ 0} is a {Ft , n ≥ 0}-martingale whose realizations
are right continuous a.s. and τ ≥ σ are two stopping times. Then for any T ≥ 0 we have
h
i
E Xτ ∧T Fσ∧T = Xσ∧T .
(9.8)
023012
This time we use the fact that {|Xt |, t ≥ 0} is a nonnegative {Ft , t ≥ 0}-submartingale
and repeat the argument for uniform integrability used in the proof of the previous theorem.
9.3. Doob’s inequality.
doob-1
Theorem 9.8. (Maximal inequality) Suppose that {Xn , n ≥ 0} is a non-negative submartingale w.r.t. the filtration {Fn , n ≥ 0}. Then,
1
P max Xn ≥ λ ≤ E XN , max Xn ≥ λ , ∀ λ > 0, N ≥ 0.
(9.9)
0≤n≤N
0≤n≤N
λ
041801
STOCHASTIC PROCESSES
39
Proof. Suppose that τ := N and σ := inf[n ≥ 0 : Xn ≥ λ], or σ = +∞ if the set under
infimum is empty. It is a stopping time (check it as an exercise!). Then, the event under
probability on the left hand side of (9.9) equals [σ ≤ N ≥ λ]. It belongs to Fσ (check it as
an exercise!). We have
i
h h
i
E[Xτ ∧N , σ ≤ N ] = E E Xτ ∧N Fσ , σ ≤ N .
According to the optional sampling theorem the right hand side is greater than, or equal
to
i
h h
i
E E Xσ∧N Fσ , σ ≤ N = E [Xσ∧N , σ ≤ N ] ≥ λP[σ ≤ N ]
and (9.9) follows.
In continuous time we have the following.
doob-2
Theorem 9.9. (Maximal inequality) Suppose that {Xt , t ≥ 0} is a non-negative, right
continuous submartingale w.r.t. the filtration {Ft , t ≥ 0}. Then,
1
P sup Xt ≥ λ ≤ E XT , sup Xt ≥ λ , ∀ λ > 0, T ≥ 0.
(9.10)
λ
0≤t≤T
0≤t≤T
051801
Proof. Suppose that Tn (T ) is the set of all n-th generation dyadic numbers in [0, T ]. Let
"
#
An :=
sup Xt ≥ λ .
t∈Tn (T )
We have An ⊂ An+1 and, from the right continuity,
[
sup Xt > λ =
An .
0≤t≤T
We have
P
n≥0
sup Xt > λ = lim P[An ].
0≤t≤T
n→+∞
On the other hand, from Theorem 9.8, we get
1
P[An ] ≤ E [XT , An ] .
λ
Letting n → +∞, we conclude by the Lebesgue dominated convergence theorem that
1
P sup Xt > λ ≤ E XT , sup Xt ≥ λ , ∀ λ > 0, T ≥ 0.
λ
0≤t≤T
0≤t≤T
This ends the proof of the theorem.
Exercise. Prove from the above the inequality in the form required in the theorem. mart-cor-1
Corollary 9.10. In addition to the assumptions of Theorem 9.9 we assume that f :
[0, +∞) → [0, +∞) is a non-decreasing, convex function such that Ef (Xt ) < +∞ for
all t ≥ 0. Then
1
P sup f (Xt ) ≥ λ ≤ E f (XT ), sup f (Xt ) ≥ λ , ∀ λ > 0, T ≥ 0.
(9.11)
λ
0≤t≤T
0≤t≤T
061801
40
mart-cor-2
T.KOMOROWSKI
Corollary 9.11. Assume that {Xt , t ≥ 0} is a martingale w.r.t. the filtration {Ft , t ≥ 0}
and f : R → [0, +∞) is a convex function such that Ef (Xt ) < +∞ for all t ≥ 0. Then
1
P sup f (Xt ) ≥ λ ≤ E f (XT ), sup f (Xt ) ≥ λ , ∀ λ > 0, T ≥ 0.
(9.12)
λ
0≤t≤T
0≤t≤T
071801
Exercise. Prove both of these corollaries.
9.4. Marcinkiewicz real interpolation theorem. We start with the following simple
observation.
lm5-1
Lemma 9.12. Suppose that (X, Σ, µ) is a measure space and F : X → [0, +∞] is a
measurable function. Then, for any p ≥ 1
Z
Z+∞
f p dµ = p
λp−1 µ[F ≥ λ]dλ.
(9.13)
111901
0
Proof. We have
Z
µ[F ≥ λ] =
1[F ≥λ] dµ
hence,
Z+∞
Z+∞Z
p−1
p
λ µ[F ≥ λ]dλ = p
λp−1 1[F ≥λ] dλdµ
0
Fubini
= p
Z
0



 +∞
Z
Z ZF
Z
p−1
p−1
 λ 1[F ≥λ] dλ dµ = p  λ dλ dµ = F p dµ.
0
0
Denote by L0 (L0 (X, Σ, µ)) the space of all measurable functions f : X → [−∞, +∞].
Suppose that 1 ≤ p1 < p2 < +∞ and T : Lp1 + Lp2 → L0 is a subadditive operator, i.e.
|T (f + g)| ≤ |T f | + |T g|,
∀ f, g ∈ Lp1 + Lp2 .
(9.14)
031402
Definition 9.13. We say that the operator T is of type (p, q) if there exists +∞ > C > 0
such that
kT f kLq ≤ Ckf kLp ∀ f ∈ Lp .
The infimum of those C for which the above inequality holds shall be denoted by kT kp,q .
Definition 9.14. Suppose that 1 ≤ p ≤ +∞ and 1 ≤ q ≤ +∞. We say that the operator
T is of weak (p, q) type if
a) in case when q < +∞ there exists a constant C > 0 such that
µ[|T f | ≥ λ] ≤
Ckf kqLp
,
λq
∀ f ∈ Lp .
Define the infimum of such C-s as kT kw
p,q .
b) in case when q = ∞: the operator is of type (p, q).
(9.15)
111902
rcinkiewicz
STOCHASTIC PROCESSES
41
The following result holds.
Theorem 9.15. (Marcinkiewicz interpolation theorem) Suppose that 1 ≤ p1 < p2 < +∞
and T is a subadditive operator acting on Lp1 + Lp2 that is of both weakly (p1 , p1 ) and
(p2 , p2 ) types. Assume also that p ∈ (p1 , p2 ) and
1
θ
1−θ
=
+
p
p1
p2
p
p1
for some θ ∈ (0, 1). Then, we have T (L ∩ (L + Lp2 )) ⊂ Lp and
kT f kLp ≤ Ckf kp .
w
The constant C depends only on kT kw
p1 ,p1 , kT kp2 ,p2 p − p1 , p2 − p.
Proof. For any λ > 0 set fλ := f 1[|f |≤λ] and f (λ) := f 1[|f |>λ] . We have
µ[|T f | ≥ 2λ] ≤ µ[|T (fλ )| ≥ λ] + µ[|T (f (λ) )| ≥ λ]
Since the operator T is both weak (p1 , p1 ) and (p2 , p2 ) types we can write that the right
hand side of the above inequality can be further estimated from above by
Z
Z
C1
C2
p1
p2
|f | dµ + p2
|f | dµ .
λp1
λ
[|f |>λ]
[|f |≤λ]
We obtain therefore
Z+∞
Z
p−1
λ µ[|T f | ≥ 2λ]dλ ≤ C1
+∞
λ
p−p1 −1
Z
Z
+∞
p−p2 −1
+C2
Z
Z
λ
|f | dµ
dλ
[|f |>λ]
0
0
p1
Z
p2
|f | dµ
dλ
[|f |≤λ]
0
|f |
p−p1 −1
Z
Z
+∞
|f | dµ
λp−p2 −1 dλ
0
|f |
Z
Z
C
C1
2
|f |p dµ +
|f |p dµ
≤
p − p1
p2 − p
and the theorem follows from Lemma 9.12.
Remark. When T is linear it extends continuously from Πp := Lp ∩ (Lp1 + Lp2 ) to the
entire Lp . Indeed, according to the above theorem we have
≤ C1
p1
|f | dµ
λ
dλ + C2
p2
kT f − T gkLp = kT (f − g)kLp ≤ Ckf − gkLp
thus, T is continuous and allows for extension to the Lp –closure of Πp , which is Lp . A
similar conclusion can be reached for |T f | if T is only sublinear.
Exercise. The above result extends also to the case when p2 = +∞
Application to martingales. For a given non-negative submartingale f := {Xt , T ≥
t ≥ 0} define its norm kf kLp := (EXTp )1/p for p > 1 and operator
T f := sup Xt .
t∈[0,S]
upcrossings
convergence
42
T.KOMOROWSKI
By Corollary 9.10 it is of weak type (p, p) for any p > 1. According to theorem 9.9 it is
also of wek type (1, 1). Using Theorem 9.15 we conclude that it is of strong type (q, q) for
all 1 < q < p.
9.5. Upcrossing-downcrossing inequalities. Let a < b. Define N0 := −1 and then by
induction
N2k−1 := inf[m > N2k−2 : Xm ≤ a],
N2k := inf[m > N2k−1 : Xm ≥ b]
with the convention that inf over an empty set equals ∞ amd once some Nk = ∞ we have
N` = +∞ for all ` ≥ k.
Exercise. Prove that all Nk , k ≥ 1 are stopping times. Define
1, for all N2k−1 < m ≤ N2k
Hm =
0, if otherwise.
In particular each Hm is Fm−1 -measurable.
Let also Un := sup[k : N2k ≤ n] be a number of upcrossing completed by time n.
Theorem 9.16. Suppose that {Xn , n ≥ 0} is a submartingale. Then
1 EUn ≤
E(Xn − a)+ − E(X0 − a)+ .
b−a
(9.16)
Proof. Note that Yn := (Xn − a)+ is a submartingale. It upcrossess [0, b − a] the same
number of times Xn upcrosses [a, b]. Define (H · Y )0 = 0 and
n
X
Hm (Ym − Ym−1 ) ∀ n ≥ 1.
(H · Y )n :=
m=1
It is a submartingale. Indeed, for n ≥ 1
h
i
h
i
E (H · Y )n Fn−1 = (H · Y )n−1 + Hn E Yn − Yn−1 Fn−1 ≥ (H · Y )n−1 .
We also have
(b − a)Un ≤ (H · Y )n .
Let Km := 1 − Hm . We have
Yn − Y0 = (H · Y )n + (K · Y )n
and by the submartingale property
E(K · Y )n ≥ E(K · Y )0 = 0
So
E[Yn − Y0 ] ≥ E(H · Y )n ≥ (b − a)EUn .
As a corollary we conclude the following martingale convergence theorem.
Corollary 9.17. Suppose that {Xn n ≥ 0} is a submartingale with supn≥0 Xn+ < +∞.
Then, as n → +∞ we have X := limn→+∞ Xn exists a.s. and E|X| < +∞.
up-estimate
convergence
STOCHASTIC PROCESSES
43
Proof. Since (X − a)+ ≤ X + |a| for any b > a we have
|a| + EXn+
.
b−a
We have U := limn→+∞ Un is the number of upcrossings of [a, b] for the martingale. Due
to the assumptions we conclude that EU < +∞. Hence in particular U <= ∞ a.s. Note
that the event N := [U < +∞, for all rational a < b] satisfies P[N ] = 1. On the other
hand
[
[lim inf Xn < a < b < lim sup Xn ] ⊂ N c
EUn ≤
a,b∈Q
hence lim inf Xn = lim sup Xn a.s. Let X := limn→+∞ Xn . By Fatou lemma we get
EX + ≤ lim inf Xn+ .
n→+∞
Note that
EXn− = EXn+ − EXn ≤ EXn+ − EX0
and another application of Fatou lemma yields
EX − ≤ lim inf Xn+ − EX0 < +∞.
n→+∞
that ends the proof.
For continuous time martingales we obtain.
Corollary 9.18. Suppose that {Xt t ≥ 0} is a submartingale. Then, the following holds
i) There exists an event N such that P[N c ] = 1 such that for every ω ∈ N c
Xt+ (ω) :=
lim
s→t+,s∈Q
Xs ,
Xt− (ω) :=
lim
s→t−,s∈Q
Xs
exist for all t ≥ 0,
ii)
E[Xt+ |Ft ] ≥ Xt , E[Xt |Ft− ] ≥ Xt− ,
iii) {Xt+ t ≥ 0} is a cadlag submartingale.
Exercise.
∀ t ≥ 0,
44
T.KOMOROWSKI
Part 2. Markov Processes
10. Markov property. Definition of a Markov process
10.1. Generalities. Suppose that (E, ρ) is a Polish metric space with the Borel σ-algebra
B(E) and (Ω, F, P) is a probability space. Denote by Bb (E) the set of all Borel measurable,
bounded functions and P(E) the set of all Borel proability measures.
Markov property. Definition nr 1. Consider the set T that can be one of the
following N = {0, 1, . . . , }, Z, R, or R+ . Suppose that {Xt , t ∈ T } is a stochastic process
taking values in E. We say that it is Markov if for any h, t ∈ T , h ≥ 0 the following holds
E[f (Xt+h )|Ft ] = E[f (Xt+h )|Xt ],
∀ f ∈ Bb (E).
(10.1)
markov
Here Ft := σ(Xs , s ≤ t).
Exercise. Show that it suffices only to verify (10.2) for f of the form f = 1A and
A ∈ B(E).
Terminology. In case T = N, Z processes are called Markov chains and shall write
{Xn n ∈ T }. When T = R, or R+ but E is countable the processes are called Markov
processes with countable state space. When both T = N, Z and E is countable we say that
{Xn n ∈ T } is a countable state space Markov chain.
Transition probability functions. A family of measures (s, t, x) 7→ Ps,t (x, ·) defined for
s ≤ t, x ∈ E and taking values in P(E) is calles transition probability functions if for any
s ≤ t, A ∈ B(E),
i) x 7→ Ps,t (x, A) is Borel measurable,
ii) Ps,s (x, A) = 1A (x)(= δx (A)),
iii) (Chapman-Kolmogorov equations)
Z
Ps,u (x, dy)Pu,t (y, A) = Ps,t (x, A),
∀ u ∈ [s, t], x ∈ E.
E
Markov property. Definition nr 2. Suppose that we are given a family Z of
transition probability functions. We say that a process {Xt , t ∈ T } is Markov w.r.t.
to the family Z, if for any h, t ∈ T , h ≥ 0 the following holds
Z
E[f (Xt+h )|Ft ] =
f (y)Pt,t+h (Xt , dy) ∀ f ∈ Bb (E).
(10.2)
E
Exercise. Show that any process that is Markovian in the sense of Definition 2 is also
Markovian in the sense of definition 1.
Remark. Converse to the above is also true under some
very mild assumptions on the
kuznetzov
space E (including all Polish spaces considered here), see [6].
In case of Markov processes with countable state space the transition probability functions can be defined as the family of functions Ps,t : E × E → [0, +∞) satisfying: for any
s ≤ t, x ∈ E,
P
i) (stochasticity) y Ps,t (x, y) = 1,
ii) Ps,s (x, y) = δx,y ,
markov
STOCHASTIC PROCESSES
iii) (Chapman-Kolmogorov equations)
X
Ps,u (x, z)Pu,t (z, y) = Ps,t (x, y),
45
∀ u ∈ [s, t], y ∈ E.
z
When E is finite E = {1, . . . , N } we can think about {Ps,t , s ≤ t} as a family of nonnegative entry matrices such that
i) (stochasticity) Ps,t 1 = 1,
ii) Ps,s = IN ,
iii) (Chapman-Kolmogorov equations)
Ps,u Pu,t = Ps,t ,
∀ u ∈ [s, t].
Here 1T = [1, . . . , 1].
10.2. Path measures. To simplify considerations suppose that T = R (other cases can
be done similarly). Fix µ ∈ E and s ∈ R. For any n ≥ 0 and s = t0 ≤ t1 ≤ . . . ≤ tn we
define a measure on E n+1 := E × . . . × E (n + 1-time cartesian product) inductively by
Pt0 (A) = µ(A),
A ∈ B(E).
Suppose that Pt0 ,...,tn ∈ P(E n+1 ). We let then for any C = A × B such that A ∈ B(E n+1 ),
B ∈ B(E)
Z
Pt0 ,...,tn+1 (C) =
Pt0 ,...,tn+1 (dy0 , . . . , dyn )Ptn ,tn+1 (yn , B).
A
Recall that E[s, +∞) is the space of all functions π : [s, +∞→ E. Denote the corresponding
algebra of cylindrical sets and cylindrical σ-algebra by Cs and σ(Cs ) respectively. Also we
can define a canonical process Πt := π(t), t ∈ R and a filtration of σ-algebras Ms,t :=
σ(Πu , u ∈ [s, t]).
Exercise.
1) Prove that Pt0 ,...,tn+1 (·) extends uniquely to a measure from P(E n+1 ). Hint. Define
a finite additive set function on the algebra of finite disjoint sum of rectangles and
then use Theorem 4.1.
2) Prove that the family Pt0 ,...,tn (·) satisfies the consistency condition (3.1).
Using a slight extension of Theorem 3.1 where Polish space is used instead of R (the proof
remains the same) we conclude that there exists a measure Ps,µ on (E[s, +∞), σ(Cs )) such
that for any s = t0 ≤ . . . ≤ tn and A ∈ B(E n+1 ) we have
Ps,µ [(Πt0 , . . . , Πtn ) ∈ A] = Pt0 ,...,tn (A).
(10.3)
In case µ = δx we shall write Ps,x instead Ps,δx . The corresponding expectations shall be
denoted by Es,µ and Es,x . We conclude easily the following.
path-meas
Theorem 10.1. Given a family of transition probability functions there exists a family of
probability measures {Ps,x , (s, x) ∈ R×E} on (E[s, +∞), σ(Cs )), s ∈ R that satisfies (10.3)
and such that:
1) Ps,x [Πs = x] = 1 for all (s, x) ∈ R × E,
2) x 7→ Es,x f (Πt ) is Borel measurable for any f ∈ Bb (E) and s ∈ R,
011402
46
T.KOMOROWSKI
3) for any t ≥ s, h ≥ 0, f ∈ Bb (E) and x ∈ E we have
Es,x [f (Πt+h )|Ms,t ] = Et,Πt f (Πt+h ).
Conversely, given a family of probability measures {Ps,x , (s, x) ∈ R×E} on (E[s, +∞), σ(Cs )),
s ∈ R that satisfies 1)-3) above there exist transition probability functions Ps,t (x, ·) such
that (10.3) holds.
Proof. Exercise. For the converse take Ps,t (x, A) := Ps,x [Πt ∈ A] = Es,x 1A (Πt ).
Corollary 10.2. Given a family of the transition probability functions, time s ∈ R and
a measure µ ∈ P(E) there exists a probability space (Ω, F, P) and a process {Xt , t ≥ 0}
given over that space such that: 1) the law of Xs equals µ and 2) (10.2) holds. Moreover,
any process satisfying 1) and 2) has the law Ps,µ .
Proof. To prove the existence take as the probability space (E[s, +∞), σ(Cs ), Ps,µ ) and let
Xt := Πt , t ≥ s. To identify the law use the Markov property (Exercise)
Markov family. By a Markov family we understand a process {Xt , t ∈ R}, a family of
measures Ps,x on (Ω, F) such that
i) Ps,x [Xs = x] = 1 for all (s, x) ∈ R × E,
ii) x 7→ Es,x f (Xt ) is Borel measurable for any f ∈ Bb (E) and s ∈ R,
iii) for any t ≥ s, h ≥ 0, f ∈ Bb (E) and x ∈ E we have
Es,x [f (Xt+h )|Fs,t ] = Et,Πt f (Xt+h ).
Here Es,x are the corresponding expectations.
Exercise. Show that {Xt , t ≥ s} is a Markov process over (Ω, F, Ps,x ). Hint. Note that
Ps,t (x, A) := Ps,x [Xt ∈ A] are the transition probability functions.
10.3. Shift operator. Another statement of Markov property. Consider a family
of operators θh : E[s, +∞) → E[s, +∞), t ≥ 0 given by θh (π)(t) := π(t), t ≥ 0. Observe
that, with F (π) := f (Πt (π)), where f ∈ Bb (E), condition 3) from Theorem 10.1 can be
rewritten as
Es,µ [F ◦ θh |Ms,t ] = Et,Πt F,
(10.4)
for any t ≥ s, h ≥ 0 and µ ∈ P(E). Using (10.2) we can show that (10.4) holds for
functions of the form
N
Y
F (π) =
fi (Πti (π)),
i=1
where fi ∈ Bb (E), i = 1, . . . , N .
Exercise. Prove it!!!
In fact using Dynkin’s theorem on π − λ systems, see Theorem 2.1, we can further
generalize (10.4) and obtain.
markov2
Proposition 10.3. Equality (10.4) holds for any bounded, Borel measurable function F :
E[0, +∞) → R.
markov1
STOCHASTIC PROCESSES
47
Proof. Exercise. Prove it!!!
Recall that given a stochastic process {Xt , t ≥ 0} over (Ω, F) we are given a mapping
X : Ω → E[0, +∞) by X(ω)(t) := Xt (ω).
Definition nr 3. Markov property. Suppose that we are given a family of measures
Ps,x on path spaces E[s, +∞) satisfying conditions 1)-3) from Theorem 10.1 (or equivalently
transition probability functions Ps,t (x, ·)). We say that a process {Xt , t ≥ 0} satisfies
Markov property with respect to the given family of measures if
E[F ◦ θh (X)|Fs,t ] = Et,X(t) F
(10.5)
markov3
for any bounded, Borel mapping F : E[0, +∞) → R, h ≥ 0, t ≥ s.
Note that this definition is equivalent with Definition nr 2. It implies the latter if we
use F (π) := π(t).
Exercise. Prove that Definition nr 2 implies Definition nr 3. Solution: Indeed, if
Definition nr 2 holds then we define the family Es,x that satisfies (10.4) for any any bounded,
Borel mapping F : E[0, +∞) → R. Let µ be the law of Xs . Then for any G that is Ms,t
measurable we can write
E [F ◦ θh (X)G(X)] = Es,µ [E[F ◦ θh |Fs,t ]G]
(10.4)
= Es,µ [Et,Πt F G] = E [Et,Xt F G(X)] .
10.4. Time homogeneous Markov process. Semigroups. In case Ps,t (x, A) = Pt−s (x, A)
we say that the process is time homogeneous. Most of our considerations shall be devoted
to study this class of processes. This family satisfies
i) x 7→ Ps (x, A) is Borel measurable,
ii) P0 (x, A) = 1A (x)(= δx (A)),
iii) (Chapman-Kolmogorov equations)
Z
Ps (x, dy)Pt (y, A) = Ps+t (x, A), ∀ s, t ≥ 0, x ∈ E.
E
In the Banach spaceR Bb (E) with the norm kf k∞ := supx∈E |f (x)| we define a family of
operators Pf f (x: = E f (y)Pt (x, dy). It satisfies
i) Pt : Bb (E) → Bb (E) is a linear operator satisfying Pt 1 = 1, Pt f ≥ 0 for any f ≥ 0
(such operators are called Markov),
ii) P0 = I,
iii) (Chapman-Kolmogorov equations) Pt Ps = Pt+s for all t, s ≥ 0.
Such a family is called a transition probablity semigroup.
Exercise. Show that 1) kPt k = 1 for all t ≥ 0 and 2) Pt f ≥ Pt g if f ≥ g.
Markov property in the homogeneous case. A process {Xt , t ≥ 0} is called a time
homogeneous Markov, corresponding to the transition probability semigroup {Pt , t ≥ 0},
if
E[f (Xt+h )|Ft ] = Ph f (Xt ) ∀ f ∈ Bb (E), t ≥ 0.
(10.6)
markovc
48
T.KOMOROWSKI
Suppose that we are given a transition probability semigroup we can define then the transition probability functions Pt (x, A) = Pt 1A (x) that, via Kolmogorov theorem, lead to the
definition of path measures Es,x , Es,µ . In fact time homogeneity allow us to consider only
the measures starting at s = 0. Denote them by Ex , Eµ . The equivalent formulation of the
Markov property, uses those measures and the shift operators and can be stated as follows.
thm012102
Theorem 10.4. A process {Xt , t ≥ 0} is a time homogeneous Markov corresponding to
the transition semigroup {Pt , t ≥ 0} iff the corresponding path measures {Px , x ∈ E}
satisfy
E[F ◦ θt (X)|Ft ] = EXt F ∀ F ∈ Bb (E[0, +∞)), t ≥ 0.
(10.7)
Here Ex is the expectation corresponding to Px .
10.5. Examples.
10.5.1. Arbitrary state space Markov chains. In this case it is enough to assign a family of
probability measures Pn (x, ·) such that
• x 7→ Pn (x, A) is measurable for any A ∈ B(E) and n ≥ 1.
They describe the probabilities of going from state x at time n to a set A at time n + 1.
We define
Pn,n (x, A) := δx (A), Pn,n+1 (x, A) := Pn (x, A)
and for any k ≥ 2
Z
Z
Pn,n+k (x, A) :=
. . . Pn (x, dy1 ) . . . Pn+k−2 (yk−2 , dyk−1 )Pn+k−1 (yk−1 , A),
E
| {z E}
k−1−times
where y0 := x.
Exercise. Show that Pn,n+k (x, A) constitute transition probability functions.
In case of homogeneous transition probabilities it is enough to define only the family
family of probability measures P (x, ·) such that
• x 7→ P (x, A) is measurable for any A ∈ B(E).
When E is countable we can drop the condition of measurability, as all functions are
measurable in this case. In fact since the measure is pure atomic it is enough to assign
only the transition probabilities pn (x, y) ≥ 0 such that
X
pn (x, z) = 1, ∀ x, y ∈ E.
z∈E
Then,
Pn (x, A) :=
X
pn (x, z)
z∈A
defines the transition probability measures. Of course in case of time homogeneous chains
it suffices only to assign p(x, y) ≥ 0 such that
X
p(x, z) = 1, ∀ x, y ∈ E.
z∈E
markovd
STOCHASTIC PROCESSES
49
10.5.2. Finite state, homogeneous Markov chains.PSuppose that N ≥ 1 is an integer and
Z := {1, . . . , N }. Suppose that p(i, j) ≥ 0 and N
j=1 p(i, j) = 1, i = 1, . . . , N . Matrix
P = [p(i, j)] is called stochastic. Denote by {Xn , n ≥ 0} the corresponding Markov chain.
Matrix L := P − I is called the generator of the chain.
Exercise. Show that L = [`(i, j)] is a generator iff
1) `(i, j) ≥ 0 for i 6= j and
X
`(i, i) = −
`(i, j).
j6=i
2) −1 ≤ `(i, i) ≤ 0 for all i = 1, . . . , N .
Exercise. Suppose that


f (1)
f =  ...  .
f (N )
Show that
N
X
Lf (i) =
p(i, j)[f (j) − f (i)].
(10.8)
j=1
Exercise. Prove the following property of the generator called a maximum principle.
Suppose that f is given by (10.8) and i0 is such that f (i0 ) = max{f (j), j = 1, . . . , N }.
Then Lf (i0 ) ≤ 0.
Exercise. A partial converse also holds. Suppose that p(i, j) > 0 for all i, j = 1, . . . , N
and Lf (i) ≤ 0 for all i = 1, . . . , N . Then f (i) ≡ f (1) for all i = 1, . . . , N . Hint. Suppose
that m = min f (i) and Z := [j : f (j) = m]. Show that if j ∈ Z then
X
p(j, k)[f (k) − f (j)] ≥ 0,
0 ≥ Lf (j) =
k
which implies that all f (k) = f (j) for all k.
10.5.3. Countable state, homogeneous Markov chains. In this case E is countable
and in
P
order to define a Markov chain it is enough to assign p(x, y) ≥ 0 such that y p(x, y) = 1.
Example 1. Random walks on lattice Zd . In this case
X
p(x, y) ≥ 0,
p(x, y) = 1.
y∈Zd
The nearest neighbor random walk p(x, y) = 0 if |x − y| > 1. In this case it suffices only
to know p(x, e) for |e| = 1 and e ∈ Zd . The symmetric simple random walk
1
p(x, e) = , |e| = 1, x ∈ Zd .
2d
Example. Birth and death chain. It is a time homogeneous Markov chain with E =
{0, 1, . . . , }(= N). Its transition probability function equals p(i, j) = 0 if |i − j| > 1. We
let p(i, i + 1) = pi , p(0, i) = ri for i ≥ 0 and p(i, i − 1) = qi for i ≥ 1. We also assume that
p0 = 1, and pi + ri + qi = 1 for all i ≥ 1.
f
50
T.KOMOROWSKI
10.5.4. Continuous time, countable state Markov processes. Suppose that Px is a path
measure on E N that corresponds to transition probabilities p(x, y), as in the previous
section. Assume also that λ : E → (0, +∞) is a bounded function that satisfies λ(x) ≥ λ0 >
0 for all x ∈ E. Given N ≥ 0 we construct a probability measure µx,N on (E ×(0, +∞))N +1
as follows: suppose that (x0 , . . . , xN ) ∈ E N +1 and A0 , . . . , AN ∈ B(0, +∞). By ξn (ω), τn (ω)
we denote mappings that to a given ω ∈ (E × (0, +∞))N assign respectively its E and
(0, +∞) coordinates of the projection onto the n-th component. Let
µx,N [{x0 } × A0 × . . . × {xN } × AN ]
Z
Z Y
N
=
...
λ(xi )e−λ(xi )τi Px σ ∈ E N : σ(0) = x0 , . . . , σ(N ) = xN dτ0 . . . τN .
A0
AN i=0
Exercise. Prove that {µx,N , N ≥ 0} satisfies the consistency condition, i.e.
X
µx,N +1 [{x0 }×A0 ×. . .×{xN }×AN ×{y}×(0, +∞))] = µx,N [{x0 }×A0 ×. . .×{xN }×AN ]
y∈E
for all (x0 , . . . , xN ) ∈ E N +1 and A0 , . . . , AN ∈ B(0, +∞).
By Kolmogorov theorem there exists a unique measure µx on the cylindrical σ-algebra
F of Ω := (E × (0, +∞))N such that
µx,N [(x0 , A0 , . . . , xN , AN )] = µx [(x0 , A0 , . . . , xN , AN )]
for all N ≥ 0, (x0 , . . . , xN ) ∈ E N +1 and A0 , . . . , AN ∈ B(0, +∞).
Exercise. Define Gn := σ(ξk , k ≤ n) and G the smallest σ-algebra generated by all Gn .
By νN (; {xn , n ≥ 0}) we denote a Borel probability measure on (0, +∞)N given by
Z
Z Y
N
νN [A0 × . . . × AN ] =
...
λ(xi )e−λ(xi )τi dτ0 . . . τN .
A0
AN i=0
This family satisfies the consistency condition. There exists therefore the measure ν(; {xn , n ≥
0}) on the cylindrical σ-algebra of (0, +∞)N . By the same symbol we denote the measure
induced on Ω by the canonical projection. Prove that
µx [·|G] = ν(·; {ξn , n ≥ 0}),
µx a.s.
In conclusion we obtain that the random variables {τn , n ≥ 0} are independent in µx [·|G].
Exercise. Let T0 := 0 and Tn+1 := Tn + τn for n ≥ 0. Prove that {(ξn , Tn ), n ≥ 0} is
a (time homogeneous) Markov chain on the state space E × (0, +∞) with the transition
probabilities
Z
P (x, s; y, A) = p(x, y) λ(x)e−λ(x)(t−s) 1[t>s] dt, A ∈ B(0, +∞).
A
On (Ω, F, µx ) define a stochastic process
Xt := ξn ,
if Tn ≤ t < Tn+1
for all n ≥ 0. To make sure that the process is defined for all times we need the following
STOCHASTIC PROCESSES
prop012202
51
Proposition 10.5. We have
T∗ := lim Tn = +∞,
n→+∞
µx a.s.
(10.9)
Proof. It is enough to show that (10.9) holds µx [·|G] a.s. for almost sure realization of
{ξn , n T
≥ 0}. Let Tn := σ(τk , k ≤ n) and Tn+ := σ(τk , k ≥ n). Observe that T∗ is
T∞ := n≥0 Tn+ measurable. On the other hand as τn are independent under µx [·|G] we
conclude that T∞ is trivial under this measure. In particular we can write for any a > 0
a ≥ Ex [T∗ , T∗ ≤ a|G] = lim Ex [Tn , T∗ ≤ a|G] = lim Ex [Tn |G]µx [T∗ ≤ a|G].
n→+∞
n→+∞
But
Ex [Tn |G] ≥ nλ−1
0 → +∞
so we have to have µx [T∗ ≤ a|G] = 0 for all a. Thus (10.9) follows.
thm012102
Theorem 10.6. The process {Xt , t ≥ 0} is Markovian.
References
[1] V. G. Asatiani, Z. A. Chanturia, The modulus of a function and the Banach indicatrix, Acta Sci.
Math., 45 (1983), 5f1-66.
[2] Yu.K. Belyaev, ”Continuity and Hólder’s conditions for sample functions of stationary Gaussian processes” , Proc. 4-th Berkeley Symp. Math. Stat. Probab., 2 , Univ. California Press (1961) pp. 2333
[3] J. Doob, Stochastic processes.
[4] R. Durrett, Probability. Theory and examples.
[5] W. Feller, Wstȩp do rachunku prawdopodonieństwa, tom 2.
[6] S.E. Kuznetsov, Any Markov process in a Borel space has a transition function Theory Probab. Appl.
, 25 : 2 (1980) pp. 384388
[7] I. P. Natanson, Theory of functions of finite variation.
012202