Non-homogeneous random walks
Lyapunov function methods for near-critical stochastic systems
Mikhail Menshikov, Serguei Popov, Andrew Wade
12th December 2015
ii
Contents
Preface
vii
Summary of notation
xv
1 Introduction
1.1 Random walks . . . . . . . . .
1.2 Simple random walk . . . . . .
1.3 Lamperti’s problem . . . . . . .
1.4 General random walk . . . . . .
1.5 Recurrence and transience . . .
1.6 Angular asymptotics . . . . . .
1.7 Centrally biased random walks
Bibliographical notes . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
2 Semimartingale approach and Markov
2.1 Definitions . . . . . . . . . . . . . . . .
2.2 An introductory example . . . . . . .
2.3 Fundamental semimartingale facts . .
2.4 Displacement and exit estimates . . .
2.5 Recurrence/transience criteria . . . . .
2.6 Positive recurrence . . . . . . . . . . .
2.7 Moments of hitting times . . . . . . .
2.8 Growth bounds on trajectories . . . .
Bibliographical notes . . . . . . . . . . . . .
3 Lamperti’s problem
3.1 Introduciton . . . . .
3.2 Markovian case . . .
3.3 General case . . . . .
3.4 Lyapunov functions .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
iii
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
chains
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
1
2
4
8
9
13
14
17
.
.
.
.
.
.
.
.
.
23
23
30
34
41
52
64
74
82
89
.
.
.
.
97
97
99
103
109
iv
Contents
3.5 Recurrence classification . . . . . . .
3.6 Irreducibility and regeneration . . .
3.7 Moments and tails of passage times .
3.8 Excursion durations and maxima . .
3.9 Almost-sure bounds on trajectories .
3.10 Transient theory in the critical case .
3.11 Nullity and weak limits . . . . . . .
3.12 Supercritical case . . . . . . . . . . .
3.13 Proofs for the Markovian case . . . .
Bibliographical notes . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
114
123
134
140
145
149
156
163
173
174
4 Many-dimensional random walks
185
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
4.1.1 Chapter overview: Anomalous recurrence . . . . . . . 185
4.1.2 Notation for many-dimensional random walks . . . . . 188
4.1.3 Beyond the Chung–Fuchs theorem . . . . . . . . . . . 189
4.1.4 Relation to Lamperti’s problem . . . . . . . . . . . . . 194
4.1.5 Sufficient conditions for recurrence and transience . . 198
4.2 Elliptic random walks . . . . . . . . . . . . . . . . . . . . . . 203
4.2.1 Recurrence classification . . . . . . . . . . . . . . . . . 203
4.2.2 Example: Increments supported on ellipsoids . . . . . 205
4.3 Controlled driftless random walks . . . . . . . . . . . . . . . . 207
4.4 Centrally biased random walks . . . . . . . . . . . . . . . . . 215
4.4.1 Model and notation . . . . . . . . . . . . . . . . . . . 215
4.4.2 Subcritical case . . . . . . . . . . . . . . . . . . . . . . 216
4.4.3 Supercritical case . . . . . . . . . . . . . . . . . . . . . 217
4.4.4 Critical case: Angular asymptotics . . . . . . . . . . . 222
4.4.5 Critical case: Recurrence classification and excursions 227
4.5 Range and local time . . . . . . . . . . . . . . . . . . . . . . . 229
Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . 234
5 Heavy tails
5.1 Chapter overview . . . . . . . . . . . . . . . .
5.2 Directional transience . . . . . . . . . . . . .
5.2.1 Introduction . . . . . . . . . . . . . .
5.2.2 A condition for directional transience
5.2.3 Almost-sure bounds . . . . . . . . . .
5.2.4 Moments of passage times . . . . . . .
5.2.5 Moments of last exit times . . . . . .
5.3 Oscillating random walk . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
239
. 239
. 240
. 240
. 241
. 244
. 247
. 251
. 254
v
Contents
5.3.1 Introduction . . . . . . . . . . . . . . . .
5.3.2 Relation to some two-dimensional random
5.3.3 Complexes of half-lines . . . . . . . . . . .
5.3.4 Lyapunov functions . . . . . . . . . . . .
5.3.5 Technical preliminaries . . . . . . . . . . .
5.3.6 Increment estimates . . . . . . . . . . . .
5.3.7 Proof of recurrence classification . . . . .
Bibliographical notes . . . . . . . . . . . . . . . . . . .
. . . .
walks
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
254
256
258
260
261
264
267
271
6 Further applications
275
6.1 Random walk in random environment . . . . . . . . . . . . . 275
6.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 275
6.1.2 Recurrence classification . . . . . . . . . . . . . . . . . 276
6.1.3 Almost-sure upper bounds . . . . . . . . . . . . . . . . 277
6.1.4 Almost-sure lower bounds . . . . . . . . . . . . . . . . 279
6.1.5 Perturbation of Sinai’s regime . . . . . . . . . . . . . . 282
6.2 Random strings in random environment . . . . . . . . . . . . 291
6.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 291
6.2.2 Lyapunov exponents and products of random matrices 292
6.2.3 Recurrence classification . . . . . . . . . . . . . . . . . 293
6.2.4 Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
6.3 Stochastic billiards . . . . . . . . . . . . . . . . . . . . . . . . 299
6.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 299
6.3.2 Reduction to Lamperti’s problem . . . . . . . . . . . . 300
6.3.3 Increment estimates . . . . . . . . . . . . . . . . . . . 302
6.4 Exclusion and voter models . . . . . . . . . . . . . . . . . . . 306
6.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 306
6.4.2 Configurations and exclusion-voter dynamics . . . . . 309
6.4.3 Lyapunov functions . . . . . . . . . . . . . . . . . . . 312
6.4.4 Exclusion process . . . . . . . . . . . . . . . . . . . . . 316
6.4.5 Voter model . . . . . . . . . . . . . . . . . . . . . . . . 318
6.4.6 Hybrid process . . . . . . . . . . . . . . . . . . . . . . 321
Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . 324
7 Markov chains in continuous time
7.1 Introduction and notation . . . . . . . .
7.2 Recurrence and transience . . . . . . . .
7.3 Existence and non-existence of moments
7.4 Explosion and implosion . . . . . . . . .
7.5 Applications . . . . . . . . . . . . . . . .
. . . . . .
. . . . . .
of passage
. . . . . .
. . . . . .
. . . .
. . . .
times
. . . .
. . . .
.
.
.
.
.
.
.
.
.
.
331
331
334
336
345
355
vi
Contents
7.5.1 Lamperti processes in continuous time . . . . . . . . . 355
7.5.2 Simple symmetric random walk . . . . . . . . . . . . . 358
Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . 359
Glossary of named assumptions
363
Bibliography
367
Index
394
Preface
What is this book about?
This book has two main goals:
• to give an up-to-date exposition of the ‘semimartingale’ or ‘Lyapunov
function’ approach to the analysis of stochastic processes;
• to present applications of the methodology to fundamental models
(classical and modern) in probability theory and related fields.
Our expository bridge between these dual aims, between methods and
models, is the d-dimensional non-homogeneous random walk, which as a
model is simple to describe, closely resembling the classical homogeneous
random walk, but displays many interesting and subtle phenomena alien to
the classical model. Non-homogeneous random walks cannot be studied by
the techniques generally used for homogeneous random walks: new methods
(and, just as importantly, new intuitions) are required.
Semimartingale and Lyapunov function ideas lead to a unified and powerful methodology in this context. As well as non-homogeneous random walks,
we present applications of the methods to several other models from modern
probability theory; while any of the models that we discuss can by studied by
several probabilistic techniques, we believe that only the Lyapunov function
method has something to say about all of them.
We emphasize that semimartingale methods are ‘robust’ in the sense
that the underlying stochastic process need not satisfy simplifying assumptions such as the Markov property, reversibility, or time-homogeneity, for
instance, and the state-space of the process need not be countable. In such
a general setting, the semimartingale approach has few rivals. In particular,
the methods presented work for non-reversible Markov chains. A general
feeling is that, if a Markov chain is reversible, then things can be done in
many possible ways: there are methods from electrical networks, spectral
vii
viii
Preface
calculations, harmonic analysis, etc. On the other hand, the non-reversible
case is usually much harder. Similarly, the Markovian setting is not essential
to the methods. In the semimartingale approach, the Markov property is a
side issue and non-Markovian processes can be treated equally well.
The Lyapunov function approach for analysis of Markov processes originated with classical work of Foster, and the theory has since expanded
greatly and proved very successful in analysis of numerous Markov models.
Aspects of Foster–Lyapunov theory are presented in the books [96, 6, 239]
(the presentation in [96] being the closest to our perspective). However,
almost all of these existing presentations are concerned with the situation
in which the process under consideration is not too close to a phase boundary in terms of its asymptotic behaviour. This book deals with analysis of
near-critical systems, which exhibit fundamental phase transitions such as
that between recurrence and transience.
Near-critical systems are exactly that: even if transient, they are not ballistic; even if positive-recurrent, they do not exhibit geometric ergodicity;
random quantities associated with the system typically have heavy (powerlaw) tails. Heavy tails have become increasingly prevalent in applications
across many fields over the last few years, including queueing theory and
finance; physicists associate heavy-tails with the presence of “self-organized
criticality”, a very fashionable topic at the moment. Naturally, the analysis of near-critical systems is more challenging and delicate than that for
systems that are far from criticality.
The fundamental contributions to the near-critical situation come from
classical work of Lamperti, and almost none of this has previously appeared
in any book, despite its age and importance. Lamperti’s basic problem concerned the asymptotic analysis of a stochastic process on the half-line with
mean drift at x of order 1/x: this is exactly the critical situation in respect
of the recurrence classification of the process. Importantly, Lamperti’s techniques are based on semimartingale ideas, and so the Markov property is
not essential. Building on Lamperti’s ideas, over the last 15 years much
work has appeared in academic journals on semimartingale methods and
near-critical probabilistic systems, and this is the theory that we present in
this book.
We want to emphasize the importance of applications of the theory. The
Lyapunov function ideology often enables one to reduce a question about
a complex model arising in applications to a question about a simpler onedimensional model by considering a function of the original process whose
image is one-dimensional. If the original model is interesting, in that its behaviour is near-critical in some sense, then the image process (for a suitably
Preface
ix
chosen Lyapunov function) will be near-critical. To give a concrete example,
the classical and fundamental model of symmetric simple random walk on Zd
(Xn , say) can be analysed through the Lyapunov function f (x) = kxk (the
Euclidean norm). Then f (Xn ) is a stochastic process on the half-line with
mean drift of order 1/x at x, i.e., precisely a critical Lamperti-type process.
Moreover, f (Xn ) is not a Markov process, demonstrating the importance of
the generality of the semimartingale approach.
Much more complicated and non-classical examples can be studied by
the same methods, and we present several such examples to give a flavour of
the power and utility of the techniques. We mention, for example, models
from queueing theory, interacting particle systems, random walks in random
environments, and so on. As mentioned above, our canonical example will
be the non-homogeneous random walk. This model is a natural generalization of the very classical and extremely well-studied homogeneous random
walk, but whose analysis requires entirely new methods. The semimartingale
approach gives a systematic and intuitive way to analyse these processes.
So, to summarize: This book is about the analysis of Markov processes
(such as random walks) via the method of Lyapunov functions; a correctly
applied Lyapunov function of a process gives rise to a semimartingale. Our
terminology here is neither particularly standard nor particularly precise;
we discuss briefly here our usage.
What is a random walk?
Many random walks are sums of i.i.d. random variables (or vectors); this
usage is too narrow for us. Many random walks take place on graphs or
groups; combinatorial or algebraic considerations are not the focus of this
book. Our random walks are (usually) Markov chains, on state-spaces that
are embedded in Euclidean space, with transitions that are in some sense
local (so that it is natural to speak of ‘jumps’ or ‘increments’). These are the
models that are best suited to the probabilistic approach of this book, and
include broad classes of models of interest in applications, such as queueing
theory or ecology.
This book is not just about random walks; we discuss other Markov
processes, including interacting particle systems and a stochastic billiards
model, for example, but random walks provide a rich set of models on which
to demonstrate some aspects of the Lyapunov function method.
x
Preface
What is a Lyapunov function?
The phrase ‘Lyapunov function’ has a quite precise technical meaning in
the theory of stability of differential equations. For us, it has a much looser
meaning: a Lyapunov function earns the name if the image under the function of a stochastic process is a process satisfying some conditions that enable one to deduce some property of the original process. For instance, if
(ξn , n ≥ 0) is a time-homogeneous Markov chain on Zd , and f : Rd → R is
a judicious choice of function such that there is a set A ⊂ Zd for which
E[f (ξn+1 ) − f (ξn ) | ξn = x] ≤ 0, for all x ∈
/ A,
(?)
then one can deduce that ξn is recurrent, or transient, if f and A satisfy
certain simple conditions. Results of this kind, which have their origins in
work of F.G. Foster in the 1950s, are known as Foster–Lyapunov conditions.
In all but the simplest cases, one cannot compute the left-hand side of (?)
exactly, but often one can estimate (?) using Taylor’s formula and coarse
properties of the increments ξn+1 − ξn ; usually one or two moments suffice.
Truncation arguments may be needed to control unusually large increments,
where Taylor’s formula will break down.
The language and tools of stopping times and martingale theory are close
at hand. If τ = min{n ≥ 0 : ξn ∈ A} denotes the first hitting time of A,
the drift condition (?) above can be interpreted as saying that f (ξn∧τ ) is
a supermartingale adapted to the natural filtration. Since recurrence and
transience properties of ξn can be related to properties of the stopping time
τ , it is natural to try to examine τ using the technology of martingale theory,
such as the optional stopping or martingale convergence theorems. This is
the basic ideology of the semimartingale method. This approach came after
Foster; fundamental work was done by J. Lamperti in the 1960s.
What is a semimartingale?
The phrase ‘semimartingale’ has a quite precise technical meaning in stochastic
analysis, as well as being an obsolete term for submartingale. For us, it has
a much looser meaning: a semimartingale is a process that satisfies some
drift condition like (?), typically only locally. Often, with the aid of a stopping time, one can convert such a process into a true supermartingale or
submartingale, as in the case of (?), but we do not make this demand on all
our semimartingales.
xi
Preface
Features of this book
As mentioned above, we use the terminology Lyapunov function in a general sense, and that in our usage the term does not presuppose any particular property (stability or otherwise) for the transformed process. When
presented with a Foster–Lyapunov result demanding verification of a drift
condition such as (?), one immediately is faced with the problem of how to
choose the Lyapunov function f . Usually there is no fixed rule about how
to discover the ‘right’ Lyapunov function, which must somehow encapsulate
one’s intuition about the process ξn under consideration. For this reason,
in this book we not only present the ‘right’ answers, but also do our best to
explain the intuition behind the choice of Lyapunov function in the context
of diverse applications and examples.
Finding a suitable Lyapunov function is not always easy, and there is
no exact algorithm for that. Nevertheless, there are several intuitive rules
and non-rigorous ideas that may help; in the text, we emphasize this kind
of heuristics in the following way:
i
Just try a good-looking Lyapunov function, work hard to
perform all the computations, and hope for the best.
We usually avoid adorning the main text with citations to the literature,
and instead collect bibliographical notes at the end of each chapter. We
have endeavoured to track down original references where possible, and have
uncovered several important works of which we were previously unaware; we
apologise in advance for any egregious omissions that remain.
Overview of content
The material is presented in logical order, but the book has several entry
points for the reader. Chapter 1 serves as a gentle introduction to the
main theme of the book (non-homogeneous random walks). Chapters 2
and 3 introduce the technical apparatus of the semimartingale approach
and describe its application to near-critical processes on the half-line. Our
intention is that these two chapters will serve as a useful reference for researchers who wish to use these tools, and so we have tried to give relatively
strong versions of the results in some generality. As an antidote to the technical demands of these two chapters, we have included many examples, as
our intention is also that Chapters 2 and 3 should prove instructive to the
student who wishes to develop an intuition for the method. Chapters 4–6
xii
Preface
present applications of the Lyapunov function method to some near-critical
stochastic processes. Thus while these chapters frequently refer to results
from Chapter 2 (and Chapter 4 also relies heavily on Chapter 3), our intention is that the reader who is so inclined can take the technical tools
for granted and read each of these later chapters as a stand-alone exposition. The final chapter, Chapter 7, switches focus to continuous-time; while
the development parallels some of the ideas in Chapter 2, this chapter is
essentially self-contained.
The section headings in the table of contents provide an indication of the
subject matter of each chapter; here we outline briefly what each chapter
contains.
Chapter 1: Introduction.
This chapter motivates the developments that follow by way of a classical
and fundamental model in probability theory: the d-dimensional random
walk. We describe the transition from (classical) homogeneous random walk
to spatially non-homogeneous random walks, and how the investigation of
such models is motivated by theoretical questions arising from trying to go
beyond the classical setting and to probe the recurrence/transience phase
transition. Immediately the relaxation of spatial homogeneity requires a significant re-adjustment of random walk intuition: one can readily construct
two-dimensional, zero-drift, bounded-jump random walks that are transient, for example, provided spatial homogeneity is not enforced, completely
contrary to classical behaviour.
Chapter 2: Semimartingale approach and Markov chains.
This section presents the basic technical apparatus that we rely on for the
rest of the book. We review some basic martingale ideas (Doob’s inequality,
martingale convergence, and the optional stopping theorem) and present a
variety of semimartingale tools, including maximal inequalities, results on
finiteness of hitting times, and existence and non-existence of moments for
hitting times. These results include Foster–Lyapunov criteria for Markov
chains, whereby a suitable Lyapunov function enables one to conclude that
the process is transient, recurrent, positive recurrent, etc. We provide many
examples of the application of these results.
Preface
xiii
Chapter 3: Lamperti’s problem.
This chapter presents applications of the semimartingale tools of Chapter 2
in the context of one-dimensional adapted processes with asymptotically
vanishing drift, the so-called Lamperti’s problem. Lamperti’s problem serves
as a first important example of a near-critical stochastic process, and is also
motivated by its ubiquity arising from the application of the Lyapunov function method to near-critical processes in higher dimensions. This chapter
studies in turn various aspects of the asymptotic behaviour of Lamperti
processes, including the recurrence classification for processes with asymptotically zero drifts, results on existence and non-existence of passage-time
moments, Gamma-type weak convergence results, and almost-sure bounds
on the trajectory of the process.
Chapter 4: Many-dimensional random walks.
This chapter presents applications of the results of Chapter 3 to manydimensional random walks. The applications in this chapter (and later on)
proceed by the Lyapunov function ideology: use a suitably chosen Lyapunov
function of the many-dimensional Markov process to obtain a (probably nonMarkov) stochastic process in one dimension, which fits into the framework
of Chapter 3. We consider in detail the recurrence classification problem
for non-homogeneous random walks, with emphasis on the possibility of anomalous recurrence behaviour. We also give results on angular asymptotics
and on the range of many-dimensional martingales.
Chapter 5: Heavy tails.
The processes considered in Chapters 3 and 4 are all assumed to have at
least one or two moments for their increments. This chapter turns to the
heavy-tailed case when the first or second increment moment is infinite.
We present results for real-valued Markov chains with heavy-tailed jumps,
focusing on the recurrence classification; we demonstrate how the Lyapunov
function approach is equally effective in this heavy-tailed setting.
Chapter 6: Further applications.
This chapter presents a selection of applications of the Lyapunov function
method to some near-critical stochastic systems. We consider Markov processes in random environments, some models of interacting particle systems,
and a stochastic billiards process. This chapter focuses on recurrence and
xiv
Preface
transience results, obtained by applications of the Foster–Lyapunov criteria
from Chapter 2.
Chapter 7: Markov chains in continuous time.
For the final chapter of the book, we switch from discrete to continuous time.
This chapter presents recent developments on semimartingale techniques for
continuous-time discrete-space Markov chains, non-homogeneous both in
space and in jump rates. For example, the (embedded) jump process might
be of Lamperti-type, while the rates are not uniformly bounded away from
0 and ∞. This gives rise to additional phenomena over the discrete-time
setting. We present conditions for existence of moments of hitting times in
this continuous-time setting, and give criteria for explosion and implosion
for such processes, again using semimartingale techniques.
Acknowledgements
We express our immense gratitude to all our co-authors. Especial thanks
are appropriate here to those whose collaborations directly relate to material presented in one or more of the chapters of this book: Sanjar Aspandiiarov, Inna Asymont, Vladimir Belitsky, Francis Comets, Guy Fayolle,
Pablo Ferrari, Nicholas Georgiou, Ostap Hryniv, Roudolf Iasnogorodski,
Iain MacPhee, Vadim Malyshev, Aleksandar Mijatović, Yuval Peres, Dimitri
Pétritis, Perla Sousi, Marina Vachkovskaia, and Stanislav Volkov; their ideas
permeate this book.
We are grateful to Nicholas Georgiou for providing Figures 4.1, 4.2,
and 4.4. Parts of the manuscript were read by Marcelo Costa, Nicholas
Georgiou, Chak Hei Lo, and James McRedmond; we are grateful for their
comments and suggestions.
This project was started when AW was at the University of Strathclyde;
AW also acknowledges support during part of this project by the Engineering
and Physical Sciences Research Council [grant number EP/J021784/1].
Summary of notation
Miscellaneous
We write a := · · · to indicate the definition of a. Occasionally we also use
· · · =: a. We use the standard abbreviation ‘a.s.’ for ‘almost surely’ (with
probability 1).
Sets, probabilities and events
The set of integers is Z. The natural numbers are N := {1, 2, 3, . . .}. The
non-negative integers are Z+ := {0} ∪ N. The real numbers are R. The
non-negative half-line is R+ := [0, ∞). It is often convenient to extend these
sets to include infinities, so we set Z+ := Z+ ∪ {+∞}, R := R ∪ {±∞}, and
R+ := R+ ∪ {+∞}. For a set S, we write #S for its cardinality (number of
elements, when finite). For a measurable subset A of Rd , we write |A| for
its d-dimensional Lebesgue measure (volume).
We will always assume an underlying probability space (Ω, F, P); expectation with respect to P will be denoted E. For A ⊆ Ω, we denote its
complement by Ac = Ω \ A.
If (An , n ∈ Z+ ) is a sequence of events, we use the standard notation
{An i.o.} = {An infinitely often}
= lim sup An = ∩m≥0 ∪n≥m An ;
{An ev.} = {An eventually} = {An all but finitely often}
= lim inf An = ∪m≥0 ∩n≥m An ;
{An f.o.} = {An finitely often} = {An i.o.}c = {Acn ev.}
= ∪m≥0 ∩n≥m Acn .
xv
xvi
Summary of notation
Conventions and empty evaluations
Unless otherwise stated, the following conventions are in force throughout.
An empty sum is zero, and an empty product is one. Also inf ∅ := +∞, and
sup ∅ := 0.
Real numbers, vectors, and matrices
For x a real number we set
x+ := max{0, x}, and x− := − min{0, x},
so x = x+ − x− and |x| = x+ + x− . For real numbers x and y, we set
x ∧ y := min{x, y}, and x ∨ y := max{x, y}.
For x ∈ R we write bxc for the largest integer not exceeding x, and dxe
for the smallest integer no less than x; so dxe − bxc ∈ {0, 1}, and is 0 if and
only if x ∈ Z.
For emphasis, we sometimes denote monotone convergence by ‘↑’ and
‘↓’, so for a sequence ax ∈ R, ax ↑ a as x → ∞ means that limx→∞ ax = a
and ax ≤ ay for all x ≤ y.
For a matrix M with real-valued entries we write M > for its transpose,
and λmax (M ) for its maximum eigenvalue. We usually use boldface letters
for vectors in Rd , and write, for example, x = (x1 , . . . , xd )> in Cartesian
components; for definiteness, vectors x ∈ Rd are viewed as column vectors
throughout. The origin in Rd is denoted by 0. We write k · k for the
Euclidean norm on Rd . We write e1 , . . . , ed for the standard orthonormal
basis of Rd , and for vectors u, v ∈ Rd we denote their scalar product by
u> v, u · v, or hu, vi. For a non-zero vector x ∈ Rd we write x̂ := x/kxk for
the corresponding unit vector, and we adopt the convention 0̂ := 0. The
unit-radius sphere in Rd is
Sd−1 := {u ∈ Rd : kuk = 1}.
The (closed) Euclidean d-ball centred at x ∈ Rd with radius r ∈ R+ is
B(x; r) := {y ∈ Rd : kx − yk ≤ r}.
We denote the d by d identity matrix by Id . For a square matrix M
with real-valued entries, we write tr M for its trace. A d by d real matrix
M acts on column vectors x ∈ Rd as an affine function from Rd to Rd via
xvii
Summary of notation
x 7→ M x. The associated matrix (operator) norm k · kop induced by the
Euclidean norm on Rd is
kM kop := sup kM uk.
u∈Sd−1
Using the variational characterization of the largest eigenvalue as λmax (M ) =
supu∈Sd−1 (u> M u), we note that
sup kM uk2 = sup
u∈Sd−1
u> M > M u = λmax (M > M ),
u∈Sd−1
so that an alternative expression for the operator norm is
kM kop = λmax (M > M )
1/2
.
Functions
The natural logarithm of x is log x. For r ∈ R, we write logr x for (log x)r .
We also write log1 x := log x, and for k ≥ 2 set logk x := log logk−1 x, so
that logk x is the k-fold iterated logarithm of x.
Random variables
We use 1 for the indicator function of an event, indicated either in curly
braces as 1{ · } or via a previously assigned symbol such as 1(A).
For a sub-σ-field G of F, G-measurable random variables X and Y , and
an event A ∈ G, the statement “X = Y on A” is equivalent to “X1(A) =
Y 1(A)”; similarly for inequalities.
We use the standard notation for essential supremum and infimum: for
a real-valued random variable X,
ess inf X := sup{x ∈ R : P[X ≥ x] = 1};
ess sup X := inf{x ∈ R : P[X > x] = 0}.
For Rd -valued random variables X, X1 , X2 , . . ., we denote convergence
by Xn → X qualified by the mode of convergence in the text; for example,
Xn → X a.s. means that P[Xn → X] = 1. Sometimes for compactness we
a.s.
p
d
write ‘−→’, ‘−→’, and ‘−→’ for convergence almost surely, in probability,
and in distribution, respectively.
xviii
Summary of notation
Asymptotics
We reserve unadorned Landau O( · ) and o( · ) symbols for the case where
implicit constants are non-random. Thus for a real-vaued function f and
a R+ -valued function g, the expression f (x) = O(g(x)) means that there
exist finite deterministic constants C and x0 such that |f (x)| ≤ Cg(x) for
all x ≥ x0 . Similarly, f (x) = o(g(x)) means that for any ε > 0 there exists
a finite deterministic xε such that |f (x)| ≤ εg(x) for all x ≥ xε .
It is convenient to extend the O, o notation to permit random variables,
but it is important to do this carefully to avoid ambiguous formulas. Given
F (Y )
a σ-field F and F-measurable random variables X and Y , we write OX
to represent an F-measurable random variable such that there exist finite
F (Y )| ≤ CY on the event
deterministic constants C and x0 such that |OX
{X ≥ x0 }. Although this notation is a little cumbersome, we feel the extra
clarity is worthwhile, as an ambiguous O( · ) can hide a multitude of sins.
Chapter 1
Introduction
1.1
Random walks
Random walks are fundamental models in probability theory that exhibit
deep mathematical properties and enjoy broad application across the sciences and beyond. Generally speaking, a random walk is a stochastic process
modelling the random motion of a particle (or random walker ) in space. The
particle’s trajectory is described by a series of random increments or jumps
at discrete instants in time. Central questions for these models involve the
long-time asymptotic behaviour of the walker.
Random walks have a rich history involving several disciplines. Classical one-dimensional random walks were first studied several hundred years
ago as models for games of chance, such as the so-called “gambler’s ruin”
problem. Similar reasoning led to random walk models of stock prices described by Jules Regnault in his 1863 book [265] and Louis Bachelier in
his 1900 thesis [14]. Many-dimensional random walks were first studied at
around the same time, arising from work of pioneers of science in diverse
applications such as acoustics (Lord Rayleigh’s theory of sound developed
from about 1880 [264]), biology (Karl Pearson’s 1906 [254] theory of random
migration of species), and statistical physics (Einstein’s theory of Brownian
motion developed during 1905–08 [86]). The mathematical importance of
the random walk problem became clear after Pólya’s work in the 1920s, and
over the last 60 years or so there have emerged beautiful connections linking random walk theory and other influential areas of mathematics, such
as harmonic analysis, potential theory, combinatorics, and spectral theory.
Random walk models have continued to find new and important applications
in many highly active domains of modern science: see for example the wide
1
2
Chapter 1. Introduction
range of articles in [287]. Specific recent developments include modelling
of microbe locomotion in microbiology [245, 23], polymer conformation in
molecular chemistry [15, 202], and financial systems in economics.
Spatially homogeneous random walks are the subject of a substantial
literature, including numerous books, such as [293, 269, 195, 139]. In many
modelling applications, the classical assumption of spatial homogeneity is
not realistic: the behaviour of the random walker may depend on the present
location in space. Applications thus motivate the study of non-homogeneous
random walks. These models are also motivated naturally from a mathematical perspective: non-homogeneous random walks are the natural setting
in which to probe near-critical behaviour and obtain a finer understanding
of phase transitions present in the classical random walk models.
The main theme of this book is the analysis of near-critical stochastic
systems using the method of Lyapunov functions. The non-homogeneous
random walk serves as a prototypical near-critical system; the Lyapunov
function methodology is robust and powerful, and can be applied to many
other near-critical models, including those with applications across modern probability and beyond, to areas such as queueing theory, interacting
particle systems, and random media. In this chapter we give an informal
introduction to non-homogeneous random walks, and how their behaviour
differs from classical random walks; we also describe some fundamental ideas
of the Lyapunov function technique. We state some theorems, but we often omit technical details and generally omit proofs. All of the results that
we mention will be stated more precisely (and proved) later in the book,
and also applied to a wide variety of near-critical stochastic systems: the
non-homogeneous random walk serves as an expository bridge between wellknown classical results and the near-critical behaviour that is the subject of
this book.
1.2
Simple random walk
The most intensively studied random walk model is symmetric simple random walk. Simple random walk is a discrete-time Markov process (Sn , n ≥ 0)
on the d-dimensional integer lattice Zd : Sn can be thought of as the location
(in the state-space Zd ) of the random walker at time n (or after n steps).
The stochastic evolution of the process is as follows. Given Sn in Zd , the
next point Sn+1 is chosen uniformly at random from among the 2d lattice
points adjacent to Sn , i.e., those points that differ from Sn by exactly ±1
in a single coordinate. In other words, the transition probabilities of the
3
1.2. Simple random walk
Markov chain for are given for x, y ∈ Zd by
(
P[Sn+1 = y | Sn = x] =
1
2d
0
if kx − yk = 1;
otherwise;
where k · k is the Euclidean norm on Rd .
For example, when d = 1 one-dimensional simple random walk jumps
one unit to the left or right, each with probability 1/2, while when d = 2
two-dimensional simple random walk jumps to one of its four neighbours,
with probability 1/4 of each.
For definiteness, suppose that the walk starts at the origin of Zd : S0 = 0.
By the Markov property and spatial homogeneity, the increments (or jumps)
Sn+1 − Sn of the walk are independent and identically distributed (i.i.d.)
random vectors. If {e1 , . . . , ed } is the standard orthonormal basis on Rd , let
Ud := {±e1 , . . . , ±ed }
for the
Ppossible values of the increments of the walk. Then we can write
Sn = nk=1 Zk , where Z1 , Z2 , . . . are i.i.d. with
P[Z1 = e] =
1
for e ∈ Ud .
2d
(With the usual convention that an empty sum is zero, S0 = 0.) Thus we
may represent the random walk Sn via a sequence of partial sums of the
i.i.d. random increments Zn .
A fundamental question, addressed by Pólya [259], concerns the recurrence or transience of the random walk: What is the probability that the
walk eventually returns to 0? If we write τd := min{n ≥ 1 : Sn = 0} for the
time of the first return to 0 (with the usual convention that min ∅ := ∞),
the recurrence question concerns
pd := P[τd < ∞].
The random walk is recurrent if pd = 1, in which case with probability one
the random walk will visit 0 infinitely often. On the other hand, if pd < 1
the random walk is transient, and will, with probability one, visit 0 only
finitely many times, before eventually leaving, never to return.
The following fundamental result is due to Pólya [259].
Theorem 1.2.1. Simple random walk is recurrent in 1 or 2 dimensions,
but transient in 3 or more; i.e., p1 = p2 = 1 but pd < 1 for all d ≥ 3.
4
Chapter 1. Introduction
The content of the theorem is nicely captured by an aphorism attributed
to Shizuo Kakutani: “A drunk man will eventually find his way home, but
a drunk bird may get lost forever” (see [83, p. 191]).
Pólya’s theorem (Theorem 1.2.1) tells us that the walk returns to 0
eventually when d = 1 or d = 2. But how long might we have to wait? The
answer is, potentially, a very long time, since τd has very heavy tails:
1
π
P[τ1 > n] ∼ √ , and P[τ2 > n] ∼
, as n → ∞;
log n
πn
see e.g. [259, p. 159] for the first expression and [84, pp. 356–357] for the
1/2
second. So in d = 1, E[τ1 ] = ∞, while in d = 2, τ2 has no moments at all.
According to Hughes [139, p. 42],
the failure of certain moments of distributions or densities to
converge . . . is pregnant with physical meaning, and indicative
of connections to scaling laws, renormalization group methods,
and fractals.
The recurrence exhibited by the simple random walk for d ∈ {1, 2} is nullrecurrence, meaning that E τd = ∞; more stable processes may exhibit
positive-recurrence, meaning that the analogue of τd is integrable.
1.3
Lamperti’s problem
There are several proofs of Theorem 1.2.1 in the literature, the most popular being those that are largely combinatorial (such as Pólya’s original
argument [259]) and those based on potential theory and electrical networks
(see e.g. [81]). A drawback of each of these approaches is that they rapidly
break down when one tries to generalize Pólya’s theorem to other random
walks. In this section we describe a robust approach to proving Pólya’s
theorem, due to Lamperti, which enables very broad generalization. This
approach is based on the methodology of Lyapunov functions.
Again let Sn be the symmetric simple random walk on Zd , starting at 0.
In the context of Pólya’s recurrence theorem, we are interested in the events
{Sn = 0}. We can reduce this d-dimensional problem to a one-dimensional
problem by considering a transformation of the process (a Lyapunov function) given by
Xn := kSn k,
(1.1)
i.e., Xn is the distance from the origin of the walker at time n. The stochastic
process (Xn , n ≥ 0) takes values in the countable set S = {kxk : x ∈ Zd },
5
1.3. Lamperti’s problem
6
6
5
5
4
4
3
3
2
2
1
1
0
bc
bc
0
bc
-1
-1
-1
0
1
2
3
4
5
6
-1
0
1
2
3
4
5
6
Figure 1.1: Illustration of the non-Markovian nature of Xn in d = 2.
a subset of the half-line R+ , and Xn = 0 if and only if Sn = 0. So we can
study the recurrence or transience of Sn via the recurrence or transience of
Xn , a one-dimensional process.
This reduction in dimensionality of the problem comes at a price: Xn
is not in general a Markov process. For instance, when d = 2, given one of
the two events {Sn = (5, 0)} and {Sn = (3, 4)} we have Xn = 5 in each case
but Xn+1 has two different distributions; Xn+1 can take the value 6 (with
probability 1/4) in the first case, but this is impossible in the second case.
See Figure 1.3. Thus the tools that we use to study Xn must not rely too
heavily on the Markov property.
First we compute the expected increment of Xn given Sn = x, namely
d
E[Xn+1 − Xn | Sn = x] =
1 X
(kx + ei k + kx − ei k − 2kxk) .
2d
(1.2)
i=1
To proceed we apply Taylor’s theorem in an elementary way. Using the
Taylor expansion
1
1
(1 + y)1/2 = 1 + y − y 2 + O(y 3 ), as y → 0,
2
8
we obtain that, for any e ∈ Sd−1 ,
p
kx + ek − kxk = (x + e) · (x + e) − kxk
1
"
#
2e · x + 1 1/2
= kxk 1 +
−1
kxk2
6
Chapter 1. Introduction
= kxk
2e · x + 1 (e · x)2
−3
−
+ O(kxk ) .
2kxk2
2kxk4
(1.3)
It follows that
kx + ek + kx − ek − 2kxk =
Thus from (1.2), using the fact that
1
(e · x)2
−
+ O(kxk−2 ).
kxk
kxk3
Pd
i=1 (ei
· x)2 = kxk2 , we obtain
d
1 X
(kx + ei k + kx − ei k − 2kxk)
2d
i=1
d−1
1
=
+ O(kxk−2 ).
(1.4)
2d
kxk
E[Xn+1 − Xn | Sn = x] =
Similarly
d
2
E[Xn+1
−
Xn2
1 X
| Sn = x] =
kx + ei k2 + kx − ei k2 − 2kxk2 = 1.
2d
i=1
2
Then since (Xn+1 − Xn )2 = Xn+1
− Xn2 − 2Xn (Xn+1 − Xn ) we obtain
E[(Xn+1 − Xn )2 | Sn = x] =
1
+ O(kxk−1 ).
d
(1.5)
Informally speaking, (1.4) says that the mean increment of Xn at x ∈ S
1
is 2x
(1 − d1 ) + O(x−2 ), and similarly (1.5) says that the second moment of
the increment at x is d1 + O(x−1 ); however, to formalize these statements
we need to be careful since Xn is not a Markov process, and we have to
clarify what we mean by the increment moments “at x”. We deal with these
technicalities later, since they complicate the notation (although they will
not complicate the proofs, which are based on martingale arguments). For
now, for the purposes of exposition, we switch to the case where Xn is a
Markov process.
Suppose now that (Xn , n ≥ 0) is a time-homogeneous Markov process on
an unbounded subset S of R+ . Consider the increment moment functions
µk (x) := E[(Xn+1 − Xn )k | Xn = x].
For simplicity, suppose that Xn has uniformly bounded increments, so that
P[|Xn+1 − Xn | ≤ B] = 1
(1.6)
7
1.3. Lamperti’s problem
for some B ∈ R+ ; under condition (1.6), the µk are well-defined functions of
x ∈ S. The first moment function, µ1 (x), is the one-step mean drift of Xn
at x.
Lamperti [190, 191, 192] investigated the extent to which the asymptotic
behaviour of such a process is determined by the µk ; essentially, µ1 and µ2
turn out to govern the asymptotic behaviour. For example, the following
result is a version Lamperti’s fundamental recurrence classification.
Theorem 1.3.1. Suppose that Xn is a Markov process on S satisfying (1.6).
Under mild conditions, the following recurrence classification holds. Let
ε > 0.
• If 2xµ1 (x) + µ2 (x) < −ε, then Xn is positive-recurrent;
• If 2x|µ1 (x)| ≤ µ2 (x) + O(x−ε ), then Xn is null-recurrent;
• If 2xµ1 (x) − µ2 (x) > ε, then Xn is transient.
The mild conditions mentioned in the statement of Theorem 1.3.1 are
related to issues of irreducibility; the exact nature of the conditions required
depends on the state-space S and the notion of recurrence and transience
desired. We leave the technical details until later in the book.
A version of Theorem 1.3.1 applies to non-Markov processes Xn when
formulated correctly, using appropriate versions of the µk . In particular, we
can use such results to study the process Xn defined by (1.1): in this case,
the analogue of 2xµ1 (x) is, by (1.4),
1−
1
+ O(x−1 )
d
and the analogue of µ2 (x) is, by (1.5),
1
+ O(x−1 ).
d
An application of the generalized version of Theorem 1.3.1 shows that Xn
is transient if and only if
1
1
1− > ,
d
d
or, in other words, d > 2. This gives a very robust strategy for proving
Pólya’s theorem (Theorem 1.2.1) and its generalizations, based on computations of increment moments for Xn defined by (1.1). These computations
use elementary Taylor’s theorem ideas, and do not rely at all on special
structure of the original process Sn .
8
1.4
Chapter 1. Introduction
General random walk
Simple random walk is an attractive model and can be studied using combinatorial methods based on counting sample paths, for example; it is, however,
a very specific model. Naturally, it is of interest to study a much broader
class of random walks. In particular, for what class of models does a result
similar to Theorem 1.2.1 hold? To put the question in another way: What
are the essential properties possessed by simple random walk that imply
Theorem 1.2.1? To answer such questions, we start by describing a much
more general model of a random walk.
By a random walk we mean a discrete-time Markov process (ξn , n ≥ 0) on
an unbounded state-space Σ ⊆ Rd . We assume that the random walk is timehomogeneous. This means that the distribution of ξn+1 given (ξ0 , ξ1 , . . . , ξn )
depends only on ξn (and not on n). We also need some form of irreducibility
to ensure that the random walk cannot get ‘trapped’ in some part of the
state-space: it is simplest to take Σ to be a locally finite set (such as Zd ) to
avoid technical issues at this point.
We are thus using the term random walk in a rather general sense, requiring that the Markov process inhabit Euclidean space. We also want to
impose some regularity assumptions on the increments of the process, to rule
out very long jumps for the random walk. For this chapter, for convenience
we often suppose that the increments of the walk are uniformly bounded, so
that there exists B ∈ R+ for which, almost surely (a.s.),
P[kξn+1 − ξn k ≤ B | ξn = x] = 1, for all x ∈ Σ.
(1.7)
In many cases this assumption can be replaced by a weaker assumption on
the existence of higher moments for the increments, without producing fundamentally new behaviour. On the other hand, the case of genuinely heavytailed increments leads to different phenomena, as discussed in Chapter 5 of
this book.
Under the assumption (1.7), the random vectors ξn+1 − ξn have welldefined moments, which may depend on ξn . In particular, an important
quantity is the one-step mean drift vector
µ(x) := E[ξn+1 − ξn | ξn = x],
which is the average change in position in a single step starting from x ∈ Σ.
(Note that the definition via the conditional expectation on {ξn = x} is clear
in the case of a countable state-space Σ, and makes sense when correctly
interpreted for uncountable Σ as well.)
9
1.5. Recurrence and transience
An important and well-studied sub-class of random walks are spatially
homogeneous, for which the distribution of the increment ξn+1 − ξn does
not depend on the current location ξn . Writing θn := ξn+1 − ξn , spatial
homogeneity implies that θ0 , θ1 , . . . are i.i.d. random vectors. Then the representation
n−1
X
ξn =
θk
(1.8)
k=0
as a sum of i.i.d. random vectors enables classical tools of probability theory,
such as Fourier methods, to be brought to bear in the analysis of the random
walk in the spatially homogeneous case.
Example 1.4.1 (Simple random walk). Using e1 , . . . , ed to denote the
standard orthonormal basis vectors of Rd , simple random walk is spatially
homogeneous with θn uniformly distributed on the set Ud = {±e1 , . . . , ±ed }.
4
Example 1.4.2 (Pearson–Rayleigh random walk). Take θn to be uniformly
distributed on the unit-radius sphere Sd−1 ⊂ Rd . The corresponding ddimensional random walk, that proceeds via a sequence of unit-length steps,
each in an independent and uniformly random direction, is sometimes called
a Pearson–Rayleigh random walk: see the bibliographical notes at the end
of this chapter.
4
Spatially homogeneous random walks have been extensively studied in
classical probability theory. Non-homogeneous random walks, on the other
hand, require new techniques and, just as importantly, new intuitions.
1.5
Recurrence and transience
For our general random walks, we need a more general definition of recurrence and transience. In the absence of any structural assumptions, our
basic use of the terminology is as follows. Note that, in general, the two
behaviours are not a priori exhaustive.
Definition 1.5.1. A stochastic process (ξn , n ≥ 0) taking values in Σ ⊆ Rd
is transient if limn→∞ kξn k = ∞, a.s. The process is recurrent if, for some
constant r0 ∈ R+ , lim inf n→∞ kξn k ≤ r0 , a.s.
If ξn is an irreducible time-homogeneous Markov chain whose state-space
is a locally finite subset of Rd (such as Zd ), then recurrence and transience in
the sense of Definition 1.5.1 coincide with the usual Markov chain definition
10
Chapter 1. Introduction
in terms of returns to any given state. Definition 1.5.1 allows more general
processes, and is the most convenient definition in the context of Lamperti’s
problem outlined in Section 1.3.
First we return to the classical spatially homogeneous random walk described in Section 1.4, in which case the increments θn in (1.8) are i.i.d. In
this case, when defined, E[ξn+1 − ξn | ξn = x] = E θ0 = µ does not depend
on x.
If µ 6= 0, then the strong law of large numbers shows that the walk
is transient, and limn→∞ n−1 ξn = µ, a.s., so the walk escapes to infinity
at positive speed. The most subtle case is that of zero drift when µ = 0.
Here, under mild conditions, Pólya’s theorem (Theorem 1.2.1) for simple
symmetric random walk extends to the case of the general spatially homogeneous random walks with zero drift. Recall that we view θ0 and other
d-dimensional vectors as column vectors throughout.
Theorem 1.5.2. For a spatially homogeneous random walk ξn on Rd , suppose that E[kθ0 k2 ] < ∞, E θ0 = 0, and E[θ0 θ0> ] is positive-definite. Then ξn
is recurrent for d ∈ {1, 2} and transient for d ≥ 3.
The positive-definite covariance condition ensures that the increments
are not supported on a lower-dimensional subspace.
How does the situation change if the walk is allowed to be spatially nonhomogeneous? In the general, non-homogeneous case µ(x) = E[θn | ξn = x]
will depend on x. Even if µ(x) = 0 for all x, however, the non-homogeneous
zero-drift walk can behave completely differently to the homogeneous zerodrift walk.
Theorem 1.5.3. Let ξn be a spatially non-homogeneous random walk on
Rd with zero drift, so that µ(x) = 0 for all x.
• If d = 2, then we can exhibit such a walk that is transient.
• If d ≥ 3, then we can exhibit such a walk that is recurrent.
In all cases, we may take these examples to have uniformly bounded increments, as at (1.7).
We emphasize that, for example, in two dimensions, zero drift does not
imply recurrence for a non-homogeneous random walk with bounded jumps.
This fact is contrary to intuition built from homogeneous random walks, but
should not be surprising to readers who have encountered random walks in
random environments: for example, Zeitouni [319, pp. 90-91] discusses an
1.5. Recurrence and transience
11
example of a transient walk in d = 2 with symmetric increments; see also
the examples in Chapter 4.
So non-homogeneous random walks can show anomalous (non-classical)
recurrence behaviour. We can reassert some control by imposing additional
regularity structure on the second moments of the increments θn = ξn+1 −ξn .
Assuming (1.7), then the matrix function
M (x) = E[θn θn> | ξn = x]
(1.9)
is well defined, since for any e ∈ Sd−1 , |e> θn θn> e| = (e · θn )2 ≤ kθn2 k; for each
x, M (x) is a symmetric, nonnegative-definite matrix. We refer to M (x)
defined at (1.9) as the increment covariance matrix ; this is a slight abuse of
terminology, which more accurately should refer to
E[(θn − µ(x))(θn − µ(x))> | ξn = x] = M (x) − µ(x)µ(x)> ,
but in our calculations it is always M (x) that appears.
Suppose that M (x) = M is fixed, for a positive-definite matrix M .
This condition is satisfied with M = σ 2 Id for some σ 2 ∈ (0, ∞), where
Id is the d by d identity matrix, by a walk with sufficient symmetry in its
increments, such as simple symmetric random walk or the Pearson–Rayleigh
walk. With this additional regularity structure, the wild behaviour shown
in Theorem 1.5.3 is ruled out, and we recapture the result of Theorem 1.5.2.
Theorem 1.5.4. Let ξn be a spatially non-homogeneous random walk on
Rd for which (1.7) holds. Suppose that, for x, µ(x) = 0 and M (x) = M for
some positive-definite matrix M . Then ξn is recurrent for d ∈ {1, 2} and
transient for d ≥ 3.
Remark 1.5.5. Given that M (x) = M for some positive-definite (necessarily,
symmetric) matrix M , there exists a (unique) positive definite, symmetric
matrix square-root M 1/2 with matrix inverse M −1/2 , such that M 1/2 M 1/2 =
M . The matrix M −1/2 defines a linear transformation of Rd via x 7→ M −1/2 x
(x ∈ Rd ). Define ζn := M −1/2 ξn . Then ϕn := ζn+1 − ζn = M −1/2 θn , where
θn = ξn+1 − ξn , and, by linearity,
E[ϕn | ζn = x] = M −1/2 E[θn | ξn = M 1/2 x] = 0, and
E[ϕn ϕ>n | ζn = x] = M −1/2 E[θn θn> | ξn = M 1/2 x]M −1/2 ,
which is the identity. The transformed process ζn is recurrent if and only if
the original process ξn is recurrent. Thus it suffices to prove Theorem 1.5.4
in the case M = Id , or indeed for any other fixed positive-definite matrix.
12
Chapter 1. Introduction
We describe the relevance of Lamperti’s problem to Theorem 1.5.4. The
basic idea is the same as in Section 1.3, but we use Taylor’s theorem in a
slightly more sophisticated way.
Consider a random walk ξn satisfying the conditions of Theorem 1.5.4.
Let Xn := kξn k. Then
E[Xn+1 − Xn | ξn = x] = E[kx + θn k − kxk | ξn = x],
where, as usual, θn = ξn+1 − ξn is the increment of the walk. We now use
Taylor’s theorem to expand the expression inside the expectation. Note first
that, by a similar argument to (1.3), for any y ∈ Rd with kyk ≤ B,
"
#
1/2
2y · x + kyk2
kx + yk − kxk = kxk 1 +
−1
kxk2
=
y · x kyk2
(y · x)2
+
−
+ O(kxk−2 ),
kxk
2kxk
2kxk3
as kxk → ∞. We are going to apply this with y = θn . In other words,
kx + θn k − kxk = x̂ · θn +
1
kθn k2 − (x̂ · θn )2 + O(kxk−2 ).
2kxk
It is important to note that this is legitimate because of the uniform jumps
bound (1.7), and that the implicit constant in the O( · ) term is non-random,
i.e., uniform in θn ; we use this notation strictly for such circumstances. Thus
we may take expectations to obtain
E[Xn+1 − Xn | ξn = x]
= E[x̂ · θn | ξn = x] +
1
E kθn k2 − (x̂ · θn )2 ξn = x + O(kxk−2 ).
2kxk
With the notation µ(x) = E[θn | ξn = x] and M (x) = E[θn θn> | ξn = x],
using linearity of expectation and a little linear algebra we can write this as
E[Xn+1 − Xn | ξn = x] = x̂ · µ(x) +
1
tr M (x) − x̂> M (x)x̂ + O(kxk−2 ).
2kxk
Suppose that the conditions of Theorem 1.5.4 hold, with M = σ 2 Id for some
σ 2 ∈ (0, ∞), which is no loss of generality by Remark 1.5.5. Then we have
µ(x) = 0, tr M (x) = dσ 2 , and e> M (x)e = σ 2 for any e ∈ Sd−1 , so that
E[Xn+1 − Xn | ξn = x] =
(d − 1)σ 2
+ O(kxk−2 ).
2kxk
1.6. Angular asymptotics
13
For symmetric simple random walk, σ 2 = 1/d, and we recover (1.4), but
now we have used only the first two moments of the increments. Similarly,
E[(Xn+1 − Xn )2 | ξn = x] = σ 2 + O(kxk−1 ),
which generalizes (1.5). Again, Lamperti’s approach yields transience when
(d − 1)σ 2 > σ 2 , i.e., d > 2.
From this perspective, the proof of Theorem 1.5.4 is no more difficult
than the proof of Theorem 1.5.2, at least if we assume the jumps bound
(1.7); this robustness is a feature of the Lyapunov function method.
Later in the book, we shall see how to relax the jumps bound (1.7) in
favour of a moments condition. The issue is clear: Taylor’s theorem for
kx + θn k is only useful if kθn k is small compared to kxk. To overcome this
obstacle, one uses a basic truncation idea: partition the increment as
Xn+1 − Xn = (Xn+1 − Xn ) 1{kθn k ≤ kxkγ } + (Xn+1 − Xn ) 1{kθn k > kxkγ }
for some constant γ ∈ (0, 1); then the first term can be expanded using
Taylor’s theorem, while the second term can be bounded in expectation
using a version of Markov’s inequality and a bound on the moments of kθn k.
We shall see these ideas carried through later in the book.
1.6
Angular asymptotics
The recurrence and transience properties described in the previous section
are properties of the radial component of the process; it is also natural to
investigate asymptotics of the angular component.
We say that a random walk ξn has a limiting direction if limn→∞ ξˆn
exists in Sd−1 . Here and elsewhere x̂ denotes the unit vector x/kxk when
x 6= 0; we adopt the convention 0̂ := 0. A walk with a limiting direction
eventually remains in an arbitrarily narrow cone with that direction as its
axis; when is this the case?
Here we review briefly the situation for classical random walks. Suppose
that ξn is a spatially homogeneous random walk, so the increments θn in
(1.8) are i.i.d.
Theorem 1.6.1. For a spatially homogeneous random walk ξn on Rd , suppose that E kθ0 k < ∞; let µ = E θ0 ∈ Rd .
(i) If µ 6= 0, then P[limn→∞ ξˆn = µ̂] = 1.
(ii) If µ = 0 and P[θ0 = 0] < 1, then P[limn→∞ ξˆn exists] = 0.
14
Chapter 1. Introduction
A proof of Theorem 1.6.1 is presented in the bibliographical notes at the
end of this chapter. Once spatially homogeneity is relaxed, the situation
becomes much more interesting.
1.7
Centrally biased random walks
To probe more precisely the recurrence-transience phase transition, it is
natural to consider the case in which µ(x) → 0 as kxk → ∞: this is known
as the asymptotically zero drift regime.
For definiteness, consider a 2-dimensional non-homogeneous random walk
whose increments have a fixed covariance structure. We have seen (Theorem 1.5.4) that if µ(x) = 0 for all x, the random walk is recurrent. On the
other hand, if the walk has uniformly positive drift radially away from 0,
i.e., µ(x) = εx̂ for any ε > 0, then it can be shown that the walk is transient
with linear rate of escape.
The natural setting to look for finer behaviour is the case in which the
magnitude of the drift tends to zero as the distance from the origin increases.
We focus on a concrete example of the asymptotically zero drift regime, in which the drift is always in the radial direction. Consider a nonhomogeneous random walk on Rd . Let r0 be a finite positive constant.
Suppose that, for ρ ∈ R,
µ(x) = ρ
x̂
, for all x with kxk ≥ r0 ;
kxk
(1.10)
in other words, the mean drift is radially towards or away from 0, vanishing in magnitude as the distance from 0 increases. Such centrally biased
random walks were studied by Lamperti; one can view these processes as
perturbations of zero-drift random walks. It is natural to investigate what
magnitude of perturbation is needed to change the asymptotic behaviour of
the process.
For simplicity, we take the state space of our random walk to be Zd .
We also need to impose an assumption to ensure that the walk cannot get
trapped in a subset of the state space. For simplicity, at this point we
suppose that the increments are uniformly elliptic:
inf P [(ξn+1 − ξn ) · u ≥ ε | ξn = x] ≥ ε, for all x ∈ Zd .
u∈Sd−1
(1.11)
We will see later in the book (Example 3.3.5) that uniform ellipticity (1.11)
implies lim supn→∞ kξn k = ∞, a.s., so questions concerning the asymptotic
behaviour of the walk are non-trivial.
1.7. Centrally biased random walks
15
The following result due to Lamperti can be viewed as a generalization
of the one-dimensional Theorem 1.3.1. Recall the definition of the increment
covariance matrix M (x) from (1.9).
Theorem 1.7.1. Let ξn be a spatially non-homogeneous random walk on
Zd for which (1.7) and (1.11) hold, with asymptotically zero drift of the
form (1.10). Suppose that M (x) = M for some positive-definite M . Then
there exist constants c1 = c1 (d, M ) and c2 = c2 (d, M ), with −∞ < c1 <
c2 < ∞, for which:
• If ρ < c1 , then ξn is positive-recurrent;
• If c1 ≤ ρ ≤ c2 , then ξn is null-recurrent;
• If ρ > c2 , then ξn is transient
In particular, for d ∈ {1, 2}, c2 ≥ 0, while for d ≥ 3, c2 < 0.
This result implies that by adding an appropriate drift of order 1/kxk
away from 0 to a zero drift random walk in d ∈ {1, 2}, we can change the walk
from recurrent to transient. On the other hand, by adding an appropriate
drift of order 1/kxk towards 0 to a zero drift random walk in d ≥ 3, we can
change the walk from transient to recurrent (even positive-recurrent).
Theorem 1.7.1 shows that the case where kµ(x)k = O(kxk−1 ) is critical
from the point of view of the recurrence classification of the non-homogeneous
random walk. It also turns out that several other important characteristics of the process undergo phase transitions in this regime; in this sense,
the centrally biased random walk is a near-critical stochastic system. We
will see this near-critical behaviour exhibited by several quantitative characteristics later in the book, including passage-time distributions, stationary
measures, and almost-sure scaling exponents; typical of near-criticality is
that scaling is polynomial and distributions are heavy-tailed.
The regime kµ(x)k = O(kxk−1 ) also turns out to be critical from the
point of view of the angular asymptotics in the sense of Section 1.6. Here,
where the walk may be transient or recurrent, by Theorem 1.7.1, the following result shows that the walk has no limiting direction.
Theorem 1.7.2. Let ξn be a spatially non-homogeneous random walk on Zd ,
d ≥ 2, for which (1.7) and (1.11) hold. Suppose that M (x) = M for some
positive-definite M and that kµ(x)k = O(kxk−1 ). Then P[limn→∞ ξˆn exists] =
0.
16
Chapter 1. Introduction
On the other hand, a radial drift of order greater than kxk−1 can lead to
completely different behaviour, in which the random walk is transient with
a limiting direction, which can (unlike in Theorem 1.6.1) be random.
In fact, we have the following strong law of large numbers:
Theorem 1.7.3. Let ξn be a spatially non-homogeneous random walk on
Zd for which (1.7) and (1.11) hold, with drift of the form
µ(x) = ρ
x̂
, for all x with kxk ≥ r0 ,
kxkβ
for some β ∈ (0, 1), ρ > 0, and r0 ∈ R+ . Suppose that M (x) = M for some
positive-definite M . Then as n → ∞,
ξn
1/(1+β)
n
→ v, a.s.,
for a random vector v ∈ Rd of fixed (deterministic, non-zero) magnitude.
In particular, limn→∞ ξˆn = v̂, a.s., a random limiting direction.
See Figure 1.2 for some simulations illustrating these results.
200
1000
150
800
100
600
50
400
0
200
-50
-100
0
0
50
100
150
200
250
-700
-600
-500
-400
-300
-200
-100
0
Figure 1.2: Two 2-dimensional centrally biased random walk simulations,
each run for 4 × 104 steps, under the conditions of Theorem 1.7.2 (left) and
Theorem 1.7.3 (right). Both walks are transient; the walk on the left has no
limiting direction, while the walk on the right has a limiting direction and
satisfies a super-diffusive law of large numbers.
Bibliographical notes
17
Bibliographical notes
Section 1.1
We give a few references to some of the many applications enjoyed by random
walks. Surveys of numerous applications can be found in [15, 202, 315, 287,
23, 139, 314].
Animal behaviour. The use of random walks to model the movement of
animals goes back about 100 years to K. Pearson [254]; subsequently random
walk models have been used both for the foraging, territorial, and migratory
behaviour of macroscopic animals (see e.g. [131]) and for the locomotion of
microbes, where experiment suggests that the motion of several kinds of
cells consists of roughly straight line segments linked by discrete changes in
direction: see e.g. Chapter 6 of [23] and [244, 245]. We refer to [49, 290] for
recent surveys and to [32] for further references to the literature.
Molecular conformation and dynamics. Random walks have been
used in chemical physics as idealizd models for the static and dynamic behaviour of flexible long chain molecules in solution. In the most basic ‘freelyjointed’ chain model, the steps of the walk are independent; more realistic
models take ‘restricted’ or ‘correlated’ random walks where the step distribution depends on the previous step(s). We refer to Chapter 15 of [178],
Section 2.6 of [139], and to [15, 202, 248] for surveys.
Theory of sound. The passage of a sound wave through an inhomogeneous medium can be modelled using a random walk in phase space formed by
summing random wave-vectors of fixed amplitude and random phases. This
approach goes back to Lord Rayleigh [264]; subsequent work includes [143].
Other applications. A selection of other applications of random walks
includes electronic converters [29], cosmic microwave background [294, 266],
and financial mathematics [94].
Section 1.2
Simple random walk was studied by Lord Rayleigh (see [35]) but the seminal
early contribution was Pólya’s 1921 paper [259] which gave Theorem 1.2.1.
Versions of the usual proof, which is essentially Pólya’s, can be found for example in Chapter 10 of [241]. Simple random walk is undoubtedly the most
18
Chapter 1. Introduction
studied random walk model; some of the key results from the vast literature
are covered in the books of Feller [98] and Révész [269], for example.
This deceptively simple model shows deep and interesting phenomena.
For example, there is a remarkable closed-from expression for p3 , given by
p3 = 1 −
1
≈ 0.340537,
u3
where
Z πZ πZ π
3
dxdydz
u3 =
3
(2π) −π −π −π 3 − cos x − cos y − cos z
√
6 1 5 7 11 =
Γ
Γ
Γ
Γ
;
32π 3 24
24
24
24
a version of the first equality for u3 was given by McCrea and Whipple [219],
the second is a result of Glasser and Zucker [118].
Section 1.3
The approach described in this section is due to Lamperti [190, 192], and
is discussed at length in Chapter 3 of this book. The statement of the
recurrence classification Theorem 1.3.1 is made precise in Theorem 3.2.3 (in
the Markovian case) and Theorems 3.5.1 and 3.5.2 (in the more general case
where the Markov property does not hold).
The observation that kSn k is not Markov is an important instance of
a general phenomenon; if Yn is a Markov chain, then f (Yn ) need not be
Markov. In the particular case where the state-space is Zd and f (x) = kxk,
it is clear what goes ‘wrong’ for kSn k, as illustrated in Figure 1.3: the function x 7→ kxk is many-to-one in some places, with pre-images that contain
‘distinguishable’ points. In this case, one can rectify the issue by considering
f (x) = kζ + xk for some ζ ∈ Rd with irrational coordinates; then f is oneto-one over Zd , and kζ + Sn k is Markov. So it is misleading to overplay the
loss of the Markov property in the case of kSn k. However, kSn k is certainly
a more natural object than kζ + Sn k, and we will encounter other situations
(more general state-spaces, more complicated Lyapunov functions) where it
is less straightforward to rescue the Markov property in such a way.
The general question of when a function f applied to a Markov chain
Yn yields a Markov chain f (Yn ) is a topic that has received much interest
over the years, under the heading of Markov functions. Here we mention,
for example, [274, 196, 272, 156].
Bibliographical notes
19
Section 1.4
The random walk with spherical increments described in Example 1.4.2 is
known as the Pearson–Rayleigh random walk after the exchange in the letters pages of Nature between Karl Pearson and Lord Rayleigh in 1905 [253]:
A man starts from a point O and walks ` yards in a straight line;
he then turns through any angle whatever and walks another `
yards in a second straight line. He then repeats this process n
times.
Carazza speculates [35, p. 419] that Pearson’s enthusiasm for this problem
may have originated in the game of golf, although Pearson’s academic interest in such problems was directed towards migration of animal species,
such as mosquitoes [254]; Rayleigh had earlier considered the acoustic ‘random walks’ in phase space produced by combinations of sound waves of the
same amplitude and random phases.
Section 1.5
Theorem 1.5.2 is a special case of the famous Chung–Fuchs theorem, which
gives a criterion for recurrence/transience for general spatially homogeneous
random walks on Rd ; see [47] or Chapter 9 of [150]. In dimension d = 1, the
second moments condition can be replaced by the weaker condition E |θ0 | <
∞, while in dimension d ≥ 3, the moments condition can be dispensed with
altogether; in these cases one must reformulate the covariance condition as
a condition on the support of the increment distribution being of the full
dimension.
Theorem 1.5.3 exhibits the possible anomalous recurrence properties of
non-homogeneous random walks; this issue is explored in detail in Chapter 4.
The statement of Theorem 1.5.3 is exhibited by both the elliptic random
walks of Theorem 4.2.1 (see also [114]) and the controlled random walks of
Section 4.3 (see also [257]).
Although certainly appreciated by experts, Theorem 1.5.3 is perhaps not
as widely known as it might be. Zeitouni (pp. 91–92 of [320]) describes an
example of a transient zero-drift random walk on Z2 , and states that the
idea “goes back to Krylov (in the context of diffusions)”.
Theorem 1.5.4 shows that under suitable regularity conditions the Chung–
Fuchs theorem extends to spatially non-homogeneous random walks with
zero drift and a fixed increment covariance. The proof of Theorem 1.5.4
20
Chapter 1. Introduction
is given as Example 3.5.4. A stronger version of Theorem 1.5.4 is Theorem 4.1.3 proved in Chapter 4, which relaxes the uniform bound on the
increments of the walk to a moments condition.
Section 1.6
We do not claim that Theorem 1.6.1 is new, but we could not find a reference.
Thus we present the proof here.
Proof of Theorem 1.6.1. Suppose first that µ 6= 0. Then the strong law of
large numbers shows that n−1 ξn → µ, a.s., and n−1 kξn k → kµk, a.s. The
statement in part (i) now follows, since
n−1 ξn
= µ̂, a.s.
n→∞ n−1 kξn k
lim ξˆn = lim
n→∞
On the other hand, suppose µ = 0. Observe that, by exchangeability, the Hewitt–Savage zero-one law (see e.g. [150, p. 53]) shows that
P[limn→∞ ξˆn exists] ∈ {0, 1}, and if the limit exists with probability 1, it
is a.s. degenerate.
Suppose, for the purpose of deriving a contradiction, that
h
i
P lim ξˆn exists > 0,
n→∞
in which case the probability must be equal to 1. Since ξˆn ∈ Sd−1 ∪ {0},
and the limit can be 0 only if ξn = 0 for all but finitely many n, which is
impossible since P[θ0 = 0] < 1, it follows that P[limn→∞ ξˆn = e] = 1 for
some deterministic e ∈ Sd−1 . Then
lim
n→∞
e · ξn
= kek2 = 1, a.s.,
kξn k
Pn−1
which implies that lim inf n→∞ e · ξn ≥ 0, a.s. But e · ξn = k=0
e · θk is
a one-dimensional random walk with mean increment E[e · θ0 ] = e · µ = 0,
which means that lim inf n→∞ e · ξn = −∞, a.s. (see e.g. [150, p. 167]). This
gives the desired contradiction, and yields the statement in part (ii).
Section 1.7
Centrally biased random walks are discussed in detail in Chapter 4; refer
to the notes to that chapter for full bibliographic details. The recurrence
classification, Theorem 1.7.1, is contained in Theorem 4.4.8. The result on
Bibliographical notes
21
angular asymptotics, Theorem 1.7.2, is contained in Theorem 4.4.5. The
result on the limiting direction in the supercritical case, Theorem 1.7.3, is
contained in Theorem 4.4.2.
22
Chapter 1. Introduction
Chapter 2
Semimartingale approach
and Markov chains
2.1
Definitions
We assume that the reader is familiar with the basic concepts of probability
theory, including convergence of random variables and uniform integrability.
In this section we recall some basic definitions related to Markov processes with discrete time and (sub-, super-)martingales, and introduce some
of our notational conventions.
Definition 2.1.1 (Basic concepts for discrete-time stochastic processes).
In the following, all random variables are defined on a common probability
space (Ω, F, P). Random variables take values usually in R or Rd , possibly
extended to allow components ±∞ to handle limiting variables. We write E
for expectation corresponding to P, which will be applied to real-valued
random variables, or componentwise in the case of random vectors in Rd .
• We take as a state space a measurable space (Σ, E). For technical
reasons (existence of Markov transition functions), one requires that
(Σ, E) is a Borel space; we will always assume that any state space is
Borel. In this book, typically Σ ⊆ Rd ; often, Σ ⊆ Rd is countable. In
these cases the σ-field E is the obvious one (i.e., the appropriate Borel
sets B(Σ) = {A ∩ Σ : A ∈ B(Rd )}, which reduces to the power set 2Σ
in the countable case), and will usually not be mentioned explicitly;
then we use the term ‘state space’ to refer simply to the set Σ.
• A discrete-time stochastic process with state space (Σ, E) is sequence
of random variables Xn : (Ω, F) → (Σ, E) indexed by n ∈ Z+ . We
23
24
Chapter 2. Semimartingale approach and Markov chains
write such sequences as (Xn , n ≥ 0), with the understanding that the
time index n is always an integer.
• A filtration (of F) is a sequence of σ-fields (Fn , n ≥ 0) such that
Fn ⊆ Fn+1 ⊆ F for all n ≥ 0. We set F∞ := σ(∪n≥0 Fn ) ⊆ F.
• A stochastic process (Xn , n ≥ 0) is adapted to a filtration (Fn , n ≥ 0)
if Xn is Fn -measurable for all n ∈ Z+ .
• It is convenient, for definiteness, to introduce the notation X∞ :=
lim supn→∞ Xn , computed componentwise if Xn ∈ Rd , so that X∞
d
has a meaning (as an extended random variable in R ) even when the
limit does not exist. Note that if (Xn , n ≥ 0) is adapted to (Fn , n ≥ 0),
then X∞ is F∞ -measurable.
• For a (possibly infinite) random variable τ ∈ Z+ , the random variable
Xτ is (as the notation suggests) equal to Xn on {τ = n} for finite
n ∈ Z+ and equal to X∞ := lim supn→∞ Xn on {τ = ∞}.
• A (possibly infinite) random variable τ ∈ Z+ is a stopping time with
respect to a filtration (Fn , n ≥ 0) if {τ = n} ∈ Fn for all n ≥ 0 (and
n = ∞). If τ is a stopping time, then so are τ + 1, τ + 2, etc.
• If τ is a stopping time, the corresponding σ-field Fτ consists of all
events A ∈ F∞ such that A ∩ {τ ≤ n} ∈ Fn for all n ≥ 0 (and
n = ∞). Note that Fτ ⊆ F∞ ; events in Fτ include {τ = ∞}, as well
as {Xτ ∈ B} for B ∈ E, and so on.
For any process (Xn , n ≥ 0), one can define a (minimal) filtration to
which this process is adapted via Fn = σ(X0 , X1 , . . . , Xn ), to give the socalled natural filtration.
If (Xn , n ≥ 0) is a process adapted to a filtration (Fn , n ≥ 0) of F and τ
is a stopping time, then, from the definitions above, (Fn , n ≥ τ ) is also a
filtration of F, to which the sequence (Xn , n ≥ τ ) is adapted, where on the
event {τ = ∞} the latter notation stands for (X∞ , X∞ , . . .).
To keep the notation concise, we will frequently write Xn and Fn instead
of (Xn , n ≥ 0) and (Fn , n ≥ 0) (and so on) when no confusion will arise.
Definition 2.1.2 (Martingales, submartingales, supermartingales). A realvalued stochastic process Xn adapted to a filtration Fn is a martingale (with
respect to the given filtration) if, for all n ≥ 0,
(i) E |Xn | < ∞, and
25
2.1. Definitions
(ii) E[Xn+1 − Xn | Fn ] = 0.
If in (ii) “=” is replaced by “≥” (respectively, “≤”), then Xn is called a
submartingale (respectively, supermartingale).
We use the term semimartingale in a less precise way, to encompass not
only martingales, submartingales and supermartingales, but also adapted
processes satisfying ‘drift’ conditions of a similar kind to (ii) above; perhaps,
for instance, only locally, i.e., on {Xn ∈
/ A} for some measurable set A.
Another measure to avoid notational clutter is that we will frequently
drop ‘almost surely’ qualifiers in statements asserting equality or inequality
of random variables, as in part (ii) above. Moreover, we often will not
mention explicitly with respect to which filtration the process is a (sub-,
super-)martingale, since normally there will be no possibility of ambiguity.
Consider an Fn -adapted stochastic process Xn taking values in a state
space (Σ, E). For A ∈ E we define
τA := min{n ≥ 0 : Xn ∈ A}, and τA+ := min{n ≥ 1 : Xn ∈ A};
we may refer to either τA or τA+ as the hitting time of A (also called the
passage time into A). Then τA+ and τA are Fn -stopping times, given that
A ∈ E (see e.g. [150, p. 123]).
In the case where Σ ⊆ R we will also frequently use notation for the
hitting times of half-lines; namely, for x ∈ R we set
ρx := τΣ∩[x,∞) = min{n ≥ 0 : Xn ≥ x},
λx := τΣ∩(−∞,x] = min{n ≥ 0 : Xn ≤ x},
for right and left passage times, and, for x ≥ 0,
σx := τΣ\(−x,x) = min{n ≥ 0 : |Xn | ≥ x} = λ−x ∧ ρx .
In the case where Σ ⊆ R+ , obviously ρx = σx for x ≥ 0; in this case our
preference is to use the notation σx .
Next we recall some fundamental definitions on Markov processes.
Definition 2.1.3 (Markov chains).
• An Fn -adapted process Xn taking values in a state space (Σ, E) is a
Markov chain if, for any B ∈ E, any n ≥ 0, and any m ≥ 1,
P[Xn+m ∈ B | Fn ] = P[Xn+m ∈ B | Xn ], a.s.
This is the Markov property.
(2.1)
26
Chapter 2. Semimartingale approach and Markov chains
• If there is no dependence on n in (2.1), the Markov chain is homogeneous in time (or time-homogeneous). Unless explicitly stated otherwise, all Markov chains considered in this book are assumed to be
homogeneous in time. In this case, the Markov property (2.1) reads
P[Xn+m ∈ B | Fn ] = Pm (Xn , B), a.s.,
(2.2)
where Pm : Σ × E → [0, 1] is the m-step Markov transition function,
which satisfies the Chapman–Kolmogorov relation Pn+m (x, B) =
R
P
Σ n (x, dy)Pm (y, B).
• We use the shorthand Px [ · ] = P[ · | X0 = x] and Ex [ · ] = E[ · | X0 =
x] for probability and expectation for the time-homogeneous Markov
chain starting from initial state x ∈ Σ.
• If the state space Σ is countable (i.e., finite, or countably infinite), we
call Xn a countable Markov chain. For a countable state space Σ, we
always suppose that E = 2Σ , the power set of Σ.
• If the Markov chain is time-homogeneous and countable, we write
p(x, y) := P[X1 = y | X0 = x] = P1 (x, {y}) for the one-step transition
probabilities of the Markov chain.
• A time-homogeneous, countable Markov chain is irreducible if for all
x, y ∈ Σ there exists m = m(x, y) ∈ N such that Px [Xm = y] > 0.
For an uncountable state space Σ, notions of irreducibility are more
technical, and not discussed in this book. For our purposes, if we refer
to an irreducible Markov chain, it is implicit that the Markov chain is
time-homogeneous and countable.
• For an irreducible Markov chain, we define its period as the greatest
common divisor of {n ∈ N : Px [Xn = x] > 0} (it is straightforward to
show that it does not depend on the choice of x ∈ Σ). An irreducible
Markov chain with period 1 is called aperiodic.
Suppose now that Xn is a countable Markov chain; recall the definitions
of the hitting times τA+ and τA , A ⊆ Σ. For x ∈ Σ, we use the notation
+
τx+ := τ{x}
and τx := τ{x} for hitting times of one-point sets. Note that for
any x ∈ A, Px [τA = 0] = 1, while τA+ ≥ 1 is then the return time to A. Also
note that Px [τA = τA+ ] = 1 for all x ∈ Σ \ A.
Definition 2.1.4. For a countable Markov chain Xn , a state x ∈ Σ is called
27
2.1. Definitions
• recurrent if Px [τx+ < ∞] = 1;
• transient if Px [τx+ < ∞] < 1.
A recurrent state x is classified further as
• positive recurrent if Ex τx+ < ∞;
• null recurrent if Ex τx+ = ∞.
It is straightforward to show that the four properties in Definition 2.1.4
are class properties, which entails the following statement.
Proposition 2.1.5. For an irreducible Markov chain, if a state x ∈ Σ
is recurrent (respectively, positive recurrent, null recurrent, transient) then
all states in Σ are recurrent (respectively, positive recurrent, null recurrent,
transient).
By the above fact, it is legitimate to call an irreducible Markov chain
itself recurrent (positive recurrent, null recurrent, transient).
A basic fact that we need about positive recurrent Markov chains is the
following. As with several other standard results in this chapter, we state
the result without proof; see the bibliographical notes at the end of the
chapter for references to the literature.
Theorem 2.1.6. Suppose that Xn is an irreducible,positive recurrent Markov
chain. Define the stationary measure π(x), x ∈ Σ by π(x) = (Ex τx+ )−1 .
P
(i) It holds that x∈Σ π(x) = 1 (so π defines a probability measure on Σ),
π(x) > 0 for all x ∈ Σ, and
X
π(x) =
π(y)p(y, x), for all x ∈ Σ.
y∈Σ
(This implies that if X0 is distributed according to the stationary measure π, then so is Xn for all n.)
(ii) If the Markov chain is also aperiodic, then Px [Xn = y] → π(y) as
n → ∞ for all x, y ∈ Σ. For periodic chains,
P convergence also takes
place, but only in the Cesàro sense: n−1 Ex n−1
k=0 1{Xk = y} → π(y).
Example 2.1.7. Let p ∈ (0, 1), and assume that Z1 , Z2 , Z3 , . . . are i.i.d.
random variables with P[Z1 = 1] = 1 − P[Z1 = −1] = p. Let F0 = {∅, Ω},
the trivial σ-field, and Fn = σ(Z1 , . . . , Zn ) for n ≥ 1. We set S0 = 0
28
Chapter 2. Semimartingale approach and Markov chains
and Sn = Z1 + · · · + Zn for n ≥ 1; the Fn -adapted process Sn is a onedimensional simple random walk (SRW) with parameter p. In the case
p = 21 , we say that Sn is (one-dimensional) symmetric SRW, or sometimes
just SRW, without any adjectives.
The following observations are straightforward.
• Sn is a Markov chain with countable state space Z, and transition
probabilities p(x, x + 1) = 1 − p(x, x − 1) = p for x ∈ Z;
• this Markov chain is irreducible and periodic with period 2;
• with respect to the filtration Fn , the process Sn is a supermartingale
for p ≤ 12 , a submartingale for p ≥ 12 , and a martingale for p = 21 ;
• for the symmetric SRW, Sn2 is a submartingale, and (Sn2 − n) is a
martingale (in fact, Sn2 = (Sn2 − n) + n is the Doob decomposition of
Sn2 : see Theorem 2.3.1 below);
x−1
1−p x+1
1−p x
• (1 − p) 1−p
+
p
=
for all x ∈ Z, so the prop
p
p
1−p Sn
cess p
is a martingale (this fact, of course, has non-trivial con4
sequences only for p 6= 12 ).
Example 2.1.8. Now, consider a modification Ŝn of the process of the
previous example, which lives on Σ = Z+ := {0, 1, 2, . . .}. Again, fix a
parameter p ∈ (0, 1); for x > 0 we keep the same transition probabilities
p(x, x + 1) = 1 − p(x, x − 1) = p, and we set p(0, 1) = 1; we will call
this process the (reflected) simple random walk on the half-line. Then, for
p ≥ 21 this process is still a submartingale, but one loses the supermartingale
property for p ≤ 21 (as well as the martingale property for p = 12 ) because
of the reflection at 0. However, these properties are readily recovered if we
consider the stopped process Ŝn∧τ0 instead of Ŝn itself.
4
i
When the desired (sub-, super-)martingale property breaks
down in some region, often it is possible (and useful) to fix
it by considering a modification of the original process by
means of a suitable stopping time.
The notions of recurrence and transience given in Definition 2.1.4 depend
only on the structure of Σ as a set (i.e., are purely ‘topological’). In the context of this book, geometric consequences of these notions are important; we
are usually concerned with a given embedding of Σ into Euclidean space Rd .
29
2.1. Definitions
A set A ⊆ Rd is bounded if supx∈A kxk < ∞; otherwise it is unbounded
(recall the convention sup ∅ = 0). A set A ⊆ Rd is locally finite if #(A∩B) <
∞ for all bounded B ⊂ Rd . Of course, a locally finite set is countable, and
an unbounded locally finite set is countably infinite.
We state here some basic results for Markov chains, which follow from
more general results whose proofs are presented later, in Section 3.6.
Theorem 2.1.9. Let Xn be an irreducible time-homogeneous Markov chain
on an unbounded, countable state space Σ ⊂ Rd .
(i) If Xn is recurrent, then, a.s.,
lim inf kXn k = inf kxk, and lim sup kXn k = ∞.
n→∞
x∈Σ
n→∞
(ii) If Σ is locally finite and Xn is transient, then limn→∞ kXn k = ∞, a.s.
Without local finiteness of Σ, part (ii) of Theorem 2.1.9 may fail: see
the bibliographical notes at the end of this chapter.
Contained in Theorem 2.1.9 is the following basic fact.
Corollary 2.1.10. Let Xn be an irreducible time-homogeneous Markov chain
on an unbounded, locally finite state space Σ ⊂ Rd . Then lim supn→∞ kXn k =
∞, a.s.
We end this section with a technical lemma, which the standard notation
suggests is obvious, and will often be used without comment.
Lemma 2.1.11. Let Fn be a filtration and τ a stopping time.
(i) If X is an integrable random variable, then for any n ∈ Z+ ,
E[X | Fτ ]1{τ = n} = E[X | Fn ]1{τ = n} = E[X1{τ = n} | Fn ].
(ii) If Xn is an Fn -adapted process such that limn→∞ Xn = X∞ exists a.s.,
then limn→∞ Xn∧τ = Xτ , a.s.
Proof. (i) Fix n ∈ Z+ . Write ξ = E[X | Fn ] and ζ = E[X | Fτ ]. Note
that ξ and ζ are integrable, and, by definition of Fτ , both ξ1{τ = n} and
ζ1{τ = n} are both Fτ -measurable and Fn -measurable. For any A ∈ Fτ ,
A∩{τ = n} is in Fτ and Fn , so that, by definition of conditional expectation,
E[ζ1(A ∩ {τ = n})] = E[X1(A ∩ {τ = n})] = E[ξ1(A ∩ {τ = n})].
30
Chapter 2. Semimartingale approach and Markov chains
Hence E[(ζ − ξ)1{τ = n}1(A)] = 0. Choosing A = {(ζ − ξ)1{τ = n} >
0} ∈ Fτ we deduce that E[(ζ − ξ)+ 1{τ = n}] = 0. A similar argument
gives E[(ζ − ξ)− 1{τ = n}] = 0, and hence |ζ − ξ|1{τ = n} = 0, a.s. Exactly
the same proof works when n = ∞, but it is worth reading through the
argument again in that case.
(ii) Suppose limn→∞ Xn = X∞ . Then limn→∞ Xn 1{τ > n} = X∞ 1{τ = ∞},
while limn→∞ Xτ 1{τ ≤ n} = Xτ 1{τ < ∞}. In other words,
lim Xn∧τ = X∞ 1{τ = ∞} + Xτ 1{τ < ∞} = Xτ ,
n→∞
as claimed.
The point of Lemma 2.1.11(i) will become clearer later on; it allows us
to extend statements like E[f (Xn+1 , . . . , Xn+k ) | Fn ] ≤ g(Xn ) (for all n) to
the ‘stopped’ version E[f (Xτ +1 , . . . , Xτ +k ) | Fτ ] ≤ g(Xτ ) for finite stopping
times τ . Here is one example of an application.
Example 2.1.12. Let Xn be an Fn -adapted time-homogeneous Markov
chain on state space (Σ, E). Let τ be a stopping time. Then, on {τ < ∞},
P[Xτ +m ∈ B | Fτ ] =
=
=
∞
X
n=0
∞
X
n=0
∞
X
P[Xτ +m ∈ B | Fτ ]1{τ = n}
P[Xn+m ∈ B | Fn ]1{τ = n},
Pm (Xn , B)1{τ = n},
n=0
by Lemma 2.1.11(i) and then (2.2). The fact that the Markov property (2.2)
extends to stopping times as
P[Xτ +m ∈ B | Fτ ] = Pm (Xτ , B), on {τ < ∞},
is the strong Markov property.
2.2
4
An introductory example
In this section we briefly describe a simple example that is rich enough to
display a range of interesting behaviour. We will refer back to this example
from time to time to illustrate the subsequent discussions in this chapter.
This example, and many of the classical methods of its analysis, are likely
31
2.2. An introductory example
to be familiar to the reader; we present some of the standard results, but
will give semimartingale-style proofs here and throughout this chapter to
illustrate our approach.
Our example is a nearest-neighbour (i.e., simple) random walk on Z+ ,
(Xn , n ≥ 0). For x ≥ 1, we set
P[Xn+1 = x − 1 | Xn = x] = px ,
P[Xn+1 = x + 1 | Xn = x] = 1 − px =: qx ,
(2.3)
and P[Xn+1 = 0 | Xn = 0] = p0 , P[Xn+1 = 1 | Xn = 0] = 1 − p0 =: q0 .
Assuming px ∈ (0, 1) for all x ∈ Z+ , Xn is an irreducible, aperiodic
Markov chain on the locally finite state space Z+ ; in particular, Corollary 2.1.10 shows that lim supn→∞ Xn = +∞, a.s.
For x ∈ Z+ we make the following definitions, recalling that an empty
sum in 0 and an empty product is 1.
dx :=
x−1
Y
y=0
ux :=
x
X
py
;
qy
(2.4)
y=0
vx :=
∞
X
y=x
x
Y
−1
qx−y
z=x−y+1
p−1
y
1
px
px px−1 · · · p1
pz
=
+
+ ··· +
;
qz
qx qx qx−1
qx qx−1 · · · q1 q0
y−1
Y
qz
1
qx
=
+
+ ··· ;
p
px px px+1
z=x z
(2.5)
(2.6)
in particular, d0 = 1 and u0 = 1/q0 . Since px ∈ (0, 1) for all x, dx and ux
are finite for all x. However, vx < ∞ if and only if
∞
X
y=x
−1
p−1
y dy < ∞
⇐⇒
∞
X
y=x
−1
(d−1
y + dy+1 ) < ∞,
−1 = d−1 + d−1 . Hence v < ∞ if and only if
using the fact that p−1
x
y dy
y
y+1
P∞ −1
d
<
∞.
x=1 x
Now for x ∈ Z+ define
h(x) :=
x
X
dy ;
(2.7)
y=0
P
then h : Z+ → R+ is increasing, so limx→∞ h(x) = ∞
x=0 dx exists in R+ .
The function h has a classical interpretation in terms of hitting probabilities.
Recall that τx := min{n ≥ 0 : Xn = x}; here x ∈ Z+ . Note that Xτx = x
a.s. and τx 6= τy for any x 6= y.
32
Chapter 2. Semimartingale approach and Markov chains
Lemma 2.2.1. Let a, b, x be integers with 0 ≤ a < b and a ≤ x ≤ b. Then
Px [τa < τb ] =
h(x) − h(b)
.
h(a) − h(b)
The classical proof of Lemma 2.2.1 goes via solving a difference equation.
We will present a different argument in Example 2.4.2 below, based on the
following fact.
Lemma 2.2.2. For any n ∈ Z+ , and any x ∈ Z+ ,
E[h(Xn+1 ) − h(Xn ) | Xn = x] = q0 1{x = 0}.
Proof. It suffices to take n = 0. For x ≥ 1,
Ex [h(X1 ) − h(X0 )] = px h(x − 1) + qx h(x + 1) − h(x)
= qx dx+1 − px dx = 0.
On the other hand, E0 [h(X1 ) − h(X0 )] = q0 (h(1) − h(0)) = q0 d0 = q0 .
For x ∈ Z+ , set
t(x) :=
x−1
X
uy , and r(x) :=
y=0
x
X
vy ,
(2.8)
y=1
where uy and vy are given by (2.5) and (2.6). By theP
remarks above, t(x) <
−1
∞ for all x, while r(x) < ∞ for x ≥ 1 if and only if ∞
y=1 dy < ∞.
Lemma 2.2.3.
(i) For x ∈ Z+ , E0 τx = t(x).
(ii) For x ∈ Z+ , Ex τ0 = r(x).
We give a proof of Lemma 2.2.3 based on the following semimartingale
properties of the functions r and t.
Lemma 2.2.4.
(i) For any n ∈ Z+ and any x ∈ Z+ ,
E[t(Xn+1 ) − t(Xn ) | Xn = x] = 1.
P∞
−1
x=1 dx
< ∞. Then, for any n ∈ Z+ and any x ∈ Z+ ,
(
−1,
if x ≥ 1,
E[r(Xn+1 ) − r(Xn ) | Xn = x] =
q0 v1 , if x = 0.
(ii) Suppose that
33
2.2. An introductory example
Proof. It suffices to take n = 0. For x ≥ 1, we have
Ex [t(X1 ) − t(X0 )] = px (t(x − 1) − t(x)) + qx (t(x + 1) − t(x))
= qx ux − px ux−1 = 1,
by (2.5). Also,
E0 [t(X1 ) − t(X0 )] = q0 t(1) = 1,
since t(1) = u0 = 1/q0 . This gives (i). Similarly, for x ≥ 1,
Ex [r(X1 ) − r(X0 )] = px (r(x − 1) − r(x)) + qx (r(x + 1) − r(x))
= qx vx+1 − px vx = −1,
by (2.6). Also,
E0 [r(X1 ) − r(X0 )] = q0 (r(1) − r(0)) = q0 v1 ,
which completes the proof of (ii).
Proof of Lemma 2.2.3. Let Fn = σ(X0 , . . . , Xn ). For part (i), let Yn =
t(Xn∧τx ). By Lemma 2.2.4(i), for m ≥ 0,
E[Ym+1 − Ym | Fm ] = 1{τx > m}.
Taking expectations conditioned on X0 = 0, summing over m from 0 to
n − 1, and then letting n → ∞ gives
E0 τx = lim
n→∞
n−1
X
m=0
P0 [τx > m] = lim E0 t(Xn∧τx ),
n→∞
using the fact that t(0) = 0. Here t(Xn∧τx ) is uniformly bounded and
converges a.s. to t(Xτx ) = t(x), so by the bounded convergence theorem we
get part (i).
P
−1
For part (ii), first suppose that ∞
x=1 dx < ∞, so that r(x) ≤ r(∞) <
∞ for all x. Let Yn = r(Xn∧τ0 ). Similarly to the preceding argument,
Lemma 2.2.4(ii) shows that Ex [n ∧ τ0 ] = Ex Y0 − Ex Yn . Here 0 ≤ Yn ≤
r(∞) < ∞, so Yn is uniformly bounded and converges to r(0) = 0, so again
we may apply the bounded convergence
theorem to deduce that Ex τ0 = r(x).
P
−1
The fact that Ex τ0 = ∞ when ∞
x=1 dx = ∞ is easiest to see from the
classical difference equation argument.
Now we can state the basic classification result, in terms of the quantities dx defined at (2.4).
34
Chapter 2. Semimartingale approach and Markov chains
P
(i) Xn is recurrent if and only if ∞
x=1 dx = ∞;
P∞ −1
(ii) Xn is positive recurrent if and only if x=1 dx < ∞.
Theorem 2.2.5.
Proof. There are numerous ways to prove Theorem 2.2.5; we describe an
instructive approach. Suppose X0 = 0. Since lim supn→∞ Xn = ∞ a.s., we
have τx < ∞ a.s. for all x ∈ Z+ . Also, since |Xn+1 − Xn | ≤ 1, we have
τx+1 ≥ τx + 1. Hence τx is strictly increasing in x, with limx→∞ τx = ∞.
Hence τ0+ = ∞ if and only if τ0+ > τx for all x ∈ N. Since {τ0 > τx } ⊇ {τ0 >
τx+1 }, we have
P0 [τ0+ = ∞] = q0 P1 [∩x≥2 {τ0 > τx }] = q0 lim P1 [τx < τ0 ],
x→∞
by continuity of probability along monotone limits. Hence by Lemma 2.2.1,
P0 [τ0+ = ∞] = q0 lim
x→∞
h(1) − h(0)
,
h(x) − h(0)
which is 0 if and only if limx→∞ h(x) = ∞, giving part (i) of Theorem 2.2.5.
Part (ii) of Theorem 2.2.5 can be derived by direct computation of the
stationary measure π, using the reversibility of the random walk, or by the
hitting time result Lemma 2.2.3, since positive recurrence is equivalent to
finiteness of E0 τ0+ = E0 τ1 + E1 τ0 .
After developing some of the necessary technology, we will give Lyapunov
function proofs of parts of Theorem 2.2.5 later, in Examples 2.5.5 and 2.5.13.
2.3
Fundamental semimartingale facts
In this section we state (often, without proof) some basic facts related to
(sub-, super-)martingales that we will need in this book. We also give a
number of examples to illustrate in a simple way some of the basic ideas of
applications of these results.
Recall that the underlying setting is a probability space (Ω, F, P) equipped
with a filtration (Fn , n ≥ 0). Whenever we refer without further qualification to a martingale, submartingale, or supermartingale Xn , it is implicit
that Xn takes values in R.
Theorem 2.3.1 (Doob decomposition). A submartingale Xn has a unique
representation Xn = Mn + An , n ≥ 0, where Mn is a martingale, and An is
a non-decreasing process such that A0 = 0 and An is Fn−1 -measurable for
n ≥ 1 (that is, An is previsible).
2.3. Fundamental semimartingale facts
35
Recall that a random variable X is integrable if E |X| < ∞ and squareintegrable if E[X 2 ] < ∞; a process (Xn , n ≥ 0) is square-integrable if
E[Xn2 ] < ∞ for all n ≥ 0.
Theorem 2.3.2 (Orthogonality of martingale increments). Let Xn be a
square-integrable martingale. If m ≤ n and Y is Fm -measurable with finite
second moment, then E[(Xn − Xm )Y ] = 0.
Observe that Theorem 2.3.2 implies that E[(Xm+1 − Xm )(Xk+1 − Xk )] =
0 for k 6= m for any square-integrable martingale Xn (thus justifying its
name).
Theorem 2.3.3 (Martingales and convexity). (i) If Xn is a martingale
and g : R → R is a convex function such that E |g(Xn )| < ∞ for all n,
then g(Xn ) is a submartingale, with respect to the same filtration.
(ii) If Xn is a submartingale and g : R → R is a non-decreasing convex
function such that E |g(Xn )| < ∞ for all n, then g(Xn ) is a submartingale, with respect to the same filtration.
As an immediate corollary, we obtain that if Xn is a square-integrable
martingale, then Xn2 is a submartingale.
A fundamental result is the following martingale convergence theorem.
Theorem 2.3.4 (Martingale convergence theorem). Assume that Xn is a
submartingale such that supn E[Xn+ ] < ∞. Then there is an integrable random variable X such that Xn → X a.s. as n → ∞.
Remarks 2.3.5. (a) Under the hypotheses of Theorem 2.3.4, the sequence
E Xn is non-decreasing (by the submartingale property) and bounded above
by supn E[Xn+ ], so limn→∞ E Xn exists and is finite; however, it is not necessarily equal to E X.
(b) Note that Xn− = Xn+ − Xn so that E[Xn− ] ≤ E[Xn+ ] − E X0 , by the
submartingale property, so that the hypothesis supn E[Xn+ ] < ∞ actually
implies that supn E |Xn | < ∞.
An important corollary to Theorem 2.3.4 and Fatou’s lemma is as follows.
Theorem 2.3.6 (Convergence of non-negative supermartingales). Assume
that Xn ≥ 0 is a supermartingale. Then there is an integrable random
variable X such that Xn → X a.s. as n → ∞, and E X ≤ E X0 .
Another fundamental result that we will use extensively is the following.
36
Chapter 2. Semimartingale approach and Markov chains
Theorem 2.3.7 (Optional stopping theorem). Suppose that σ ≤ τ are
stopping times, and Xτ ∧n is a uniformly integrable submartingale. Then
E Xσ ≤ E Xτ < ∞ and Xσ ≤ E[Xτ | Fσ ] a.s.
Remarks 2.3.8. (a) From the conclusion Xσ ≤ E[Xτ | Fσ ] a.s., it follows on
taking conditional expectations that E[Xσ | F0 ] ≤ E[Xτ | F0 ] a.s., which is
another useful form of optional stopping.
(b) If Xn is a uniformly integrable submartingale and τ is any stopping time,
then it can be shown that Xτ ∧n is also uniformly integrable: see e.g. Section
5.7 of [83].
(c) Observe that two applications of Theorem 2.3.7, one with σ = 0 and
one with τ = ∞, show that for any uniformly integrable submartingale
Xn and any stopping time τ , E X0 ≤ E Xτ ≤ E X∞ < ∞, where X∞ :=
lim supn→∞ Xn = limn→∞ Xn exists and is integrable, by Theorem 2.3.4.
Theorem 2.3.7 has the following corollary, obtained on setting σ = 0
and using well-known sufficient conditions for uniform integrability (see e.g.
Sections 4.5 and 4.7 of [83]).
Corollary 2.3.9. Let Xn be a submartingale and τ a finite stopping time.
For a constant c > 0, suppose that at least one of the following holds:
(i) τ ≤ c a.s.;
(ii) |Xn∧τ | ≤ c a.s. for all n ≥ 0;
(iii) E τ < ∞ and E[|Xn+1 − Xn | | Fn ] ≤ c a.s. for all n ≥ 0.
Then E Xτ ≥ E X0 . If Xn is a martingale and at least one of the above
conditions (i)–(iii) holds, then E Xτ = E X0 .
Remarks 2.3.10. (a) Corollary 2.3.9(i) exhibits the important fact that for
any stopping time τ , if Xn is a (sub-, super-)martingale, then so is Xn∧τ .
(b) Suppose that τ is a stopping time and Xn is a non-negative supermartingale, so that X∞ = limn→∞ Xn exists by Theorem 2.3.6. Fatou’s lemma
with Lemma 2.1.11(ii) shows that E[Xτ ] ≤ lim inf n→∞ E[Xn∧τ ] ≤ E[X0 ],
which is a simple version of optional stopping, without the need for uniform
integrability. A more general statement of this kind is the next result.
Theorem 2.3.11. Suppose that Xn is a non-negative supermartingale and
σ ≤ τ are stopping times. Then E Xτ ≤ E Xσ < ∞ and E[Xτ | Fσ ] ≤ Xσ , a.s.
2.3. Fundamental semimartingale facts
37
Proof. Under the conditions of the theorem, Theorem 2.3.6 shows that
limn→∞ Xn = X∞ exists and is integrable. Hence by Lemma 2.1.11(ii)
and the (conditional) Fatou lemma, for any σ-field G ⊆ F,
E[Xτ | G] ≤ lim inf E[Xn∧τ | G].
n→∞
In particular, for any fixed m ≥ 0,
E[Xτ | Fm∧σ ] ≤ lim inf E[Xn∧τ | Fm∧σ ] ≤ Xm∧σ ,
n→∞
by Theorem 2.3.7 applied to the uniformly integrable submartingale Yn =
−Xm∧n , n ≥ 0. Now, by an application of Lemma 2.1.11(i),
E[Xτ | Fσ ]1{σ < ∞} = lim E[Xτ | Fσ ]1{σ ≤ m}
m→∞
= lim E[Xτ | Fm∧σ ]1{σ ≤ m}
m→∞
≤ lim Xm∧σ 1{σ < ∞} = Xσ 1{σ < ∞}.
m→∞
This gives the claim E[Xτ | Fσ ] ≤ Xσ on {σ < ∞}. On {σ = ∞}, τ = ∞
too and the claim is justified by the n = ∞ case of Lemma 2.1.11(i):
E[Xτ | Fσ ]1{σ = ∞} = E[Xτ | F∞ ]1{σ = ∞} = X∞ 1{σ = ∞}.
Example 2.3.12. Let Sn be simple random walk on Z with parameter
p ∈ (0, 1), and, for a fixed x > 0, let σ = σx = min{n ≥ 0 : |Sn | ≥ x}.
First suppose p = 12 . Then the martingale Sn∧σ is uniformly bounded,
so Sn∧σ converges, by Theorem 2.3.4. Also, Sn∧σ 1{σ < ∞} converges (to
Sσ 1{σ < ∞}). Hence Sn∧σ − Sn∧σ 1{σ < ∞} converges as well. But this
last quantity is Sn 1{σ = ∞}. Since |Sn+1 − Sn | = 1, the only way that this
convergence can happen is if P[σ < ∞] = 1.
Sn∧σ (see
If p 6= 12 , a similar argument based on the martingale ( 1−p
p )
Example 2.1.7) gives the same conclusion.
This is an amusing way of showing that lim supn→∞ |Sn | = +∞, a.s. 4
Example 2.3.13. Let Xn be an Fn -adapted stochastic process on state
space (Σ, E), and let f : Σ → R be measurable. Suppose that f (Xn ) is
integrable for all n. Then
Mn = f (Xn ) − f (X0 ) −
n−1
X
m=0
E[f (Xm+1 ) − f (Xm ) | Fm ]
38
Chapter 2. Semimartingale approach and Markov chains
is an Fn -adapted martingale. Hence, for any stopping time τ ,
(n∧τ
)−1
X
E f (Xn∧τ ) = E f (X0 ) + E
E[f (Xm+1 ) − f (Xm ) | Fm ] ,
(2.9)
m=0
which is a version of Dynkin’s formula. Under additional conditions, optional
stopping (Corollary 2.3.9) permits n → ∞ preserving equality in (2.9). 4
The following maximal inequality, named after Doob, is important.
Theorem 2.3.14 (Doob’s inequality). Let Xn be a submartingale. For any
λ > 0 and any n ≥ 0 we have
i
h
P max Xm ≥ λ F0 ≤ λ−1 E[Xn+ | F0 ].
0≤m≤n
Proof. We give a (short) proof, since this version, conditional on F0 , is rarely
stated explicitly. Fix λ > 0. Let τ = min{n ≥ 0 : Xn ≥ λ}. Then
+
Xn∧τ
≥ Xτ 1{τ ≤ n} ≥ λ1{τ ≤ n}.
Hence
+
P[τ ≤ n | F0 ] ≤ λ−1 E[Xn∧τ
| F0 ].
Since x 7→ x+ is non-decreasing and convex, and E[Xn+ ] ≤ E |Xn |, the fact
that Xn is a submartingale implies that Xn+ is also a submartingale, by
Theorem 2.3.3(ii). Then optional stopping (Theorem 2.3.7) applied at the
+
| F0 ] ≤ E[Xn+ | F0 ];
bounded stopping times n ∧ τ and n shows that E[Xn∧τ
see Remark 2.3.8(a).
Theorems 2.3.2, 2.3.3, and 2.3.14 have the following corollary.
Corollary 2.3.15. For Xn a square-integrable martingale, for any λ > 0,
n−1
h
i
X
P max |Xm | ≥ λ ≤ λ−2 E[Xn2 ] = λ−2 E[X02 ] +
E[(Xm+1 − Xm )2 ] .
0≤m≤n
m=0
Example 2.3.16. Suppose that Xn is a square-integrable martingale with
X0 = 0 such that E[(Xn+1 − Xn )2 | Fn ] ≤ K for all n ≥ 0, for some constant
K. Recall σx = min{n ≥ 0 : |Xn | ≥ x}. Then, Corollary 2.3.15 implies that
h
i
P[σn ≤ an2 ] = P max |Xm | ≥ n
≤n
0≤m≤an2
−2
2
Kan = aK.
2.3. Fundamental semimartingale facts
39
This confirms the intuition that martingales with well-behaved increments
usually move diffusively: the process is not likely to go out of the interval (−n, n) before time an2 if a is small. We return to the question of
quantifying the finite-time displacement of a process in Section 2.4.
4
Example 2.3.17. Continuing the theme of Example 2.3.16, let Sn be symmetric simple random walk on Z with S0 = 0. Then Sn2 is a non-negative submartingale with E[Sn2 ] = n, and Doob’s inequality, Theorem 2.3.14, which
in this case reduces to an instance of Kolmogorov’s inequality, gives
h
i
P max |Sm | ≥ λ ≤ λ−2 n.
0≤m≤n
For ε > 0, let u(n) = n1/2 (log n)(1/2)+ε ; then
h
i
P max |Sm | ≥ u(n) ≤ (log n)−1−2ε .
0≤m≤n
Although this seems a rather weak bound, we can still extract a reasonable
result by considering the subsequence n = 2k , k ≥ 0. The Borel–Cantelli
lemma then implies that, a.s., max0≤m≤2k |Sm | ≤ u(2k ) for all but finitely
many k. Any n ∈ N has 2kn ≤ n ≤ 2kn +1 for some kn ∈ Z+ , where kn → ∞
as n → ∞. Hence, a.s., for all but finitely many n,
max |Sm | ≤
0≤m≤n
max
0≤m≤2kn +1
|Sm | ≤ u(2 · 2kn ) ≤ 2u(n).
So we have shown that for any ε > 0, a.s., for all but finitely many n,
max |Sm | ≤ n1/2 (log n)(1/2)+ε .
0≤m≤n
This result is certainly not as sharp as the ultimate ‘lim sup’ result for simple
random walk, the law of the iterated logarithm, which states that
lim sup √
n→∞
|Sn |
= 1, a.s.,
2n log log n
but the proof here is also certainly much simpler, and readily generalizes.
We return in some detail to these ideas in Section 2.8.
4
We collect some remarks on quadratic variation for square-integrable
martingales, which will be useful occasionally later on. Let Xn be a martingale with X0 = 0, such that E[Xn2 ] < ∞ for all n. Then the process Xn2 is
40
Chapter 2. Semimartingale approach and Markov chains
a submartingale, with Doob decomposition Xn2 = Mn + An , where Mn is a
martingale with M0 = 0 and
An =
n−1
X
m=1
2
2
E[Xm+1
− Xm
| Fm ] =
n−1
X
m=1
E[(Xm+1 − Xm )2 | Fm ],
using the fact that Xn is a martingale. The non-decreasing process An is the
quadratic variation process associated to Xn , and the notation hXin = An
is used. Let τ be a finite stopping time. Then An∧τ ↑ Aτ as n → ∞ and
hence, by monotone convergence, E An∧τ → E Aτ .
2 ]=
Since Mn = Xn2 − An is a martingale, so is Mn∧τ , and hence E[Xn∧τ
E An∧τ because M0 = 0. Then, by Fatou’s lemma,
E[Xτ2 ] ≤ lim inf E An∧τ = E Aτ .
n→∞
(2.10)
2
is a submartingale, so by optional stopping (Theorem 2.3.7)
But Xn∧τ
2
2 ] = EA
2
E[Xτ ] ≥ E[Xn∧τ
n∧τ provided Xn∧τ is uniformly integrable. Since
E An∧τ → E Aτ , we note the following result.
Lemma 2.3.18. Let Xn be a square-integrable martingale with X0 = 0,
2
is uniformly integrable,
and let τ be a finite stopping time. Then, if Xn∧τ
2
E[Xτ ] = E[hXiτ ].
We finish this section with a result that we will use several times, which
is a refinement due to Lévy of the Borel–Cantelli lemma.
Theorem 2.3.19. Let (Fn , n ≥ 0) be a filtration and (En , n ≥ 0) a sequence
of events with En ∈ Fn . Then, up to sets of probability zero,
{En i.o.} =
∞
nX
n=1
o
P[En | Fn−1 ] = ∞ .
This result is useful for, among other things, formalizing the intuition
that an event that has positive probability of occurring soon, having not yet
occurred, will eventually happen; see the next example.
Example 2.3.20. Let Xn be an irreducible Markov chain on a countable
state space Σ. Suppose that there is a finite non-empty set A ⊂ Σ such that
Py [τA < ∞] = 1 for all y ∈ Σ \ A. We deduce that Py [τy+ < ∞] = 1 for all
y ∈ Σ \ A, i.e., Xn is recurrent (see Definition 2.1.4).
Note that for any y ∈ Σ \ A, irreducibility and the finiteness of A imply
that there exist m = m(y) < ∞ and δ = δ(y) > 0 such that P[τy < m |
2.4. Displacement and exit estimates
41
X0 ∈ A] > δ. Fix one such y ∈ Σ \ A (and corresponding m and δ) for the
duration of this proof.
Let ηA = τΣ\A , the first exit time from A. Then, on {ηA > n}, P[ηA <
n + m | Fn ] > δ. In particular, setting Gn = Fmn and En = {ηA < nm}, we
c
have En ∈ Gn and P[En | Gn−1 ] > δ on En−1
. Also, En−1 ⊆ En so
c
P[En | Gn−1 ] ≥ δ1(En−1
) + 1(En−1 ) ≥ δ, a.s.
Hence Theorem 2.3.19 implies that En occurs infinitely often, so P[ηA < ∞ |
X0 ∈ A] = 1.
Now take X0 = y ∈ Σ \ A. Let κ0 = τA and, for k ≥ 1, κk = min{n >
κk−1 + m : Xn ∈ A}, (a subsequence of) return times to A. It follows from
the argument above and the hypothesis that Pz [τA < ∞] = 1 for all z ∈ Σ\A
that κk < ∞, a.s., for all k.
A similar argument now shows that Xn will eventually return to y having
visited A. To spell this out, for k ≥ 1 let Ek = {τy+ ≤ κk } and now set
Gk = Fκk ; then Ek ∈ Gk and Ek−1 ⊆ Ek so that
P[Ek | Gk−1 ] ≥ P[τy+ < κk−1 + m | Fκk−1 ]1{τy+ > κk−1 } + 1{τy+ ≤ κk−1 } ≥ δ,
a.s. Thus Theorem 2.3.19 implies that Py [τy+ < ∞] = Py [En i.o.] = 1.
2.4
4
Displacement and exit estimates
A basic ingredient of many of the arguments that we use later in this book
is some estimate of how far a process can travel in a certain time, or the
probability that a process exits an interval by one end point rather than
the other. In this section we collect some examples and results in this vein,
to illustrate some important applications of the semimartingale ideas from
Section 2.3.
The main tool in this context is the optional stopping theorem. We
give several examples showing how optional stopping may be used to deduce
information on displacement. We will return to many of the ideas illustrated
in these examples later in the book.
Example 2.4.1. For SRW Sn on Z with parameter p ∈ (0, 1), p 6= 12 ,
Sn
abbreviate β = 1−p
p and Xn = β ; as observed in Example 2.1.7, Xn is a
martingale. Also, Xn∧τ{a,b} is bounded and τ{a,b} is finite, as observed in
Example 2.3.12. So, with a < x < b, optional stopping (Corollary 2.3.9)
gives
β x = Ex X0 = Ex Xτ{a,b} = β a Px [τa < τb ] + β b Px [τb < τa ],
42
Chapter 2. Semimartingale approach and Markov chains
and so
Px [τa < τb ] =
βx − βb
,
βa − βb
solving the classical gambler’s ruin problem.
In the case p = 12 the process Sn is itself a martingale, and the same
calculation gives Px [τa < τb ] = b−x
b−a . It is elementary but instructive to note
that one may recast this result in the form, say, for x ≥ 1,
1
+
P0 [σx < τ0 ] = P0 max |Sn | ≥ x = P1 [τx < τ0 ] = ,
(2.11)
+
x
0≤n<τ0
a statement about the excursion maximum of the walk. (Here σx = τ{−x,x} .)
Hence, the random variable M := max0≤n<τ + |Sn | is almost surely finite and
0
has E0 [M s ] < ∞ if and only if s < 1.
With a little extra work, this argument demonstrates the null recurrence
of the random walk Sn . Indeed, the σx are a.s. finite (by e.g. the argument
in Example 2.3.12), and a similar argument to the proof of Theorem 2.2.5
shows that
P0 [τ0+ < ∞] = P0 ∪x≥1 {τ0+ < σx } = lim P0 [τ0+ < σx ] = 1,
x→∞
by (2.11), proving recurrence. Moreover, under P0 , |Sn | ≤ n so M ≤ τ0+ ,
and
X
E0 [τ0+ ] ≥ E0 [M ] =
P0 [M ≥ x] = ∞, by (2.11).
4
x≥1
The martingale solution to the gambler’s ruin problem from Example 2.4.1
extends to give the proof of Lemma 2.2.1 from Section 2.2, as the next example shows.
Example 2.4.2. In the setting of the general nearest-neighbour random
walk on Z+ from Section 2.2, let a, b, x be integers with 0 ≤ a < b and
a ≤ x ≤ b; let X0 = x. Then h(Xn∧τa ∧τb ) is a bounded martingale, by
Lemma 2.2.2, and τa ∧ τb < ∞ a.s., by Corollary 2.1.10. Then, by optional
stopping (Corollary 2.3.9),
h(x) = Ex h(X0 ) = Ex h(Xτa ∧τb ) = h(a) Px [τa < τb ] + h(b) Px [τa > τb ].
Re-arranging this gives Lemma 2.2.1.
4
The next example returns to simple symmetric random walk to illustrate
another idea.
2.4. Displacement and exit estimates
43
Example 2.4.3. Again consider Sn simple symmetric random walk on Z;
write Fn = σ(S0 , . . . , Sn ). Example 2.4.1 showed that E0 [τ0+ ] = ∞. In fact,
E0 [(τ0+ )1/2 ] = ∞ (and this is best possible).
To show this, we elaborate the idea of the previous example. Roughly
speaking, we know that Sn reaches ±x before returning to 0 with probability 1/x; but, if it does so, it should take time about x2 to come back
to the origin, by the idea of Example 2.3.16. Formally, let x ∈ N and set
Ex = {max0≤n≤x2 |2x − Sσ2x +n | < 2x}. Then,
P0 [τ0+ ≥ x2 ] ≥ P0 [{σ2x < τ0+ } ∩ Ex ]
= E0 1{σ2x < τ0+ } P[Ex | Fσ2x ] ,
since {σ2x < τ0+ } ∈ Fσ2x . Let Xn = (Sσ2x +n − 2x)2 . Then, Xn ≤ n2 so
E Xn < ∞, and E[Xn+1 − Xn | Fσ2x +n ] = 1. Hence (Xn , n ≥ 0) is a nonnegative submartingale adapted to (Fσ2x +n , n ≥ 0) with X0 = 0, a.s., and
we may apply Doob’s inequality (Theorem 2.3.14) to show that
E Xx2 Fσ2x
1
2
P max Xn ≥ 4x Fσ2x ≤
= , a.s.,
2
2
4x
4
0≤n<x
so that P[Ex | Fσ2x ] ≥ 3/4, a.s. Thus by (2.11),
3
3
P0 [σ2x < τ0+ ] =
.
4
4x
√
Hence, for any n ∈ Z+ , P0 [τ0+ ≥ n] ≥ 34 (1 + n)−1 , which shows that
E0 [(τ0+ )1/2 ] = ∞. It is clear that the idea of this example can be generalized
to martingales satisfying uniformly bounded increments, say; we return to
this general theme in Section 2.7.
4
P0 [τ0+ ≥ x2 ] ≥
It is important to observe that the hitting probability calculation of
Example 2.4.1 is “robust”, in the sense that it gives non-trivial and often
useful bounds on hitting probabilities with only partial information. The
next example takes up this theme.
Example 2.4.4. Suppose that Xn is a submartingale with X0 = x and
|Xn+1 − Xn | ≤ K a.s. for all n. Recall also that λa = min{n : Xn ≤ a} and
ρb = min{n : Xn ≥ b}. Suppose a < x < b. Suppose that P[λa ∧ρb < ∞] = 1
(this will usually be an easy consequence of some form of non-degeneracy or
irreducibility). Then, analogously to Example 2.4.1,
x = E X0 ≤ E Xλa ∧ρb ≤ a P[λa < ρb ] + (b + K) P[ρb < λa ],
44
Chapter 2. Semimartingale approach and Markov chains
so
b+K −x
.
b+K −a
One would need, however, a supermartingale in order to obtain a lower
bound for P[λa < ρb ]: see Corollary 2.4.6 below for a closely related result.
4
P[λa < ρb ] ≤
i
For estimating hitting probabilities with a suitable Lyapunov function to hand, the optional stopping theorem is
one’s friend.
At this point, we present a simple maximal inequality for supermartingales, which is in some sense more elementary than Doob’s inequality (Theorem 2.3.14) and which will be useful at several points later on.
Theorem 2.4.5. Suppose that Xn is a non-negative supermartingale adapted to Fn . Then, for any x > 0,
i X
h
0
P max Xm ≥ x F0 ≤
, a.s.
m≥0
x
In particular, P[maxm≥0 Xm ≥ x] ≤ x−1 E X0 .
Proof. Fix x > 0. Theorem 2.3.11 applied to the non-negative supermartingale Xn at stopping times 0 and σx = min{n ≥ 0 : Xn ≥ x} yields
X0 ≥ E[Xσx | F0 ] ≥ E[Xσx 1{σx < ∞} | F0 ] ≥ x P[σx < ∞ | F0 ].
But σx < ∞ if and only if maxm≥0 Xm ≥ x.
Theorem 2.4.5 has a useful corollary on the same theme as the preceding
examples, but which imposes no assumptions on the size of the increments
of the process.
Corollary 2.4.6. Suppose that Xn is an Fn adapted process taking values
in R+ . Suppose that there exists y ∈ R+ for which, a.s.,
E[Xn+1 − Xn | Fn ] ≤ 0, on {Xn > y}.
Then, with λy = min{n ≥ 0 : Xn ≤ y}, for any x > 0,
h
i EX
0
P max Xm∧λy ≥ x ≤
.
m≥0
x
2.4. Displacement and exit estimates
45
Proof. Apply Theorem 2.4.5 to the supermartingale Xn∧λy ∈ R+ .
The next result includes another important maximal inequality, which
shows how an upper bound on the expected increments of a non-negative
process gives a probabilistic upper bound on its finite-time maximum. The
inequality (2.14) is closely related to the submartingale inequality of Doob
(Theorem 2.3.14) and also to the supermartingale inequality Theorem 2.4.5
(take B = 0).
Theorem 2.4.7. Let Xn be an integrable Fn -adapted process on R+ . Suppose that for some B ∈ R+ ,
E[Xn+1 − Xn | Fn ] ≤ B, a.s.
Then, for any stopping time ν with E ν < ∞, and any x > 0, a.s.,
i B E[ν | F ] + X
h
0
0
P max Xm ≥ x F0 ≤
;
0≤m≤ν
x
x − X0
and E[σx | F0 ] ≥
.
B
In particular, for any n ≥ 0 and any x > 0,
h
i Bn + E X
0
.
P max Xm ≥ x ≤
0≤m≤n
x
(2.12)
(2.13)
(2.14)
We deduce Theorem 2.4.7 from the following important result.
Theorem 2.4.8. Let Xn be an integrable Fn -adapted process on R, and let
τ be a stopping time. Suppose that for some B ∈ R,
E[Xn+1 − Xn | Fn ] ≤ B, on {n < τ }.
(2.15)
Then, for any n ≥ 0, a.s.,
E[Xn∧τ | F0 ] ≤ X0 + B E[n ∧ τ | F0 ].
(2.16)
Proof. We may rewrite the condition (2.15) as
E[X(m+1)∧τ − Xm∧τ | Fm ] ≤ B1{τ > m}, for all m ≥ 0.
(2.17)
Indeed, (2.17) is clearly equivalent to (2.15) on {τ > m}, since then Xm∧τ =
Xm and X(m+1)∧τ = Xm+1 , while on {τ ≤ m} the statement (2.17) is trivial.
Conditioning on F0 and taking expectations in (2.17), we obtain
E[X(m+1)∧τ | F0 ] − E[Xm∧τ | F0 ] ≤ B P[τ > m | F0 ].
46
Chapter 2. Semimartingale approach and Markov chains
Then summing from m = 0 to m = n − 1 we obtain
E[Xn∧τ | F0 ] − X0 ≤ B E
which gives (2.16) since
Pn−1
m=0 1{τ
n−1
X
1{τ > m},
m=0
> m} = n ∧ τ .
Remark 2.4.9. A variation of Dynkin’s formula (see Example 2.3.13) gives
E[Xn∧τ
(n∧τ
)−1
X
| F0 ] = X0 + E
E[Xm+1 − Xm | Fm ] F0 .
m=0
Theorem 2.4.8 can be viewed as a consequence of this; the elementary proof
presented above is still instructive, and demonstrates how weakly optional
stopping is required.
Now we complete the proof of Theorem 2.4.7.
Proof of Theorem 2.4.7. Take τ = ν ∧ σx ; then it follows from (2.16) that
X0 + B E[n ∧ ν ∧ σx | F0 ] ≥ E[Xn∧ν∧σx | F0 ] ≥ x P[σx ≤ n ∧ ν | F0 ], (2.18)
since Xn∧ν∧σx ≥ x1{σx ≤ n ∧ ν}, given that Xn ≥ 0 for all n. In particular,
since ν ≥ n ∧ ν ∧ σx ,
X0 + B E[ν | F0 ] ≥ x P[σx ≤ n ∧ ν | F0 ],
which gives the maximal inequality (2.12) on taking n → ∞. The simpler
version (2.14) follows from (2.12) applied at the deterministic stopping time
ν = n.
To prove (2.13) it suffices to suppose that P[σx < ∞] = 1. An application
of (2.18) with ν = n gives
X0 + B E[σx | F0 ] ≥ x P[σx ≤ n | F0 ],
which on letting n → ∞ yields (2.13).
i
A Lyapunov function whose drift is uniformly bounded
above gives control over the maximum of a process.
Theorems 2.4.7 and 2.4.8 will form basic tools later on; for now we make
an aside to give a corollary pertaining to the so-called finite time stochastic
stability of Markov chains.
47
2.4. Displacement and exit estimates
Corollary 2.4.10 (Kushner–Kalashnikov inequality). Let Xn be a timehomogeneous Markov chain on state-space (Σ, E). Suppose that for measurable f : Σ → R+ , A ∈ E, and B ∈ R+ ,
E[f (Xn+1 ) − f (Xn ) | Xn = x] ≤ B, whenever x ∈ A.
Then, if σA = min{n ≥ 0 : Xn ∈
/ A}, for n ∈ Z+ and x ∈ A,
P [σA ≤ n | X0 = x] ≤
Bn + f (x)
.
inf y∈A
/ f (y)
Proof. Let Fn = σ(X0 , . . . , Xn ). Apply (2.12) to the process f (Xn ) at the
stopping time ν = n ∧ σA to get
i
h
P[σA ≤ n | F0 ] ≤ P
max f (Xm ) ≥ inf f (y) F0
0≤m≤n∧σA
≤
y ∈A
/
B E[n ∧ σA | F0 ] + f (X0 )
,
inf y∈A
/ f (y)
which gives the result.
Here is a result in the opposite direction, giving an upper bound on the
expected time to exit an interval. Again for x > 0 we use the notation
σx = min{n ≥ 0 : |Xn | ≥ x}.
Theorem 2.4.11. Let Xn be an Fn -adapted process on R+ and let η be a
stopping time. Fix x ≥ 0. Suppose that there exist a non-decreasing function
f : R+ → R+ and a constant ε > 0 for which f (Xn ) is integrable and
E[f (Xn+1 ) − f (Xn ) | Fn ] ≥ ε, on {n < η ∧ σx }.
(2.19)
E[η ∧ σx ] ≤ ε−1 E f (Xσx ).
(2.20)
Then
Proof. Let x ≥ 0, and write Yn = f (Xn∧η∧σx ). We may express (2.19) as
E[Ym+1 − Ym | Fm ] ≥ ε1{η ∧ σx > m},
for any m ≥ 0. We obtain E Ym+1 − E Ym ≥ ε P[η ∧ σx > m] on taking
expectations, and then summing for m from 0 up to n we get
E Yn+1 − E Y0 ≥ ε
n
X
m=0
P[η ∧ σx > m].
48
Chapter 2. Semimartingale approach and Markov chains
Since Y0 = f (X0 ) ≥ 0, it follows that
E[η ∧ σx ] = lim
n→∞
n
X
m=0
P[η ∧ σx > m] ≤ ε−1 lim sup E Yn+1 .
n→∞
Since f is non-decreasing, Yn = f (Xn∧η∧σx ) ≤ f (Xσx ), a.s., for all n ∈ Z+ .
The result now follows.
i
A Lyapunov function whose drift is uniformly bounded below quantifies that a process is unlikely to stay in a bounded
region for too long.
Some refinements of Theorem 2.4.11 are given in Section 3.10 below.
Although the proof of Theorem 2.4.11 is elementary, it is quite powerful.
For example, an easy consequence is the following result which shows that
a martingale with sufficient variability will exit an interval at least ‘diffusively’. In the particular case where η = ∞, Theorem 2.4.12 reduces to a
semimartingale version of Kolmogorov’s ‘other’ inequality.
Theorem 2.4.12. Suppose that Xn is a square-integrable Fn -adapted process on R with X0 = 0. Suppose that there exist a stopping time η and
constants v > 0, B ∈ R+ such that, for all n ≥ 0,
E[(Xn+1 − Xn )2 | Fn ] ≥ v1{n < η},
P[|Xn+1 − Xn | ≤ B | Fn ] = 1.
(2.21)
(2.22)
Suppose also that either (i) Xn is a martingale; or (ii) Xn is a non-negative
submartingale. Then for any n ≥ 0 and any x > 0,
h
i
(x + B)2
P max |Xm | ≥ x ≥ 1 −
− P[η ≤ n].
0≤m≤n
vn
The lower bound on second moments in (2.21) is a form of uniform
non-degeneracy. The proof of Theorem 2.4.12 uses the following elementary
algebraic identity:
a2 − b2 = (a − b)2 + 2b(a − b),
(a, b ∈ R).
(2.23)
Although trivial, (2.23) will be used repeatedly in this book, so we refer to
it as the square-difference identity.
2.4. Displacement and exit estimates
49
Proof of Theorem 2.4.12. By (2.21), (2.23), and the (sub)martingale assumption, we have that
2
E Xn+1
− Xn2 Fn = E (Xn+1 − Xn )2 Fn + 2Xn E Xn+1 − Xn Fn ≥ v,
on {n ≤ η}, while |Xσx | ≤ x + B, a.s., by (2.22). So we may apply Theorem 2.4.11 to the process Xn2 ∈ R+ with f (x) = x, to obtain E[η ∧ σx ] ≤
v −1 (x + B)2 . Then, P[σx ≤ n] ≥ P[η ∧ σx ≤ n] − P[η ≤ n], which, with
Markov’s inequality, gives the desired result.
Some generalizations of Theorem 2.4.12 are given in Chapter 4.
Example 2.4.13. Let Xn be an Fn -adapted process on R+ . Suppose that
there exists an increasing f : R+ → R+ such that E f (X0 ) < ∞ and
E[f (Xn+1 ) − f (Xn ) | Fn ] = B ∈ (0, ∞), a.s.
Since f is increasing, σx = min{n ≥ 0 : f (Xn ) ≥ f (x)}. Then applications
of (2.13) and Theorem 2.4.11 to the process f (Xn ) show that
f (x) − E f (X0 )
E f (Xσx )
≤ E σx ≤
.
B
B
Assuming that f (x) → ∞ as x → ∞, and
lim
x→∞
E f (Xσx )
= 1,
f (x)
it follows that
lim
x→∞
(2.24)
E σx
1
= ,
f (x)
B
which is sometimes known as a weak renewal theorem. Sufficient for (2.24)
is that Xn+1 − Xn is uniformly bounded above and f does not grow too fast;
see also Section 3.10. Lemma 2.2.4(i) provides an example.
4
Another very useful result for bounding probabilities of displacement,
typically in the situation when the size of displacement is much bigger than
one would normally expect, is the Azuma–Hoeffding inequality. Here we
state the traditional inequality alongside its one-sided versions; since the
latter are not easily found in the literature, we give the proof (which is
essentially the standard one).
50
Chapter 2. Semimartingale approach and Markov chains
Theorem 2.4.14 (Azuma–Hoeffding inequalities). Suppose that (ck , k ≥ 1)
are finite positive constants. Suppose that Xn is a supermartingale, such
that |Xk − Xk−1 | ≤ ck a.s. for all k ≥ 1. Then for all a > 0 and all n ≥ 0,
a2
.
P[Xn − X0 ≥ a | F0 ] ≤ exp − Pn
2 k=1 c2k
(2.25)
If Xn is a submartingale with |Xk − Xk−1 | ≤ ck , then
a2
P[Xn − X0 ≤ −a | F0 ] ≤ exp − Pn
.
2 k=1 c2k
(2.26)
Finally, if Xn is a martingale with |Xk − Xk−1 | ≤ ck , then
a2
P |Xn − X0 | ≥ a F0 ≤ 2 exp − Pn
.
2 k=1 c2k
(2.27)
Proof. We prove only (2.25); then, (2.26) follows by considering (−Xn ),
and (2.27) by the union bound.
For any fixed λ, g(x) = eλx is a convex function. Let c > 0. Since the
line segment with endpoints (−c, e−λc ) and (c, eλc ) lies above the graph of g,
we have that for all x ∈ [−c, c],
eλx ≤
eλc − e−λc
eλc + e−λc
x+
.
2c
2
So, since Xn is a supermartingale, we obtain
h eλck − e−λck
λ(X −X ) eλck + e−λck
k
k−1 Fk−1 ≤ E
(Xk − Xk−1 ) +
E e
2ck
2
λc
−λc
e k +e k
≤
2
λ2 c2k /2
≤e
,
where we used the elementary inequality
∞
∞
k=0
k=0
X y 2k
X y 2k
ey + e−y
2
=
≤
= ey /2 .
k
2
(2k)!
2 k!
With (2.28), we obtain by induction
n
1 X
λ(Xn −X0 ) 2
2
E e
F0 ≤ exp λ
ck .
2
k=1
i
Fk−1
(2.28)
51
2.4. Displacement and exit estimates
Now the (conditional) Markov inequality gives, for a > 0 and λ > 0,
P[Xn − X0 ≥ a | F0 ] ≤ e−λa E eλ(Xn −X0 ) F0
n
1 X 2
≤ exp −λa + λ2
ck ,
2
k=1
Pn
2 −1 we conclude the proof.
and taking λ = a
k=1 ck
We finish this section with an example of particularly wide-ranging excursions.
Example 2.4.15. Let Sn be symmetric SRW on Z2 , and consider f (x) =
(log(1 + kxk2 ))γ for γ ∈ R. Let U2 := {±e1 , ±e2 } denote the possible jumps
of the walk. Then for e ∈ U2 , we have
γ


2e·x+1


log 1 + 1+kxk
2
 −1 .
f (x + e) − f (x) = f (x) 1 +


log(1 + kxk2 )
Here, we have from Taylor’s formula that
2e · x + 1 2(e · x)2
2e · x + 1
=
−
+ O(kxk−3 ),
log 1 +
1 + kxk2
kxk2
kxk4
so that, with the Taylor formula for (1 + x)γ , we obtain
f (x + e) − f (x) = γkxk−2 (log(1 + kxk2 ))γ−1
2(e · x)2
2(γ − 1)(e · x)2
−1
× 2e · x + 1 −
+
+ O(kxk ) .
kxk2
kxk2 log(1 + kxk2 )
Using the fact that
X
1 = 4,
e∈U2
X
e∈U2
e · x = 0, and
X
e∈U2
(e · x)2 = 2kxk2 ,
it follows that, as kxk → ∞,
E[f (Sn+1 ) − f (Sn ) | Sn = x] =
= γ(γ − 1)kxk
−2
1 X
f (x + e) − f (x)
4
e∈U2
(log(1 + kxk2 ))γ−1 (1 + o(1)),
which, for γ > 1, is positive for all kxk sufficiently large.
(2.29)
52
Chapter 2. Semimartingale approach and Markov chains
Suppose S0 6= 0. Then, if τ = min{n ≥ 0 : Sn = 0} and σx = min{n ≥
0 : kSn k ≥ x}, we have that Yn = f (Sn∧τ ∧σx ) is a non-negative submartingale, bounded above by (log(2 + x2 ))γ since kSn∧σx k2 ≤ 1 + x2 . Thus Yn is
uniformly integrable, and by optional stopping (Theorem 2.3.7),
log 2 ≤ f (S0 ) = Y0 ≤ E Yτ ∧σx ≤ P[σx < τ ](log(2 + x2 ))γ ,
since Yτ = f (0) = 0. In other words, for any ε > 0, there exists c > 0 such
that, for all x ≥ 2,
h
i
P max kSm k ≥ x ≥ c(log x)−1−ε .
4
0≤m≤τ
2.5
Recurrence and transience criteria for Markov
chains
We start with a technical result that will allow us to convert results on
hitting times of sets (such as we obtain by Lyapunov function arguments)
to results on return times to points, and hence statements about recurrence
and transience as defined at Definition 2.1.4.
Lemma 2.5.1. Let Xn be an irreducible Markov chain on a countable state
space Σ.
(i) If for some x ∈ Σ and some non-empty A ⊆ Σ, Px [τA < ∞] < 1, then
Xn is transient.
(ii) If for some finite non-empty A ⊆ Σ and all x ∈ Σ\A, Px [τA < ∞] = 1,
then Xn is recurrent.
Proof. We prove part (i). If A is non-empty and Px [τA = ∞] > 0, then
there is some y ∈ A, y 6= x (since Px [τx = ∞] = 0). But τy ≥ τA , so
Px [τy = ∞] > 0. By irreducibility, for any x 6= y, Py [τx < τy+ ] > 0; otherwise
the strong Markov property would show that Py [τx < ∞] = 0, contradicting
irreducibility. So, by the strong Markov property,
Py [τy+ = ∞] ≥ Py [τx < τy+ ] Px [τy = ∞] > 0,
which shows transience by Definition 2.1.4.
Part (ii) is demonstrated in Example 2.3.20.
The two basic criteria for recurrence and transience of Markov chains
that we present in this section have at their foundation the supermartingale
convergence result Theorem 2.3.6. We first present the recurrence criterion.
2.5. Recurrence/transience criteria
53
Theorem 2.5.2 (Recurrence criterion). An irreducible Markov chain Xn
on a countably infinite state space Σ is recurrent if and only if there exist a
function f : Σ → R+ and a finite non-empty set A ⊂ Σ such that
E[f (Xn+1 ) − f (Xn ) | Xn = x] ≤ 0, for all x ∈ Σ \ A,
(2.30)
and f (x) → ∞ as x → ∞.
We emphasize an elementary but important point about the notation
here. The key to correct interpretation of the statement ‘f (x) → ∞ as
x → ∞’ is the fact that the domain of f is precisely the countable set Σ,
so while the first ‘∞’ in the statement is sup R+ , the second is #Σ. Thus
the statement ‘f (x) → ∞ as x → ∞’ entails that for any enumeration
x1 , x2 , . . . of the elements of Σ, limn→∞ f (xn ) = ∞. Indeed, given such
an enumeration, limn→∞ inf m≥n f (xm ) ≥ M for any M ∈ R+ , so the set
{x ∈ Σ : f (x) ≤ M } is finite for any M ∈ R+ . Hence the choice of
enumeration of Σ did not matter.
If Σ ⊂ Rd is unbounded and locally finite, then any enumeration x1 , x2 , . . .
of Σ has limn→∞ kxn k = ∞; thus we may restate an ‘embedded’ version of
Theorem 2.5.2 as follows.
Corollary 2.5.3. An irreducible Markov chain Xn on an unbounded and
locally finite state space Σ ⊂ Rd is recurrent if and only if there exist a
function f : Rd → R+ and a bounded non-empty set A ⊂ Σ such that (2.30)
holds and f (x) → ∞ as kxk → ∞.
Example 2.5.4. As in Example 2.4.15, let Sn be symmetric SRW on Z2 ,
and consider f (x) = (log(1 + kxk2 ))γ for γ ∈ (0, 1). Then, by (2.29),
E[f (Sn+1 ) − f (Sn ) | Sn = x] = γ(γ − 1)kxk−2 (log(1 + kxk2 ))γ−1 (1 + o(1)),
which is negative for all kxk sufficiently large. Thus Corollary 2.5.3 shows
that SRW on Z2 is recurrent; this is a direct Lyapunov-function proof of the
d = 2 case of Pólya’s theorem (Theorem 1.2.1).
4
Proof of Theorem 2.5.2. To prove that having a function that satisfies (2.30)
is sufficient for the recurrence, take X0 = x ∈ Σ. Set Yn = f (Xn∧τA ) and
observe that Yn is a non-negative supermartingale. Then, by Theorem 2.3.6,
there exists a random variable Y∞ such that Yn → Y∞ a.s. and
Ex Y∞ ≤ Ex Y0 = f (x),
(2.31)
for any x ∈ Σ. On the other hand, since f → ∞, it holds that the set
{y ∈ Σ : f (y) ≤ M } is finite for any M ∈ R+ ; so, the irreducibility implies
54
Chapter 2. Semimartingale approach and Markov chains
that lim supn→∞ f (Xn ) = +∞ a.s. on {τA = ∞}. Hence, on {τA = ∞}, we
must have Y∞ = limn→∞ Yn = +∞, a.s. This would contradict (2.31) if we
assume that Px [τA = ∞] > 0, because then Ex [Y∞ ] ≥ Ex [Y∞ 1{τA = ∞}] =
∞. Hence Px [τA = ∞] = 0 for all x ∈ Σ, which means that the Markov
chain is recurrent, by Lemma 2.5.1(ii).
For the ‘only if’ part, see the proof of Theorem 2.2.1 of [96].
Example 2.5.5. Consider the nearest-neighbour random walk on Z+ from
Section 2.2, and take f (x) = h(x) as defined at (2.7). Then Lemma 2.2.2
shows that
E[f (Xn+1 ) − f (Xn ) | Xn = x] = 0, for all x 6= 0,
so Theorem 2.5.2 shows that Xn is recurrent if limx→∞ h(x) = ∞. This is a
Lyapunov function proof of the ‘if’ half of Theorem 2.2.5(i).
4
Example 2.5.6. Let Xn be an irreducible Markov chain on a locally finite
subset of R+ . Suppose that for some bounded set A,
E[Xn+1 − Xn | Xn = x] = 0, for all x ∈
/ A,
(2.32)
i.e., the process has zero drift outside A. Corollary 2.5.3 (with f (x) = x)
shows that Xn is recurrent. However, without additional conditions the
same result is not true if the state-space is allowed to be a subset of R
rather than just R+ : see the notes at the end of this chapter. This contrasts
with the situation when Xn is a spatially homogeneous random walk on R,
i.e., a sum of i.i.d. random variables, in which case zero drift does imply
recurrence; see e.g. [150, Ch. 9].
4
To be able to conclude that zero drift implies recurrence for a Markov
chain on R, we need to impose additional moments assumptions on the
increments, including a uniform non-degeneracy condition.
Theorem 2.5.7. Let Xn be an irreducible Markov chain on a locally finite
subset of R. Suppose that for some bounded set A, (2.32) holds. Suppose
that for some p > 2 and v > 0,
sup E[(Xn+1 − Xn )p | Xn = x] < ∞;
x
inf E[(Xn+1 − Xn )2 | Xn = x] ≥ v.
x
Then Xn is recurrent.
55
2.5. Recurrence/transience criteria
Proof. We will apply Corollary 2.5.3 with f (x) = log(1 + |x|). Write ∆ =
X1 − X0 and Ex = {|∆| < |x|}. We compute
E[f (Xn+1 ) − f (Xn ) | Xn = x] = Ex [(f (x + ∆) − f (x))1(Ex )]
+ Ex [(f (x + ∆) − f (x))1(Exc )].
On {|∆| < |x|} we have that x and x + ∆ have the same sign, so
Ex [(f (x + ∆) − f (x))1(Ex )]
1 + |x + ∆|
= Ex log
1(Ex )
1 + |x|
∆ sgn(x)
= Ex log 1 +
1(Ex )
1 + |x|
sgn(x)
1
≤
Ex [∆1(Ex )] − (1 + |x|)−2 Ex [∆2 1(Ex )],
1 + |x|
6
using the inequality log(1 + y) ≤ y − 16 y 2 for all −1 < y ≤ 1. Here, since
Ex ∆ = 0 for x ∈
/ A, as |x| → ∞,
| Ex [∆1(Ex )]| ≤ Ex [|∆|1(Exc )] ≤ Ex [|∆|p |x|1−p ] = o(|x|−1 ).
Similarly,
Ex [∆2 1(Ex )] ≥ v − Ex [∆2 1(Exc )] ≥ v − o(1), as |x| → ∞.
Finally we estimate the term
|Ex [(f (x + ∆) − f (x))1(Exc )]| ≤ Ex [(log(1 + |∆|) + log(1 + 2|∆|)) 1(Exc )] .
Here,
log(1 + 2|∆|)1(Exc ) = log(1 + 2|∆|)|∆|p |∆|−p 1(Exc )
≤ |x|−p log(1 + 2|x|)|∆|p ,
for all x with |x| greater than some x0 sufficiently large, using the fact that
y 7→ y −p log(1 + 2y) is eventually decreasing. It follows that
|Ex [(f (x + ∆) − f (x))1(Exc )]| ≤ 2|x|−p log(1 + 2|x|) Ex [|∆|p ] = o(|x|−2 ).
Combining these calculations we obtain
v
E[f (Xn+1 ) − f (Xn ) | Xn = x] ≤ − (1 + |x|)−2 + o(|x|−2 ),
6
which is negative for all x with |x| sufficiently large. Thus we may apply
Corollary 2.5.3 to deduce recurrence.
56
Chapter 2. Semimartingale approach and Markov chains
The idea of Theorem 2.5.7, and the interplay between first and second
moments of increments for determining recurrence, will be picked up again
in detail in Chapter 3. When second moments fail to exist, heavy tailed
phenomena come into play; some of these are explored in Chapter 5. In
higher dimensions, the analogue of Theorem 2.5.7 fails, and either recurrence
or transience is possible: see Chapter 4
Next we formulate a criterion for transience.
Theorem 2.5.8 (Transience criterion). An irreducible Markov chain Xn on
a countable state space Σ is transient if and only if there exist a function f :
Σ → R+ and a non-empty set A ⊂ Σ such that
E[f (Xn+1 ) − f (Xn ) | Xn = x] ≤ 0, for all x ∈ Σ \ A,
(2.33)
f (y) < inf f (x), for at least one site y ∈ Σ \ A.
(2.34)
and
x∈A
Note that, unlike the recurrence criterion, here the set A need not be
finite. The following simple example is important.
Example 2.5.9. Let Sn be symmetric SRW on Zd . Write e1 , . . . , ed for the
standard orthonormal basis vectors and Ud = {±e1 , . . . , ±ed } for the set of
possible jumps for the random walk. Let α > 0. Consider the Lyapunov
function f : Zd → (0, 1] defined by f (0) = 1 and f (x) = kxk−2α for x 6= 0.
Then, for any x ∈ Zd with kxk > 1, for any e ∈ Ud ,
"
#
−α
kx + ek2
−2α
f (x + e) − f (x) = kxk
−1
kxk2
"
#
2e · x + 1 −α
−2α
= kxk
1+
−1
kxk2
(2e · x + 1)
−2α
2
−3
= kxk
−α
+ 2α(α + 1)(e · x) + O(kxk ) ,
kxk2
P
P
by Taylor’s formula. Using the facts that e∈Ud 1 = 2d, e∈Ud e · x = 0,
P
and e∈Ud (e · x)2 = 2kxk2 , we obtain
E[f (Sn+1 ) − f (Sn ) | Sn = x] =
1 X
(f (x + e) − f (x))
2d
e∈Ud
α
= kxk−2−2α 2(α + 1) − d + O(kxk−1 ) ,
d
57
2.5. Recurrence/transience criteria
which is negative for all x with kxk sufficiently large provided we choose
α ∈ (0, d−2
2 ), which we may do for any d ≥ 3. Thus Theorem 2.5.8 shows
that SRW on Zd is transient for d ≥ 3; this is a direct Lyapunov-function
proof of the ‘transience’ part of Pólya’s theorem (Theorem 1.2.1).
4
The central semimartingale idea of the proof of Theorem 2.5.8 is very
simple, and will be useful in other contexts; thus we extract it as the following lemma.
Lemma 2.5.10. Let Xn be an Fn -adapted process with state space (Σ, E).
Suppose that f : Σ → R+ is measurable, and A ∈ E is such that
E[f (Xn+1 ) − f (Xn ) | Fn ] ≤ 0, on {Xn ∈ Σ \ A},
(2.35)
for all n ≥ 0. Then, a.s.,
P[τA < ∞ | F0 ] ≤
f (X0 )
.
inf x∈A f (x)
Proof. Define the process Yn = f (Xn∧τA ). Then (2.35) implies that Yn is a
supermartingale adapted to Fn . Since Yn is also non-negative, Theorem 2.3.6
implies that there is a random variable Y∞ ∈ R+ such that limn→∞ Yn = Y∞
a.s. Also, by Theorem 2.3.11,
Y0 ≥ E[Y∞ | F0 ] ≥ E[Y∞ 1{τA < ∞} | F0 ].
Here we have that, a.s.,
Y∞ 1{τA < ∞} = lim Yn 1{τA < ∞} = f (XτA )1{τA < ∞}
n→∞
≥ inf f (x)1{τA < ∞}.
x∈A
So we obtain
f (X0 ) = Y0 ≥ P[τA < ∞ | F0 ] inf f (x),
x∈A
which yields the result.
i
A Lyapunov function that is a supermartingale outside a set
gives an upper bound on the probability the the processes
started outside that set ever reaches its destination.
58
Chapter 2. Semimartingale approach and Markov chains
Proof of Theorem 2.5.8. For the ‘if’ part, we apply Lemma 2.5.10. In this
case, with y ∈ Σ \ A as in (2.34), we obtain
Py [τA < ∞] ≤
f (y)
< 1,
inf x∈A f (x)
proving the transience of the Markov chain Xn , by Lemma 2.5.1(i).
For the ‘only if’ part, fix an arbitrary x0 ∈ Σ, set A = {x0 }, and f (x) =
Px [τx0 < ∞] for x ∈ Σ (so, in particular, f (x0 ) = 1). Then (2.33) holds with
equality for all x 6= x0 , and, by transience, one can find y ∈ Σ such that
f (y) < 1 = f (x0 ).
An inspection of the ‘only if’ part of the preceding proof shows that, if
a Markov chain is transient, a function f will be an apt Lyapunov function
if the value f (x) is given by the probability that the process reaches some
fixed set starting from x. This suggests the following heuristic rule:
i
To find a Lyapunov function that proves transience, fix a
well-chosen A ⊂ Σ, and set f (x) to be your reasonable guess
for the probability to ever reach A starting from x.
Example 2.5.11. Consider the one-dimensional SRW Sn , as in Example 2.1.7.
• To prove recurrence for p = 21 , take A = {0} and f (x) = |x|; then,
(2.30) holds with equality for all x 6= 0.
• To prove transience of the SRW for p > 21 , take A = (−∞, 0] and
x
; then f (x) → 0 as x → +∞ and (2.33) holds with
f (x) = 1−p
p
equality. For p < 21 , take A = [0, +∞) and the same f . Or just take
A = {0} in both cases, and choose any y > 0 in the first case and any
y < 0 in the second one to have (2.34).
4
Theorem 2.5.8 has the following consequence, which is sometimes useful.
Corollary 2.5.12. An irreducible Markov chain Xn on a countable state
space Σ is transient if and only if there exist a non-constant function h :
Σ → R and a state x0 ∈ Σ such that supx∈Σ |h(x)| < ∞ and
E[h(Xn+1 ) − h(Xn ) | Xn = x] = 0, for all x 6= x0 .
(2.36)
Proof. For the ‘if’ part of the proof, by swapping h with −h, it suffices
to consider the non-constant condition in the form h(x) < h(x0 ) for some
59
2.5. Recurrence/transience criteria
x 6= x0 . Then taking f (x) = h(x) − inf x∈Σ h(x) we have f : Σ → R+ satisfies
the conditions of Theorem 2.5.8 with A = {x0 }, and transience follows.
For the ‘only if’ statement, the construction in the proof of Theorem 2.5.8
serves again: for fixed x0 ∈ Σ, set A = {x0 }, and h(x) = Px [τx0 < ∞] for
x ∈ Σ. Then (2.36) holds for all x 6= x0 , and, by transience, one can find
y ∈ Σ such that h(y) < 1 = h(x0 ).
Example 2.5.13. Consider the nearest-neighbour random walk on Z+ from
Section 2.2. With h as defined at (2.7), we see that if supx h(x) < ∞, then
Lemma 2.2.2 shows that the conditions of Corollary 2.5.12 are satisfied, so
that Xn is transient. This is a Lyapunov function proof of the ‘onlf if’ half
of Theorem 2.2.5(i).
4
Next, we formulate a sufficient condition for transience, first for general
processes, and then for Markov chains.
Theorem 2.5.14. For a real-valued Fn -adapted process Xn , define τa =
min{n ≥ 1 : Xn − X0 ≤ −a}, a ≥ 0, and suppose that, for some ε > 0,
E[Xn+1 − Xn | Fn ] ≥ ε, on {n < τa },
(2.37)
for all n ≥ 0. Suppose also there exists K > 0 such that, for all n ≥ 0,
|Xn+1 − Xn | ≤ K, on {n < τa }.
(2.38)
Then there exist positive constants c1 , c2 depending only on ε, K, such that
P[τa < ∞ | F0 ] ≤ c1 e−c2 a , for all a ≥ 0.
(2.39)
Also, we have
P[τ0 = ∞ | F0 ] = P[Xn > X0 for all n ≥ 1 | F0 ] ≥ c3 > 0,
(2.40)
for some constant c3 depending only on ε and K.
Proof. Define the process Yn = Xn − εn; (2.37) implies that Yn∧τa is a
submartingale. By (2.37), it holds a.s. that |Y(n+1)∧τa − Yn∧τa | ≤ K + ε,
so, using the one-sided Azuma–Hoeffding inequality (2.26) applied to the
submartingale Yn∧τa , together with the union bound, we obtain
P[τa < ∞ | F0 ] = P[there exists n ≥ 1 such that Yn∧τa − Y0 ≤ −a − εn | F0 ]
∞
(a + εn)2 X
≤
exp −
2n(ε + K)2
n=1
60
Chapter 2. Semimartingale approach and Markov chains
≤
∞
X
n=1
2aεn + (εn)2 exp −
2n(ε + K)2
= exp −
∞
X
ε
ε2
a
exp −
n ,
(ε + K)2
2(ε + K)2
n=1
P
ε2
ε
proving (2.39) with c1 = ∞
n=1 exp − 2(ε+K)2 n and c2 = (ε+K)2 .
To prove (2.40), observe first that, by (2.38), on {n < τa },
ε
Xn+1 − Xn ≤ 1{Xn+1 − Xn ∈ (0, ε/2)} + K1{Xn+1 − Xn ≥ ε/2},
2
so, taking expectation conditional on Fn and using (2.37), we obtain
ε1{τa > n} ≤
ε
+ K P[Xn+1 − Xn ≥ ε/2 | Fn ]
2
and thus
P[Xn+1 − Xn ≥ ε/2 | Fn ] ≥
ε
1{τa > n}.
2K
(2.41)
Now, choose a0 large enough so that c1 e−c2 a < 12 for all a ≥ a0 . Denote
a0
n0 = d ε/2
e. The idea now is that, by (2.41), the process can initially make
at least n0 steps to the right of amount at least ε/2 with uniformly positive
probability (so that its distance from the starting point is at least a0 ), and
then by (2.39) the process will not fall below its initial position with probability at least 12 . To make this argument rigorous, observe that by (2.41),
P[Xk − Xk−1 ≥ ε/2 for all k = 1, . . . , n | F0 ]
= E P[Xk − Xk−1 ≥ ε/2 for all k = 1, . . . , n | Fn−1 ] F0
= E 1{Xk − Xk−1 ≥ ε/2 for all k = 1, . . . , n − 1}
P[Xn − Xn−1 ≥ ε/2 | Fn−1 ] F0
ε
P[Xk − Xk−1 ≥ ε/2 for all k = 1, . . . , n − 1 | F0 ],
≥
2K
and so, by induction,
P[Xk − Xk−1 ≥ ε/2 for all k = 1, . . . , n | F0 ] ≥
ε n
.
2K
(2.42)
Abbreviating X̃k = Xk+n0 and τ̃a = min{k ≥ 1 : X̃k − X̃0 ≤ −a}, we have
P[τ0 = ∞ | F0 ]
= E E[1{τ0 = ∞} | Fn0 ] F0
61
2.5. Recurrence/transience criteria
≥ E E[1{τ0 = ∞, Xk − Xk−1 ≥ ε/2 for all k = 1, . . . , n0 } | Fn0 ] F0
≥ E 1{Xk − Xk−1 ≥ ε/2 for all k = 1, . . . , n0 } P[τ̃a0 = ∞ | Fn0 ] F0
1
≥ P[Xk − Xk−1 ≥ ε/2 for all k = 1, . . . , n0 | F0 ]
2
1 ε n0
,
≥
2 2K
proving (2.40).
A consequence of the above result is the following sufficient condition for
the transience of Markov chains.
Theorem 2.5.15. Let Xn be an irreducible Markov chain on a countable
state space Σ. Suppose that there exist a function f : Σ → R, some h ∈ R
for which the set A := {x ∈ Σ : f (x) ≤ h} is neither ∅ nor Σ, and constants
ε > 0 and K ∈ R+ such that
E[f (Xn+1 ) − f (Xn ) | Xn = x] ≥ ε, for all x ∈ Σ \ A; and
P[|f (Xn+1 ) − f (Xn )| ≤ K | Xn = x] = 1, for all x ∈ Σ \ A.
(2.43)
(2.44)
Then, the Markov chain is transient.
Proof. Choose any x0 ∈ Σ \ A, so that f (x0 ) > h, and take X0 = x0 . Let
a = f (x0 ) − h > 0, and consider the process Yn = f (Xn ). Then
τa = min{n ≥ 0 : Yn − Y0 ≤ −a} = min{n ≥ 0 : Xn ∈ A},
so that an application of Theorem 2.5.14 to the process Yn shows that with
positive probability Xn started from X0 = x0 never reaches A, i.e., the
process is transient.
Example 2.5.16. Again, consider the one-dimensional SRW Sn , as in
Example 2.1.7. Assume that p > 21 ; to prove the transience using Theorem 2.5.15, take f (x) = x and a = 0. Then, (2.43) holds with ε = 2p−1 > 0
and (2.44) holds with K = 1.
4
i
Heuristically, when something has drift in a fixed direction, it should go to that direction in the limit, and Theorem 2.5.15 is a convenient tool to make this intuition rigorous; however, the restriction on the size of the jumps cannot
be completely removed, as the following example shows.
62
Chapter 2. Semimartingale approach and Markov chains
Example 2.5.17. Now, consider the random walk on the half-line Ŝn of
Example 2.1.8, with, say, p = 13 (so that it is positive recurrent). It it
straightforward to check, however, that (2.43) still holds with uniformly
positive ε for f (x) = 3x and a = 1:
E[f (Xn+1 ) − f (Xn ) | Xn = x] =
2
3
· 3x−1 +
1 x+1
·3
− 3x
3
2
= 2 · 3x−2 ≥ , for all x ≥ 1.
3
4
Example 2.5.17 shows that one cannot omit the condition (2.44) from
Theorem 2.5.15. However, if one is interested only in transience, rather than
exponential estimates such as those in Theorem 2.5.14, then the uniform
bounds on the increments in Theorems 2.5.14 and 2.5.15 can be relaxed and
replaced by a moments assumption, as in the following result. The idea
of the proof is elementary, combining truncation and a Taylor’s formula
computation, but important, and previews the methods of Chapter 3.
Theorem 2.5.18. Let Xn be a stochastic process on R+ adapted to a filtration Fn . Suppose that there exist constants ε > 0, δ > 0, B < ∞, and
x0 < ∞ such that, a.s.,
E[Xn+1 − Xn | Fn ] ≥ ε, on {Xn ≥ x0 };
E[|Xn+1 − Xn |
1+δ
| Fn ] ≤ B, on {Xn ≥ x0 }.
(2.45)
(2.46)
There exist α > 0 and x1 ∈ R+ such that, with λx1 = min{n ≥ 0 : Xn ≤ x1 },
1 + X0 −α
P[λx1 < ∞ | F0 ] ≤
, a.s.
1 + x1
Proof. It suffices to take δ ∈ (0, 1). Let α > 0, to be specified later. The
idea is to show that, for suitable choice of α we can find x1 ∈ R+ for which
E[(1 + Xn+1 )−α − (1 + Xn )−α | Fn ] ≤ 0, on {Xn ≥ x1 },
(2.47)
and then use Lemma 2.5.10. To ease notation, write ∆n = Xn+1 − Xn for
the duration of this proof. Let γ ∈ (0, 1), to be specified later, and let
An = {|∆n | ≤ (1 + Xn )γ }. Then, by Taylor’s formula,
(1 + Xn+1 )−α − (1 + Xn )−α 1(An )
#
"
−α
∆
n
− 1 1(An )
= (1 + Xn )−α
1+
1 + Xn
63
2.5. Recurrence/transience criteria
= −α(1 + Xn )−1−α ∆n 1(An ) + ζn ,
where there is a constant C < ∞ (depending on γ) for which the error
term ζn satisfies
|ζn | ≤ C(1 + Xn )−α−2 |∆n |2 1(An ), a.s.
Note that, a.s.,
|E[∆n 1(An ) | Fn ] − E[∆n | Fn ]| ≤ E[|∆n |1(Acn ) | Fn ]
≤ (1 + Xn )−γδ E[|∆n |1+δ | Fn ]
≤ B(1 + Xn )−γδ ,
on {Xn ≥ x0 }, by (2.46). Moreover, a.s.,
E[|ζn | | Fn ] ≤ C(1 + Xn )−α−2 E[|∆n |2 1(An ) | Fn ]
≤ C(1 + Xn )−2−α+γ(1−δ) E[|∆n |1+δ | Fn ]
≤ BC(1 + Xn )−2−α+γ(1−δ) ,
on {Xn ≥ x0 }. So we obtain, for some constant C 0 < ∞, on {Xn ≥ x0 },
E (1 + Xn+1 )−α − (1 + Xn )−α 1(An ) | Fn
≤ −αε(1 + Xn )−1−α + C 0 (1 + Xn )−1−α−γδ + C 0 (1 + Xn )−2−α+γ(1−δ) .
The first term on the right-hand side of the last display dominates as Xn →
∞ provided γ(1 − δ) < 1, which is indeed the case. Hence we can find x2
with x0 ≤ x2 < ∞ for which
E (1 + Xn+1 )−α − (1 + Xn )−α 1(An ) | Fn ≤ −(αε/2)(1 + Xn )−1−α ,
on {Xn ≥ x2 }. On the other hand,
E (1 + Xn+1 )−α − (1 + Xn )−α 1(Acn ) | Fn
≤ P[|∆n |1+δ ≥ (1 + Xn )γ(1+δ) | Fn ]
≤ B(1 + Xn )−γ(1+δ) ,
on {Xn ≥ x0 }, by Markov’s inequality and (2.46). The result (2.47) now
follows provided γ(1 + δ) > 1 + α, i.e., γ ≥ 1+α
1+δ , a choice of γ ∈ (0, 1) that
we can achieve provided we take α ∈ (0, δ).
Theorem 2.5.18 has the following consequence for Markov chains.
64
Chapter 2. Semimartingale approach and Markov chains
Theorem 2.5.19. Assume that, for an irreducible Markov chain Xn on a
countable state space Σ, one can find a function f : Σ → R+ and a ∈ (0, ∞),
such that the set A := {x ∈ Σ : f (x) < a} =
6 ∅ is a proper subset of Σ, and
for some ε > 0, δ > 0, and B ∈ R+ , (2.43) holds, and
E[|f (Xn+1 ) − f (Xn )|1+δ | Xn = x] ≤ B, for all x ∈ Σ \ A.
(2.48)
Then, the Markov chain is transient.
Proof. The idea is to set Yn = f (Xn ), Fn = σ(X0 , . . . , Xn ), and apply
Theorem 2.5.18 to the process Yn . Then (2.45) and (2.46) hold for Yn with
x0 = a. With x1 the constant arising from the conclusion of Theorem 2.5.18,
set A0 = {x ∈ Σ : f (x) ≤ x1 }; then Theorem 2.5.18 yields
1 + f (x) −α
Px [τA0 < ∞] ≤
< 1,
1 + f (x1 )
for all x ∈ Σ \ A0 . Then Lemma 2.5.1(i) gives transience.
i
2.6
A Lyapunov function that has strictly positive drift outside some finite set yields transience provided it satisfies a
bounded moments condition.
Expectations of hitting times and positive recurrence
We start this section with a technical lemma for Markov chains that relates
the moments of hitting times of finite sets to those of individual states.
Lemma 2.6.1. Let Xn be an irreducible Markov chain on a countable state
space Σ. Let α > 0 and A be a finite non-empty subset of Σ such that
Ex (τA+ )α < ∞ for all x ∈ A. Then Ex (τx+ )α < ∞ for all x ∈ Σ.
Proof. Define σ0 := 0 and, for k ≥ 0,
σk+1 = min{n > σk : Xn ∈ A},
so that σk is the k-th passage time of A. Fix an arbitrary x ∈ A and assume
that X0 = x; then, the process Yn = Xσn is a (finite) Markov chain on A
starting from Y0 = x. It is also not hard to see that irreducibility of Yn
follows from irreducibility of Xn . Define
w = min{n ≥ 1 : Yn = x}
65
2.6. Positive recurrence
to be the return time to x for the process Yn .
Now, since the state space of Yn is finite, there exist constants K1 > 0
and δ > 0 such that
Px [w = k] ≤ K1 e−δk .
(2.49)
Let q(y, z) be the one-step transition probabilities from y ∈ A to z ∈ A for
the Markov chain Yn , and set
r1 := min{q(y, z) : y, z ∈ A, q(y, z) > 0},
which is positive since A is finite. Set uk := σk+1 − σk . By assumption,
E[uαk | Yk = y] < ∞, for all y ∈ A.
(2.50)
Also, we have for all y, z ∈ A with q(y, z) > 0,
E[uαk | Yk = y] ≥ E[uαk 1{Yk+1 = z} | Yk = y]
= q(y, z) E[uαk | Yk = y, Yk+1 = z],
and hence, since r1 > 0,
E[uαk | Yk = y, Yk+1 = z] ≤
1
E[uαk | Yk = y] < ∞.
r1
So, since A is finite, there exists r2 ∈ R+ such that
E[uαk | Yk , Yk+1 ] ≤ r2 , a.s.
Then, using the simple inequality
(a1 + · · · + an )α ≤ nα (aα1 + · · · + aαn )
for all α > 0 and ai ≥ 0,
(2.51)
we have for all k,
E[σkα | w = k] = E[(u0 + · · · + uk−1 )α | w = k]
≤ k α E[uα0 + · · · + uαk−1 | Y1 6= x, . . . , Yk−1 6= x, Yk = x]
≤ k α+1 r2 .
α and we have by (2.49)
Since Ex (τx+ )α = Ex σw
α
Ex σw
=
∞
X
k=1
Px [w = k] E[σkα | w = k]
66
Chapter 2. Semimartingale approach and Markov chains
≤
∞
X
k=1
K1 e−δk r2 k α+1 < ∞,
we obtain that Ex (τx+ )α < ∞ for all x ∈ A.
Now, fix any x0 ∈ A, consider an arbitrary y ∈ Σ, and define A0 :=
{x0 , y}. Since the Markov chain is irreducible, there exist ε > 0 and k ≥ 1
such that
/ A0 for all m = 1, . . . , k − 1] ≥ ε.
Px0 [Xk = y, Xm ∈
Then, we have
Ex0 (τx+ )α ≥ Ex0 (τx+ )α ; Xk = y, Xm ∈
/ A0 for all m = 1, . . . , k − 1
≥ ε Ey [(τx+ )α ],
so we must have Ey (τx+ )α < ∞. But this means that Ez (τA+0 )α ≤ Ez (τx+ )α <
∞ for all z ∈ A0 . Repeating the above argument with A0 on the place of A,
we obtain that Ey (τy+ )α < ∞, and this concludes the proof.
Next, we state a couple of simple results, which are, however, very useful
for proving positive recurrence of (Markov) processes.
Theorem 2.6.2. Let Xn be a real-valued integrable Fn -adapted process, and
let τ be a stopping time with respect to Fn ; suppose also that E Xn∧τ ≥ 0
a.s. for all n. Assume that for some ε > 0 and all n ≥ 0, on {τ > n},
E[Xn+1 − Xn | Fn ] ≤ −ε, a.s.
Then it holds that
(2.52)
E X0
< ∞.
(2.53)
ε
Proof. The result is a version of Theorem 2.4.8, but we recapitulate the
(simple) proof. We may rewrite the condition (2.52) as
Eτ ≤
E[X(m+1)∧τ − Xm∧τ | Fm ] ≤ −ε1{τ > m}, a.s., for any m ≥ 0.
We take expectations
E[X(m+1)∧τ ] − E[Xm∧τ ] ≤ −ε P[τ > m],
and then sum the above over m from 0 to n to obtain that
0 ≤ E X(n+1)∧τ ≤ −ε
n
X
m=0
P[τ > m] + E X0 .
67
2.6. Positive recurrence
So, passing to the limit, we have
E τ = lim
n→∞
n
X
m=0
P[τ > m] ≤
E X0
< ∞,
ε
which yields (2.53).
Remark 2.6.3. Under the conditions of Theorem 2.6.2, Dynkin’s formula
(2.9) (with f the identity) gives E[n ∧ τ ] ≤ ε−1 E X0 , which first shows that
P[τ = ∞] = 0 and then, with Fatou’s lemma, provides an alternative proof
of the theorem.
A consequence of Theorem 2.6.2 is the following positive recurrence criterion for Markov chains.
Theorem 2.6.4 (Foster’s criterion). An irreducible Markov chain Xn on
a countable state space Σ is positive recurrent if and only if there exist a
positive function f : Σ → R+ , a finite non-empty set A ⊂ Σ, and ε > 0 such
that
E[f (Xn+1 ) − f (Xn ) | Xn = x] ≤ −ε, for all x ∈ Σ \ A,
E[f (Xn+1 ) | Xn = x] < ∞, for all x ∈ A.
(2.54)
(2.55)
Proof. Let p(x, y) be the transition probabilities of the Markov chain. For
y ∈ Σ define the stopping times
τy = min{n ≥ 0 : Xn = y},
and for B ⊂ Σ let
τB = min{n ≥ 0 : Xn ∈ B} = min τy .
y∈B
To prove the “only if” part, it is enough to set A = {x0 } (where x0 ∈ Σ is
any fixed state), and then notice that for x 6= x0 ,
X
Ex τx0 =
p(x, y) Ey [1 + τx0 ]
y∈Σ
and that
X
y∈Σ
p(x0 , y) Ey τx0 = Ex0 τx+0 < ∞,
so the function f (x) = Ex τx0 satisfies (2.54)–(2.55) with ε = 1.
68
Chapter 2. Semimartingale approach and Markov chains
To prove the “if” part, take X0 = x ∈ Σ. Then f (X0 ) is integrable,
and f (Xn ) is integrable by an induction based on (2.54) and (2.55). Now
Theorem 2.6.2 applied to the non-negative, integrable process f (Xn ) with
the stopping time τA implies that
Ex τA ≤
f (x)
, for all x ∈
/ A.
ε
(2.56)
So, for any y ∈ A, by (2.55) and (2.56)
Ey τA =
X
p(y, z) +
z∈A
=1+
X
p(y, z) Ez [1 + τA ]
z ∈A
/
X
p(y, z) Ez τA
z ∈A
/
≤ 1 + ε−1
X
z ∈A
/
p(y, z)f (z) < ∞.
Then the α = 1 case of Lemma 2.6.1 implies that the Markov chain Xn is
positive recurrent, and this concludes the proof of Theorem 2.6.4.
Of course, a successful application of Theorem 2.6.4 for proving the positive recurrence of a Markov chain depends on one’s ability to find a Lyapunov
function f that works. As observed in the above proof, such a function always exists and is given by the expected time to reach some fixed state.
While it is not always possible to obtain an exact expression for this expected time, one may try to guess an approximation for it, and check if it works
as a Lyapunov function in Theorem 2.6.4:
i
When looking for the Lyapunov function that proves positive
recurrence, fix a state (or a group of states), and set f (x) to
be proportional to your guess for the expected time (starting
from x) to hit that fixed state.
Example 2.6.5. Consider Ŝn , the nearest-neighbour random walk on the
half-line from Example 2.1.8, with p < 21 . Intuitively speaking, the particle
just drifts towards the origin with constant speed, so the time of reaching the
origin from x should be proportional to x (i.e., the distance to the origin).
So, consistently with the above heuristic rule, one can set A = {0} and
f (x) = x; then, (2.54) holds with ε = 1 − 2p > 0, thus proving the positive
4
recurrence of Ŝn with p < 21 .
69
2.6. Positive recurrence
Theorem 2.6.2 has many uses beyond the case where τ is a hitting time
of a finite set, as the next two examples illustrate.
Example 2.6.6. Let (ξn , n ≥ 0) be a Markov chain on Σ ⊆ R2+ ; denote its
(1) (2)
increments by θn := θn+1 − θn , as in Section 1.4. Write ξn = (ξn , ξn ) in
(1) (2)
Cartesian coordinates. Let τ = min{n ≥ 0 : ξn ξn = 0}, the first hitting
time of the boundary of the quarter-plane Σ0 = {(x, y) ∈ R2+ : xy = 0}.
Suppose that for some B ∈ R+ ,
E[θn | ξn = x] = 0, for all x ∈ Σ \ Σ0 ;
E[kθn k2 | ξn = x] ≤ B for all x ∈ Σ \ Σ0 .
Suppose also that the components of the increments have a constant covariance, i.e.,
(1)
(2)
E[(ξn+1 − ξn(1) )(ξn+1 − ξn(2) ) | ξn = x] = ρ, for all x ∈ Σ \ Σ0 ,
for a constant ρ; with the notation at (1.9), this means that the off-diagonal
element of M (x) is equal to ρ. Then, for x ∈ Σ \ Σ0 ,
(1)
(2)
E[ξn+1 ξn+1 − ξn(1) ξn(2) | ξn = x] = ρ.
(2.57)
(1) (2)
Hence if ρ < 0 we may apply Theorem 2.6.2 with Xn = ξn ξn to deduce
that E τ < ∞.
Remarkably, one may also explicitly compute E τ when it exists. Sup(k)
pose that E τ < ∞. For each k ∈ {1, 2}, the process ξn∧τ is a non-negative
(k)
martingale adapted to Fn = σ(ξ0 , . . . , ξn ), and converges to ξτ . The
associated quadratic variation process satisfies hξ (k) in∧τ ≤ B(n ∧ τ ), so
(k)
(k)
E[(ξn∧τ )2 ] ≤ B E τ < ∞. Thus the martingales ξn∧τ are uniformly bounded
in L2 , and hence converge in L2 (see e.g. Theorem 5.4.5 of [83]). Hence,
(1) (2)
ξn∧τ ξn∧τ converges in L1 , so
(1)
(2)
lim E ξn∧τ ξn∧τ = E ξτ(1) ξτ(2) = 0.
n→∞
(2.58)
(1) (2)
Moreover, it follows from (2.57) that ξn ξn − ρn is a martingale, so
(1) (2)
(1)
(2)
ξ0 ξ0 = E ξn∧τ ξn∧τ − ρ E[n ∧ τ ].
Re-arranging, taking n → ∞ and using (2.58) and the monotone convergence
theorem, we obtain
(1) (2)
ξ ξ
E τ = lim E[n ∧ τ ] = 0 0 , when ρ < 0.
n→∞
|ρ|
70
Chapter 2. Semimartingale approach and Markov chains
A similar argument shows that the result E τ < ∞ when ρ < 0 is sharp:
we show that if ρ ≥ 0 and ξ0 ∈
/ Σ0 , then E τ = ∞. For the purposes of
deriving a contradiction, suppose that E τ < ∞. Since ρ ≥ 0, (2.57) shows
(1) (2)
that ξn∧τ ξn∧τ is a submartingale, which converges in L1 by (2.58). Hence
(1) (2)
(1) (2)
0 = E ξτ ξτ ≥ E ξ0 ξ0 > 0, since ξ0 ∈
/ Σ0 , which is a contradiction. So
E τ = ∞.
4
Example 2.6.7. Let Xn be a square-integrable martingale with X0 = 0,
such that |Xn+1 − Xn | ≤ K, a.s., and E[(Xn+1 − Xn )2 | Fn ] = v 2 , a.s., for
some K, v ∈ (0, ∞). Fo a ∈ R+ , define
κa := min{n ≥ 0 : |Xn | > an1/2 },
(2.59)
the exit time from a square-root boundary. In this example we prove that
E κa < ∞ if and only if a < v.
(2.60)
To obtain the ‘only if’ half of (2.60), it suffices to show that E κv =
∞. To this end, let Xn be a square-integrable martingale with E[(Xn+1 −
Xn )2 | Fn ] = v 2 (for this part of the argument the uniform jumps bound is
unnecessary).
Suppose, for the purpose of deriving a contradiction, that E κv < ∞.
We make use of the discussion leading up to Lemma 2.3.18. In this case,
hXin = nv 2 , and hXiκv = κv v 2 . Then, from (2.10), E[Xκ2v ] ≤ v 2 E κv < ∞.
2
It is straightforward to show that a sufficient condition for Xn∧κ
to be
v
uniformly integrable is that
E[Xκ2v ] < ∞,
and Xn2 1{κv > n} is uniformly integrable.
Moreover, by definition of κv ,
Xn2 1{κv > n} ≤ v 2 n1{κv > n} ≤ v 2 κv ,
2
which is integrable under the assumption E κv < ∞. So Xn∧κ
is uniformly
v
integrable, and we may apply Lemma 2.3.18 to conclude
E[Xκ2v ] = E[hXiκv ] = v 2 E κv .
But, by definition, Xκ2v − v 2 κv > 0 then has expectation 0, and so must be
0 a.s., providing the desired contradiction. Thus E κv = ∞.
For the ‘if’ half of (2.60), we will actually prove the stronger result that
E κa < ∞ for any a < v, but where we replace (2.59) by
κa := min{n ≥ 0 : |Xn | > a(n + c)1/2 } for some c ∈ R+ .
71
2.6. Positive recurrence
We permit c > 0 to downgrade the role of trivial cases where P[|X1 | > a] = 1;
but note that E[|X1 |2 ] = v 2 so P[|X1 | > v] < 1, and there can be no trivial
contradiction to E κv = ∞.
Suppose a ∈ (0, v). Take b ∈ (a, v) and c0 > c. Define Yn = b2 (n + c0 ) −
2
Xn . Then, using the square-difference identity (2.23),
Yn+1 − Yn = b2 − 2Xn (Xn+1 − Xn ) − (Xn+1 − Xn )2 ,
and hence, by our hypotheses on Xn , E[Yn+1 − Yn | Fn ] = b2 − v 2 , which is
uniformly (strictly) negative since b < v. Moreover, a.s.,
2
Yn∧κa ≥ b2 (n ∧ κa + c0 ) − a(n ∧ κa + c)1/2 + K ≥ 0, for all n ≥ 0,
provided c0 > c is chosen sufficiently large (depending on a, b, c and K).
Then we can apply Theorem 2.6.2 to see that, if a ∈ (0, v), E κa < ∞. 4
Foster’s original argument in [108] for his version of Theorem 2.6.4 is
rather different from the proof given above; instead it is more similar in spirit
to the following result, which gives a lower bound on expected occupation
times. Note that the set A in this formulation need not be finite.
Lemma 2.6.8. Let Xn be an Fn -adapted process on state space (Σ, E).
Suppose that there exist f : Σ → R+ , a non-empty A ∈ E, and ε > 0 such
that f (X0 ) is integrable, and
E[f (Xn+1 ) − f (Xn ) | Fn ] ≤ −ε, on {Xn ∈ Σ \ A}.
Suppose also that for some constant K < ∞,
E[f (Xn+1 ) − f (Xn ) | Fn ] ≤ K, on {Xn ∈ A}.
Then it holds that
n−1
1 X
ε
lim inf E
1{Xm ∈ A} ≥
.
n→∞ n
ε+K
m=0
Proof. Combining the conditions in the lemma, we have
E[f (Xn+1 ) − f (Xn ) | Fn ] ≤ −ε + (ε + K)1{Xn ∈ A}.
In particular, it follows that f (Xn ) is integrable for all n, given that f (X0 )
is integrable, and we obtain
E f (Xn+1 ) − E f (Xn ) ≤ −ε + (ε + K) P[Xn ∈ A],
72
Chapter 2. Semimartingale approach and Markov chains
and then summing yields
E f (Xn ) − E f (X0 ) ≤ −εn + (ε + K)
n−1
X
m=0
P[Xm ∈ A].
Re-arranging and using the non-negativity of f , we obtain
n−1
1
ε+K X
P[Xm ∈ A] ≥ ε − E f (X0 ).
n
n
m=0
Taking n → ∞ yields the result.
Example 2.6.9. Suppose that Xn is an irreducible Markov chain on a
countable state space Σ, and A ⊂ Σ is finite. It is a classical result that, for
π a sub-probability measure on Σ,
n−1
1 X
E
1{Xm ∈ A} = π(A);
n→∞ n
lim
m=0
Xn is positive recurrent if and only if π(x) > 0 for some (hence all) x ∈ Σ,
in which case π is a proper probability measure. Thus in this case the
conclusion of Lemma 2.6.8 shows that π(A) > 0 and hence π(x) > 0 for some
x ∈ A, which yields positive recurrence. So the “if” half of Theorem 2.6.4
can be deduced from Lemma 2.6.8.
4
We end this section with a partial result in the opposite direction.
Theorem 2.6.10. Let Xn be an Fn -adapted process taking values in R+ ,
and let τ be a stopping time. Suppose that there exists B ∈ R+ such that
E[Xn+1 − Xn | Fn ] ≥ 0, on {n < τ };
E[(Xn+1 − Xn )+ | Fn ] ≤ B, on {n < τ }.
Suppose that there is some c ≥ 0 such that P[Xτ ≤ c | τ < ∞] = 1 and
E X0 > c. Then E τ = ∞.
Proof. Let Yn = Xn∧τ . Then, by hypothesis,
E[(Yn+1 − Yn )+ | Fn ] ≤ B1{n < τ }.
(2.61)
Suppose, for the purpose of deriving a contradiction, that E τ < ∞; so
P[τ < ∞] = 1. Now, taking expectations in (2.61) and summing from
m = 0 to n − 1 we obtain
E
n−1
X
+
(Ym+1 − Ym ) ≤ B
m=0
n−1
X
m=0
P[τ > m] ≤ B E τ.
73
2.6. Positive recurrence
Moreover, since Y0 = X0 ,
0 ≤ Yn = Y0 +
n−1
X
(Ym+1 − Ym )
m=0
∞
X
≤ X0 +
(Ym+1 − Ym )+ =: Z.
m=0
Here Z ≥ 0 is an integrable random variable, since, by Fatou’s lemma,
E Z ≤ X0 + lim inf E
n→∞
n−1
X
(Ym+1 − Ym )+ ≤ X0 + B E τ < ∞.
m=0
Hence |Xn∧τ | is dominated by an integrable random variable, so the submartingale Xn∧τ is uniformly integrable. Then by optional stopping (Theorem 2.3.7),
E Xτ ≥ E X0 > c,
by hypothesis. But, since P[τ < ∞] = 1, Xτ ≤ c, a.s., by hypothesis, giving
the desired contradiction. So E τ = ∞.
This leads to a sufficient condition for a Markov chain to be null, i.e.,
null recurrent or transient.
Corollary 2.6.11. Let Xn be an irreducible Markov chain on a countable
state space Σ. Suppose that there exists a non-empty A ⊂ Σ and a function
f : Σ → R+ such that f (Xn ) is integrable, and
E[f (Xn+1 ) − f (Xn ) | Xn = x] ≥ 0, for all x ∈ Σ \ A;
E[(f (Xn+1 ) − f (Xn ))+ | Xn = x] ≤ B, for all x ∈ Σ \ A;
f (y) > max f (x), for some y ∈ Σ \ A.
x∈A
Then Ex τx+ = ∞ for any x ∈ Σ, i.e., Xn is not positive recurrent.
Proof. Take X0 = y and let τ = τA . Then f (Xn ) satisfies the hypotheses of
Theorem 2.6.10 (with c = maxx∈A f (x)), which implies that Ey τA = ∞.
But τA = minx∈A τx , so that Ey τx = ∞ for any x ∈ A. Take one such
x ∈ A. Then Px [τy < τx+ ] > 0, by irreducibility. Hence, by the strong
Markov property,
Ex τx+ ≥ Px [τy < τx+ ] Ey τx+ = ∞.
But then we must have Ex τx+ = ∞ for all x ∈ Σ, since (by Proposition 2.1.5)
the property Ey τy+ < ∞ is a class property.
74
Chapter 2. Semimartingale approach and Markov chains
Example 2.6.12. Let Sn be symmetric simple random walk on Z. The
hypotheses of Corollary 2.6.11 are satisfied with Xn = Sn , f (x) = |x|,
A = {0}, B = 21 , and any y ≥ 1, so we conclude E1 τ0 = ∞. Hence
E0 τ0+ ≥ 1 +
2.7
1
E1 τ0 = ∞.
2
4
Moments of hitting times
If a Markov chain is recurrent, it is of interest to quantify recurrence beyond
the distinction between null and positive recurrent by estimating tails of
the return time or determining which moments of return times exist. The
question of existence and non-existence of moments of hitting times is also
of interest beyond the Markov setting. This section gives some results in
this direction.
Recall that λx = min{n ≥ 0 : Xn ≤ x}. The first result provides a
sufficient condition for existence of moments for λx , which, by Markov’s
inequality, yield upper tail bounds on λx .
Theorem 2.7.1. Let Xn be an Fn -adapted stochastic process, taking values
in an unbounded subset of R+ . Suppose that there exist δ > 0, x > 0, and
s0 > 0 such that for any n ≥ 0, Xn2s0 is integrable and
2s0
E[Xn+1
− Xn2s0 | Fn ] ≤ −δXn2s0 −2 , on {n < λx }.
(2.62)
For any s ∈ [0, s0 ), there exists a positive constant c = c(δ, s, s0 ) such that
E[λsx | F0 ] ≤ cX02s0 .
Moreover, in the case when s0 ≥ 1, for any s ∈ [0, s0 ], there exists a positive
constant c0 = c0 (δ, s, s0 ) such that
E[λsx | F0 ] ≤ c0 X02s .
Remark 2.7.2. It follows from (2.62) that Xn ≥ δ −1/2 on {n < λx }. So
implicit in the hypotheses of Theorem 2.7.1 is that x ≥ δ −1/2 is not too
small.
Before proving Theorem 2.7.1, we state a useful corollary, which can be
seen as a direct generalization of Theorem 2.6.2 (the case γ = 0).
75
2.7. Moments of hitting times
Corollary 2.7.3. Let Xn be an integrable Fn -adapted stochastic process,
taking values in an unbounded subset of R+ , with X0 = x0 fixed. Suppose
that there exist δ > 0, x > 0, and γ < 1 such that for any n ≥ 0,
E[Xn+1 − Xn | Fn ] ≤ −δXnγ , on {n < λx }.
(2.63)
Then, for any p ∈ [0, 1/(1 − γ)), E[λpx ] < ∞.
(1−γ)/2
Proof. Let s0 = 1/(1 − γ) > 0, and Yn = Xn
. Then Xn = Yn2s0 and
γ
Xn = Yn2s0 −2 , so (2.63) implies that (2.62) holds for Yn , and Theorem 2.7.1
yields the result.
Proof of Theorem 2.7.1. It suffices to suppose that X0 > x. We treat separately the two cases: s0 ≥ 1 and s0 < 1.
(s)
Case 1. s0 ≥ 1. For s ∈ (0, s0 ] define Un for n ≥ 0 by
s
δ
2
Un(s) = Xn∧λ
+ (n ∧ λx ) .
x
s0
(s )
We show that Un 0 is a supermartingale. By (2.62), on {n < λx },
2s0
E[Xn+1
| Fn ] ≤ Xn2s0 − δXn2s0 −2 = Xn2s0 1 − δXn−2 .
The last quantity is non-positive if Xn ≤
may apply the simple inequality
p
p
δ/s0 , while if Xn > δ/s0 we
1 − sy ≤ (1 − y)s for s ≥ 1 and 0 < y < 1,
with y = (δ/s0 )Xn−2 ∈ (0, 1) and s = s0 to see that
p
1 − δXn−2 ≤ 1 − δXn−2 1{Xn > δ/s0 } ≤
s0
δ
1 − Xn−2
,
s0
on {n < λx }. It follows that
δ s 0
2s0
E[Xn+1
| Fn ] ≤ Xn2 −
, on {n < λx }.
s0
(2.64)
Now, on {n < λx }, we apply the Ls0 version of Minkowski’s inequality for
conditional expectations to obtain
h
s 0 i
δ
(s0 )
2
E[Un+1
| Fn ] = E Xn+1
+ (n + 1)
Fn
s0
76
Chapter 2. Semimartingale approach and Markov chains
s0
2s0
, on {n < λx }.
| Fn ]1/s0 + (δ/s0 )(n + 1)
≤ E[Xn+1
Then using (2.64), we obtain on {λx > n},
δ s 0
(s0 )
E[Un+1
| Fn ] ≤ Xn2 + n
= Un(s0 ) .
s0
(s )
Hence Un 0 is a non-negative supermartingale, and it follows from The(s)
orem 2.3.3 that for s ∈ (0, s0 ], the process Un is also a non-negative supermartingale. Hence, by optional stopping (Theorem 2.3.11),
δ s
(s)
E[(n ∧ λx )s | F0 ] ≤ E[Un(s) | F0 ] ≤ E[U0 | F0 ] = X02s ,
s0
and it follows from the (conditional) Fatou lemma that
s s
0
X02s , for any s ∈ (0, s0 ].
E[λsx | F0 ] ≤
δ
Case 2. s0 < 1. We fix some s ∈ (0, s0 ), and define, for n ≥ 0,
2s0
Vn(s) = Xn∧λ
+
x
δ
(n ∧ λx )s .
s0
First we observe, on {λx > n},
2s0
2s0
E[Xn+1
− Xn2s0 | Fn ] = E (Xn+1
− Xn2s0 )1{Xn ∈ (x, n1/2 )} Fn
2s0
+ E (Xn+1
− Xn2s0 )1{Xn ≥ n1/2 } Fn ;
note that for small n the event {Xn ∈ (x, n1/2 )} may be empty. A simple
consequence of (2.62) is
2s0
E[Xn+1
− Xn2s0 | Fn ] ≤ 0, on {n < λx }.
(2.65)
So applying (2.65) and (2.62) we have, on {n < λx },
2s0
E[Xn+1
− Xn2s0 | Fn ] ≤ −δXn2s0 −2 1{Xn ∈ (x, n1/2 )}.
Hence, on {n < λx },
(s)
E[Vn+1 − Vn(s) | Fn ] ≤ −δXn2s0 −2 1{Xn ∈ (x, n1/2 )} +
δ
((n + 1)s − ns ) .
s0
So, using the simple inequality
(n + 1)s − ns ≤ sns−1 , for all n ≥ 1,
77
2.7. Moments of hitting times
we obtain, for s < s0 , for n ≥ 1,
(s)
E[Vn+1 − Vn(s) | Fn ] ≤ δ ns−1 − Xn2s0 −2 1{Xn ∈ (x, n1/2 )} 1{n < λx }
2s0 −2
1{Xn∧λx ∈ (x, n1/2 )}
≤ δ ns−1 − Xn∧λ
x
+ δns−1 1{Xn∧λx ≥ n1/2 }.
≤ δ ns−1 − ns0 −1 + δns−1 1{Xn∧λx ≥ n1/2 }
≤ δns−1 1{Xn∧λx ≥ n1/2 }.
Now taking expectations conditioned on F0 and using Markov’s inequality
we have, for n ≥ 1,
(s)
2s0
E[Vn+1 − Vn(s) | F0 ] ≤ δns−s0 −1 E[Xn∧λ
| F0 ]
x
≤ δns−s0 −1 X02s0 ,
as follows from (2.65). Thus, for n ≥ 1,
E[Vn(s)
| F0 ] ≤
(s)
E[V1
≤
(s)
E[V1
| F0 ] +
| F0 ] +
δX02s0
∞
X
k s−s0 −1
k=1
2s0
c1 X0 ,
for c1 = c1 (δ, s, s0 ) < ∞. Here, by (2.62),
(s)
E[V1
| F0 ] =
E[X12s0
δ
δ −2s0
2s0
−2
| F0 ] +
≤ X0
1 − δX0 + X0
.
s0
s0
Since X0 > x ≥ δ −1/2 (see Remark 2.7.2), the term in the last displayed
bracket is bounded above by some c2 = c2 (δ, s0 ) < ∞, so finally we obtain,
for some c0 = c0 (δ, s, s0 ) < ∞,
E[Vn(s) | F0 ] ≤ c0 X02s0 , on {X0 > x}, for all n ≥ 0.
The statement now follows similarly to the case s0 ≥ 1.
Now we turn to the opposite problem. The first result states sufficient
conditions for non-existence of moments of hitting times λx = min{n ≥ 0 :
Xn ≤ x}.
Theorem 2.7.4. Let Xn be a Fn -adapted stochastic process taking values in
an unbounded subset of R+ . Suppose that there exist constants x ∈ (0, ∞),
B ∈ (0, ∞) and c ∈ R such that, for any n ≥ 0,
E[(Xn+1 − Xn )2 | Fn ] ≤ B, on {Xn ≥ x};
(2.66)
78
Chapter 2. Semimartingale approach and Markov chains
E[Xn+1 − Xn | Fn ] ≥ −
c
, on {Xn ≥ x}.
Xn
(2.67)
2s0
is a submartinSuppose in addition that for some s0 > 0, the process Xn∧λ
x
s
gale. Then, for any s > s0 , we have E[λx | X0 = x0 ] = ∞ for any x0 > x.
A key role in the proof of this theorem is the following estimate, which
gives conditions to ensure that a process in unlikely to return rapidly to a
neighbourhood of the origin. The proof is based on an application of the
maximal inequality from Theorem 2.4.8.
Lemma 2.7.5. Let Xn be an Fn -adapted stochastic process taking values in
R+ . Let δ ∈ (0, 1). Suppose that there exist finite positive constants x, B,
and c such that, for any n ≥ 0, (2.66) and (2.67) hold. There exists ε > 0
(depending only on B, c, and δ) so that, for any n,
i
h
1
P
min
Xm ≥ 2 Xn Fn ≥ 1 − δ, on {Xn > 2x}.
(2.68)
n≤m≤n+εXn2
In particular, on {Xn∧λx > 2x},
P[λx > n + εXn2 | Fn ] ≥ 1 − δ.
(2.69)
Proof. To ease notation, we prove (2.68) and (2.69) for n = 0; the argument
in the general case is the same. Hence suppose X0 > 2x. Write λ = λX0 /2 ,
and note that λx ≥ λ provided X0 > 2x.
Write ∆k = Xk+1 −Xk for the duration of this proof. Define the function
w(x) := (X0 − x)2 1{x < X0 } for x ∈ R+ , and let Wk = w(Xk ). Since w is
non-increasing, on {Xk ≥ X0 }, w(Xk + ∆k ) ≤ w(X0 + ∆k ) ≤ ∆2k . Hence,
on {Xk ≥ X0 },
E[Wk+1 − Wk | Fk ] ≤ E[∆2k | Fk ] ≤ B, a.s.,
by (2.66). On the other hand, on {Xk < X0 },
Wk+1 − Wk ≤ (Wk+1 − Wk )1{Xk+1 < X0 }
= −2(X0 − Xk )∆k + ∆2k 1{∆k < X0 − Xk }.
Here we have that
− 2(X0 − Xk )∆k 1{∆k < X0 − Xk }
= −2(X0 − Xk )∆k + 2(X0 − Xk )∆k 1{∆k ≥ X0 − Xk }
≤ −2(X0 − Xk )∆k + 2∆2k 1{∆k ≥ X0 − Xk }.
2.7. Moments of hitting times
79
Combining these bounds we get, on {Xk < X0 },
Wk+1 − Wk ≤ −2(X0 − Xk )∆k + 2∆2k .
Taking expectations we see that, on {X0 /2 ≤ Xk < X0 }, a.s.,
E[Wk+1 − Wk | Fk ] ≤ −2(X0 − Xk ) E[∆k | Fk ] + 2 E[∆2k | Fk ]
2c(X0 − Xk )
≤
+ 2B.
Xk
In particular, there exists C < ∞, depending only on B and c, such that,
on {Xk > X0 /2}, E[Wk+1 − Wk | Fk ] ≤ C, a.s. Hence we conclude that,
for any k ≥ 0, E[W(k+1)∧λ − Wk∧λ | Fk ] ≤ C, a.s. We apply the maximal
inequality (2.12) from Theorem 2.4.8 to the process Xk∧λ at deterministic
stopping time ν = m to obtain
i Cm + W
h
Cm
0
P max Wk∧λ ≥ y 2 F0 ≤
= 2 , a.s.,
2
0≤k≤m
y
y
on {X0 > 2x}. With y 2 = X02 /4 and m = εX02 we see that
h
X 2 i
P max Wk∧λ < 0 F0 ≥ 4Cε,
4
0≤k≤εX02
on {X0 > 2x}. Now, Wk < y 2 implies Xk > X0 − y, while Wλ = (X0 −
Xλ )2 ≥ X02 /4, so the event in the last display implies min0≤k≤εX02 Wk >
δ
X0 /2. Thus we obtain (2.68), on taking ε < 4C
, and this in turn implies (2.69).
Proof of Theorem 2.7.4. Let X0 = x0 > x. It suffices to suppose that λx <
∞ a.s., otherwise E λsx = ∞ is trivial. We proceed by contradiction. Suppose
that there exists some s > s0 such that E λsx < ∞. Lemma 2.7.5 with δ = 12
yields the existence of positive ε such that for all n ≥ 0,
E λsx ≥ E λsx 1{Xn∧λx > 2x}
εs 2s ≥
E Xn∧λx 1{Xn∧λx > 2x}
2
εs
εs
2s
≥
E Xn∧λ
−
(2x)2s .
x
2
2
Under the hypothesis E λsx < ∞, the last inequality shows that there ex2s0
2s
ists K < ∞ such that E Xn∧λ
≤ K for all n. Hence Xn∧λ
is uniformly
x
x
integrable, and
2s0
lim E Xn∧λ
= E Xλ2sx0 ≤ x2s0 ,
x
n→∞
80
Chapter 2. Semimartingale approach and Markov chains
2s0
is a submartingale,
by definition of λx . On the other hand, since Xn∧λ
x
2s0
2s0
2s0
we have E Xn∧λx ≥ E X0 = x0 . Since x0 > x, this gives the desired
contradiction.
Example 2.7.6. Let Sn be one-dimensional SRW. Take a positive α < 1/2,
and fix c ∈ (0, α(1 − 2α)). We have then
1 2α
1
1 2α 2α
+ 1−
−2
E[Xn+1
− Xn2α | Xn = x] = x2α 1 +
2
x
x
2α−2
≤ −cx
,
for all large enough x, so Theorem 2.7.1 implies that E0 [(τ0+ )α ] < ∞. On the
other hand, in order to apply Theorem 2.7.4, take Xn = |Sn |. The conditions
(2.66) and (2.67) are easily verified, and Xn is itself a submartingale, so we
may apply Theorem 2.7.4 with s0 = 1/2 to deduce that E[(τ0+ )β ] = ∞ for
all β > 1/2.
4
As Example 2.7.6 demonstrates when compared to Example 2.4.3, Theorem 2.7.4 is not quite sharp; in compensation, it is robust. In some situations it is possible to do better by using Lemma 2.7.5 in a slightly different
way: the next result shows how tail information about the maximum of an
excursion can be used to deduce a lower bound on the tail of λx . The idea
is basically an extension of that in Example 2.4.3. In fact, the idea readily yields a lower bound on any non-decreasing additive functional of the
excursion, so we state the result in that generality.
Lemma 2.7.7. Let Xn be an Fn -adapted stochastic process taking values in
an unbounded subset of R+ . Suppose that there exist finite positive constants
x, B, and c such that, for any n ≥ 0, (2.66) and (2.67) hold. Let α : R+ →
R+ be a non-decreasing function. Then there exists ε > 0 (not depending on
α or x) such that, for all y > x,
"
P
λx
X
m=1
#
1
2
α(Xm ) ≥ 4εy α(y) F0 ≥ P max Xm ≥ 2y F0 , a.s.
2 0≤m≤λx
In particular, there exists C < ∞ (not depending on α or x) such that, for
all n sufficiently large,
1
1/2 P λx ≥ n F0 ≥ P max Xm ≥ Cn F0 , a.s.
2 0≤m≤λx
81
2.7. Moments of hitting times
Example 2.7.8. Let y > x. Lemma 2.7.7 applied to the function α(x) =
1{x ≥ y} gives the estimate
#
"λ
x
X
1
2
1{Xm ≥ y} ≥ 4εy ≥ P max Xm ≥ 2y ,
P
0≤m≤λx
2
m=1
and hence, for the expected number of visits to [y, ∞) before reaching [0, x],
#
"λ
x
X
2
4
1{Xm ≥ y} ≥ 2εy P max Xm ≥ 2y .
E
0≤m≤λx
m=1
Proof of Lemma 2.7.7. Let y > x and define the stopping time κ = min{n ≥
0 : Xn > 2y}. Define the event
n
o
E(y, ε) =
min
Xm ≥ 21 Xκ .
κ≤m≤κ+εXκ2
Then by (2.68) applied at the stopping time κ (if finite), which is justified
by Lemma 2.1.11(i), there exists ε > 0 such that
1
P[E(y, ε) | Fκ ] ≥ , on {κ < ∞}.
2
(2.70)
Now, {κ < λx } ∩ E(y, ε) implies that λx > κ + εXκ2 , so that
λx
X
m=1
α(Xm ) ≥ 1{κ < λx }1(E(y, ε))
X
α(Xm )
κ≤m≤κ+εXκ2
≥ εXκ2 α(Xκ /2)1{κ < λx }1(E(y, ε))
≥ 4εy 2 α(y)1{κ < λx }1(E(y, ε)), a.s.,
since α is non-decreasing. Hence, since {κ < λx } is Fκ measurable,
#
"λ
x
X
α(Xm ) ≥ 4εy 2 α(y) F0 ≥ P {κ < λx } ∩ E(y, ε) F0
P
m=1
= E 1{κ < λx } P[E(y, ε) | Fκ ] F0
1
≥ P[κ < λx | F0 ],
2
by (2.70). But κ < λx if and only if max0≤m≤λx Xm ≥ 2y. The result now
follows.
82
Chapter 2. Semimartingale approach and Markov chains
i
A general strategy to prove that certain moments of a hitting
time do not exist is to bound below the probability that the
process ends up a long way away from its destination, and
then takes a while to come back. A local submartingale can
help with the first part, and a maximal inequality with the
second.
Example 2.7.9. As in Example 2.4.15, let Sn be symmetric SRW on Z2 .
Let τ = min{n ≥ 1 : Sn = 0}. It is not hard to show that Xn = kSn k
satisfies the conditions of Lemma 2.7.7, so that with the excursion lower
bound in Example 2.4.15, we obtain that for any ε > 0 there is a constant
c > 0 such that, for all n ≥ 2,
P[τ ≥ n] ≥ c(log n)−1−ε .
In particular, E[τ s ] = ∞ for all s > 0.
2.8
4
Growth bounds on trajectories
In this section we turn to the question of quantifying the ‘speed’ of an R+ valued discrete-time stochastic process (Xn , n ≥ 0) adapted to a filtration
(Fn , n ≥ 0); later we will apply these results to multidimensional models.
Under mild conditions, lim supn→∞ Xn = ∞, i.e., max0≤m≤n Xm tends
to infinity. To quantify the ‘speed’ of the process, we present general semimartingale criteria for obtaining upper and lower almost-sure bounds on
max0≤m≤n Xm . For example, we give conditions for finding an increasing
function h such that max0≤m≤n Xm ≤ h(n) for all but finitely many n ∈ Z+ ,
a.s. (which implies lim supn→∞ Xn /h(n) ≤ 1). Similarly, we present lower
bounds that may be cast as ‘lim inf’ statements.
To start with, we consider almost-sure upper bounds. The statement
involves a function a satisfying the following condition.
(A0) Suppose that for some na ∈ Z+ the function a : [na , ∞) → (0, ∞) is
such thatP
(i) x 7→ a(x) is increasing on x ≥ na ; (ii) limx→∞ a(x) = ∞;
1
and (iii) n≥na a(n)
< ∞.
Condition (A0) is satisfied, for example, by a(x) = x1+ε (with na = 1) or
a(x) = x(log x)1+ε (with na = 2), where ε > 0.
83
2.8. Growth bounds on trajectories
Theorem 2.8.1. Let Xn be an Fn -adapted process on R+ . Suppose that
there exist a non-decreasing function f : R+ → R+ and a constant B ∈ R+
for which f (X0 ) is integrable, and
E[f (Xn+1 ) − f (Xn ) | Fn ] ≤ B, a.s.,
(2.71)
for all n ≥ 0. Define the non-decreasing function f −1 for x > 0 by
f −1 (x) := sup{y ≥ 0 : f (y) < x}.
(2.72)
Let a satisfy (A0). Then, a.s., for all but finitely many n ≥ 0,
max Xm ≤ f −1 (a(2n)).
0≤m≤n
(2.73)
Proof. Note that
E f (Xn ) ≤ E f (X0 ) +
n−1
X
m=0
E[f (Xm+1 ) − f (Xm )] ≤ E f (X0 ) + Bn,
by (2.71), so that f (Xn ) is integrable for all n provided f (X0 ) is integrable.
The maximal inequality (2.14) from Theorem 2.4.7 applied to the process
f (Xn ) shows that, for any n ≥ na ,
h
i Bn + E f (X )
0
P max f (Xm ) ≥ a(n) ≤
.
(2.74)
0≤m≤n
a(n)
Also, since f is non-negative and non-decreasing, for n ≥ na ,
h
i
h i
P max f (Xm ) ≥ a(n) = P f max Xm ≥ a(n) .
0≤m≤n
0≤m≤n
(2.75)
With f −1 defined by (2.72), let En denote the event
n
o
En := max Xm > f −1 (a(n)) .
0≤m≤n
Then since z > f −1 (r) implies f (z) ≥ r, we obtain from (2.74) and (2.75)
that for all n ≥ na ,
h i Bn + E f (X )
0
P[En ] ≤ P f max Xm ≥ a(n) ≤
.
(2.76)
0≤m≤n
a(n)
Now for any `a ≥ log2 na ,
∞
X
`=`a
∞
X
2`
1
<
∞
⇐⇒
< ∞.
`
a(`)
a(2 )
`=na
(2.77)
84
Chapter 2. Semimartingale approach and Markov chains
By (2.76) and (2.77), along the subsequence n = 2` for ` ∈ Z+ , (A0) and
the Borel–Cantelli lemma imply that, a.s., the event En occurs only finitely
often, so, for all but finitely many `,
max Xm ≤ f −1 a(2` ) .
0≤m≤2`
Every n ≥ 1 has 2`n ≤ n < 2`n +1 for some `n ≥ 0 with limn→∞ `n = ∞;
then,
max Xm ≤
max Xm ≤ f −1 a(2`n +1 ) , a.s.,
0≤m≤n
0≤m≤2`n +1
for all but finitely many n. Now since 2`n +1 ≤ 2n and f −1 and a are
eventually non-decreasing, (2.73) follows.
We give an application of the preceding result to a many-dimensional
random walk with asymptotically vanishing drift.
Example 2.8.2. Let (ξn , n ≥ 0) be a Markov process with state space
Σ ⊆ Rd and increments θn := ξn+1 − ξn . Suppose that there exists C < ∞
such that, for all x ∈ Σ,
kE[θn | ξn = x]k ≤ C(1 + kxk)−1 , and E[kθn k2 | ξn = x] ≤ C.
Set Xn = kξn k2 . Then, writing ‘·’ for the scalar product on Rd ,
Xn+1 − Xn = ξn+1 · ξn+1 − ξn · ξn = 2ξn · (ξn+1 − ξn ) + kξn+1 − ξn k2 .
It follows that
E[Xn+1 − Xn | ξn = x] = 2x · E[θn | ξn = x] + E[kθn k2 | ξn = x]
≤ 3C < ∞,
by hypothesis. In other words, if Fn = σ(ξ0 , ξ1 , . . . , ξn ), we have that
E[Xn+1 − Xn | Fn ] ≤ 3C, a.s.
Hence an application of Theorem 2.8.1 with f (x) = x shows that for any
ε > 0, a.s., for all but finitely many n,
max kξm k ≤ n1/2 (log n)(1/2)+ε .
0≤m≤n
4
The next result is a lower bound for max0≤m≤n Xm . In addition to the
condition (A0), the statement involves the following condition.
2.8. Growth bounds on trajectories
85
(A1) Suppose that for some na ∈ Z+ the function a : [na , ∞) → (0, ∞) is
such thatP
(i) x 7→ a(x) is increasing on x ≥ na ; (ii) limx→∞ a(x) = ∞;
1
and (iii) n≥na na(n)
< ∞.
Note P
that the summability condition in (A1) is equivalent to the condition
that n≥na a(r1n ) < ∞ for some (hence all) r > 1. Condition (A1) is satisfied, for example, by a(x) = (log x)1+ε or (log x)(log log x)1+ε , ε > 0.
Theorem 2.8.3. Let Xn be an Fn -adapted process on R+ . Suppose that
there exists b ∈ R+ such that, for all n ≥ 0,
P[Xn+1 − Xn ≤ b] = 1.
(2.78)
Suppose that there exist a non-decreasing function f : R+ → R+ and some ε >
0 for which, for all n ≥ 0,
E[f (Xn+1 ) − f (Xn ) | Fn ] ≥ ε, a.s.
(2.79)
For a function a : [na , ∞) → (0, ∞) with na ∈ Z+ , define ra : R+ → R+ by
ra (x) := inf{y ≥ na : ε−1 a(y)f (y + b) ≥ x}, (x ≥ 0).
(2.80)
(i) Let a satisfy (A0). Then, a.s., for all but finitely many n ≥ 0,
max Xm ≥ ra (n) − 1.
0≤m≤n
(2.81)
(ii) Let a satisfy (A1). For any δ > 0, a.s., for all but finitely many n ≥ 0,
max Xm ≥ (1 − δ)ra (n).
0≤m≤n
(2.82)
Remark 2.8.4. The two parts of the theorem are complementary. If one
expects the growth rate of max0≤m≤n Xm with n to be polynomial, then one
should try a polynomial f , and part (ii) of the theorem will usually provide
the better bound; for slower growth, one should try a super-polynomial f ,
and part (i) will usually provide the better bound.
Proof of Theorem 2.8.3. Recall that for Xn ∈ R+ , σx = min{n ≥ 0 : Xn ≥
x}. Note that under either (A0) or (A1), the fact that f is non-decreasing
ensures that x 7→ ra (x) is non-decreasing for all x ≥ 0, and ra (x) → ∞ as
x → ∞.
86
Chapter 2. Semimartingale approach and Markov chains
We now apply Theorem 2.4.11. Since f satisfies (2.79), we have that (2.19)
holds for all x > 0 and all n ≥ 0, while (2.78) and the fact that f is nondecreasing implies that f (Xσx ) ≤ f (x + b), a.s. Then Theorem 2.4.11 (with
stopping time η = ∞) implies that E σx ≤ ε−1 f (x + b) for all x > 0.
In particular, for all x > 0, σx < ∞, and σx ≤ σy for x ≤ y. Moreover,
the jumps bound (2.78) implies that, for all x ≥ X0 , σx+b ≥ 1 + σx a.s., so
that limx→∞ σx = ∞, a.s.
By Markov’s inequality, for any x ≥ na ,
P σx > a(x) E σx ≤ (a(x))−1 .
(2.83)
To prove part (i), suppose that (A0) holds. Then (2.83) and the Borel–
Cantelli lemma imply that, a.s., for all but finitely many ` ∈ Z+ ,
σ` ≤ a(`) E σ` ≤ ε−1 a(`)f (` + b).
Here, since σx → ∞ as x → ∞, σ` ≥ na for all but finitely many `, so, with
the definition of ra at (2.80), a.s., for all but finitely many `,
ra (σ` ) ≤ ra (ε−1 a(`)f (` + b)) ≤ `.
(2.84)
For n ∈ Z+ , define `n = min{` ≥ 0 : σ` > n}. Then `n → ∞ as n → ∞, and
σ`n −1 ≤ n < σ`n . Moreover, a.s.,
max Xm ≥
0≤m≤n
max
0≤m≤σ`n −1
Xm ≥ Xσ`n −1
≥ `n − 1 ≥ ra (σ`n ) − 1,
(2.85)
for all but finitely many n, by (2.84). Since σ`n > n, (2.81) follows.
The proof of part (ii) is similar. Fix δ > 0 and K > 1 such that
K −1 > 1 − δ. Supposing that (A1) holds, (2.83) and the Borel–Cantelli
lemma imply that, a.s., for all but finitely many ` ∈ Z+ ,
σK ` ≤ a(K ` ) E σK ` ≤ ε−1 a(K ` )f (K ` + b).
Similarly to (2.84), we obtain ra (σK ` ) ≤ K ` , a.s., for all but finitely many
`. Now setting `n = min{` ≥ 0 : σK ` > n}, we have σK `n −1 ≤ n < σK `n .
Then, similarly to (2.85), a.s.,
max Xm ≥ XσK `n −1
0≤m≤n
≥ K `n −1 ≥
1
1
ra (σK `n ) ≥ ra (n),
K
K
for all n sufficiently large, which yields (2.82).
87
2.8. Growth bounds on trajectories
For the rest of this section we return to the nearest-neighbour random
walk on Z+ introduced in Section 2.2 in order to illustrate the results of this
section. Consider the case px = p ∈ (0, 1) for all x ∈ Z+ . If p > 1/2, the
random walk Xn is positive-recurrent; this fact is an easy consequence of
Theorem 2.2.5, or can be seen with the argument in Example 2.6.5, which
is for a model with a slightly different reflection rule at the origin. Quantitatively, we will show that
(2.86)
lim (log n)−1 max Xm = %, a.s.,
n→∞
0≤m≤n
where % ∈ (0, ∞) is given by
% := (log(p/(1 − p)))−1 .
(2.87)
We will prove the following more refined result, from which (2.86) is
immediate. Recall that logk x denotes the k-fold iterated logarithm.
Proposition 2.8.5. Let px = p ∈ (1/2, 1) for all x ∈ Z+ . Then with %
given by (2.87), for any integer k ≥ 3 and any ε > 0, a.s., for all but finitely
many n ∈ Z+ , the following inequalities hold:
k−1
X
max Xm ≤ % log n +
logj n + (1 + ε) logk n ,
(2.88)
k−1
X
≥ % log n −
logj n − (1 + ε) logk n .
(2.89)
0≤m≤n
max Xm
0≤m≤n
j=2
j=2
Proof. From (2.5) with px = p ∈ (1/2, 1), qx = q = 1 − p for all x ∈ Z+ ,
ux = q −1
x
X
(p/q)y =
y=0
1
((p/q)x+1 − 1).
p−q
Hence from (2.8) we obtain, for x ∈ N,
x−1
1 X
p
p
x
t(x) =
((p/q)y+1 − 1) =
(p/q)x −
−
.
2
p−q
(p − q)
p − q (p − q)2
y=0
Thus there is a constant C0 ∈ (1, ∞) such that for all x ≥ 1,
t0 (x) := C0−1 (p/q)x ≤ t(bxc) ≤ C0 (p/q)x =: t1 (x).
(2.90)
88
Chapter 2. Semimartingale approach and Markov chains
The functions t0 and t1 are continuous and monotone, and so can be inverted.
Then, with % given by (2.87), we have that for a constant C1 = % log C0 ∈
(0, ∞) and all x ≥ 1,
t−1
0 (x) = % log x + C1 ,
t−1
1 (x) = % log x − C1 .
(2.91)
We will apply Theorems 2.8.1 and 2.8.3 with Xn as given, with Fn =
σ(X0 , . . . , Xn ), f : R+ → R+ the non-decreasing function given by f (x) =
t(bxc). In particular, (2.71) holds with B = 1, (2.79) holds with ε = 1,
and (2.78) holds with b = 1.
For the upper bound we apply Theorem 2.8.1. For k ≥ 2 and ε > 0, set
a(x) = x(logk x)
1+ε
k−1
Y
logj x.
j=1
Then a satisfies (A0). Define f −1 as at (2.72). Since f (x) = t(bxc) ≥ t0 (x),
we have by continuity of t0 that f −1 (x) ≤ t−1
0 (x). Then Theorem 2.8.1
and (2.91) imply that a.s.
max Xm ≤ t−1
0 (a(2n)) = %(log(a(2n))) + C1 ,
0≤m≤n
(2.92)
for all but finitely many n, and here
log(a(2n)) =
k
X
logj n + (1 + ε) logk+1 n + O(1),
j=1
and so from (2.92) we obtain (2.88).
On the other hand, for the lower bound we apply Theorem 2.8.3(i). For
k ≥ 3 and ε > 0, set
k
X
c(x) := % log x −
logj x − (1 + ε) logk+1 x .
j=2
Then, for some C2 ∈ (0, ∞), for all x > 0 sufficiently large,
sup
0≤y≤c(x)
a(y) ≤ a(c(x)) ≤ a(% log x)
≤ C2 (log x)(log2 x) · · · (logk x)(logk+1 x)1+ε .
Also, from (2.90), for all y ≤ c(x),
t1 (y) ≤ t1 (c(x)) = C0 x(log x)−1 (log2 x)−1 · · · (logk−1 x)−1 (logk x)−1−ε .
Bibliographical notes
89
Thus there exists C3 ∈ (0, ∞) such that for all y with y + 1 ≤ c(x),
a(y)f (y + 1) ≤ a(y)t1 (y + 1) ≤ C3 x(logk x)−ε (logk+1 x)1+ε ≤ x,
for all x sufficiently large. Hence
ra (x) = inf{y ≥ na : a(y)f (y + 1) ≥ x} ≥ c(x) − 1,
for all x sufficiently large. Now (2.89) follows from Theorem 2.8.3(i).
Bibliographical notes
Sections 2.1 and 2.3
There are many books that provide the necessary background in probability theory, e.g. [25, 83, 267, 286], and there also one can find the systematic treatment of martingales and countable Markov chains. The books of
Durrett [83] and Shiryaev [286] are excellent sources for many of the results quoted without proof in Section 2.3; martingale inequalities are treated
thoroughly by Chow and Teicher [41].
Theorem 2.1.6 can be obtained by combining Theorems 1.8.3 and 1.10.2
of [243] for example. Theorem 2.1.6 is a simple Markov chain ergodic theorem; similar results in a more general setting are given in Section 3.6, which
also contains the proof of Theorem 2.1.9 and related results.
Without local finiteness of Σ, part (ii) of Theorem 2.1.9 may fail. To
see this, note that a countable set S ⊆ Rd is locally finite if and only if it
contains no finite limit points. Indeed, it is easy to see that if S ⊆ Rd has
a finite limit point, it is not locally finite. On the other hand, suppose that
S ⊆ Rd is not locally finite. Then there is some ball B0 of radius r0 = 1
such that #(B0 ∩ S) = ∞. Cover B0 by a finite number of balls of radius
r1 = 2−1 whose centres lie in B0 . At least one of the balls in this covering
contains infinitely many points of S; let B1 be one such ball. Iterating this
procedure gives a decreasing sequence of balls Bn , each containing infinitely
many points of S, in which Bn has radius 2−n . Let xn ∈ Rd be the centre
of Bn . Then kxn − xn+1 k ≤ 2−n so limn→∞ xn = x ∈ Rd exists. For any
ε > 0, we can take n large enough so that kx − xn k < ε/2 and rn < ε/2,
so Bn (and hence an infinite subset of S) is contained in a ball of radius ε
centred at x. So x is a finite limit point of S.
Many of the fundamental results on discrete-time martingales in Section 2.3 are due to Doob, and can be found in his classic book [78]. In
90
Chapter 2. Semimartingale approach and Markov chains
the more modern reference [83], Theorem 2.3.1 is Theorem 5.2.10, Theorem 2.3.2 is Theorem 5.4.6, Theorem 2.3.3 is a combination of Theorems 5.2.3 and 5.2.4, Theorem 2.3.4 is Theorem 5.2.8, Theorem 2.3.6 is Theorem 5.2.9, Theorem 2.3.7 is Theorem 5.7.4, and Theorem 2.3.19 is Theorem 5.3.2. Theorems 2.3.11 and 2.3.14 generalize slightly Theorems 5.7.6
and 5.4.2 of [83], respectively.
Section 2.2
The recurrence classification of nearest-neighbour random walks on Z+ is
classical, with early contributions being [127] and [129]. A standard presentation is given by Chung [45, §I.12]
Section 2.4
Of the numerous examples in Section 2.4, several are standard, and can also
be found in [83], for instance; others are more specialized and foreshadow
later developments in the present chapter and the rest of the book.
Theorem 2.4.7 is from a vein of results that can be traced back to Kushner [187, 188] and Spitzer [292]. The maximal inequality (2.14) is Lemma 3.1
in [232]. The expectation lower bound (2.13) is inspired by Spitzer [292]; a
related result is Lemma 2.2 in [230]. Theorem 2.4.8 extends the result from
[232]; inequality (2.12) is a slight generalization of Lemma 4.3 from [133].
Theorem 2.4.5 was used by Kushner [187], who showed that a supermartingale Lyapunov function provides an upper bound on the probability
of eventual escape from a set, similar to the B = 0 case of Corollary 2.4.10.
A version of Corollary 2.4.10 for Markov chains on Rd was given by Kushner [188], assuming continuity of f ; Kalashnikov [145, p. 1406] remarked
that this continuity assumption was unnecessary. Kalashnikov [146, 147]
later provided some generalizations of the idea of Corollary 2.4.10 and gave
applications to reliability theory and to queueing systems.
An alternative proof of (2.14) is to apply Theorem 2.4.5 to the nonnegative supermartingale (Xm∧n + B(m − m ∧ n), n ≥ 0), where m is fixed;
this is the method used by Kushner [188] in the Markov setting, but it is
awkward to extend to admit the stopping time ν. The argument in [232, 133]
is based on Doob’s inequality. The proof of the more general Theorem 2.4.8
given here is entirely elementary.
The elementary approach to Theorem 2.4.8 is essentially the same idea
used in the proofs of the close relatives Theorems 2.4.11 and 2.6.2; as
described in Remark 2.4.9, all of these results can essentially be seen as
Bibliographical notes
91
consequences of Dynkin’s formula. Theorem 2.4.11, which is a relative of
Lemma 3.2 from [232], can be seen as a ‘reverse’ Foster-type result. Theorem 2.4.11 provides an unusually simple approach to Kolmogorov’s ‘other’
inequality (Theorem 2.4.12); the standard approach via optional stopping
can be found in [123, p. 502], for instance. Example 2.4.13 generalizes a result of Spitzer [292] for Markov chains on Z+ ; the terminology ‘weak renewal
theorem’ is Spitzer’s.
The Azuma–Hoeffding martingale concentration inequality (2.27) in Theorem 2.4.14 is due to Hoeffding [130] and Azuma [13]. The submartingale
and supermartingale versions given here are not often explicitly stated in
the literature. A variety of related concentration inequalities can be found
in e.g. [43].
Section 2.5
The basic criteria in Theorems 2.5.2 and 2.5.8 have their origin in work of
F. G. Foster [108], who was a student of D. G. Kendall. Foster [108] proved
a version of the ‘if’ half of the recurrence criterion Theorem 2.5.2, in the
case where the exceptional set A is a single point. Extension to the case of
a finite set A appears in Pakes [250]. The ‘only if’ part of Theorem 2.5.2 is
due to Mertens et al. [238].
The transience criterion Theorem 2.5.8 is also due to Foster [108], again
in the case where A is a singleton. The case of finite A is treated by Harris
and Marlin [126] and Mertens et al. [238]; the condition that A be finite
is not needed, as had already been realised in a different but analogous
context by Kalashnikov [148], who proved a version of Theorem 2.5.8 with
an inessential restriction on the form of f .
Theorem 2.5.15, which gives transience of a Markov chain under a positive drift condition, assuming a uniform jumps bound, can be found in
[213], following earlier work of Malyshev [214]. Both Theorem 2.5.2 and
Theorem 2.5.15 were usefully extended to multiple-step drift conditions by
Menshikov and Malyshev [226, 213]; see also Chapter 2 of Fayolle et al. [96].
Theorem 2.5.19 replaces the jumps bound in Theorem 2.5.15 by a (much
weaker) moments assumption.
As mentioned in the text, Theorems 2.5.7 and 2.5.18 prefigure the developments in Chapter 3, and the basic ideas behind these two results owe
much to Lamperti [190]. Theorem 2.5.7 is contained in Theorem 3.1 of
[190]. A counterexample showing that zero drift for a Markov chain on R
does not imply recurrence (unlike on R+ , as discussed in Example 2.5.6)
can be found in Rogozin and Foss [273], who give a particular example of
92
Chapter 2. Semimartingale approach and Markov chains
Kemperman’s [157] oscillating random walk in which the increment law is
one of two distributions (with mean zero but infinite variance) depending
on the walk’s present sign. The oscillating random walk and related heavytailed phenomena are discussed in Chapter 5. The Markov chain transience
condition, Theorem 2.5.19, is close to best possible for a one-step drift condition of this kind: the version here is given by Zachary [318], a (weaker)
version requiring second moments is in Robert [271, p. 218], while some
improvements are given by Denisov and Foss [106, 73].
The arguments of Foster, and other early authors, are simple enough,
but are based on algebraic manipulations with Markov transition probabilities. The more general, and more probabilistic, approach via stopping times
and adapted processes came later. The systematic use of martingale and
stopping time ideas for investigating recurrence and transience seems to begin with Lamperti [190], who (while acknowledging a debt to ideas of Doob)
treated more general processes than Markov chains. It took some time (see
Mertens et al. [238] and Filonov [103]) for martingale ideas to become the
norm in the proofs of the Markov chain versions of the classification theorems, although Blackwell [27] and Breiman [33], for example, had earlier
made use of martingale ideas in proving structural results about Markov
chains relating to recurrence.
The presentation in Section 2.5, and also Section 2.6, based on adapted processes, stopping times, and semimartingales, follows the spirit of
Chapter 2 of [96], which contains additional results in a similar vein.
Section 2.6
Foster’s positive-recurrence criterion for Markov chains, Theorem 2.6.4, was
first proved by Foster in [108] in the case where A is a singleton, and ε = 1
(the latter being no loss of generality here). In fact, Foster appeals to a
result of Feller (from the 1950 first edition of [98]) for the ‘only if’ part.
Foster himself was aware that his argument could be extended to the case
of finite A (see his contribution to the discussion of [158, p. 175]), as was
also remarked by Moustafa [242] and Kingman [170]; explicit proofs for
finite A were given by Pakes [250] and Kalashnikov [148], although the
finite A result is also a consequence of a condition of Mauldon [218] for
a (not necessarily irreducible) Markov chain to be non-dissipative, which is
equivalent to positive-recurrence in the irreducible case. (The terminology
was introduced by Foster [107].)
The formulation of a Foster-type condition for integrability of a stopping
time for an adapted process, as in Theorem 2.6.2, can be found for example
Bibliographical notes
93
in Theorem 2.1.1 of [96]; a version for processes with bounded increments
was given by Malyshev [214]. Theorems 2.6.2 and 2.6.4 may both be extended to a condition concerning increments over non-unit time intervals:
see Theorems 2.1.2 and 2.2.4 of [96], which are based on earlier results of
[226, 213, 103].
The results in Example 2.6.6 on the exit time of a zero drift Markov
chain from a quadrant were first announced in the case of nearest-neighbour
jumps on Z2+ by Klein Haneveld [175], who used generating functions and
an analytical method from unpublished work of J. Groeneveld; see [176] for
a systematic account. In the generality in Example 2.6.6, the results are
due to Klein Haneveld and Pittenger [177], whose proof is the basis for the
argument given here, and Cohen [51], who used another elaborate generating
function technique; see also [50, 298].
Example 2.6.7 on the exit time of a martingale from a square-root boundary gives a semimartingale approach to a result that dates back to Blackwell
and Freedman [28], who showed for one-dimensional symmetric SRW that
E κa < ∞ if and only if a < 1. The result of Blackwell and Freedman
was generalized by Chow et al. [40] and Gundy and Siegmund [122]. The
approach here to integrability, via Foster’s criterion, is different from these
original papers, while the approach to non-integrability, via quadratic variation, is closely related to the approach in [40].
As mentioned in the main text, Foster’s original argument in [108], and
indeed the arguments of Pakes [250] and Kalashnikov [148], are closer to that
presented in the proof of Lemma 2.6.8, although all those results are stated
only in the Markov chain case. Lemma 2.6.8 is the analogue of Foster’s
original argument for adapted processes.
The idea of proving that a stopping time has infinite expectation by exhibiting a contradiction with the optional stopping theorem, as in Theorem
2.6.10, goes back at least to Doob [78, p. 308], who considered a martingale with uniformly bounded increments. The submartingale result stated
as Theorem 2.6.10 here is a variation on Theorem 2.4 of Fayolle et al. [95].
The Markov chain formulation, Corollary 2.6.11, is a version of a result
of Tweedie [304]; an earlier version, assuming a uniform bound on the increments of f (Xn ), was given by Malyshev in his doctoral thesis of 1973.
Other results on ‘non-ergodicity’ of Markov chains include those of Sennott
et al. [281], which make use of the less elementary Kaplan’s condition [151].
94
Chapter 2. Semimartingale approach and Markov chains
Section 2.7
Results of the form of those in Section 2.7 on existence and non-existence of
moments of passage times originate with fundamental work of Lamperti [192],
who considered the case of E[λpx ] for positive integer p. Lamperti’s ideas were
extended in [10] to include the case of non-integer p, including the important case p ∈ (0, 1); the presentation here is based closely on [10]. In the
Appendix of [10] it is shown that Theorems 2.7.1 and 2.7.4 of Section 2.7
improve the results of Lamperti (cf. Theorems 2.1, 2.2, 3.1, and 3.2 in [192]).
General criteria in the vein of Theorem 2.7.1 for existence of generalized moments E g(τ ) are given in [9].
Lemma 2.7.5, showing that the process takes quadratic time to return to
a neighbourhood of zero from a distant point, is closely related to Lemma
2 of [10], which improved Lemma 3.1 of [192]. The proof in [10] is different
to the proof of Lemma 2.7.5 given here, which is based on the maximal
inequality Theorem 2.4.8.
While existence of moments results are essentially equivalent, via Markov’s
inequality, to upper tail bounds on passage times, non-existence of moments
results do not directly yield any lower tail bounds. However, both types
of result are obtained by combining, with slightly different technical ideas,
two elements, namely (i) Lemma 2.7.5 and (ii) a suitable submartingale.
The first method, as in the proof of Theorem 2.7.4, uses the submartingale
property and the quadratic time estimate to exhibit a contradiction with the
optional stopping theorem if one assumes existence of sufficient moments,
an argument similar in spirit to Theorem 2.6.10 and an idea that goes back
to Doob [78, p. 308]. A second method uses the submartingale to instead
provide a lower tail bound on the maximum of an excursion, which with
the quadratic time estimate yields a lower tail bound on the return time via
Lemma 2.7.7. A similar idea goes back to Lamperti, who obtained a lower
tail bound in the course of the proof of his Theorem 3.2 [192]; a variation is
used to obtain lower tail bounds in [7]. The explicit connection to excursion
maxima, as in Lemma 2.7.7, follows a similar presentation in [60] and [135].
For an irreducible Markov chain Xn on a countable state-space Σ, the
property that Ex [τxγ ] < ∞ for x ∈ Σ was studied under the name “γrecurrence” (also translated as “γ-reflexivity”) by Kalashnikov [149] and
Popov [260], apparently unaware of Lamperti’s work, who both provided
some conditions for Ex [τxγ ] < ∞, γ ≥ 1, in terms of space-time Lyapunov
functions of the form f (Xn , n); these conditions seem to be much less applicable than the results presented here based on [192, 10]. Popov [260] went
on to show that if Xn is γ-recurrent with γ > 1, then the rate of convergence
Bibliographical notes
95
of P[Xn = y | X0 = x] to the stationary probability πy is O(nγ−1 ).
For Markov chains on general state-spaces, drift conditions similar to
those in Theorem 2.7.1 are used by Jarner and Roberts [142] to establish
polynomial rates of convergence to stationarity, in the positive-recurrent
case; see also [79].
The idea of Example 2.7.9 is clearly applicable much more widely than
the case of SRW on Z2 , and the statement is close to sharp. For SRW,
Erdős and Taylor [88], refining an earlier result of Dvoretzky and Erdős
[84], showed that P[τ ≥ n] = logπ n + O((log n)−2 ). See also [80].
Analogues of Theorems 2.7.1 and 2.7.4 for diffusion processes on R+ are
given in [224].
Section 2.8
The results of Section 2.8 are based on [232]. The upper bound result, Theorem 2.8.1, is Theorem 3.2 in [232]. The lower bound result, Theorem 2.8.3,
is based on Theorem 3.3 of [232]; the proof here is simpler, and the statement corrects the erroneous assertion from [232] that one may set δ = 0 in
(2.82).
Example 2.8.2 gives a diffusive upper bound for many-dimensional random walks whose drift decays suitably rapidly with distance from the origin;
such walks may be transient, null-recurrent, or positive-recurrent (in which
case this upper bound is typically not sharp) depending on the details: see
Chapter 4. The application of the results in this section to nearest-neighbour
random walk on Z with negative drift (Proposition 2.8.5) may well be known,
but we could find no reference in the literature.
Further remarks
Additional historical and bibliographic details related to this chapter may
be found in [96] and [239].
It is worth emphasizing that a large and sophisticated body of work exists
on Foster–Lyapunov results for Markov chains in general state spaces. Here
the interaction between the process, the space, and the different possible
notions of recurrence and transience leads to a rich theory, but one with
rather different focus and methodology from this book. Early work in this
direction, starting in the mid 1970s, includes that of Tweedie [302, 304], and
a recent treatment, including thorough bibliographic details, is the book of
Meyn and Tweedie [239].
96
Chapter 2. Semimartingale approach and Markov chains
It is also worth mentioning that there is a well-developed analogue of the
Foster–Lyapunov theory for diffusion processes (and Itô processes more generally). Here early contributions include fundamental work of Khas’minskii,
such as [167]; see also [316, 317]. Recent accounts of aspects of the theory
are presented in the books of Pinsky [258] (for diffusions) and Mao [216] (for
Itô processes).
Chapter 3
Lamperti’s problem
3.1
Introduciton
The law of a one-dimensional time-homogeneous Markov chain is determined
by the collection, over all points x in the state space, of laws of its increments
at x, i.e., the one-step distributions for the chain started from x. A very
broad question is to investigate the extent to which the asymptotic behaviour
of the process is determined by coarser information, such as the first few
moments of the increments at x, as functions of x: can some general form
of ‘invariance principle’ be discerned? To obtain an intuitive grounding, it
is sensible that the variable x should describe a natural scale for the process
in some way. Formalizing such vague notions, and generalizing beyond the
Markovian setting, one is led almost inevitably to what has become known
as Lamperti’s problem, after pioneering work of J. Lamperti published in
the early 1960s (see the bibliographical notes at the end of this chapter for
details).
Let us describe informally Lamperti’s problem. Consider a discrete-time
stochastic process (Xn , n ≥ 0) on R+ . For now suppose that Xn is a timehomogeneous Markov process whose increment moment functions
µk (x) = E[(Xn+1 − Xn )k | Xn = x]
(3.1)
are well-defined for k ≥ 0; one way to ensure this is to impose a uniform
bound on the increments. (We will relax all of these conditions shortly.)
Lamperti’s problem is to determine how the asymptotic behaviour of Xn
depends upon µ1 and µ2 .
It is possible to proceed more generally, but to establish some intuition
we suppose for now that Xn is already on a natural scale, meaning that
97
98
Chapter 3. Lamperti’s problem
µ2 (x) is bounded away from 0 and ∞ by finite constants. This is not too
much of a restriction, since if this second moments condition does not hold
for Xn , one can often choose a suitable f : R+ → R+ for which the condition
does hold for the process Yn = f (Xn ). If f is bijective, then the Markov
property of Xn transfers to Yn . Even if Yn is not itself Markov, we may
still proceed; indeed, there are other, more pressing reasons why we wish to
relax the Markov assumption, as we describe below.
Assuming that µ2 (x) is bounded away from 0 and ∞, the behaviour
of Xn is rather standard when, outside some bounded set, µ1 (x) ≡ 0 (the
zero drift case) or µ1 (x) is uniformly bounded to one side of 0. In the zerodrift case, the Markov chain is null-recurrent; recurrence follows, under mild
conditions, from Theorem 2.5.7 and the null property by Corollary 2.6.11. In
the case of uniformly negative drift, the chain is positive-recurrent by Theorem 2.6.4. In the case of uniformly positive drift, the chain is transient, by
Theorem 2.5.19. Typically, these regimes are also far from critical; positiverecurrence is accompanied by exponential estimates on hitting times and
tails of stationary distributions, and transience is accompanied by positive
speed of escape.
This motivates the study of the asymptotically zero drift regime, in which
µ1 (x) → 0 as x → ∞, to investigate phase transitions. We shall examine
in this chapter part of the rich spectrum of possible behaviours for Xn . It
turns out that from the point of view of the recurrence classification of Xn ,
the case where |µ1 (x)| is of order 1/x is critical.
These Lamperti processes serve as prototypical near-critical stochastic
systems, and provide an excellent setting in which to demonstrate the application of the ideas from Chapter 2. There is, however, another reason,
of at least equal importance, for studying these processes: they appear regularly in the analysis of near-critical stochastic processes via the Lyapunov
function method. This context also provides the main motivation for the
desire to work in some generality without imposing, for instance, assumptions of the Markov property, a countable state-space, or uniformly bounded
increments. The following simple example demonstrates several important
points.
Example 3.1.1. As in Example 2.8.2, let (ξn , n ≥ 0) be a time-homogeneous
Markov chain on state space Σ ⊆ Rd with increments θn := ξn+1 − ξn . Suppose that there exists B ∈ R+ such that, for all x ∈ Σ,
E[kθn k2 | ξn = x] ≤ B, and E[θn | ξn = x] = 0.
For example, ξn might be symmetric simple random walk on Zd ; in this
99
3.2. Markovian case
case, as is typically true in general, Xn is not Markov: see the discussion in
Section 1.3 and the accompanying bibliographical notes in Chapter 1.
Consider Xn = kξn k. First note that, by the triangle inequality,
kξn+1 k − kξn k ≤ kξn+1 − ξn k, a.s.,
so that E[(Xn+1 − Xn )2 | ξn = x] is uniformly bounded. To get an estimate
on the mean increment of Xn given ξn 6= 0, we write
Xn+1 − Xn =
kξn+1 k2 − kξn k2 (kξn+1 k − kξn k)2
−
.
2kξn k
2kξn k
Taking expectations, we see | E[Xn+1 − Xn | ξn = x]| is bounded above by
1
E[kξn+1 k2 − kξn k2 | ξn = x] + E[(kξn+1 k − kξn k)2 | ξn = x] ,
2kxk
which is at most Bkxk−1 , by a similar calculation to that in Example 2.8.2.
So, roughly speaking, the process Xn has mean drift O(Xn−1 ). To study, for
instance, the recurrence properties of ξn , this Lyapunov function approach
thus leads to the study of the R+ -valued process Xn , which has asymptotically zero drift, and so is of Lamperti type. However, since Xn is typically
not Markov, some care is needed to formulate this approach precisely. Given
additional assumptions on ξn , we can estimate the drift of Xn more carefully, using Taylor’s theorem: we return to this idea in Example 3.5.4, and
in detail in Chapter 4.
4
i
Natural Lyapunov functions for near-critical Markov processes often lead to Lamperti-type stochastic processes.
Relaxing the Markov assumption leads to a slight complication in defining the correct analogues of (3.1), but does not complicate the proofs which
are based on the semimartingale ideas of Chapter 2. For expository purposes, we start with the Markov case and then proceed to the more general
setting. Throughout this chapter we use the notation
∆n := Xn+1 − Xn .
3.2
Markovian case
For orientation, we start with the Markov setting. We state in this section
the Markov versions of some of the key results in this chapter; we do not
100
Chapter 3. Lamperti’s problem
give the proofs at this stage. The results presented in this section will follow
from the more general results presented in the main body of this chapter;
we defer the proofs until Section 3.13.
For this section, our basic assumptions on our process Xn and on the
structure of its state space Σ are as follows.
(M0) Let Xn be an irreducible, time-homogeneous Markov chain on Σ, a
locally finite, unbounded subset of R+ , with 0 ∈ Σ.
We also make some assumptions on the increments of Xn . We will
typically assume that for some p > 2 (at least),
sup E[|∆n |p | Xn = x] < ∞.
(3.2)
x∈Σ
(M1) Suppose that (3.2) holds for some p > 2.
Remark 3.2.1. As stated in Corollary 2.1.10, a consequence of (M0) is that
lim supn→∞ Xn = ∞, a.s.; see Section 3.6 below for a proof.
Given (M1), µk (x) as defined at (3.1) is finite for k ∈ {1, 2}.
Example 3.2.2. Although the proofs of the results in this section are deferred till later, we present, by way of an example, an application of Foster’s
criterion (Theorem 2.6.4) to demonstrate a condition for positive recurrence.
Suppose that (M0) and (M1) hold, and there exist positive constants ε and
x0 so that 2xµ1 (x) + µ2 (x) < −ε for all x ≥ x0 . Consider the Lyapunov
function f (x) = x2 . Then
E[f (Xn+1 ) − f (Xn ) | Xn = x] = E[2Xn ∆n + ∆2n | Xn = x]
= 2xµ1 (x) + µ2 (x) ≤ −ε,
outside the set [0, x0 ] ∩ Σ, which is finite since Σ is locally finite. Thus we
may apply Theorem 2.6.4 to deduce that Xn is positive recurrent.
4
The basic recurrence classification is as follows; part (iii) is the statement
from Example 3.2.2.
Theorem 3.2.3. Suppose that (M0) and (M1) hold. Then the following
recurrence conditions are valid.
(i) Xn is transient if there exist ε, x0 ∈ (0, ∞) such that
2xµ1 (x) − µ2 (x) > ε, for all x ≥ x0 .
101
3.2. Markovian case
(ii) Xn is null-recurrent if there exist x0 ∈ (0, ∞), θ > 0, and v > 0 such
that for all x ≥ x0 , µ2 (x) ≥ v and
1−θ
2x|µ1 (x)| ≤ 1 +
µ2 (x).
log x
(iii) Xn is positive-recurrent if there exist ε, x0 ∈ (0, ∞) such that
2xµ1 (x) + µ2 (x) < −ε, for all x ≥ x0 .
Remark 3.2.4. More compactly, (i) says lim inf x→∞ (2xµ1 (x) − µ2 (x)) > 0,
and similarly (iii) says lim supx→∞ (2xµ1 (x) + µ2 (x)) < 0.
Example 3.2.5. As in Section 2.2, consider the simple random walk on Z+
with transition probabilities p(x, x − 1) = px = 1 − p(x, x + 1) for x ≥ 1 and
p(0, 0) = p0 = 1 − p(0, 1), where px ∈ (0, 1) for all x ≥ 0. Suppose that, for
some c ∈ R,
1
c
px =
1+
+ o(x−1 ).
2
x
Then µ1 (x) = 1 − 2px = −(c + o(1))x−1 and µ2 (x) = 1, so Theorem 3.2.3
says that this random walk is
• transient if c < −1/2;
• null recurrent if |c| < 1/2;
• positive recurrent if c > 1/2.
The boundary cases c = ±1/2 can go either way depending on the o(x−1 )
term in px ; if this term is actually o (x log x)−1 , say, Theorem 3.2.3(ii)
shows that the case |c| = 1/2 is null recurrent.
4
Other results that we present in this chapter include existence of passagetime moments and almost-sure bounds on trajectories. To simplify the statements in the Markovian case, it is convenient to assume the following.
(M2) Suppose that there exist a ∈ R and b ∈ (0, ∞) such that
lim µ2 (x) = b, and lim xµ1 (x) = a.
x→∞
x→∞
In the case where (M0), (M1) and (M2) all hold, Theorem 3.2.3 implies
that Xn is
102
Chapter 3. Lamperti’s problem
• transient if 2a > b;
• null recurrent if |2a| < b;
• positive recurrent if 2a < −b.
As in Example 3.2.5, to classify the boundary cases needs stronger assumptions. The next result gives existence and non-existence of moments of
return times. Recall from Section 2.1 that τx+ = min{n ≥ 1 : Xn = x}.
Theorem 3.2.6. Suppose that (M0), (M1) and (M2) hold for p > max{2, 2s},
where s > 0.
(i) For any x ∈ Σ, Ex [(τx+ )s ] < ∞ if s <
b−2a
2b .
(ii) For any x ∈ Σ, Ex [(τx+ )s ] = ∞ if s >
b−2a
2b .
The next result gives growth bounds on trajectories. For simplicity, we
state the result on the logarithmic scale.
Theorem 3.2.7. Suppose that (M0), (M1) and (M2) hold.
(i) If 2a > b, then, a.s.,
log Xn
1
= .
n→∞ log n
2
lim
(ii) If |2a| ≤ b, then, a.s.,
lim sup
n→∞
1
log Xn
= .
log n
2
(iii) If 2a < −b and (M1) holds with p >
lim sup
n→∞
b−2a
b ,
then, a.s.,
log Xn
b
=
∈ 0, 12 .
log n
b − 2a
Example 3.2.8. Continuing from Example 3.2.5, consider the simple random walk on Z+ with
1
c
p(x, x − 1) = px = 1 − p(x, x + 1) =
1+
+ o(x−1 ).
2
x
Then µ1 (x) = 1 − 2px = −(c + o(1))x−1 and µ2 (x) = 1, so (M2) holds
with a = −c and b = 1, and Theorem 3.2.7 gives corresponding almost-sure
bounds. For example, in the case c > 1/2 (positive recurrence), a.s.,
lim sup
n→∞
1
log Xn
=
∈ 0, 12 ,
log n
1 + 2c
demonstrating the sub-diffusive behaviour of the walk’s upper envelope.
4
103
3.3. General case
3.3
General case
We now describe the more general setting that will occupy most of this
chapter. Let (Xn , n ≥ 0) be a discrete-time stochastic process adapted to a
filtration (Fn , n ≥ 0) and taking values in a subset S of R that is bounded
in one direction and unbounded in the other. Without loss of generality, we
locate the ordinate such that S ⊆ R+ with inf S = 0 and sup S = +∞.
Central objects in this chapter will be the conditional increment moments
E[∆kn | Fn ], specifically for k = 1 and k = 2. Many of the conditions in the
theorems will suppose that an inequality hold involving the Fn -measurable
random variables E[∆n | Fn ], E[∆2n | Fn ], and Xn ; such inequalities will
have to hold a.s. and in an appropriate asymptotic sense (for large values
of Xn ). It is convenient therefore to introduce some notation for upper and
lower bounds on the increment moments E[∆kn | Fn ] as ‘functions’ of Xn .
To this end, shortly we will define µk : S → R and µ̄k : S → R such that
µk (Xn ) ≤ E[∆kn | Fn ] ≤ µ̄k (Xn ), a.s., for all n ≥ 0.
(3.3)
If Xn is Markovian, then E[∆kn | Fn ] = E[∆kn | Xn ], a.s., and we can take
µk (x) = inf E[∆kn | Xn = x], and µ̄k (x) = sup E[∆kn | Xn = x];
n≥0
n≥0
if additionally Xn is time-homogeneous then µk (x) ≡ µ̄k (x) ≡ µk (x) as
defined at (3.1).
Loosely speaking, in the general case E[∆kn | Fn ] involves additional
randomness in Fn , once Xn has been fixed. Thus µ̄k (x) should be the
(essential) supremum over this additional randomness given {Xn = x}. For
µk the situation is analogous.
Let us now formally define µk and µ̄k . Let k be a positive integer. Suppose that E[∆kn | Fn ] exists for all n ≥ 0. By standard theory of conditional
expectations (see e.g. Section 9.1 of [46]), for each n ≥ 0 there exist a Borelmeasurable function ϕk,n : S → R and an Fn -measurable random variable
ψk,n such that, a.s., E[ψk,n | Xn ] = 0 and
E[∆kn | Fn ] = E[∆kn | Xn ] + ψk,n = ϕk,n (Xn ) + ψk,n .
(3.4)
Here ϕk,n (x) = E[∆kn | Xn = x]. Then for x ∈ S define
µk (x) := inf ϕk,n (x) + ess inf(ψk,n 1{Xn ≥ x}) ;
n≥0
µ̄k (x) := sup ϕk,n (x) + ess sup(ψk,n 1{Xn ≥ x}) .
(3.5)
n≥0
(3.6)
104
Chapter 3. Lamperti’s problem
Provided that the expectations in question exist, (3.5) and (3.6) define µk (x)
and µ̄k (x) as R-valued functions of x ∈ S such that µk (x) ≤ µ̄k (x) for all
x ∈ S, and the property (3.3) is satisfied.
Example 3.3.1. With the notation at (3.4), we may write
Xn+1 = Xn + ϕ1,n (Xn ) + ψ1,n + ζn+1 ,
(3.7)
where ζn+1 = ∆n − E[∆n | Fn ] is Fn+1 -measurable with E[ζn+1 | Fn ] = 0;
i.e., ζn is a martingale difference sequence.
4
If ϕk,n (x) = ϕk (x) does not depend on n, then
|µ̄k (x) − µk (x)| ≤ 2 sup ess sup(|ψk,n |1{Xn ≥ x}),
n≥0
which, in many situations of interest, tends to zero as x → ∞ (and does so
sufficiently fast if µ̄k (x) and µk (x) also tend to zero) so that µ̄k (x) ∼ µk (x).
The following simple example is one such case, and also serves to show that
even in elementary situations, some technical machinery along the lines of
that presented in this section is not easily avoided.
Example 3.3.2. Let Sn be symmetric SRW on Zd , d ∈ N, and set Xn =
kSn k. Let Fn = σ(S0 , . . . , Sn ). We consider E[∆n | Fn ] = E[∆n | Sn ]. Recalling the notation from Section 1.2, for e1 , . . . , ed the standard orthonormal basis vectors, let Ud = {±e1 , . . . , ±ed } denote the possible jumps of the
walk. Then for x ∈ Zd ,
E[∆n | Sn = x] =
1 X
(kx + ek − kxk).
2d
e∈Ud
Here, a Taylor’s formula calculation (as in Section 1.3) shows that
!
2e · x 1/2
kx + ek − kxk = kxk
1+
−1
kxk2
1 2e · x + 1
1 (e · x)2
=
−
+ O(kxk−2 ),
2
kxk
2
kxk3
where the implicit constant in the O( · ) term depends only on d. So
1
1
E[∆n | Sn = x] =
1−
+ O(kxk−2 );
2kxk
d
(3.8)
105
3.3. General case
see equation (1.4). It follows that
1
1
1−
+ O(x−2 ), and |ψ1,n |1{Xn ≥ x} = O(x−2 ),
ϕ1,n (x) =
2x
d
where, again, implicit constants in the error terms depend only on d. Hence
1
1
1−
.
lim xµ1 (x) = lim xµ̄1 (x) =
x→∞
x→∞
2
d
Note that if d ≥ 2 the difference µ̄1 (x) − µ1 (x) is genuinely of order x−2 : for
P
instance, the third-order Taylor term in (3.8) contains kxk−5 e∈Ud (e·x)3 =
θ(x)kxk−2 for θ(x) whose range includes a positive interval for all x with
kxk sufficiently large.
4
i
If Yn is Markov and Xn = f (Yn ), then although Xn is typically not Markov, if Yn has reasonably homogeneous behaviour around level curves of f , then one can expect that
µ̄k (x) and µk (x) have similar asymptotics.
Some further discussion of the definitions in (3.5) and (3.6), and some
illustrative examples, can be found at the end of this section.
Our basic assumption is now as follows.
(L0) Let Xn be a stochastic process adapted to a filtration Fn and taking
values in S ⊆ R+ with inf S = 0 and sup S = +∞.
We impose a moments condition analogous to, but somewhat weaker
than, condition (M1).
(L1) Suppose that for some p > 2, δ ∈ (0, p − 2], and C ∈ R+ ,
E[|∆n |p | Fn ] ≤ C(1 + Xn )p−2−δ , a.s., for all n ≥ 0.
(3.9)
Note that a sufficient condition for (L1) is that E[|∆n |p | Fn ] ≤ C, a.s., for
some p > 2, some C < ∞, and all n ≥ 0; i.e., the δ = p − 2 case of (3.9).
We also will typically assume the following non-confinement condition:
(L2) Suppose that lim supn→∞ Xn = +∞, a.s.
Remarks 3.3.3. (a) Since Xn < ∞ for all n, lim supn→∞ Xn = ∞ is equivalent to supn≥0 Xn = ∞.
106
Chapter 3. Lamperti’s problem
(b) Condition (L2) is generally necessary for our questions of interest to be
non-trivial, and is usually straightforward to verify in a particular application, as we discuss in Section 3.6 below. For example, if Xn is an irreducible
time-homogeneous Markov chain and S is locally finite, then (L2) holds
automatically: see Corollary 2.1.10.
We state one simple but useful sufficient condition for (L2). This does not
rely on irreducibility as such, but the following related local escape property,
which says that the process exits any interval [0, x] in a finite number of steps
with positive probability, depending only on x.
(L3) Suppose that for each x ∈ R+ there exist rx ∈ N and δx > 0 such
that, for all n ≥ 0,
i
h
P
max Xm ≥ x Fn ≥ δx , on {Xn ≤ x}.
(3.10)
n≤m≤n+rx
Recall that for Xn a process on R+ , σx = min{n ≥ 0 : Xn ≥ x}.
Proposition 3.3.4. Suppose that (L0) and (L3) hold. Then (L2) holds.
Moreover, there exist Kx ∈ R+ and cx > 0 such that, for all n ≥ 0,
P[σx ≥ n | F0 ] ≤ Kx e−cx n , on {X0 < x}.
(3.11)
Finally, a sufficient condition for (L3) is that for each x ∈ R+ there exist
mx ∈ N and εx > 0 such that, for all n ≥ 0,
P[Xn+mx − Xn ≥ εx | Fn ] ≥ εx , on {Xn ≤ x}.
(3.12)
Example 3.3.5. As in Example 3.1.1, let (ξn , n ≥ 0) be a time-homogeneous
Markov chain on state space Σ ⊆ Rd with increments θn := ξn+1 − ξn . Suppose that there exists ε > 0 for which
inf P [θn · u ≥ ε | ξn = x] ≥ ε, for all x ∈ Σ;
u∈Sd−1
(3.13)
the property (3.13) is uniform ellipticity. Then, for Xn = kξn k, we have
P [Xn+1 − Xn ≥ ε | ξn = x] ≥ P [x̂ · (x + θn ) − kxk ≥ ε | ξn = x]
≥ P [x̂ · θn ≥ ε | ξn = x] ≥ ε,
by (3.13). Hence (3.12) holds for all x ∈ R+ with mx = 1 and εx = ε, and
Proposition 3.3.4 shows that lim supn→∞ kξn k = ∞, a.s.
4
107
3.3. General case
Proof of Proposition 3.3.4. Fix x ≥ 0. Write r = rx , δ = δx , and σ = σx for
ease of notation. By (3.10), P[maxn≤m≤n+r Xm ≥ x | Fn ] ≥ δ, on {n < σ}.
Hence,
P[σ > n + r | Fn ] ≤ 1 − δ, on {n < σ}.
(3.14)
Now, the n = 0 case of (3.14) says that P[σ > r | F0 ] ≤ 1 − δ on {X0 < x}.
Hence, conditioning on Fr and using the n = r case of (3.14),
P[σ > 2r | F0 ] = E 1{σ > r} P[σ > 2r | Fr ] F0
≤ (1 − δ) P[δ > r | F0 ]
≤ (1 − δ)2 , on {X0 < x}.
Iterating this argument shows that, on {X0 < x}, P[σ > kr | F0 ] ≤ (1 − δ)k
for any integer k ≥ 0. Any integer n has kn r < n ≤ (kn + 1)r for some
integer kn , and then, on {X0 < x},
P[σ > n | F0 ] ≤ P[σ > kn r | F0 ]
≤ (1 − δ)kn ≤ (1 − δ)(n/r)−1 ,
which yields (3.11) for constants cx and Kx depending on δ = δx and r = rx .
Now we deduce (L2). Let λ0 = min{n ≥ 0 : Xn ≤ x}, and define
recursively, for k ∈ N, λk = min{n ≥ λk−1 + r : Xn ≤ x}, with the usual
convention min ∅ = ∞. Set L = inf{k ≥ 0 : λk = ∞}. Thus we have defined
stopping times λk such that λk−1 < λk for all k ≤ L, and λk = ∞ for all
k ≥ L. Also define for k ∈ N the event
n
o
Ek =
max
Xn ≥ x ∪ {λk = ∞}.
λk−1 ≤n≤λk−1 +r
If λk < ∞, then Xλk ≤ x and so, by (3.10), P[Ek+1 | Fλk ] ≥ δ on {λk < ∞}.
On the other hand, if λk = ∞ then λk+1 = ∞ as well, by definition, so that
P[Ek+1 | Fλk ] = 1 on {λk = ∞}. Hence we obtain
P[Ek+1 | Fλk ] ≥ δ, a.s.
Since Ek ∈ Fλk , it follows from Lévy’s version of the Borel–Cantelli lemma
(Theorem 2.3.19) that {Ek i.o.} occurs a.s. Hence either L < ∞ or each
of {Xn ≤ x} and {Xn ≥ x} occurs for infinitely many n. In either case,
lim supn→∞ Xn ≥ x, a.s. Since x ∈ R+ was arbitrary, lim supn→∞ Xn = ∞,
a.s.
108
Chapter 3. Lamperti’s problem
It remains to show that (3.12) implies (3.10). Fix x ≥ 0. Write m = mx ,
ε = εx , and σ = σx . For ease of notation, with no loss of generality, we take
n = 0. Define for k ∈ N the events
Ak = {Xkm − X(k−1)m ≥ ε}, and Bk = Ak ∪ {σ ≤ m(k − 1)}.
Then Bk ∈ Fkm and
P[Bk+1 | Fkm ] ≥ P[Ak+1 | Fkm ]1{km < σ} + P[σ ≤ mk | Fkm ]1{km ≥ σ}
≥ ε, a.s.,
by (3.12), since on {km < σ} we have {Xkm ≤ x}. Hence by a similar
iterated conditioning argument to above, we obtain P[∩rk=1 Bk | F0 ] ≥ εr .
Taking rx = min{r ∈ N : rε > x} and δx = εrx , we have that with probx
x
Ak ocBk occurs, which implies that either ∩rk=1
ability at least δx , ∩rk=1
curs, or {σ < mrx }. In the former case, Xmrx ≥ X0 + rx ε > x. Hence
P[max0≤k≤mrx Xk ≥ x | F0 ] ≥ δx , which implies (3.10).
To end this section, we briefly discuss further the definitions in (3.5) and
(3.6), and give some examples for particular classes of process Xn .
Moments Markov property
Somewhat weaker than the full Markov property is the assumption that
E[∆n | Fn ] = µ1 (Xn ), and E[∆2n | Fn ] = µ2 (Xn ), a.s.,
(3.15)
for measurable functions µ1 and µ2 . In the notation at (3.4), (3.15) asserts
that ψn,k = 0 a.s. for k ∈ {1, 2}. This moments Markov property is the
standing assumption in papers such as [155] motivated by growth models
and phrased in terms of stochastic difference equations. Indeed, under (3.15)
one may write, similarly to (3.7),
Xn+1 = Xn + µ1 (Xn ) + ζn+1 ,
2
where E[ζn+1 | Fn ] = 0 and E[ζn+1
| Fn ] = µ2 (Xn ) − µ1 (Xn )2 , so ζn is a
martingale difference sequence adapted to Fn .
History-dependent processes
Suppose that Fn = σ(X0 , . . . , Xn ) and the law of Xn+1 depends only upon
(X0 , . . . , Xn ). For convenience, take S to be countable. Then we can write
X
E[∆n | Fn ] =
E[∆n | X0 = x0 , . . . , Xn = xn ]
x0 ,...,xn ∈S
109
3.4. Lyapunov functions
× 1{X0 = x0 , . . . , Xn = xn }.
In this case
µ̄1 (x) = sup sup E[∆n | X0 = x0 , . . . , Xn−1 = xn−1 , Xn = x],
n∈Z+
where the second supremum is taken over x0 , . . . , xn−1 ∈ S for which P[X0 =
x0 , . . . , Xn−1 = xn−1 | Xn = x] > 0. There is an analogous expression for µ1 .
Functions of Markov processes
Suppose that Yn is a time-homogeneous Markov process on a general statespace Σ, and that for some measurable function f : Σ → R+ , Xn = f (Yn ).
Set Fn = σ(Y0 , . . . , Yn ). Then Xn has state-space S = f (Σ). Now E[∆n |
Fn ] = E[∆n | Yn ], a.s., and if Σ (hence S) is countable, we may write
X
X
E[∆n | Fn ] =
E[∆n | Yn = y]1{Yn = y, Xn = x}.
x∈S y∈Σ:f (y)=x
In this case,
µ̄1 (x) = sup
sup
n∈Z+ y∈Σ:f (y)=x, P[Yn =y|Xn =x]>0
E[∆n | Yn = y],
and similarly for µ1 . This situation often arises in applications, where f may
be, for instance, a Lyapunov-type function applied to a multi-dimensional
Markov process.
3.4
Lyapunov functions
The results in this chapter will be obtained by application of the semimartingale results of Chapter 2 to suitably chosen Lyapunov functions of
the process Xn . For γ ∈ R and ν ∈ R we define
(
xγ logν x if x ≥ e;
fγ,ν (x) :=
(3.16)
eγ
if x < e.
The piecewise definition in (3.16) ensures that the function fγ,ν : R+ →
(0, ∞) is well defined for all γ and ν.
The analysis in this chapter will be built on the fact that fγ,ν (Xn ) satisfies a submartingale or supermartingale condition, for Xn outside some
110
Chapter 3. Lamperti’s problem
bounded set, for appropriate γ and ν. The next result gives the necessary
increment estimates that we will need throughout the chapter; we state
the result in some generality to avoid repeating very similar computations
several times.
Lemma 3.4.1. Let γ ∈ R and ν ∈ R. Suppose that (L0) and (L1) hold,
with p > 2 and δ ∈ (0, p − 2] such that −δ < γ < p. Then there exists
ε0 = ε0 (p, δ, γ) ∈ (0, 1/2) such that, for any ε ∈ (0, ε0 ), a.s.,
E[fγ,ν (Xn+1 ) − fγ,ν (Xn ) | Fn ]
γ−1
2
= γ Xn E[∆n | Fn ] +
E[∆n | Fn ] fγ−2,ν (Xn )
2
2γ − 1
2
E[∆n | Fn ] fγ−2,ν−1 (Xn )
+ ν Xn E[∆n | Fn ] +
2
Fn
γ−2−ε
2
n
). (3.17)
+ ν(ν − 1) + oF
Xn (1) E[∆n | Fn ]fγ−2,ν−2 (Xn ) + OXn (Xn
Moreover, if γ ∈ (0, p) then, for any ε ∈ (0, ε0 ), a.s.,
γ
− Xnγ | Fn ]
E[Xn+1
γ−1
Fn
2
= γ Xn E[∆n | Fn ] +
E[∆n | Fn ] Xnγ−2 + OX
(Xnγ−2−ε ). (3.18)
n
2
To prepare for the proof of Lemma 3.4.1, we first state a technical result.
For ε ∈ (0, 1), define the event
Eε (n) := |∆n | ≤ (1 + Xn )1−ε .
(3.19)
Denote the complementary event by Eεc (n).
Lemma 3.4.2. Suppose that (L0) and (L1) hold, with p > 2, δ ∈ (0, p − 2],
and C ∈ R+ . Then for any ε ∈ (0, 1), any q ∈ [0, p], and all n ≥ 0,
E[|∆n |q 1(Eεc (n)) | Fn ] ≤ C(1 + Xn )q−2−δ+pε , a.s.
(3.20)
E[|∆n |q 1(Eεc (n)) | Fn ] ≤ C(1 + Xn )q−2−ε , a.s.
(3.21)
δ
Moreover, for any ε ∈ (0, 1+p
) and any q ∈ [0, p], for all n ≥ 0,
Proof. For q ∈ [0, p], by definition of Eε (n),
|∆n |q 1(Eεc (n)) = |∆n |p |∆n |q−p 1(Eεc (n)) ≤ |∆n |p (1 + Xn )(1−ε)(q−p) .
Taking expectations conditioned on Fn and using (3.9) we obtain
E[|∆n |q 1(Eεc (n)) | Fn ] ≤ C(1 + Xn )p−2−δ+(1−ε)(q−p) , a.s.,
which implies (3.20), from which (3.21) also follows.
111
3.4. Lyapunov functions
Proof of Lemma 3.4.1. Fix γ, ν ∈ R. Throughout the proof we write just
f for fγ,ν . As we are interested in asymptotics when Xn is large, we may
suppose throughout that Xn > 2e. For any ε ∈ (0, 1),
f (Xn+1 ) − f (Xn ) = (f (Xn + ∆n ) − f (Xn )) 1(Eε (n))
+ (f (Xn + ∆n ) − f (Xn )) 1(Eεc (n)).
We estimate the expectation on Eε (n) here using a Taylor expansion, while
we use Lemma 3.4.2 to control the expectation on Eεc (n). Write
ε0 = ε0 (p, δ, γ) =
min{δ, γ + δ, 1 + p}
,
2 + 2p
(3.22)
and suppose that ε ∈ (0, 2ε0 ); note that since −δ < γ < p, we have 2ε0 ∈
(0, 1). Differentiation of f with respect to x > e gives
f 0 (x) = γ + ν log−1 x x−1 f (x);
00
fγ,ν
(x) = γ(γ − 1) + ν(2γ − 1) log−1 x + ν(ν − 1) log−2 x x−2 f (x);
000 (x) = O(xγ−3 logν x). Taylor’s formula with Lagrange remainder
and fγ,ν
says that for any x, h ∈ R with x > e and x + h > e,
f (x + h) − f (x) = hf 0 (x) +
h2 00
h3
f (x) + f 000 (x + ϕh),
2
6
for some ϕ = ϕ(x, h) ∈ [0, 1]. Taking x = Xn and h = ∆n , we have on
Eε (n) that Xn + ϕ∆n > Xn /2 > e, say, for all Xn sufficiently large. Hence
we can find constants C and x1 so that the third-order term in the Taylor
expansion of f (Xn + ∆n ) − f (Xn ) on Eε (n) satisfies
|∆3n f 000 (Xn + ϕ∆n )|1(Eε (n)) ≤ C|∆n |3 Xnγ−3 logν Xn 1(Eε (n))
≤ C∆2n Xnγ−2−ε logν Xn
≤ C∆2n Xnγ−2−(ε/2) , on {Xn > x1 }.
Hence
(f (Xn + ∆n ) − f (Xn )) 1(Eε (n))
∆2n 00
Fn
0
= ∆n f (Xn ) +
f (Xn ) 1(Eε (n)) + ∆2n OX
(Xnγ−2−(ε/2) ).
n
2
Collecting terms, we obtain
(f (Xn + ∆n ) − f (Xn )) 1(Eε (n))
112
Chapter 3. Lamperti’s problem
γ−1 2
= γ Xn ∆n +
∆n f (Xn )Xn−2 1(Eε (n))
2
2γ − 1 2
∆n f (Xn )Xn−2 log−1 Xn 1(Eε (n))
+ ν Xn ∆n +
2
n
(1)
∆2n f (Xn )Xn−2 log−2 Xn 1(Eε (n)),
+ ν(ν − 1) + oF
Xn
n
where the oF
Xn (1) error term encompasses the third-order Taylor term. To
take (conditional) expectations here, we observe that, since (3.9) holds for
δ
some p > 2, and ε ∈ (0, 2ε0 ) satisfies ε < 1+p
, we have from the q ∈ {1, 2}
cases of Lemma 3.4.2 that, a.s.,
Fn
(Xn−1−ε );
E[∆n 1(Eε (n)) | Fn ] = E[∆n | Fn ] + OX
n
Fn
(Xn−ε ).
E[∆2n 1(Eε (n)) | Fn ] = E[∆2n | Fn ] + OX
n
It follows that
E f (Xn + ∆n ) − f (Xn ) 1(Eε (n)) Fn
γ−1
2
= γ Xn E[∆n | Fn ] +
E[∆n | Fn ] Xnγ−2 logν Xn
2
2γ − 1
E[∆2n | Fn ] Xnγ−2 logν−1 Xn
+ ν Xn E[∆n | Fn ] +
2
2
γ−2
n
+ ν(ν − 1) + oF
logν−2 Xn
Xn (1) E[∆n | Fn ]Xn
Fn
(Xnγ−2−(ε/2) ).
+ OX
n
(3.23)
On the other hand, we claim that for any ε ∈ (0, 2ε0 ), a.s.,
Fn
(Xnγ−2−(ε/2) ).
E[|f (Xn + ∆n ) − f (Xn )| 1(Eεc (n)) | Fn ] = OX
n
(3.24)
To verify (3.24), first suppose that γ < 0. Then fγ,ν is eventually decreasing,
and there exists C ∈ R+ such that 0 ≤ fγ,ν (x) ≤ C for all x ∈ R+ . Hence
E[|f (Xn + ∆n ) − f (Xn )| 1(Eεc (n)) | Fn ] ≤ C P[Eεc (n) | Fn ],
which is bounded above by a constant times Xn−2−δ+pε , by the q = 0 case
of (3.20). By choice of ε ∈ (0, 2ε0 ), we have ε < γ+δ
1+p , so that −δ +pε < γ −ε;
this establishes the claim (3.24) in the case γ ∈ (−δ, 0).
Next suppose that γ ≥ 0. Now, for any ε0 > 0, there exists C ∈ R+ such
0
that 0 ≤ fγ,ν (x) ≤ C(1 + x)γ+ε for all x ∈ R+ . Hence for any ε0 > 0 there
exists C < ∞ such that
0
0
|f (Xn + ∆n ) − f (Xn )| ≤ C(1 + Xn )γ+ε + C|∆n |γ+ε , a.s.
(3.25)
113
3.4. Lyapunov functions
0
To see (3.25), note f (X + ∆n ) ≤ C(1 + 32 Xn )γ+ε on {|∆n | ≤ 21 Xn }, and
0
0
f (Xn + ∆n ) ≤ C(1 + 3|∆n |)γ+ε ≤ C(4|∆n |)γ+ε , on {|∆n | > 21 Xn },
since Xn > 2. Then by (3.25), E[|f (Xn + ∆n ) − f (Xn )| 1(Eεc (n)) | Fn ] is
bounded above by
0
0
C(1 + Xn )γ+ε P[Eεc (n) | Fn ] + C E[|∆n |γ+ε 1(Eεc (n)) | Fn ].
(3.26)
Given that (3.9) holds for p > γ, we may choose ε0 > 0 small enough so that
δ
0 ≤ γ + ε0 ≤ p. By choice of ε ∈ (0, 2ε0 ), we have ε < 1+p
, so that both
0
terms in (3.26) are bounded above by a constant times Xnγ+ε −2−ε , by the
q = 0 and q = γ + ε0 cases of (3.21) respectively. Taking ε0 > 0 small enough
(ε0 < ε/2, say) this establishes the claim (3.24) in the case γ ∈ [0, p).
Combining (3.24) with (3.23) we obtain (3.17), but with ε in place of
ε/2, which lies in the interval (0, ε0 ). The expression in (3.18) is essentially
the ν = 0 case of (3.17); the difference near zero between fγ,0 as defined at
(3.17) and the function xγ does not affect the Taylor estimate in the analogue
of (3.23), and the analogue of (3.24) goes through provided γ > 0.
For some of our results we need to study the case where fγ,ν as defined at
(3.16), with positive γ, satisfies a local submartingale property; application
often requires uniform integrability. A convenient technical device is to work
with a truncated version the function. Specifically, for x > 0 define
x
fγ,ν
(y) := min{fγ,ν (y), fγ,ν (2x)}.
(3.27)
Lemma 3.4.3. Let γ ∈ (0, ∞) and ν ∈ R+ . Suppose that (L0) and (L1)
hold, with p > max{2, γ}. Then there exists ε0 = ε0 (p, δ, γ) > 0 such that,
for any ε ∈ (0, ε0 ), there exists x0 = x0 (ε) ∈ R+ such that, for any x ≥ x0 ,
on {Xn ≤ x},
x
x
E[fγ,ν
(Xn+1 ) − fγ,ν
(Xn ) | Fn ]
γ−1
2
≥ γ Xn E[∆n | Fn ] +
E[∆n | Fn ] fγ−2,ν (Xn )
2
2γ − 1
2
+ ν Xn E[∆n | Fn ] +
E[∆n | Fn ] fγ−2,ν−1 (Xn )
2
Fn
n
+ ν(ν − 1) + oF
(1)
E[∆2n | Fn ]fγ−2,ν−2 (Xn ) + OX
(Xnγ−2−ε ), (3.28)
Xn
n
where the implicit constants in the error terms do not depend on x.
114
Chapter 3. Lamperti’s problem
x ; note that both f and f x are nonProof. Write f for fγ,ν and f x for fγ,ν
decreasing, and they coincide on [0, 2x]. Let ε0 ∈ (0, 1/2) be the constant
in the statement of Lemma 3.4.1, as given at (3.22). Fix ε ∈ (0, 2ε0 ). Then
{Xn ≤ x} ∩ Eε (n) implies that Xn+1 ≤ 2x provided x ≥ x0 for some
x0 = x0 (ε) sufficiently large, so that f x (Xn+1 ) = f (Xn+1 ). Fix x ≥ x0 .
Then, on {Xn ≤ x},
f x (Xn+1 ) − f x (Xn ) ≥ f (Xn+1 ) − f (Xn ) 1(Eε (n))
− f (Xn )1{∆n > (1 + Xn )1−ε },
so that, on {Xn ≤ x},
E[f x (Xn+1 ) − f x (Xn ) | Fn ] ≥ E (f (Xn+1 ) − f (Xn ))1(Eε (n)) Fn
− f (Xn ) P[Eεc (n) | Fn ].
Since ε ∈ (0, 2ε0 ) satisfies ε <
δ
1+p ,
the q = 0 case of (3.21) shows that
Fn
(Xnγ−2−(ε/2) ).
f (Xn ) P[Eεc (n) | Fn ] = OX
n
Combined with (3.23) this shows that, on {Xn ≤ x}, (3.28) holds, once ε/2
is replaced by ε ∈ (0, ε0 ).
3.5
Recurrence classification
We need to explain what we mean by ‘recurrence’ or ‘transience’ in the
general setting introduced in Section 3.3. In this section we give conditions
under which one or other of the following two behaviours (which are a priori
not exhaustive) occurs (cf. Definition 1.5.1):
• limn→∞ Xn = +∞, a.s., in which case we say that Xn is transient;
• lim inf n→∞ Xn ≤ r0 , a.s., for some constant r0 ∈ R+ , when we say Xn
is recurrent.
First we give a condition for transience.
Theorem 3.5.1. Suppose that (L0), (L1) and (L2) hold. Suppose that
lim sup µ̄2 (x) < ∞, and lim inf (2xµ1 (x) − µ̄2 (x)) > 0.
x→∞
Then limn→∞ Xn = +∞, a.s.
x→∞
(3.29)
3.5. Recurrence classification
115
The next result gives a condition for recurrence.
Theorem 3.5.2. Suppose that (L0), (L1) and (L2) hold. Suppose that
lim inf x→∞ µ2 (x) > 0, and that there exist x0 ∈ R+ and θ > 0 such that
2xµ̄1 (x) ≤
1−θ
1+
µ2 (x), for all x ≥ x0 .
log x
(3.30)
Then there exists r0 ∈ R+ such that lim inf n→∞ Xn ≤ r0 , a.s.
We give two important examples.
Example 3.5.3. Consider Sn symmetric SRW on Zd , and let Xn = kSn k
and Fn = σ(S0 , . . . , Sn ). Either Sn is transient and limn→∞ Xn = ∞ a.s., or
else Sn is recurrent and lim inf n→∞ Xn = 0 a.s.: see Section 3.6 for a more
general discussion. Example 3.3.2 shows that
2xµ1 (x) = 1 −
1
1
+ O(x−2 ), and 2xµ̄1 (x) = 1 − + O(x−2 ).
d
d
Second moments can be computed similarly to in Example 3.3.2 using Taylor’s
formula. Alternatively, we can use the square-difference identity (2.23) in
the form
2
E[∆2n | Sn ] = E[Xn+1
− Xn2 | Sn ] − 2Xn E[∆n | Sn ],
and the fact, obtained by expanding kSn + (Sn+1 − Sn )k2 , that
2
Xn+1
− Xn2 = 2Sn · (Sn+1 − Sn ) + kSn+1 − Sn k2 ,
(3.31)
2
so E[Xn+1
− Xn2 | Sn ] = 1, to obtain
µ2 (x) =
1
1
+ O(x−1 ), and µ̄2 (x) = + O(x−1 ).
d
d
Thus (3.29) holds provided d ≥ 3, so Theorem 3.5.1 implies transience.
On the other hand, (3.30) holds if d ∈ {1, 2} and Theorem 3.5.2 implies
recurrence. This is Lamperti’s proof of Pólya’s theorem on simple random
walk, as advertised in Section 1.3.
4
Example 3.5.4. We extend Pólya’s theorem from Example 3.5.3 to a class
of zero-drift random walks, as advertised in Section 1.5. Similarly to Example 3.1.1, let (ξn , n ≥ 0) be a time-homogeneous Markov chain with state
116
Chapter 3. Lamperti’s problem
space Σ ⊆ Rd and increments θn := ξn+1 − ξn such that, for some B ∈ R+
and all x ∈ Σ,
P[kθn k ≤ B | ξn = x] = 1, and E[θn | ξn = x] = 0.
Suppose in addition that the increment covariance matrix is constant, i.e.,
with θn as a column vector, for all x ∈ Σ,
E[θn θn> | ξn = x] = M,
for some positive definite, symmetric d by d matrix M . The argument in
Remark 1.5.5 shows that it is no loss of generality to take M = Id , the
identity.
So suppose M = Id . Let Xn = kξn k. Similarly to (3.31), we have
2
E[Xn+1
− Xn2 | ξn ] = 2ξn · E[θn | ξn ] + E[kθn k2 | ξn ]
= 0 + tr M = d.
(3.32)
Moreover, using the bound kθn k ≤ B and the Taylor expansion (1 + y)1/2 =
1 + (y/2) − (y 2 /8) + O(y 3 ), similarly to the calculation in Section 1.5,
i
2ξn · θn + kθn k2 1/2
−
1
ξn
kξn k2
ξn · E[θn | ξn ] E[kθn k2 | ξn ] E[(ξn · θn )2 | ξn ]
=
+
−
+ O(kξn k−2 ).
kξn k
2kξn k
2kξn k3
1 =
tr M − ξˆn> M ξˆn + O(Xn−2 ),
2Xn
E[Xn+1 − Xn | ξn ] = kξn k E
h
1+
using the fact that E[θn | ξn ] = 0. Hence, since M = Id , we obtain
E[Xn+1 − Xn | ξn ] =
d−1
+ O(Xn−2 ).
2Xn
(3.33)
Thus with (3.32), (3.33), and the usual trick based on (2.23), we obtain
2xµ1 (x) = d − 1 + O(x−1 ) = 2xµ̄1 (x), and µ2 (x) = 1 + O(x−1 ) = µ̄2 (x).
Thus (3.29) holds provided d ≥ 3, so Theorem 3.5.1 implies transience,
while (3.30) holds if d ∈ {1, 2} and Theorem 3.5.2 implies recurrence. This
example is a preview of Chapter 4; there we relax the uniform increment
bound, as well considering walks with non-zero drift.
4
3.5. Recurrence classification
117
Remarks 3.5.5. (a) The calculations in Examples 3.5.3 and 3.5.4 suggest
informally that the recurrence phase transition for this class of zero-drift
many-dimensional random walks occurs at d = 2 (rather than elsewhere in
the interval [2, 3]). This transition is probed more precisely in Chapter 4.
(b) For condition (3.30), it is sufficient (take θ = 1) that 2xµ̄1 (x)−µ2 (x) ≤ 0
for all x large enough; however, we needed to take θ < 1 to settle the critical
d = 2 case in Examples 3.5.3 and 3.5.4.
We now turn to the proofs of Theorems 3.5.1 and 3.5.2. We proceed
under the minimal assumption (L0) for much of the argument, in part to
highlight exactly where the non-confinement assumption (L2) enters, but
also so that the results that follow can be applied to situations where (L2)
fails (which is the case, for example, in the context of branching processes
or other processes with absorption: see Example 3.5.9).
Consider the random variable X ? := supn≥0 Xn ∈ R+ . In principle,
there are three possibilities:
(a) ess sup X ? < ∞, i.e., X ? is uniformly bounded, a.s.;
(b) ess inf X ? < ∞ but ess sup X ? = ∞;
(c) ess inf X ? = ∞, i.e., X ? = ∞ a.s.
Here, (c) is the case when (L2) holds, case (a) is generally uninteresting, and
case (b) pertains in, for instance, the case where there is some non-trivial
probability that the process gets absorbed or trapped.
The following semimartingale result is a ‘transience’ result related to
Theorem 2.5.8, but which applies in case (b) as well as case (c); we prove
Theorem 3.5.1 by applying this result with a suitable Lyapunov function.
Recall that σx = min{n ≥ 0 : Xn ≥ x}.
Theorem 3.5.6. Suppose that (L0) holds. Let f : R+ → R+ be such that
supx f (x) < ∞, limx→∞ f (x) = 0, and inf y≤x f (y) > 0 for any x ∈ R+ .
Suppose also that there exists x0 ∈ R+ for which, for all n ≥ 0,
E[f (Xn+1 ) − f (Xn ) | Fn ] ≤ 0, on {Xn > x0 }.
(i) If P[σx < ∞] > 0 for all x ∈ R+ , then P[lim inf n→∞ Xn ≥ x] > 0 for
all x ∈ R+ .
(ii) If (L2) holds, then P[limn→∞ Xn = ∞] = 1.
The core of Theorem 3.5.6 is a hitting probability estimate, which is a
variation on Lemma 2.5.10.
118
Chapter 3. Lamperti’s problem
Lemma 3.5.7. Let Xn be an Fn -adapted process taking values in R+ . Let
f : R+ → R+ be such that supx f (x) < ∞ and limx→∞ f (x) = 0. Suppose
that there exists x1 ∈ R+ for which inf y≤x1 f (y) > 0 and, for all n ≥ 0,
E[f (Xn+1 ) − f (Xn ) | Fn ] ≤ 0, on {Xn > x1 }.
Then for any ε > 0 there exists x ∈ (x1 , ∞) for which, for all n ≥ 0,
i
h
P inf Xm ≥ x1 Fn ≥ 1 − ε, on {Xn > x}.
m≥n
Proof. Fix n ∈ Z+ . For x1 ∈ R+ in the hypothesis of the lemma, write
λ = min{m ≥ n : Xm ≤ x1 } and set Ym = f (Xm∧λ ). Then (Ym , m ≥
n) is an (Fm , m ≥ n)-adapted non-negative supermartingale, and so, by
Theorem 2.3.6, converges a.s. as m → ∞ to some Y∞ ∈ R+ . Moreover, by
Theorem 2.3.11,
Yn ≥ E[Y∞ | Fn ] ≥ E[Y∞ 1{λ < ∞} | Fn ], a.s.
Here we have that, a.s.,
Y∞ 1{λ < ∞} = lim Ym 1{λ < ∞} = f (Xλ )1{λ < ∞}
m→∞
≥ inf f (y)1{λ < ∞}.
y≤x1
Combining these inequalities we obtain
inf f (y) P[λ < ∞ | Fn ] ≤ Yn , a.s.
y≤x1
In particular, on {Xn > x ≥ x1 }, we have Yn = f (Xn ) and so
inf f (y) P[λ < ∞ | Fn ] ≤ f (Xn ) ≤ sup f (y).
y≤x1
y>x
Since limy→∞ f (y) = 0 and inf y≤x1 f (y) > 0, given ε > 0 we can choose
x > x1 large enough so that
supy>x f (y)
< ε;
inf y≤x1 f (y)
the choice of x depends only on f , x1 , and ε, and, in particular, does not
depend on n. Then, on {Xn > x}, P[λ < ∞ | Fn ] < ε, as claimed.
We now use the local result Lemma 3.5.7 to deduce the sample-path
result Theorem 3.5.6.
119
3.5. Recurrence classification
Proof of Theorem 3.5.6. Under the conditions of Theorem 3.5.6, the hypotheses of Lemma 3.5.7 are satisfied for any x1 > x0 . So for any x1 > x0 and
any ε > 0, there exists x ∈ R+ , x > x1 such that, on {Xn ≥ x},
i
h
P inf Xm > x1 Fn ≥ 1 − ε, a.s.
m≥n
As usual, let σx = min{n ≥ 0 : Xn ≥ x}. Then, on {σx < ∞},
h
i
P inf Xm > x1 Fσx ≥ 1 − ε, a.s.
m≥σx
But on {σx < ∞} ∩ {inf m≥σx Xm > x1 } we have lim inf m→∞ Xm ≥ x1 , so
h
h
i
i
P lim inf Xm ≥ x1 ≥ E P inf Xm > x1 Fσx 1{σx < ∞}
m→∞
m≥σx
≥ (1 − ε) P[σx < ∞].
Under the conditions of part (i) of the theorem, P[σx < ∞] > 0, and so
the claim in part (i) follows. On the other hand, under the conditions of
part (ii) of the theorem, P[σx < ∞] = 1, and so, since ε > 0 was arbitrary,
lim inf m→∞ Xm ≥ x1 , a.s. Then since x1 > x0 was arbitrary, the result
follows.
Proof of Theorem 3.5.1. We apply Theorem 3.5.6 with Lyapunov function
f = f0,ν for ν < 0 as defined at (3.16). Taking γ = 0 in (3.17) and using
the fact that lim supx→∞ µ̄2 (x) < ∞, we see that
E[f (Xn+1 ) − f (Xn ) | Fn ]
ν
≤
2Xn E[∆n | Fn ] − E[∆2n | Fn ] Xn−2 logν−1 Xn
2
−2
ν−1
n
+ oF
Xn ).
Xn (Xn log
(3.34)
The assumption (3.29) says that there exist x1 ∈ R+ and ε0 > 0 such that
2Xn µ1 (Xn ) − µ̄2 (Xn ) ≥ ε0 on {Xn > x1 }. Hence, by definition of µ̄ and µ,
2Xn E[∆n | Fn ] − E[∆2n | Fn ] ≥ 2Xn µ1 (Xn ) − µ̄2 (Xn ) ≥ ε0 ,
for all Xn sufficiently large. Since ν < 0, it follows that the right-hand side
of (3.34) is negative for all Xn sufficiently large. Thus the conditions of
Theorem 3.5.6 are satisfied, and hence limn→∞ Xn = ∞, a.s.
Now we move on to recurrence. First we state a semimartingale relative
of Theorem 2.5.2.
120
Chapter 3. Lamperti’s problem
Theorem 3.5.8. Suppose that (L0) holds. Let f : R+ → R+ be such that
f (x) → ∞ as x → ∞ and E f (X0 ) < ∞. Suppose that there exist x0 ∈ R+
and C < ∞ for which, for all n ≥ 0,
E[f (Xn+1 ) − f (Xn ) | Fn ] ≤ 0, on {Xn > x0 }, a.s.;
E[f (Xn+1 ) − f (Xn ) | Fn ] ≤ C, on {Xn ≤ x0 }, a.s.
Then
h
i
P lim sup Xn < ∞ ∪ lim inf Xn ≤ x0 = 1.
n→∞
n→∞
In particular, if (L2) also holds, we have lim inf n→∞ Xn ≤ x0 , a.s.
Proof. First note that, by hypothesis, E f (X1 ) ≤ E f (X0 ) + C < ∞, and,
iterating this argument, it follows that E f (Xn ) < ∞ for all n ≥ 0.
Fix n ∈ Z+ . For x0 ∈ R+ in the hypothesis of the lemma, write λ =
min{m ≥ n : Xm ≤ x0 }. Let Ym = f (Xm∧λ ). Then (Ym , m ≥ n) is a nonnegative supermartingale. Hence, by Theorem 2.3.6, there exists a finite
random variable Y∞ ∈ R+ such that limm→∞ Ym = Y∞ , a.s. In particular,
this means that
lim sup f (Xm ) ≤ Y∞ , on {λ = ∞}.
m→∞
Setting ζ = sup{x ≥ 0 : f (x) ≤ Y∞ }, which satisfies ζ < ∞ a.s. since
limx→∞ f (x) = ∞, it follows that lim supm→∞ Xm ≤ ζ on {λ = ∞}. Hence
h
i
P lim sup Xn < ∞ ∪ inf Xm ≤ x0 = 1.
n→∞
m≥n
Since n ∈ Z+ was arbitrary, it follows that
h
\
i
P lim sup Xn < ∞ ∪
inf Xm ≤ x0 = 1,
n→∞
n≥0
m≥n
which gives the result.
We can now complete the proof of Theorem 3.5.2; again we use a Lyapunov function of the form fγ,ν .
Proof of Theorem 3.5.2. We apply Theorem 3.5.8 with Lyapunov function
f = f0,ν for ν ∈ (0, 1) as defined at (3.16). Taking γ = 0 in (3.17), we have
E[f (Xn+1 ) − f (Xn ) | Fn ]
ν
=
2Xn E[∆n | Fn ] − E[∆2n | Fn ] Xn−2 logν−1 Xn
2
121
3.5. Recurrence classification
n
+ ν(ν − 1) + oF
(1)
E[∆2n | Fn ]Xn−2 logν−2 Xn
Xn
−2
ν−2
n
Xn ).
+ oF
Xn (Xn log
(3.35)
Here, since ν > 0 and ν − 1 < 0, we have that, for x1 ∈ R+ sufficiently large,
1
n
ν(ν − 1) + oF
Xn (1) ≤ − ν(1 − ν), on {Xn ≥ x1 }.
2
Hence, by definition of µ̄1 and µ2 ,
E[f (Xn+1 ) − f (Xn ) | Fn ]
ν
≤
2Xn µ̄1 (Xn ) − µ2 (Xn ) Xn−2 logν−1 Xn
2
1
−2
ν−2
n
Xn ).
− ν(1 − ν)µ2 (Xn )Xn−2 logν−2 Xn + oF
Xn (Xn log
2
Hence, by (3.30),
E[f (Xn+1 ) − f (Xn ) | Fn ] ≤
ν −2
n
Xn logν−2 Xn (ν − θ)µ2 (Xn ) + oF
Xn (1) .
2
Choosing ν ∈ (0, θ), the fact that lim inf x→∞ µ2 (x) > 0 shows that the last
display is negative for all Xn sufficiently large, i.e., for some r0 ∈ R+ ,
E[f (Xn+1 ) − f (Xn ) | Fn ] ≤ 0, on {Xn ≥ r0 }.
Hence we may apply Theorem 3.5.8 with Lyapunov function f to conclude
that lim inf n→∞ Xn ≤ r0 , a.s.
We end this section with a branching process example, which is instructive in two respects: how the above ideas may be applied when condition (L2)
fails, and how a Lamperti-type problem can often be obtained via an appropriate scaling.
Example 3.5.9. Consider the following state-dependent branching process.
Let ζz , z ∈ N, be a family of random variables, representing the offspring
distribution of an individual in a population of size z. The total population
size (Zn , n ≥ 0) is a Markov process on Z+ constructed as follows. Let
Z0 = z0 ∈ N. Given Zn = z ∈ Z+ , Zn+1 is distributed as the sum of Zn
independent copies of ζz , as generation n is succeeded by generation n + 1.
For simplicity, suppose that P[ζz ≤ B] = 1 for some B ∈ R+ . Suppose
also, to avoid degenerate cases, that P[ζz = 1] < 1. The Markov chain Zn
has an absorbing state at 0 (extinction), and there are two possibilities for
122
Chapter 3. Lamperti’s problem
the long-term behaviour: either Zn → 0, a.s., or else P[Zn → ∞] > 0 and
the process survives with positive probability.
The classical Galton–Watson process has ζz = ζ independent of z, and
a.s.-extinction occurs if and only if m = E ζ ≤ 1. In the state-dependent
setting, the case where E ζz → 1 as z → ∞ is thus of particular interest. So
from now on we suppose that the offspring distribution is given by
ζz = 1 + m z + θz ,
where, for a deterministic mz and a random θz , mz ∼ c/z, E θz = 0 and
E[θz2 ] → d, for constants c ∈ R and d > 0.
In this case, given Zn = z we have
Dn := Zn+1 − Zn =
z X
mz +
θzn,k
= zmz +
k=1
z
X
θzn,k ,
k=1
where θzn,k , n ≥ 1, z ≥ 1, are independent copies of θz , representing the
offspring of individual k in generation n. We compute directly,
E[Dn | Zn = z] = c + o(1);
E[Dn2 | Zn = z] = zd + o(z);
E[Dn4 | Zn = z] = O(z 2 ).
A useful change of scale is provided by f (z) = z 1/2 . Then, Xn = f (Zn ) is a
Markov chain, and, given Xn = x (so Zn = x2 ), by Taylor’s formula,
Xn+1 − Xn =
Dn
D2
− n3 + O(|Dn |3 x−5 ).
2x
8x
Here, by Lyapunov’s inequality,
E[|Dn |3 | Zn = x2 ] ≤ E[Dn4 | Zn = x2 ]
3/4
= O(x3 ).
It follows that, with the notation at (3.1),
2xµ1 (x) = c −
µ2 (x) =
d
+ o(1);
4
d
+ o(1).
4
In this example the condition (L2) does not hold, so we cannot apply Theorems 3.5.1 and 3.5.2 directly, but it is not hard to adapt the proofs to this
setting (Theorems 3.5.6 and 3.5.8 are sufficient), and we see that
3.6. Irreducibility and regeneration
123
• if c > d/2, then P[Xn → ∞] > 0;
• if c < d/2, then P[Xn → 0] = 1.
The boundary case can not be classified without sharper assumptions on the
rates of convergence of mz and E[θz2 ].
4
3.6
Irreducibility and regeneration
To translate results from the general semimartingale setting into the Markovian
setting, one needs to convert between the notions of recurrence and transience described in Section 3.5 and the usual recurrence and transience for
Markov chains involving visits to finite sets. This section explores conditions
under which the two descriptions are equivalent. In the Markov chain context, sufficient structure is provided by the (strong) Markov property and
irreducibility, although even here there are some subtleties over the necessary
nature of the state space. The most convenient general framework is in fact
not to assume a full Markov structure but a weaker notion of regeneration.
For most of this section, Xn will be an Fn -adapted process on a countable
state-space S, not assumed to be a subset of R+ . The form of irreducibility
that we will use is as follows.
(I) Suppose that for each x, y ∈ S there exist constants m(x, y) ∈ N
and ϕ(x, y) > 0, and a collection of Fn -measurable random variables
Mn (y) ∈ Z+ , such that, for all n ≥ 0, Mn (y) ≤ m(Xn , y), a.s., and
P[Xn+Mn (y) = y | Fn ] ≥ ϕ(Xn , y), a.s.
(3.36)
In words, assumption (I) says that for every y, given Fn , the process visits y
with probability bounded below, in a number of steps bounded above, where
the bounds depend on Xn and y. A variant of (I) is the following.
(I0 ) Suppose that for each x, y ∈ S there exist constants m(x, y) ∈ N and
ϕ(x, y) > 0 such that, for all n ≥ 0,
i
h
m(X ,y)
P ∪k=0 n {Xn+k = y} Fn ≥ ϕ(Xn , y), a.s.
Proposition 3.6.1. Conditions (I) and (I0 ) are equivalent.
124
Chapter 3. Lamperti’s problem
Proof. Assuming (I), we have that
i
h
m(X ,y)
P ∪k=0 n {Xn+k = y} Fn ≥ P Xn+Mn (y) = y, Mn (y) ≤ m(Xn , y) Fn
which is bouned below by ϕ(Xn , y), giving (I0 ). On the other hand, assuming (I0 ), take
Mn (y) = arg max P[Xn+k = y | Fn ],
0≤k≤m(Xn ,y)
which is Fn -measurable and has Mn (y) ≤ m(Xn , y), a.s. Then
P Xn+Mn (y) = y Fn =
max
P[Xn+k = y | Fn ]
0≤k≤m(Xn ,y)
i
h
1
m(X ,y)
P ∪k=0 n {Xn+k = y} Fn
1 + m(Xn , y)
ϕ(Xn , y)
=: ϕ0 (Xn , y),
≥
1 + m(Xn , y)
≥
with ϕ0 (x, y) > 0 for all x, y, which gives (I) after a relabelling.
If Xn is a time-homogeneous Markov process on a countable state-space
S, then (3.36) reduces to the usual sense of irreducibility that, for any x, y ∈
S, there exists m(x, y) ∈ N such that P[Xm(x,y) = y | X0 = x] > 0. The
assumption (3.36) allows us to work with more general processes, such as
functions of Markov process, as described in the next example.
Example 3.6.2. Suppose that ξn is an irreducible Markov chain on a countable state-space Σ, and that for some measurable function f : Σ → R+ ,
Xn = f (ξn ). Set Fn = σ(ξ0 , . . . , ξn ). Then Xn is Fn -adapted and has the
countable state-space S = f (Σ). By irreducibility, given u, v ∈ Σ there exist
k(u, v) ∈ N and ω(u, v) > 0 such that P[ξn+k(u,v) = v | ξn = u] ≥ ω(x, y).
Fix y ∈ S; choose and fix an arbitrary v ∈ f −1 (y) := {u ∈ Σ : f (u) = y}.
For x ∈ S, let
ϕ(x, y) =
inf
u∈Σ:f (u)=x
ω(u, v), and Mn (y) = k(ξn , v).
Then Mn (y) is Fn -measurable, and
P[Xn+Mn (y) = y | ξn ] ≥ P[ξn+k(ξn ,v) = v | ξn ] ≥ ω(ξn , v)
≥ ϕ(Xn , y),
which is (3.36). Here Mn (y) ≤ m(Xn , y) where
m(x, y) =
sup
u∈f −1 (x),v∈f −1 (y)
k(u, v).
3.6. Irreducibility and regeneration
125
Assuming that f −1 (x) is finite for each x, we have m(x, y) < ∞ and
ϕ(x, y) > 0. Hence (I) is satisfied for Xn and this countable S ⊆ R+ .
In particular, if Σ ⊆ Rd is locally finite and f (x) = kxk for x ∈ Σ, then
for x ∈ S = f (Σ) we have f −1 (x) = {x ∈ Σ : kxk = x} is finite, and so
Xn = kξn k satisfies (I).
4
We first note the following basic result. Here ‘i.o.’ and ‘f.o.’ stand for
‘infinitely often’ and ‘finitely often’, respectively.
Lemma 3.6.3. Suppose that (I) holds for a countable S.
(i) Let R ⊂ S be finite and non-empty. Then, up to sets of probability 0,
{Xn ∈ R i.o.} ⊆ ∩S⊆S, 0<#S {Xn ∈ S i.o.}.
(ii) Let R ⊂ S be non-empty. Then, up to sets of probability 0,
{Xn ∈ R f.o.} ⊆ ∩S⊆S, 0<#S<∞ {Xn ∈ S f.o.}.
Hence, a.s., either Xn ∈ R i.o. for all finite non-empty R ⊂ S, or for none.
Proof. First we prove part (i). Let R ⊂ S be finite. Suppose that Xn ∈
R i.o., and let y ∈ S. Then, since R is finite and non-empty, we may choose
x ∈ R (any x ∈ R will do) such that Xn = x i.o.; by (3.36) there exist
stopping times n1 < n2 < · · · with ni+1 > ni + m(x, y) such that Xni = x
and, for all i,
P[Xni +Mni (y) = y | Fni ] ≥ ϕ(x, y) > 0, and {Xni +Mni (y) = y} ∈ Fni+1 ,
since Mni (y) ≤ m(x, y). Then Lévy’s extension of the Borel–Cantelli lemma
(Theorem 2.3.19) shows that Xn = y i.o., a.s. So Xn = y i.o. for all y in
the countable set S. In particular, Xn ∈ S i.o. for any non-empty S ⊆ S, as
claimed.
Next we prove part (ii). Let R ⊂ S be non-empty. Suppose Xn ∈ R f.o.
If there exists S a finite non-empty subset of S for which Xn ∈ S i.o., then
part (i) would say that Xn ∈ R i.o., a.s., giving a contradiction. So part (ii)
follows.
The next result returns to the case S ⊆ R+ , and relates the present notion of irreducibility to the earlier local escape condition for non-confinement
given in Section 3.3, assuming that S is locally finite.
Lemma 3.6.4. Suppose that (L0) and (I) hold for a locally finite S ⊂ R+ .
Then (L3) holds.
126
Chapter 3. Lamperti’s problem
Proof. Since S is locally finite, 0 = inf S ∈ S. Fix x ∈ R+ ; then, since S is
locally finite and contains 0, S∩[0, x] is finite and non-empty. Choose y ∈ S∩
(x, ∞). Take δx = minz∈S∩[0,x] ϕ(z, y) > 0 and rx = maxz∈S∩[0,x] m(z, y) ∈
N, with m and ϕ as given by (3.36). Then, by (3.36), on {Xn ∈ S ∩ [0, x]},
i
h
P
max Xk ≥ x Fn ≥ P[Xn+Mn (y) = y | Fn ]
n≤k≤n+rx
≥ ϕ(Xn , y) ≥ δx ,
as required for (L3).
In particular, under the conditions of Lemma 3.6.4, Proposition 3.3.4
applies, so that the non-confinement condition (L2) holds. In this section
we explore neighbouring results, including a direct proof of this implication,
included in the next lemma.
Lemma 3.6.5. Suppose that (L0) and (I) hold for a countable S ⊂ R+ .
Then for any (hence every) finite, non-empty R ⊂ S, the following equality
holds up to sets of probability 0:
{Xn ∈ R i.o.} = lim inf Xn = 0, lim sup Xn = ∞ .
(3.37)
n→∞
n→∞
If, in addition, S is locally finite, then for any (hence every) finite, nonempty R ⊂ S, the following equality holds up to sets of probability 0:
{Xn ∈ R f.o.} = lim Xn = ∞ .
(3.38)
n→∞
In particular, if S is locally finite, then
P {Xn → ∞} ∪ lim inf Xn = 0, lim sup Xn = ∞ = 1.
n→∞
n→∞
Proof. Suppose that S is countable. By Lemma 3.6.3, there are two possibilities, up to events of probability 0: either (i) Xn ∈ R i.o. for any non-empty
R ⊂ S; or (ii) Xn ∈ R f.o. for any finite non-empty R ⊂ S.
First suppose case (i) occurs. Then, for any x ∈ S, Xn = x i.o., so that
lim inf n→∞ Xn ≤ x ≤ lim supn→∞ Xn ; since x ∈ S was arbitrary, together
with the obvious bound Xn ≥ 0, we obtain
lim inf Xn = inf S = 0, and lim sup Xn = sup S = ∞.
n→∞
This establishes (3.37).
n→∞
3.6. Irreducibility and regeneration
127
On the other hand, suppose case (ii) occurs, and that S is locally finite; in
particular inf S = 0 ∈ S in this case. Now, for any K ∈ R+ , RK = S ∩ [0, K]
is finite and non-empty, and so Xn ∈ RK f.o. Hence
lim inf Xn ≥ K.
n→∞
Since K was arbitrary, it follows that limn→∞ Xn = ∞. This establishes
(3.38). Combining (3.37) and (3.38) we deduce the final statement in the
lemma.
The next ingredient will be a notion of regeneration, for which we need to
introduce notation for the excursions of Xn . Consider a general countable
state-space S, with an identified state 0 ∈ S. For convenience, we will
assume X0 = 0; we consider excursions away from state 0.
Set ν0 := 0 and for k ∈ N define
νk := min{n > νk−1 : Xn = 0},
with the usual convention that min ∅ = ∞. That is, ν0 , ν1 , ν2 , . . . are the
successive times of visit to 0 by Xn . Let N := min{k ∈ N : νk = ∞}; if
N = ∞ then all the νk are finite and Xn visits 0 infinitely often. Provided
νk−1 < ∞, we denote the kth excursion (k ∈ N) by Ek := (Xn )νk−1 ≤n<νk .
For k ∈ N, let κk := νk − νk−1 if νk−1 < ∞ and let κk = ∞ if νk−1 = ∞;
when νk−1 is finite, κk is the duration of the kth excursion.
(Ra) Suppose that, for all k ∈ N, the distribution of Ek on {νk−1 < ∞} is
the same as the distribution of E1 .
(Rb) Suppose that, on {N = ∞}, (Ek )k∈N is an i.i.d. sequence.
(R) Suppose that X0 = 0 and both (Ra) and (Rb) hold.
Example 3.6.6. In the setting of Example 3.6.2, suppose that S = f (Σ)
has 0 ∈ S. If f −1 (0) is a singleton, then the strong Markov property for
the underlying Markov chain ξn yields the regenerative structure required
for (R).
4
Our irreducibility and regenerative assumptions have the following basic
consequence, which includes the recurrence/transience dichotomy in this
setting.
Lemma 3.6.7. Suppose that (I) and (Ra) hold for a countable S with 0 ∈ S.
Then either:
128
Chapter 3. Lamperti’s problem
(i) P[κ1 < ∞] < 1 and P[Xn ∈ R f.o.] = 1 for every finite, non-empty
R ⊂ S; or
(ii) P[κ1 < ∞] = 1 and P[Xn ∈ R i.o.] = 1 for every non-empty R ⊂ S.
Proof. A consequence of (Ra) is that
P[κk = n | νk−1 < ∞] = P[κ1 = n], for all n ∈ Z+ .
(3.39)
It follows from (3.39), with a repeated conditioning argument, that
P[N > k] = P[κk < ∞, νk−1 < ∞, . . . , ν1 < ∞]
=
k
Y
j=1
P[κj < ∞ | νj−1 < ∞] = (P[κ1 < ∞])k .
If P[κ1 < ∞] < 1, this implies that N < ∞ a.s., so that Xn = 0 f.o.,
a.s. The statement in part (i) follows from Lemma 3.6.3(ii). On the other
hand, if P[κ1 < ∞] = 1 we have that P[N > k] = 1 for any k, so N = ∞
a.s. and hence Xn = 0 i.o., and the statement in part (ii) follows from
Lemma 3.6.3(i).
In the case where S ⊆ R+ is locally finite, the recurrence/transience
dichotomy reads as follows.
Theorem 3.6.8. Suppose that (L0) and (I) hold for a locally finite S ⊂ R+ .
Then lim supn→∞ Xn = +∞, a.s. If (Ra) also holds, then either:
(i) (transience) P[κ1 < ∞] < 1 and limn→∞ Xn = +∞, a.s.; or
(ii) (recurrence) P[κ1 < ∞] = 1 and lim inf n→∞ Xn = 0, a.s.
Proof. Lemma 3.6.5 shows that lim supn→∞ Xn = +∞, and also shows that
the statements (i) and (ii) in Lemma 3.6.7 translate into statements (i) and
(ii) in the theorem, in the case where S ⊂ R+ is locally finite.
Now we can also complete the proof of Theorem 2.1.9 (a close relative
of Theorem 3.6.8) that was deferred from Chapter 2.
Proof of Theorem 2.1.9. Let ξn be an irreducible time-homogeneous Markov
chain on Σ ⊂ Rd , unbounded and countable. Write r = inf y∈Σ kyk and set
Xn = kξn k − r and Fn = σ(ξ0 , ξ1 , . . . , ξn ). Then (L0) and (I) hold, with
S = {kyk − r : y ∈ Σ} countable. The result now follows from Lemma 3.6.5,
since recurrence of the Markov chain ξn corresponds to (3.37) holding with
probability 1, while transience of ξn corresponds to (3.38) holding with probability 1.
3.6. Irreducibility and regeneration
129
The concepts of irreducibility and regeneration also play an important
role in the discussion of positive- and null-recurrence. For a countable statespace S, define for x ∈ S the occupation time (also called local time)
Ln (x) :=
n
X
m=0
1{Xm = x}, for n ∈ N.
Also define the occupation times during the kth excursion for x ∈ S by
`k (x) :=
νX
k −1
m=νk−1
1{Xm = x}, for k ∈ N.
The following ergodic theorem shows that our regeneration and irreducibility
assumptions imply that normalized occupation times converge; if the limit
is zero, the process is null, otherwise it is positive.
Theorem 3.6.9. Suppose that (I) and (R) hold for a countable S with
0 ∈ S. Then, for any x ∈ S,
(
E `1 (x)
1
E κ1 ∈ (0, 1) if E κ1 < ∞;
lim Ln (x) = πx :=
n→∞ n
0
if E κ1 = ∞;
the limit statement
holding a.s. and in Lq for any q ≥ 1. In the case E κ1 <
P
∞, we have x∈S πx = 1.
If Xn is a positive-recurrent, irreducible, time-homogeneous Markov process, then the πx in Theorem 3.6.9 is the usual (unique) stationary distribution; Theorem 3.6.9 thus extends (the Cesàro version of) Theorem 2.1.6.
The proof of Theorem 3.6.9 will be accomplished by a sequence of lemmas. The first simple lemma will also be used frequently later on. Recall
that τy = min{n ≥ 0 : Xn = y}.
Lemma 3.6.10. Suppose that (I) and (Ra) hold for a countable S with
0 ∈ S. Then P[κ1 > 1] > 0, and, for any y ∈ S \ {0}, there exists c(y) > 0
so that
P[τy < κ1 | F0 ] = c(y), a.s.
Proof. First suppose, for the purpose of deriving a contradiction, that P[κ1 =
1] = 1. Then it follows from an induction on (3.39) that P[κk = 1] = 1 for all
k, which implies that Xn = 0 for all n, a.s. But this contradicts Lemma 3.6.3.
Hence P[κ1 > 1] > 0.
130
Chapter 3. Lamperti’s problem
Next the irreducibility assumption (3.36) implies that for any k,
P[Xνk +Mνk (y) = y | Fνk ] ≥ ϕ(0, y), on {νk < ∞}.
(3.40)
By the regenerative assumption (Ra), P [hit y before returning to 0 | Fνk ] is
constant on {νk < ∞}; call this probability c(y). Then, an upper bound on
the probability that Xn ever visits y is
P
∞
h[
k=0
∞
i X
({νk < ∞} ∩ {hit y between νk and νk+1 }) ≤
c(y).
k=0
Thus if c(y) = 0, the probability of eventually hitting y is also 0, which
contradicts (3.40). Hence c(y) > 0.
Lemma 3.6.10 shows that any x ∈ S \ {0} is visited at least once during
an excursion with positive probability. The next key result shows that the
expected number of visits is finite.
Lemma 3.6.11. Suppose that (I) and (R) hold for a countable S with 0 ∈ S.
Suppose that P[κ1 < ∞] = 1. Then, for any x ∈ S, `k (x), k ≥ 1, are
i.i.d. with E `1 (x) ∈ (0, ∞).
Proof. Assuming P[κ1 < ∞] = 1, the regenerative assumption (R) shows
that Xn = 0 i.o., and hence Lemma 3.6.3 shows that Xn = x i.o. also for
any x ∈ S. Also, (R) shows that the `k (x) are i.i.d.
The fact that E `1 (x) < ∞ will follow from irreducibility. Indeed, fix
x ∈ S. Write m = m(x, 0) and ϕ = ϕ(x, 0), the constants given by (I). Set
τ1 = min{n ≥ 0 : Xn = x} and for k ≥ 2 set τk = min{n > τk−1 + m : Xn =
x}, so that τ1 , τ2 , . . . are a subsequence of the times of successive visits to x.
(Note that τk < ∞ for all k, since Xn = x i.o.) Set
Jx = min{k ≥ 1 : Xτk +Mτk (0) = 0}.
Recall that Mτk (0) ≤ m a.s. Then Xn must visit 0 by time τJx + m, and by
that time Xn has visited x at most (m + 1)Jx times, since for each visit to
x recorded as some τk , other (unrecorded) visits may occur anywhere in the
interval [τk , τk + m]. Hence `1 (x) ≤ (m + 1)Jx , a.s. However, by (I) applied
at time τ1 ,
P[Jx ≥ 2 | Fτ1 ] ≤ P[Xτ1 +Mτ1 (0) 6= 0 | Fτ1 ] ≤ 1 − ϕ, a.s.,
and, since {Jx ≥ 2} ∈ Fτ2 by the fact that τ1 + Mτ1 (0) ≤ τ2 , we have
P[Jx ≥ 3 | Fτ1 ] = E[1{Jx ≥ 2} P[Jx ≥ 3 | Fτ2 ] | Fτ1 ]
3.6. Irreducibility and regeneration
131
= E[1{Jx ≥ 2} P[Xτ2 +Mτ2 (0) 6= 0 | Fτ2 ] | Fτ1 ]
≤ (1 − ϕ)2 ,
by (I) applied at time τ2 . Iterating this argument gives P[Jx > k] ≤ (1 − ϕ)k
for any k ≥ 0, so in particular E `1 (x) ≤ (m + 1) E Jx < ∞.
Finally, we have E `1 (x) > 0 since `1 (0) ≥ 1, a.s., and, for x ∈ S \ {0},
E `1 (x) ≥ P[τx < κ1 ], which is positive by Lemma 3.6.10.
Let Nn := max{k ≥ 1 : νk ≤ n} denote the number of completed
excursions by time n ≥ 0.
Lemma 3.6.12. Suppose that (I) and (R) hold for a countable S with 0 ∈ S.
Then, a.s.,
(
1
∈ (0, 1) if E κ1 < ∞;
1
lim Nn = E κ1
n→∞ n
0
if E κ1 = ∞.
Proof. If P[κ1 = ∞] > 0, then limn→∞ Nn = N −1 < ∞, a.s., and n−1 Nn →
0, trivially. So suppose that P[κ1 < ∞] = 1.
The result is essentially
an inversion of the strong law of large numbers
Pk
for νk = νk − ν0 = `=1 κ` , where κ1 , κ2 , . . . are i.i.d., by (R).
Suppose first that E κ1 < ∞; then, since P[κ1 > 1] > 0 (see Lemma 3.6.10)
this means E κ1 ∈ (1, ∞). Then the strong law of large numbers shows that
k −1 νk → E κ1 , a.s. Hence for any ε ∈ (0, E κ1 ), there exists an a.s.-finite
Kε such that νk ≥ k(E κ1 − ε) for all k ≥ Kε . Then if k ∈ N is such that
k(E κ1 − ε) > n for an integer n with n > Kε (E κ1 − ε), we have νk > n;
hence Nn ≤ E κn1 −ε for all n sufficiently large. Since ε > 0 was arbitrary, we
obtain
1
lim sup n−1 Nn ≤
∈ (0, 1).
E
κ1
n→∞
A similar argument in the other direction gives the corresponding lim inf
statement.
On the other hand, suppose that E κ1 = ∞; now the strong law implies
that k −1 νk → ∞, a.s. So, for any ε > 0, νk > k/ε for all k sufficiently large,
which yields lim supn→∞ n−1 Nn ≤ ε by a similar argument to above. Since
ε > 0 was arbitrary, the result follows.
Now we can complete the proof of Theorem 3.6.9.
Proof of Theorem 3.6.9. Suppose that P[κ1 = ∞] > 0. Then limn→∞ Ln (x) =
P
∞
−1
m=0 1{Xm = x} < ∞, a.s., by Lemma 3.6.7, so that n Ln (x) → 0.
132
Chapter 3. Lamperti’s problem
So suppose that P[κ1 < ∞] = 1. Lemma 3.6.11 and the strong law of
large numbers shows that
m
1 X
lim
`k (x) = E `1 (x), a.s.
m→∞ m
(3.41)
k=1
First note that, by definition of Nn and `k (x), a.s.,
Nn
X
k=1
`k (x) ≤ Ln (x) =
Nn
X
`k (x) +
k=1
n
X
m=νNn
1{Xm = x} ≤
NX
n +1
`k (x),
(3.42)
k=1
since νNn ≤ n < νNn +1 . In particular, the upper bound in (3.42) gives
NX
n +1
Nn + 1
1
1
`k (x).
Ln (x) ≤
·
n
n
Nn + 1
k=1
Hence, since Nn → ∞ a.s., it follows from (3.41) that
1
Nn + 1
lim sup Ln (x) ≤ E `1 (x) lim sup
, a.s.
n
n
n→∞
n→∞
(3.43)
If E κ1 = ∞, Lemma 3.6.12 shows that the limit on the right-hand side of
(3.43) is zero, so that n−1 Ln (x) → 0, a.s. Suppose now that E κ1 < ∞. In
this case, (3.43) with Lemma 3.6.12 shows that
1
E `1 (x)
lim sup Ln (x) ≤
, a.s.
E κ1
n→∞ n
In the other direction, the lower bound in (3.42) gives
Nn
1
Nn 1 X
Ln (x) ≥
·
`k (x),
n
n Nn
k=1
so that by Lemma 3.6.12 and (3.41),
lim inf
n→∞
1
E `1 (x)
Ln (x) ≥
, a.s.
n
E κ1
This completes the proof.
To end this section, we state a technical lemma on almost-sure bounds
for sums and maxima of i.i.d. random variables; under the regenerative
assumption, these results will be useful for our analysis of excursions of
Lamperti-type processes presented later in this chapter.
133
3.6. Irreducibility and regeneration
Lemma 3.6.13. Let ζ1 , ζ2 , . . . be i.i.d. R+ -valued random variables.
(i) Suppose that, for some θ ∈ (0, 1) and ϕ ∈ R,
lim sup(xθ (log x)−ϕ P[ζ1 ≥ x]) < ∞.
(3.44)
x→∞
Then, for any ε > 0, a.s., for all but finitely many n,
n
X
i=1
1
ζi ≤ n θ (log n)
ϕ+1
+ε
θ
.
(ii) Suppose that, for some θ ∈ (0, ∞) and ϕ ∈ R,
lim inf (xθ (log x)−ϕ P[ζ1 ≥ x]) > 0.
x→∞
(3.45)
Then, for any ε > 0, a.s., for all but finitely many n,
1
max ζi ≥ n θ (log n)
1≤i≤n
ϕ−1
−ε
θ
.
Proof. Part (i) belongs to a family of classical results related to the strong
laws of large numbers of Marcinkiewicz and Zygmund (see e.g. [150, p. 73]):
it follows from Theorem 2 of Feller [97] (see also [200, p. 253] for a more
general result).
For part (ii), we have from (3.45) that for some c > 0 and all x large
enough, P[ζ1 ≥ x] ≥ cx−θ (log x)ϕ , so that
n
P max ζi < x ≤ 1 − cx−θ (log x)ϕ .
1≤i≤n
Taking x = n1/θ (log n)q we obtain
h
n
i P max ζi < n1/θ (log n)q ≤ 1 − cn−1 (log n)ϕ−θq (1 + o(1))
1≤i≤n
= O exp −c(log n)ϕ−θq (1 + o(1)) ,
which is summable over n ≥ 2 if q < (ϕ − 1)/θ; now use the Borel–Cantelli
lemma.
134
3.7
Chapter 3. Lamperti’s problem
Moments and tails of passage times
Recall that for x ≥ 0, λx = min{n ≥ 0 : Xn ≤ x}. In the recurrent
cases identified in Section 3.5, we have that λx < ∞ a.s., at least for all x
sufficiently large. To quantify recurrence, it is natural to ask for more information about these hitting times, such as tail or moments bounds. In
this section we use the general results from Section 2.7 to investigate this
question. We start with a result on existence of moments.
Theorem 3.7.1. Suppose that (L0) holds. Suppose that, for some α > 0,
(L1) holds with p > max{2, 2α}, and that lim supx→∞ µ̄2 (x) < ∞. Suppose
also that one of the following two conditions holds.
(i) We have α ∈ (0, 21 ] and lim supx→∞ 2xµ̄1 (x) − (1 − 2α)µ2 (x) < 0.
(ii) We have α ∈ ( 12 , ∞) and lim supx→∞ 2xµ̄1 (x) + (2α − 1)µ̄2 (x) < 0.
Then there exists γ0 ∈ (2α, p) and x0 ∈ R+ for which, for any γ ∈ (2α, γ0 ),
any s ∈ (0, γ/2), and any x ≥ x0 , E[λsx | F0 ] ≤ CX0γ , where C = C(γ, s) <
∞. In particular, for any x ≥ x0 , E[λαx ] < ∞.
Now we turn to non-existence of moments. The next result also gives a
lower tail bound.
Theorem 3.7.2. Suppose that (L0) and (L2) hold. Suppose that, for some
β > 0, (L1) holds with p > max{2, 2β}, and that lim supx→∞ µ̄2 (x) < ∞.
Suppose also that one of the following two conditions holds.
(i) We have β ∈ (0, 21 ] and lim inf x→∞ 2xµ1 (x) − (1 − 2β)µ̄2 (x) > 0.
(ii) We have β ∈ ( 12 , ∞) and lim inf x→∞ 2xµ1 (x) + (2β − 1)µ2 (x) > 0.
Then there exists x0 ∈ R+ such that for any x ≥ x0 , E[λβx | F0 ] = ∞ on
{X0 > x}. Moreover, for any x1 > x0 and any x < x1 , there is a constant
c = c(x0 , x1 , x, β) > 0 such that, for all n sufficiently large,
P[λx ≥ n | F0 ] ≥ cn−β , on {X0 = x1 }.
The following more specific result gives a sufficient condition to ensure
that E λx = ∞ which is sharper than the β = 1 case of Theorem 3.7.2.
Theorem 3.7.3. Suppose that (L0) and (L2) hold. Suppose that (L1) holds
with p > 2, and that lim supx→∞ µ̄2 (x) < ∞. Suppose also that there exist
3.7. Moments and tails of passage times
135
x0 ∈ (0, ∞), v ∈ (0, ∞), and θ ∈ (0, 1) such that for all x ≥ x0 , µ2 (x) ≥ v
and
1−θ
2xµ1 (x) + 1 +
µ2 (x) ≥ 0.
log x
Then there exists x0 ∈ R+ such that, for any x ≤ x0 , E[λx | F0 ] = ∞ on
{X0 > x}. Moreover, for any ν > 1 − θ there exists x0 ∈ R+ such that, for
any x1 > x0 and any x < x1 , there is a constant c = c(x0 , x1 , x, ν, θ) > 0
such that, for all n sufficiently large,
P[λx ≥ n | F0 ] ≥ cn−1 (log n)−ν , on {X0 = x1 }.
Note that Theorem 3.7.3 is a considerably stronger result than one can
obtain from Theorem 2.6.10; one would like to apply the latter result to Xn2
satisfying the conditions of Theorem 3.7.3, but the moments condition in
Theorem 2.6.10 will not be satisfied.
Example 3.7.4. As in Example 3.5.4, let (ξn , n ≥ 0) be a time-homogeneous
Markov chain on state space Σ ⊆ R2 with increments θn := ξn+1 − ξn such
that kθn k ≤ B, a.s., for some B ∈ R+ , and, for all x ∈ Σ,
E[θn | ξn = x] = 0, and E[θn θn> | ξn = x] = I2 ,
the 2 by 2 identity matrix. In Example 3.5.4, we saw that ξn is recurrent.
Moreover, for Xn = kξn k,
lim inf 2xµ1 (x) − (1 − 2β)µ̄2 (x) = 1 − (1 − 2β) > 0,
x→∞
for any β > 0, so Theorem 3.7.2 shows that, for suitable x1 > 0, E[λβx ] = ∞
for any kξ0 k > x1 > x. Example 2.7.9 gave a version of this conclusion for
symmetric SRW on Z2 .
4
The rest of this section is devoted to the proofs of the preceding theorems.
The first step is to apply the results of Section 2.7 with suitable Lyapunov
functions. First we work towards a proof of Theorem 3.7.1.
Lemma 3.7.5. Suppose that (L0) holds. Suppose that, for some α > 0,
(L1) holds with p > max{2, 2α}, and that lim supx→∞ µ̄2 (x) < ∞. Suppose
also that one of the following two conditions holds.
(i) We have α ∈ (0, 21 ] and lim supx→∞ 2xµ̄1 (x) − (1 − 2α)µ2 (x) < 0.
(ii) We have α ∈ ( 21 , ∞) and lim supx→∞ 2xµ̄1 (x) + (2α − 1)µ̄2 (x) < 0.
136
Chapter 3. Lamperti’s problem
There exist γ0 ∈ (2α, p), x0 ∈ R+ , and ε0 > 0 such that, for any γ ∈ (2α, γ0 ),
γ
E[Xn+1
− Xnγ | Fn ] ≤ −ε0 Xnγ−2 , on {Xn ≥ x0 }.
Proof. In this case, for any γ ∈ (0, p), (3.18) gives
γ
2Xn E[∆n | Fn ] + (γ − 1) E[∆2n | Fn ] Xnγ−2
2
γ−2
n
+ oF
Xn (Xn ),
γ
E[Xn+1
− Xnγ | Fn ] =
using the fact that lim supx→∞ µ̄2 (x) < ∞. First suppose condition (i) holds,
so that for any ε1 > 0 sufficiently small there exists x1 ∈ R+ such that
2xµ̄1 (x) − (1 − 2α)µ2 (x) < −ε1 for all x ≥ x1 . Take γ0 = 2α + ε2 ∈ (2α, p).
Then, for γ ∈ (2α, γ0 ),
2Xn E[∆n | Fn ] + (γ − 1) E[∆2n | Fn ]
≤ 2Xn µ̄1 (Xn ) − (1 − 2α)µ2 (Xn ) + ε2 µ̄2 (Xn )
≤ −ε1 + ε2 µ̄2 (Xn ),
on {Xn ≥ x1 }. Since lim supx→∞ µ̄2 (x) < ∞, we may choose ε2 > 0 small
enough so that this last display is less than −ε1 /2, say, on {Xn ≥ x1 },
uniformly for γ ∈ (2α, γ0 ). Hence
γ
E[Xn+1
− Xnγ | Fn ] ≤ −
ε1
γ−2
n
γXnγ−2 + oF
Xn (Xn ),
4
on {Xn ≥ x1 }. Since γ is bounded uniformly away from 0, there exist ε0 > 0
and x0 < ∞, not depending on γ ∈ (2α, γ0 ), such that
γ
− Xnγ | Fn ] ≤ −ε0 Xnγ−2 , on {Xn ≥ x0 }.
E[Xn+1
A similar argument applies if condition (ii) holds.
Proof of Theorem 3.7.1. Let γ0 , x0 , and ε0 be the constants in Lemma 3.7.5,
and let γ ∈ (2α, γ0 ). Then Lemma 3.7.5 shows that we may apply Theorem 2.7.1 with s0 = γ/2 to conclude that E[λsx | F0 ] ≤ CX0γ for any
s < γ/2 and any x ≥ x0 . In particular, since γ > 2α, we may take s = α to
conclude that E[λαx ] < ∞.
Next we turn to the non-existence of moments results. We will obtain
Theorems 3.7.2 and 3.7.3 from applications of Lemma 2.7.7. Thus we need
lower bounds on the extent of an excursion away from a bounded set. The
following result is required for Theorems 3.7.2, and will also be useful in
Section 3.8 below.
3.7. Moments and tails of passage times
137
Lemma 3.7.6. Suppose that (L0) and (L2) hold. Suppose that, for some
β > 0, (L1) holds with p > max{2, 2β}, and that lim supx→∞ µ̄2 (x) < ∞.
Suppose also that one of the following two conditions holds.
(i) We have β ∈ (0, 12 ] and lim inf x→∞ 2xµ1 (x) − (1 − 2β)µ̄2 (x) > 0.
(ii) We have β ∈ ( 21 , ∞) and lim inf x→∞ 2xµ1 (x) + (2β − 1)µ2 (x) > 0.
Then there exist x0 ∈ R+ and a constant c = c(β) > 0 such that, for any
x ≥ x0 , for all y > x,
i
h
P max Xm ≥ y F0 ≥ c(X02β − x2β )y −2β , on {x < X0 < y}.
0≤m≤λx
y
, where f2β,0 is the Lyapunov funcProof. Write f = f2β,0 and f y = f2β,0
y
tion defined at (3.16) and f2β,0 is the truncated version defined at (3.27).
Lemma 3.4.3 shows that, for any y > e,
E[f y (Xn+1 ) − f y (Xn ) | Fn ]
2β−2
n
),
≥ β 2Xn E[∆n | Fn ] + (2β − 1) E[∆2n | Fn ] Xn2β−2 + oF
Xn (Xn
on {Xn ≤ y}, where the implicit constant in the error term does not depend
on y. Hence under either condition (i) or condition (ii) in the lemma, we
have that there exists x1 ∈ R+ such that, for any y > x1 ,
E[f y (Xn+1 ) − f y (Xn ) | Fn ] ≥ 0, on {x1 ≤ Xn ≤ y}.
Let x1 ≤ x < y, and set Yn = f y (Xn∧λx ∧σy ). Then Yn is a non-negative
submartingale, which is uniformly bounded by f (2y), and hence uniformly
integrable; moreover, (L2) implies that λx ∧ σy < ∞, a.s. Hence, by optional
stopping (Theorem 2.3.7) we have, on {x < X0 < y},
f (X0 ) = Y0 ≤ E[Yλx ∧σy | F0 ]
≤ f (2y) P[σy < λx | F0 ] + f (x) P[λx < σy | F0 ],
since f y is non-decreasing and bounded above by f (2y). Hence
P[σy < λx | F0 ] ≥
f (X0 ) − f (x)
f (X0 ) − f (x)
≥
,
f (2y) − f (x)
f (2y)
which yields the statement in the lemma.
138
Chapter 3. Lamperti’s problem
Proof of Theorem 3.7.2. Let x0 be the constant from Lemma 3.7.6. We
apply Lemma 2.7.7 with the excursion lower bound from Lemma 3.7.6 to
obtain, for any x ≥ x0 ,
i
1 h
P[λx ≥ n | F0 ] ≥ P max Xm ≥ Cn1/2 F0
2 0≤m≤λx
≥ c(X02β − x2β )n−β 1{x < X0 < Cn1/2 },
for constants c > 0 (depending on β) and C < ∞. It follows that, for some
c > 0 depending on β,
X
n−1 = ∞, on {X0 > x}.
E[λβx | F0 ] ≥ c(X02β − x2β )
n>X0 /C
Moreover, for any x1 > x0 , for any x < x1 , on {X0 = x1 },
2β −β
P[λx ≥ n | F0 ] ≥ P[λx∨x0 ≥ n | F0 ] ≥ c(x2β
,
1 − (x ∨ x0 ) )n
for all n ≥ n0 sufficiently large. This yields the result.
Now we work towards a proof of Theorem 3.7.3. First we need an excursion bound analogous to Lemma 3.7.6.
Lemma 3.7.7. Suppose that (L0) and (L2) hold. Suppose that (L1) holds
with p > 2, and that lim supx→∞ µ̄2 (x) < ∞. Suppose also that there exist
x0 ∈ (0, ∞), v ∈ (0, ∞), and θ ∈ (0, 1) such that for all x ≥ x0 ,
1−θ
µ2 (x) ≥ v, and 2xµ1 (x) + 1 +
µ2 (x) ≥ 0.
log x
Then for any ν > 1 − θ there exist x1 ∈ R+ and a constant c = c(ν, θ) > 0
such that, for any x ≥ x1 , for all y > x, on {x < X0 < y},
i
h
P max Xm ≥ y F0 ≥ c(X02 logν X0 − x2 logν x)y −2 (log y)−ν .
0≤m≤λx
y
y
Proof. Write f = f2,ν and f y = f2,ν
, where f2,ν and f2,ν
are defined at
(3.16) and (3.27) respectively. Take ν > 1 − θ. Lemma 3.4.3 shows that, for
any y > 2e,
E[f y (Xn+1 ) − f y (Xn ) | Fn ]
≥ 2Xn E[∆n | Fn ] + E[∆2n | Fn ] logν Xn
ν
+
2Xn E[∆n | Fn ] + 3 E[∆2n | Fn ] logν−1 Xn
2
139
3.7. Moments and tails of passage times
ν−1
n
+ oF
Xn ),
Xn (log
on {Xn ≤ y}, where the implicit constant in the error term does not depend
on y. Using the fact that on {Xn ≥ x0 },
2Xn E[∆n | Fn ] + E[∆2n | Fn ] ≥ −
1−θ
µ (Xn ),
log Xn 2
we thus obtain
ν−1
n
E[f y (Xn+1 ) − f y (Xn ) | Fn ] ≥ ν − (1 − θ) + oF
Xn ,
Xn (1) µ2 (Xn ) log
which is non-negative for all Xn sufficiently large, by the choice of ν > 1 − θ
and the fact that µ2 is eventually positive. In other words, there exists
x1 ∈ R+ such that, for any y > x1 ,
E[f y (Xn+1 ) − f y (Xn ) | Fn ] ≥ 0, on {x1 ≤ Xn ≤ y}.
The remainder of the proof now follows the same lines as the proof of
Lemma 3.7.6.
Proof of Theorem 3.7.3. Choose ν ∈ (1 − θ, 1), and let x0 be as in the statement of Lemma 3.7.7. We apply Lemma 2.7.7 with the excursion lower
bound from Lemma 3.7.7 to obtain, for any x ≥ x0 , on {x < X0 < Cn1/2 },
i
1 h
P max Xm ≥ Cn1/2 F0
2 0≤m≤λx
≥ c(X02 log−ν X0 − x2 log−ν x)n−1 log−ν n,
P[λx ≥ n | F0 ] ≥
for constants c > 0 (depending on ν and θ) and C < ∞. It follows that, for
some c > 0, on {X0 > x},
E[λx | F0 ] ≥ c(X02 log−ν X0 − x2 log−ν x)
X
n>X0 /C
n−1 log−ν n = ∞,
as claimed. Moreover, for any x1 > x0 , on {X0 = x1 },
P[λx ≥ n | F0 ] ≥ P[λx∨x0 ≥ n | F0 ] ≥ cn−1 log−ν n,
for some constant c = c(x0 , x1 , x, θ, ν) > 0 and all n ≥ n0 sufficiently large.
140
3.8
Chapter 3. Lamperti’s problem
Excursion durations and maxima
The results in Section 3.7 deal with bounds for passage times into bounded
intervals, and the extent of trajectories up until such a passage time, for
general Lamperti processes started far enough away from the origin. In this
section we impose the additional structure provided by the regeneration and
irreducibility assumptions of Section 3.6, and obtain analogous results for
full excursions in that setting.
First we consider tail bounds for the excursion duration κ1 . The first
result is an analogue of the upper tail bound result for passage times into
bounded intervals, Theorem 3.7.1.
Theorem 3.8.1. Suppose that (L0), (I), and (R) hold for a locally finite
S ⊂ R+ . Suppose that, for some α > 0, (L1) holds with p > max{2, 2α},
and that lim supx→∞ µ̄2 (x) < ∞. Suppose also that one of the following two
conditions holds.
(i) We have α ∈ (0, 21 ] and lim supx→∞ 2xµ̄1 (x) − (1 − 2α)µ2 (x) < 0.
(ii) We have α ∈ ( 12 , ∞) and lim supx→∞ 2xµ̄1 (x) + (2α − 1)µ̄2 (x) < 0.
Then there exists ε > 0 for which P[κ1 ≥ n] = O(n−α−ε ). In particular,
E[κα1 ] < ∞.
Note that the case α = 1 of Theorem 3.8.1 generalizes the positiverecurrence result Theorem 3.2.3(iii). The analogue of the lower tail bound
result for passage times into bounded intervals, Theorem 3.7.2, is the following.
Theorem 3.8.2. Suppose that (L0), (I), and (R) hold for a locally finite
S ⊂ R+ . Suppose that, for some β > 0, (L1) holds with p > max{2, 2β},
and that lim supx→∞ µ̄2 (x) < ∞. Suppose also that one of the following two
conditions holds.
(i) We have β ∈ (0, 21 ] and lim inf x→∞ 2xµ1 (x) − (1 − 2β)µ̄2 (x) > 0.
(ii) We have β ∈ ( 12 , ∞) and lim inf x→∞ 2xµ1 (x) + (2β − 1)µ2 (x) > 0.
Then there exists c = c(β) > 0 such that P[κ1 ≥ n] ≥ cn−β for all n ≥ 1. In
particular, E[κβ1 ] = ∞.
We also present the following analogue of Theorem 3.7.3, which sharpens
the β = 1 case of Theorem 3.8.2.
141
3.8. Excursion durations and maxima
Theorem 3.8.3. Suppose that (L0), (I), and (R) hold for a locally finite
S ⊂ R+ . Suppose that (L1) holds with p > 2, and that lim supx→∞ µ̄2 (x) <
∞. Suppose also that there exist x0 ∈ (0, ∞) and θ ∈ (0, 1) such that for all
x ≥ x0 ,
1−θ
2xµ1 (x) + 1 +
µ2 (x) ≥ 0.
log x
Then for any ν > 1 − θ, there exists c = c(ν) > 0 such that P[κ1 ≥ n] ≥
cn−1 (log n)−ν for all n ≥ 2. In particular, E κ1 = ∞.
In this section we also present an almost-sure lower bound on the maximum of the process after a large number of excursions; the basis for this
result is Lemma 3.7.6.
Theorem 3.8.4. Suppose that (L0), (I), and (R) hold for a locally finite
S ⊂ R+ . Suppose that, for some β > 0, (L1) holds with p > max{2, 2β},
and that lim supx→∞ µ̄2 (x) < ∞. Suppose also that one of the following two
conditions holds.
(i) We have β ∈ (0, 21 ] and lim inf x→∞ 2xµ1 (x) − (1 − 2β)µ̄2 (x) > 0.
(ii) We have β ∈ ( 21 , ∞) and lim inf x→∞ 2xµ1 (x) + (2β − 1)µ2 (x) > 0.
Then for any ε > 0, a.s., for all but finitely many k,
1
max Xm ≥ k 2β (log k)
0≤m≤νk
1
− 2β
−ε
.
The proofs of the preceding theorems occupy the rest of this section. We
first need a technical result that says that, with uniformly positive probability, the process will reach the origin before exceeding a fixed level, provided
it is not too close to the starting point.
Lemma 3.8.5. Suppose that (L0), (I), and (R) hold for a locally finite
S ⊂ R+ . Suppose that E[∆n | Fn ] ≤ B, a.s., for some B ∈ R+ . Then for
any x ∈ R+ there exists y ∈ (x, ∞) such that, for all n ≥ 0,
P max Xm < y Fn ≥ δ, on {Xn ≤ x}.
n≤m≤κ1
Proof. Fix x ∈ R+ . Since S is locally finite and inf S = 0, S ∩ [0, x] is finite
and non-empty; by (I), m := maxy∈S∩[0,x] m(y, 0) and ϕ := miny∈S∩[0,x] ϕ(y, 0)
satisfy m < ∞ and ϕ > 0. Hence P[κ1 ≤ n + m | Fn ] ≥ ϕ on {Xn ≤ x}. In
142
Chapter 3. Lamperti’s problem
addition, the first moment bound in the lemma with the maximal inequality
in Theorem 2.4.7 gives
Bm + x
P
max Xk ≥ y Fn ≤
,
n≤k≤n+m
y
on {Xn ≤ x}. Choosing y = 2(Bm + x)/ϕ it follows that
Bm + x
ϕ
P {κ1 ≤ m}∩
max Xk < y Fn ≥ ϕ−
≥ , on {Xn ≤ x},
n≤k≤n+m
y
2
which gives the result.
We can now give the proof of Theorem 3.8.1.
Proof of Theorem 3.8.1. Lemma 3.7.5 shows that there exist γ0 ∈ (2α, p),
C ∈ R+ , and x0 < ∞ such that, for any γ ∈ (2α, γ0 ),
γ
− Xnγ | Fn ] ≤ C1{Xn < x0 }.
E[Xn+1
It follows from Theorem 2.4.8 that, for any x ∈ R+ ,
γ
E[Xn∧σ
| F0 ] ≤ X0γ + C E[σx | F0 ].
x
Moreover, Lemma 3.6.4 shows that (L0), (I), and the fact that S is locally
finite imply that (L3) holds; hence it follows from Proposition 3.3.4 that
E[σx | F0 ] ≤ Kx , a.s., for some Kx < ∞ depending on x. Hence
γ
E[Xn∧σ
| F0 ] ≤ X0γ + Kx , a.s.,
x
(3.46)
for some Kx < ∞ depending on x.
Now take x1 ∈ (x0 , ∞). Define stopping times λk and ρk recursively by
ρ0 = 1 and, for k ∈ N,
λk = min{n ≥ ρk−1 : Xn ≤ x0 }, and ρk = min{n ≥ λk : Xn ≥ x1 }.
Lemma 3.8.5 shows that we may choose x1 sufficiently large so that, for a
constant δ > 0,
P[κ1 < ρk | Fλk ] ≥ δ, a.s.
(3.47)
An application of Theorem 3.7.1 shows that, for all k ≥ 0,
E[(λk+1 − ρk )s | Fρk ] ≤ CXργk , a.s.,
143
3.8. Excursion durations and maxima
for any s ∈ (0, γ/2). Here, it follows from (3.46) and Fatou’s lemma that,
for all k ≥ 0,
E[Xργk | Fλk ] ≤ xγ0 + Kx1 , a.s.,
so that, for all k ≥ 0, E[(λk+1 − ρk )s ] ≤ C for some C < ∞ depending on s,
γ, x0 , and x1 .
Another application of Proposition 3.3.4 shows that, for all k ≥ 1, E[(ρk −
s
λk ) ] ≤ C for some C < ∞ depending on γ and x1 . Hence, by Minkowski’s
inequality, for all k ≥ 1, E[(λk+1 − λk )s ] ≤ C for some C < ∞ depending on
s, γ, x0 , and x1 . Then, by Markov’s inequality, for all n ≥ 0 and all k ≥ 1,
P[λk+1 − λk ≥ n] ≤ Cn−s .
Let N = min{k ≥ 0 : ρk > κ1 }. Then P[N > 1] = E[P[κ1 > ρ1 | Fλ1 ]] ≤
1 − δ, by (3.47). Similarly, two applications of (3.47) show that
P[N > 2] = E P[κ1 > ρ2 | Fλ2 ]1{κ1 > ρ1 } ≤ (1 − δ)2 ;
iterating this argument gives P[N > k] ≤ (1 − δ)k ≤ e−δk for all k ≥ 0.
Moreover,
by definition of N , we have κ1 < ρN ≤ λN +1 , so that κ1 ≤
PN
(λ
k=1 k+1 − λk ). Hence
 ε

bn c
X
P[κ1 ≥ n] ≤ P[N ≥ nε ] + P 
(λk+1 − λk ) ≥ n
k=1
−δnε
≤e
ε
+ n sup P[λk+1 − λk ≥ n1−ε ].
k
Thus for any ε > 0 and any s ∈ (0, γ/2), P[κ1 ≥ n] = O(nε−(1−ε)s ). Since
γ > 2α, we may choose s > α and then choose ε > 0 small enough to see
that P[κ1 ≥ n] = O(n−α−ε ).
The proofs of Theorems 3.8.2 and 3.8.3 are accomplished by converting
lower bounds on the return time to a bounded interval starting far enough
away from the origin (given in Theorems 3.7.2 and 3.7.3 respectively) into
lower bounds for the duration of excursions away from the origin, with the
aid of Lemma 3.6.10.
Proof of Theorem 3.8.2. Recall that τy = min{n ≥ 0 : Xn = y}. Theorem 3.7.2 shows that there exists x0 ∈ R+ such that, for any x1 > x0 , there
is a constant c = c(x0 , x1 , β) > 0 such that, for all n ≥ n0 sufficiently large,
P[λx0 ≥ n | F0 ] ≥ cn−β , on {X0 = x1 }.
144
Chapter 3. Lamperti’s problem
It follows that, for n ≥ n0 ,
P[κ1 ≥ n] ≥ E P[κ1 ≥ n | Fτx1 ]1{τx1 < κ1 }
≥ cn−β P[τx1 < κ1 ];
(3.48)
the result now follows from Lemma 3.6.10.
Proof of Theorem 3.8.3. Let ν > 1 − θ. Theorem 3.7.3 shows that there
exists x0 ∈ R+ such that, for any x1 > x0 , there is a constant c > 0 such
that, for all n sufficiently large,
P[λx0 ≥ n | F0 ] ≥ cn−1 log−ν n, on {X0 = x1 }.
Another application of (3.48) and Lemma 3.6.10 now completes the proof.
Finally, we can complete the proof of Theorem 3.8.4.
Proof of Theorem 3.8.4. Under the stated conditions, Lemma 3.7.6 applies;
let x0 ∈ R+ be the constant in the statement of Lemma 3.7.6. Recall that,
assuming (R), we have X0 = 0. Define for k ∈ Z+ ,
Mk :=
max
νk ≤m<νk+1
Xm ,
the maximum extent of the kth excursion. Then choose (and fix) x =
inf(S ∩ (x0 , ∞)); since S is locally finite, x > x0 .
P[M0 ≥ y] ≥ P {τx < κ1 } ∩
max Xm ≥ y
τx ≤m≤κ1
= E 1{τx < κ1 } P max Xm ≥ y Fτx .
τx ≤m≤κ1
Here, by Lemma 3.7.6, for all y > x,
P max Xm ≥ y Fτx ≥ c1 y −2β , a.s.,
τx ≤m≤κ1
for some constant c1 > 0 depending only on β and x, but not on y. Hence,
P[M0 ≥ y] ≥ c1 y −2β P[τx < κ1 ]. Also, Lemma 3.6.10 shows that P[τx <
κ1 ] = c2 for a constant c2 > 0 depending only on x. Hence there exists
c = c(β, x) > 0 such that
P[M0 ≥ y] ≥ cy −2β , for all y > x.
(3.49)
145
3.9. Almost-sure bounds on trajectories
By the regenerative assumption (R), the random variables M0 , M1 , M2 . . .
are i.i.d. and satisfy the tail bound (3.49). Hence an application of the
θ = 2β, ϕ = 0 case of Lemma 3.6.13(ii) shows that, for any ε > 0, a.s., for
all but finitely many k,
1
1
− 2β
−ε
max M` ≥ k 2β (log k)
0≤`≤k
.
The result now follows, since max0≤m≤νk Xm ≥ max0≤`≤k−1 M` .
3.9
Almost-sure bounds on trajectories
In this section we present almost-sure upper and lower bounds on the running maximum max0≤m≤n Xm for a Lamperti process Xn . We start with
the upper bounds, which we deduce from the results of Section 2.8. The
first result gives a general diffusive upper bound.
Theorem 3.9.1. Suppose that (L0) holds and that for some constant C ∈
R+ , E[∆2n | Fn ] ≤ C, a.s. Suppose also that lim supx→∞ (xµ̄1 (x)) < ∞.
Then for any ε > 0, a.s., for all but finitely many n ≥ 0,
1
1
max Xm ≤ n 2 (log n) 2 +ε .
0≤m≤n
Proof. Given that E[∆2n | Fn ] ≤ C, µ̄2 (x) and µ̄1 (x) are well defined and
uniformly bounded. We will apply Theorem 2.8.1 with f (x) = x2 . Here
E[f (Xn+1 ) − f (Xn ) | Fn ] = 2Xn E[∆n | Fn ] + E[∆2n | Fn ]
≤ 2Xn µ̄1 (Xn ) + µ̄2 (Xn ), a.s.,
which is bounded above by a finite constant, a.s., by assumption. Then
Theorem 2.8.1 yields the result.
The bound in Theorem 3.9.1 is not always sharp. In some cases, a
stronger sub-diffusive upper bound holds.
Theorem 3.9.2. Let α > 1. Suppose that (L0) holds and that (L1) holds
with p > 2α. Suppose also that lim supx→∞ µ̄2 (x) < ∞ and
lim sup 2xµ̄1 (x) + (2α − 1)µ̄2 (x) < 0.
x→∞
There exists ε > 0 such that, a.s., for all but finitely many n ≥ 0,
1
max Xm ≤ n 2α −ε .
0≤m≤n
146
Chapter 3. Lamperti’s problem
Remark 3.9.3. In the Markov chain setting, Theorem 3.9.2 applies in the
case of positive-recurrence. Indeed, the condition for positive-recurrence
given in Theorem 3.2.3(iii) is
lim sup 2xµ1 (x) + µ2 (x) < 0;
x→∞
since lim supx→∞ µ2 (x) < ∞, it follows that for some α > 1,
lim sup 2xµ1 (x) + (2α − 1)µ2 (x) < 0.
x→∞
Proof of Theorem 3.9.2. Let f (x) = xγ . Under the assumptions of the theorem, Lemma 3.7.5 shows that there exists γ > 2α for which f (Xn ) in
integrable and
E[f (Xn+1 ) − f (Xn ) | Fn ] < 0, on {Xn ≥ x0 },
for some x0 ∈ R+ . Thus we may apply Theorem 2.8.1 to give the result.
With additional regularity assumptions on the increment moment functions, a consequence of the preceding results in the following corollary, which
uses log-log scaling to identify (upper bounds on) scaling exponents.
Corollary 3.9.4. Suppose that (L0) holds, and that (L1) holds with p > 2.
Suppose also that, for some a, b ∈ R with b > 0,
lim µ2 (x) = lim µ̄2 (x) = b;
x→∞
x→∞
lim xµ1 (x) = lim xµ̄1 (x) = a.
x→∞
x→∞
(i) If 2a + b ≥ 0, then, a.s.,
lim sup
n→∞
1
log Xn
≤ .
log n
2
(ii) If 2a + b < 0 and (L1) holds with p >
lim sup
n→∞
b−2a
b ,
then, a.s.,
log Xn
b
≤
∈ 0, 12 .
log n
b − 2a
Proof. First of all, for all a and b, Theorem 3.9.1 implies that
max log Xm ≤
0≤m≤n
1
log n + O(log log n), a.s.,
2
147
3.9. Almost-sure bounds on trajectories
which gives part (i). For part (ii), suppose that 2a + b < 0. Then, since
b > 0, we have a < 0 and b − 2a > 2b > 0. Moreover, for α > 0,
lim sup 2xµ̄1 (x) + (2α − 1)µ̄2 (x) = 2a + (2α − 1)b < 0,
x→∞
provided α < b−2a
2b . Hence Theorem 3.9.2 shows that, a.s., for all but finitely
1
many n, max0≤m≤n Xm ≤ n 2α , so that
lim sup
n→∞
log Xn
1
≤
.
log n
2α
Since α ∈ (0, b−2a
2b ) was arbitrary, we obtain part (ii).
Obtaining lower bounds requires stronger assumptions, and it is most
convenient to use the concepts of irreducibility and regeneration introduced
in Section 3.6. Under these assumptions, we can complement the upper
bounds in Corollary 3.9.4 by corresponding lower bounds, to fully identify
scaling exponents.
Theorem 3.9.5. Suppose that (L0), (I), and (R) hold for a locally finite
S ⊂ R+ , and that (L1) holds with p > 2. Suppose that, for some a, b ∈ R
with b > 0,
lim µ2 (x) = lim µ̄2 (x) = b;
x→∞
x→∞
lim xµ1 (x) = lim xµ̄1 (x) = a.
x→∞
x→∞
(i) If 2a + b ≥ 0, then, a.s.,
lim sup
n→∞
log Xn
1
= .
log n
2
(ii) If 2a + b < 0 and (L1) holds with p >
lim sup
n→∞
b−2a
b ,
then, a.s.,
log Xn
b
=
∈ 0, 12 .
log n
b − 2a
Proof. To prove the equalities in parts (i) and (ii), it suffices to prove only
the corresponding ‘≥’ statements, since the ‘≤’ statements are given in Corollary 3.9.4 (under weaker conditions).
For part (i), suppose that 2a + b ≥ 0. Then, for α > 0,
lim sup 2xµ̄1 (x) + (2α − 1)µ̄2 (x) = lim sup 2xµ̄1 (x) + (1 − 2α)µ2 (x)
x→∞
x→∞
148
Chapter 3. Lamperti’s problem
= 2a + (2α − 1)b < 0,
provided α ∈ (0, b−2a
2b ); fix such an α (so α < 1, since 2a + b ≥ 0). Theorem 3.8.1 bounds applies, and shows that P[κ1 ≥ n] = O(n−α−ε ). An
application of Lemma 3.6.13(i) shows that νk ≤ k 1/α , a.s., for all but finitely many k. In other words, νbkα c ≤ k for all but finitely many k.
On the other hand, for β > 0,
lim inf 2xµ1 (x) + (2β − 1)µ2 (x) = lim inf 2xµ1 (x) − (1 − 2β)µ̄2 (x)
x→∞
x→∞
= 2a + (2β − 1)b > 0,
provided β > b−2a
2b ; fix such a β. Then, by Theorem 3.8.4(i), for any ε > 0,
a.s., for all but finitely many n,
max Xm ≥
0≤m≤n
max
0≤m≤νbnα−ε c
≥n
α−2ε
2β
.
It follows that, for any ε > 0, a.s.,
lim sup
n→∞
log Xn
α
≥
− ε,
log n
2β
which yields the ‘≥’ half of part (i), since ε > 0, α < b−2a
2b , and β >
were arbitrary.
For part (ii), suppose that 2a + b < 0. Then, for α > 0,
lim sup 2xµ̄1 (x) + (2α − 1)µ̄2 (x) = 2a + (2α − 1)b < 0,
b−2a
2b
x→∞
b−2a
provided α < b−2a
2b . Since 2a + b < 0, we may choose α ∈ (1, 2b ); fix such
an α. Theorem 3.8.1 bounds applies, and shows that E κ1 < ∞. Then the
strong law of large numbers shows that k −1 νk → E κ1 , a.s. Choosing δ > 0
so that δ E κ1 < 1, we have that νbδkc ≤ k, a.s., for all but finitely many k.
On the other hand, for β > 0,
lim inf 2xµ1 (x) + (2β − 1)µ2 (x) = 2a + (2β − 1)b > 0,
x→∞
provided β > b−2a
2b ; fix such a β (necessarily, β > α > 1). Then, by Theorem 3.8.4(ii), for any ε > 0, a.s., for all but finitely many n,
max Xm ≥
0≤m≤n
It follows that, a.s.,
lim sup
n→∞
max
0≤m≤νbδnc
1
≥ n 2β
−ε
.
log Xn
1
≥
− ε,
log n
2β
giving the ‘≥’ half of part (ii), since ε > 0 and β >
b−2a
2b
were arbitrary.
149
3.10. Transient theory in the critical case
3.10
Transient theory in the critical case
The main result of this section is the following diffusive lower bound on the
rate of escape in the transient case.
Theorem 3.10.1. Suppose that (L0) and (L3) hold, and that (L1) holds
with p > 2 and δ ∈ (0, p − 2]. Suppose also that there exists θ ∈ (0, δ) such
that
lim sup µ̄2 (x) < ∞, and lim inf 2xµ1 (x) − (1 + θ)µ̄2 (x) > 0.
x→∞
x→∞
(3.50)
Then for any ε > 0, a.s., for all but finitely many n ≥ 0,
1
1
1
Xn ≥ n 2 (log n)− 2 − θ −ε .
Remark 3.10.2. The conditions in Theorem 3.10.1 are the same as those
in the transience classification result, Theorem 3.5.1, except that the nonconfinement condition (L2) is replaced by the stronger local escape condition
(L3), which ensures that the process does not spend too much time near zero:
see also Remark 3.10.9 below. Note that if the Lamperti condition (3.29)
for transience from Theorem 3.5.1 holds, then (3.50) holds for some θ > 0.
We give two examples.
Example 3.10.3. Consider again Sn symmetric SRW on Zd , and let Xn =
kSn k and Fn = σ(S0 , . . . , Sn ). Take d ≥ 3 (the transient case). The calculations in Example 3.5.3 show that
2+θ
lim 2xµ1 (x) − (1 + θ)µ̄2 (x) = 1 −
,
x→∞
d
which is strictly positive if θ ∈ (0, d − 2). Since in this case (L1) holds with
δ = p − 2 for all p > 2, Theorem 3.10.1 shows that, for any ε > 0, a.s., all
but finitely often,
1
1
1
kSn k ≥ n 2 (log n)− 2 − d−2 −ε .
This result is not quite sharp. A theorem of Dvoretzky and Erdős [84] says
that the sharp bound is, for any ε > 0, a.s., all but finitely often,
1
1
kSn k ≥ n 2 (log n)− d−2 −ε ,
and this inequality fails infinitely often for ε = 0.
4
150
Chapter 3. Lamperti’s problem
Example 3.10.4. Continuing from Example 3.2.5, consider the simple random walk on Z+ with
1
c
p(x, x − 1) = px = 1 − p(x, x + 1) =
1+
+ o(x−1 ).
2
x
Then µ1 (x) = 1 − 2px = −(c + o(1))x−1 and µ2 (x) = 1. In the transient
case c < −1/2, we thus have that (3.50) holds for any θ ∈ (0, −1 − 2c), and
hence Theorem 3.10.1 shows that, for any ε > 0, a.s., all but finitely often,
1
1
1
Xn ≥ n 2 (log n)− 2 + 1+2c −ε .
Again, this result is not quite sharp: a result of Csáki et al. [65] shows that
4
the − 21 in the exponent of the logarithm can be dropped.
The rest of this section is devoted to the proof of Theorem 3.10.1. The
starting point is the following. Recall that σy = min{n ≥ 0 : Xn ≥ y} is the
first passage time into [y, ∞). To quantify the transience of the process Xn ,
we study
ηx := max{n ≥ 0 : Xn ≤ x},
(3.51)
the last exit time from [0, x]. Note that ηx is not a stopping time. For y > x,
let λy,x := min{n ≥ σy : Xn ≤ x} be the return time to [0, x] having reached
[y, ∞). Then we have, for any y > x,
P[ηx > n] = P[ηx > n, σy ≤ n] + P[ηx > n, σy > n].
Here
P[ηx > n, σy ≤ n] ≤ P[ηx > σy , σy ≤ n] ≤ P[σy ≤ n, λy,x < ∞].
Hence we obtain the simple upper bound, valid for any y > x,
P[ηx > n] ≤ P[λy,x < ∞] + P[σy > n].
(3.52)
Theorem 3.10.1 will follow from an analysis of the last exit estimate (3.52).
i
A decreasing Lyapunov function that gives a supermartingale outside a set provides a bound on the probability of
returning to that set having reached a more distant region.
An increasing Lyapunov function that gives a strict submartingale outside a bounded set provides an upper bound
on the hitting time of the distant region. Together these
estimates can be used to estimate the rate of escape for a
transient process.
151
3.10. Transient theory in the critical case
To bound P[σy > n] we use an extension of the idea of the ‘reverse Foster’
Theorem 2.4.11. The main shortcoming of Theorem 2.4.11 is the fact that
(2.19) does not permit an exceptional set. Also, to get a suitable bound on
the right-hand side of (2.20) we previously imposed a uniform upper bound
on the increments (see Theorem 2.8.3). In this section we address these
points; in fact, the second necessitates dealing with the first, which is the
purpose of the next result.
Lemma 3.10.5. Suppose that (L0) holds. Suppose that there exist a nondecreasing function f : R+ → R+ , and constants x0 ≥ 0 and ε > 0, for
which, for all n ≥ 0,
E[f (Xn+1 ) − f (Xn ) | Fn ] ≥ ε, on {Xn ≥ x0 }.
(3.53)
Then for any x > 0,
E σx ≤ ε−1 E f (Xσx ) + (1 + ε−1 f (x0 )) E
σx
X
n=0
1{Xn ≤ x0 }.
Proof. Let Yn = Xn∧σx . Note first that, on {Xm ≤ x0 } ∩ {m < σx },
E[f (Ym+1 ) − f (Ym ) | Fm ] ≥ −f (Ym ) = −f (Xm ) ≥ −f (x0 ),
since f is non-negative and non-decreasing. So, with K = ε + f (x0 ), we may
express (3.53) as
E[f (Ym+1 ) − f (Ym ) | Fm ] ≥ 1{m < σx } (ε − K1{Xm ≤ x0 }) .
(3.54)
The rest of the proof now follows closely the proof of Theorem 2.4.11; taking
expectations in (3.54) and summing for m from 0 up to n − 1 we get
E f (Yn ) ≥ ε
n−1
X
m=0
P[σx > m] − K E
∞
X
m=0
1{m < σx , Xm ≤ x0 }.
Rearranging this inequality and using the fact that f (Yn ) ≤ f (Xσx ) we get
n−1
X
m=0
σ
P[σx > m] ≤
x
1
K X
E f (Xσx ) +
E
1{Xm ≤ x0 }.
ε
ε
m=0
The statement in the lemma follows on taking n → ∞.
152
Chapter 3. Lamperti’s problem
To apply Lemma 3.10.5, one must also bound E f (Xσx ). This may be
done by assuming an upper bound on jumps as in Theorem 2.8.3, but such
an assumption is often too restrictive. The next result imposes a weaker
‘uniform f -integrability’ condition on the increments instead.
Lemma 3.10.6. Suppose that (L0) holds. Suppose that there exist a nondecreasing function f : R+ → R+ , and constants x0 ≥ 0 and ε > 0, for
which (3.53) holds for all n ≥ 0. Let δ > 0. Suppose that for any ν > 0
there exists y ∈ R+ such that, for all n ∈ Z+ ,
+
E[f ((1 + δ −1 )∆+
n )1{∆n > y} | Fn ] ≤ ν, a.s.
(3.55)
Then there is a constant x1 ≥ x0 such that for all x > x1 ,
E σx ≤ 2ε−1 f ((1 + δ)x) + 2(1 + ε−1 )f (x1 ) E
σx
X
n=0
1{Xn ≤ x1 }.
Remark 3.10.7. For example, if f (x) = xp for some p > 0, then a sufficient
condition for (3.55) is that there exist q > p and C < ∞ for which
q
E[(∆+
n ) | Fn ] ≤ C, a.s.,
(3.56)
for all n ≥ 0. Indeed, in this case,
+
−1 p
+ q
+ p−q
f ((1 + δ −1 )∆+
1{∆+
n )1{∆n > y} = (1 + δ ) (∆n ) (∆n )
n > y}
q p−q
≤ (1 + δ −1 )p (∆+
,
n) y
since p − q > 0, so that
+
−1 p p−q
E[f ((1 + δ −1 )∆+
,
n )1{∆n > y} | Fn ] ≤ C(1 + δ ) y
by (3.56). Hence (3.55) is satisfied.
Proof of Lemma 3.10.6. We use a truncation argument. Let
fx (y) := min{f (y), f ((1 + δ)x)},
where δ > 0 is as in the statement of the lemma. We will show that a version
of (3.53) holds with fx instead of f , up to modification of the constants ε
and x0 . Specifically, we claim that there is some x1 > x0 such that
E[fx (Xn+1 ) − fx (Xn ) | Fn ] ≥ ε/2, on {x1 ≤ Xn < x},
(3.57)
153
3.10. Transient theory in the critical case
for all x > x1 and all n ≥ 0.
Assuming the claim (3.57), we may apply Lemma 3.10.5, since (3.57)
implies that (3.54) now holds with f replaced by fx , x0 replaced by x1 ,
and ε replaced by ε/2. Then we conclude from Lemma 3.10.5 that, for all
x > x1 ,
E σx ≤ 2ε−1 E fx (Xσx ) + 2(1 + ε−1 )fx (x1 ) E
σx
X
n=0
1{Xn ≤ x1 }.
The statement of the lemma now follows since, by definition of fx , fx (Xσx ) ≤
f ((1 + δ)x) and fx (x1 ) = f (x1 ) for x > x1 .
It remains to verify the claim (3.57). Note first that, on {Xn < x},
fx (Xn+1 ) − fx (Xn )
= f (Xn+1 ) − f (Xn ) + (f ((1 + δ)x) − f (Xn+1 )) 1{Xn+1 > (1 + δ)x}
≥ f (Xn+1 ) − f (Xn ) − f (x + ∆n )1{∆n > δx},
using the fact that {Xn < x} ∩ {Xn+1 > (1 + δ)x} implies that ∆n > δx
and f (Xn + ∆n ) ≤ f (x + ∆n ). Hence, again using the fact that f is nondecreasing, on {Xn < x},
fx (Xn+1 ) − fx (Xn ) ≥ f (Xn+1 ) − f (Xn ) − f ((1 + δ −1 )∆n )1{∆n > δx}.
Here we have E[f (Xn+1 ) − f (Xn ) | Fn ] ≥ ε on {Xn ≥ x0 }. Also
E[f ((1 + δ −1 )∆n )1{∆n > δx} | Fn ] ≤ ε/2,
for all x > x1 sufficiently large, by (3.55). So we may choose x1 > x0 such
that (3.57) holds, as claimed.
Denote the renewal function associated to Xn by
H(x) := E
∞
X
n=0
1{Xn ≤ x} =
∞
X
n=0
P[Xn ≤ x].
(3.58)
Note that H(x) ≤ 1 + E ηx . It is important for application of Lemma 3.10.5
or Lemma 3.10.6 to show that H(x1 ) < ∞. The next result establishes this
in the present setting.
Lemma 3.10.8. Suppose that (L0) and (L3) hold, and that (L1) holds with
p > 2. Suppose also that (3.29) holds. Then, for any x ∈ R+ , E ηx < ∞.
154
Chapter 3. Lamperti’s problem
Remark 3.10.9. This is the only point in the proof of Theorem 3.10.1 where
the local escape condition (L3) (rather than just the fact that lim supn→∞ Xn =
∞) is used; presumably this condition is stronger than is necessary, but it
is not too restrictive.
Proof of Lemma 3.10.8. Fix x ∈ R+ . Define E(n) = {ηx ≤ n}, the event
that the process never visits [0, x] after time n. For the duration of this
proof, write f = f0,ν as defined at (3.16) It was shown in the proof of
Theorem 3.5.1 that (L0) and the p > 2 case of (L1) imply that for some
ν < 0 and some x1 ∈ R+ ,
E[f (Xn+1 ) − f (Xn ) | Fn ] ≤ 0, on {Xn ≥ x1 }.
An application of Lemma 3.5.7 shows that there exists y > x such that
1
P[E(n) | Fn ] ≥ , on {Xn ≥ y}.
2
For this choice of y, write r = ry and δ = δy for the constants in (L3). Then,
on {Xn < y}, if κn,y = min{k ≥ n : Xk ≥ y},
P[E(n + r) | Fn ] ≥ P[{κn,y ≤ n + r} ∩ E(κn,y ) | Fn ]
= E 1{κn,y ≤ n + r} P[E(κn,y ) | Fκn,y ] Fn
1
δ
≥ P[κn,y ≤ n + r | Fn ] ≥ , a.s.,
2
2
by (L3). On the other hand, on {Xn ≥ y}, P[E(n + r) | Fn ] ≥ P[E(n) |
Fn ] ≥ 1/2. It follows that, for all n ≥ 0,
P[E(n + r) | Fn ] ≥ δ/2, a.s.
(3.59)
For n ∈ N, let A(n) = ∪2r(n−1)≤k≤2rn {Xk ≤ x} and Gn = F2rn . Then
A(n) ∈ Gn . For n ≥ 0, let τ1 = min{n ≥ 1 : A(n) occurs}, and for k ≥ 2 set
τk = min{n > τk−1 : A(n) occurs}. Then, on {τk < ∞},
P[τk+2 = ∞ | Gτk ] ≥ P[E(2rτk + r) | F2rτk ] ≥ δ/2 =: q,
by (3.59), while τk = ∞ implies that τk+2 = ∞. Hence
P[τk+2 < ∞] = E[P[τk+2 < ∞ | Gτk ]1{τk < ∞}]
≤ (1 − q) P[τk < ∞].
3.10. Transient theory in the critical case
155
In other words, if N = max{k ≥ 1 : τk < ∞} denotes the number of times
A(n) occurs, we have, setting τ0 = 0,
P[N ≥ k + 2 | N ≥ k] = P[τk+2 < ∞ | τk < ∞] ≤ 1 − q < 1,
for all k ≥ 0. It follows that P[N ≥ k] ≤ (1 − q)bk/2c for all k ≥ 0, so in
particular E ηx ≤ 2r E N < ∞.
Now we can complete the proof of Theorem 3.10.1.
Proof of Theorem 3.10.1. Let f (x) = x2 . Then
E[f (Xn+1 ) − f (Xn ) | Fn ] = 2Xn E[∆n | Fn ] + E[∆2n | Fn ]
≥ 2Xn µ1 (Xn ) ≥ ε, on {Xn ≥ x0 },
for some constants ε > 0 and x0 ∈ R+ , by assumption. Thus we can apply
Lemma 3.10.6 with f (x) = x2 and Lemma 3.10.8 to see that there exists
a constant C ∈ R+ such that E σy ≤ C(1 + y)2 , for all y ≥ 0. Hence, by
Markov’s inequality,
P[σy > n] ≤
C(1 + y)2
, for all y > 0.
n
Now we turn to the other term on the right-hand side of (3.52). Let f =
f−θ,0 . Provided θ ∈ (0, δ), we may apply the γ = −θ case of Lemma 3.4.1
to obtain, using the fact that lim supx→∞ µ̄2 (x) < ∞,
E[f (Xn+1 ) − f (Xn ) | Fn ]
θ
−2−θ
n
≤ − 2Xn µ1 (Xn ) − (1 + θ)µ̄2 (x) Xn−2−θ + oF
).
Xn (Xn
2
By the hypothesis of the theorem, there exist ε > 0 and x0 ∈ R+ such that,
for x ≥ x0 , µ1 (x) ≥ 0 and 2xµ1 (x) − (1 + θ)µ̄2 (x) > ε. Hence
θ
−2−θ
n
E[f (Xn+1 ) − f (Xn ) | Fn ] ≤ − εXn−2−θ + oF
).
Xn (Xn
2
So there exists x1 ∈ R+ such that
E[f (Xn+1 ) − f (Xn ) | Fn ] ≤ 0, on {Xn ≥ x1 }.
Then an application of Lemma 2.5.10 shows that, for any x > x1 ,
P[λy,x < ∞ | Fσy ] ≤
f (Xσy )
f (y)
≤
,
inf z≤x f (z)
f (x)
156
Chapter 3. Lamperti’s problem
since z 7→ f (z) is non-increasing.
Thus we have from (3.52) that, for all x sufficiently large,
θ
x
C(1 + y)2
P[ηx > n] ≤
+
.
y
n
(3.60)
Fix ε > 0. Choose
1
2
y = y(x) = x(log x) θ +ε , and n = n(x) = x2 (log x)1+ θ +4ε .
Then, for all x sufficiently large,
P[ηx > n(x)] ≤ (log x)−1−θε + (log x)−1−ε .
P
So k∈Z+ P[η2k > n(2k )] < ∞. Hence the Borel–Cantelli lemma shows that,
a.s., η2k ≤ n(2k ) for all but finitely many k ∈ Z+ .
Any x ∈ R+ has 2k(x) ≤ x < 2k(x)+1 for some k(x) ∈ Z+ with k(x) → ∞
as x → ∞. Then for all x sufficiently large, a.s.,
ηx ≤ η2k(x)+1 ≤ n(2 · 2k(x) ) ≤ n(2x) ≤ 5n(x),
since n( · ) is eventually increasing. So we get that for any ε > 0, a.s., for all
x sufficiently large,
2
ηx ≤ x2 (log x)1+ θ +ε .
In other words, since (by Theorem 3.5.1) Xn → ∞, we have that, a.s., for
all but finitely many n,
2
n ≤ ηXn ≤ Xn2 (log Xn )1+ θ +ε .
The statement of the theorem follows.
3.11
Nullity and weak limits
For this section, we restrict attention to the Markov case as described in
Section 3.2. We are interested here in the null case, when, roughly speaking,
the probability mass of the process escapes to infinity. The following result
gives a sufficient condition for the null property, formalized
Pn by recalling from
Section 3.6 the notation for occupation times Ln (x) = m=0 1{Xm = x}.
157
3.11. Nullity and weak limits
Theorem 3.11.1. Suppose that (M0) and (M1) hold. Suppose that 0 <
lim inf x→∞ µ2 (x) ≤ lim supx→∞ µ2 (x) < ∞, and, for some constants θ > 0
and x0 ∈ R+ , for all x ≥ x0 ,
1−θ
2xµ1 (x) + 1 +
µ2 (x) ≥ 0.
log x
Then for any x ∈ S, limn→∞ n−1 Ln (x) = 0 a.s. and in Lq for any q ≥ 1.
Proof. Under the conditions of the theorem, Theorem 3.7.3 applies, and
shows that there exists x ∈ S such that E[λx | F0 ] = ∞ on {X0 > x}.
Choose y ∈ S with y > x; then, on {τy < τ0+ }, we have
E[τ0+ | Fτy ] ≥ E[min{n ≥ 0 : Xτy +n ≤ x} | Fτy ] = ∞.
Moreover, by Lemma 3.6.10, P0 [τy < τ0+ ] > 0, and so
E0 [τ0+ ] ≥ E0 [E[τ0+ | Fτy ]1{τy < τ0+ }] = ∞.
In the notation of Section 3.6, we have shown that E κ1 = ∞, and then the
statement follows from Theorem 3.6.9.
To proceed further, we impose the additional assumption (M2); Theorem 3.11.1 applies when b > 0 and 2a + b ≥ 0. To obtain a non-trivial
distributional limit theorem, we must scale Xn . The diffusive bounds given
in Theorem 3.2.7 suggest n−1/2 Xn as the most likely candidate for a nontrivial limit; we will see that this is indeed the case. The limit statement
will involve the distribution function Fα,θ defined for parameters α > 0 and
θ > 0 as the (normalized) lower incomplete Gamma function,
Z x2 /θ
γ(α, x2 /θ)
1
Fα,θ (x) =
=
z α−1 e−z dz, (x ≥ 0).
(3.61)
Γ(α)
Γ(α) 0
Note that, if Z ∼ Γ(α, θ) is a Gamma random
√ variable with shape parameter
α > 0 and scale parameter θ > 0, then P[ Z ≤ x] = Fα,θ (x). (In the special
case with α = 1/2 and θ = 2, Fα,θ is the distribution of the square-root of
a χ2 random variable with one degree of freedom, i.e., the absolute value of
a standard normal random variable.)
Theorem 3.11.2. Suppose that (M0) and (M1) hold, and that (M2) holds
for b > 0 and 2a + b > 0. Set
α=
Then, for any x ≥ 0,
b + 2a
, and θ = 2b.
2b
lim P[n−1/2 Xn ≤ x] = Fα,θ (x).
n→∞
(3.62)
158
Chapter 3. Lamperti’s problem
The remainder of this section is devoted to the proof of Theorem 3.11.2.
Let m ∈ N. We define an auxiliary process (X̃m,n , n ≥ 0) coupled to
(Xn , n ≥ 0) via truncation of increments. As usual, we write ∆n = Xn+1 −
˜ m,n = X̃m,n+1 − X̃m,n . For some γ ∈ (0, 1) to be specified
Xn ; also write ∆
later, let
bm (x) := (max{x, m1/2 })γ .
We will define our truncation via the function


if |h| ≤ bm (x)
h
Tm (x, h) := bm (x)
if h > bm (x)


−bm (x) if h < −bm (x).
The pair (Xn , X̃m,n ) for n ≥ 0 will be a time-homogeneous Markov process
on R2+ . We take X̃m,0 = X0 ∈ R+ . The transition law of (Xn , X̃m,n ) is as
follows. Given Xn = X̃m,n ∈ R+ , we take
Xn+1 = Xn + ∆n , and X̃m,n+1 = Xn + Tm (Xn , ∆n ).
˜ m,n to be independent of ∆n but with the
Given Xn 6= X̃m,n , we take ∆
appropriate truncated distribution, i.e., for any Borel subsets B1 , B2 of R,
˜ m,n ) ∈ B1 × B2 | (Xn , X̃m,n = (x, y)]
P[(∆n , ∆
= P[∆n ∈ B1 | Xn = x] P[Tm (y, ∆n ) ∈ B2 | Xn = y], for all x 6= y.
With this coupling, the process (Xn , n ≥ 0) is our original Markov process, and (X̃m,n , n ≥ 0) is also time-homogeneous and Markov. Note that
˜ m,n | ≤ bm (X̃m,n ), a.s., by construction. We write
|∆
˜ k | X̃m,n = x],
µ̃k (m, x) = E[∆
m,n
for the moments of the increments of X̃m,n .
By construction of the coupled Markov processes,
P[X̃m,n+1 6= Xn+1 | Xn = X̃m,n ] ≤ P[|∆n | > bm (Xn ) | Xn = X̃m,n ]
≤ P[|∆n | > mγ/2 | Xn ]
≤ Cm−pγ/2 ,
by Markov’s inequality and (M1). From this point onwards, we choose (and
fix) γ ∈ (2/p, 1), which is possible since p > 2. Then
m
h
i
X
P ∪0≤n≤m {Xn 6= X̃m,n } ≤ C
m−pγ/2 = o(1),
n=0
159
3.11. Nullity and weak limits
as m → ∞. Hence, for any ε > 0,
lim P[|Xm − X̃m,m | > ε] = 0.
m→∞
Thus to complete the proof of the theorem it suffices to show that
lim P[m−1/2 X̃m,m ≤ x] = Fα,θ (x).
m→∞
(3.63)
Note from (M2) that 2xµ1 (x) + µ2 (x) = 2a + b + o(1), and so Xn satisfies
the conditions of Theorem 3.11.1. In particular, since Σ is locally finite,
m−1
1 X
P[X` ≤ x] = 0,
m→∞ m
lim
`=0
for any x ∈ R+ . Moreover,
m−1
1 m−1
h
i
1 X
X
P[X` ≤ x] −
P[X̃m,` ≤ x] ≤ P ∪0≤n≤m {Xn 6= X̃m,n } ,
m
m
`=0
`=0
which tends to 0, so that
m−1
1 X
P[X̃m,` ≤ x] = 0.
m→∞ m
lim
(3.64)
`=0
˜ m,n , we have that
By definition of ∆
|µ̃k (m, x) − µk (x)| ≤ E[|∆n |k 1{∆n > xγ } | Xn = x],
so that for k ∈ {1, 2}, by (M1) and a similar argument to Lemma 3.4.2,
|µ̃k (m, x) − µk (x)| ≤ Cxγ(k−p) = ox (xk−2 ),
(3.65)
1
since γ > p2 > p−1
, where the notation ox emphasizes that the error term is
uniform in m as x → ∞. Moreover, for integer k ≥ 3,
˜ m,n |k | X̃m,n = x] ≤ E[|∆
˜ m,n |2 |∆
˜ m,n |k−2 | X̃m,n = x]
E[|∆
˜ 2 | X̃m,n = x]
≤ bm (x)k−2 E[∆
m,n
≤ Cbm (x)
k−2
.
(3.66)
The distributional convergence of X̃m,m will be deduced by the method of
moments, based on the following result.
160
Chapter 3. Lamperti’s problem
Lemma 3.11.3. For any j ∈ N,
!
j
Y
2j
lim m−j sup E[X̃m,n
] − nj
(2a + (2i − 1)b) = 0.
m→∞
0≤n≤m
(3.67)
i=1
Proof. We use induction on j. First, for j = 1, for ` ∈ Z+ ,
2
2
˜ m,` )2 − x2 | X̃m,` = x]
E[X̃m,`+1
− X̃m,`
| X̃m,` = x] = E[(x + ∆
= 2xµ̃1 (m, x) + µ̃2 (m, x).
(3.68)
By (3.65) and our assumptions on µ1 and µ2 ,
lim sup |xµ̃1 (m, x) − a| = 0, and lim sup |µ̃2 (m, x) − b| = 0.
x→∞ m∈N
x→∞ m∈N
(3.69)
For compactness, we write these last expressions as xµ̃1 (m, x) = a + ox (1)
and µ̃2 (m, x) = b + ox (1), where again the subscript x on the error term
indicates that it is uniform in m as x → ∞. Hence for any ε > 0 there exist
x ∈ R+ and C ∈ R+ such that
2
2
− X̃m,`
| X̃m,` ] − (2a + b) ≤ ε + C1{X̃m,` ≤ x}.
E[X̃m,`+1
Hence, taking expectations,
2
2
] − (2a + b) ≤ ε + C P[X̃m,` ≤ x].
] − E[X̃m,`
E[X̃m,`+1
Summing from ` = 0 to ` = n − 1 we obtain
m−1
X
2
2
sup E[X̃m,n ] − E[X̃m,0 ] − n(2a + b) ≤ mε + C
P[X̃m,` ≤ x]
0≤n≤m
`=0
≤ 2mε,
for all m ≥ m0 sufficiently large, by (3.64). In other words, since ε > 0 was
arbitrary,
2
m−1 sup E[X̃m,n
] − n(2a + b) → 0,
0≤n≤m
as m → ∞, which is the j = 1 case of (3.67).
For the inductive step, suppose that (3.67) holds for all j ∈ {1, 2, . . . , k −
1} for some k ≥ 2. Consider
2k
2k
˜ m,` )2k − x2k | X̃m,` = x]
E[X̃m,`+1
− X̃m,`
| X̃m,` = x] = E[(x + ∆
161
3.11. Nullity and weak limits
=
2k X
2k
i=1
i
x2k−i µ̃i (m, x),
by the binomial formula. Hence, using (3.69) to estimate the terms i = 1
and i = 2 in the sum, we obtain
2k
2k
E[X̃m,`+1
− X̃m,`
| X̃m,` = x]
2k−2
= k 2a + (2k − 1)b + ox (1) x
2k X
2k 2k−i
+
x
µ̃i (m, x).
i
(3.70)
i=3
Also, we have from (3.66) that, for all i ≥ 3,
|µ̃i (m, x)| ≤ Cxγ(i−2) + Cmγ(i−2)/2 1{x2 < m}
≤ Cxγ(i−2) + Cm(i−2)/2 m−(1−γ)/2 1{x2 < m}
≤ Cxγ(i−2) + Cm(i−2)/2 x−(1−γ)/4 .
Here, since γ < 1,
2k X
2k 2k−i γ(i−2)
x
x
= Ox (x2k+γ−3 ) = ox (x2k−2 ),
i
i=3
so we have from (3.70) that
2k−2 2k
2k
E[X̃m,`+1 − X̃m,` | X̃m,` = x] − k 2a + (2k − 1)b + ox (1) x
2k X
2k 2k−i (i−2)/2
≤ ox (1)
x
m
.
i
i=3
In other words, for any ε > 0 there exist constants x ∈ R+ and C ∈ R+
(depending on k but not on ` or m) so that
2k−2 2k
2k
− X̃m,`
| X̃m,` ] − k 2a + (2k − 1)b X̃m,`
E[X̃m,`+1
2k X
2k
2k−i (i−2)/2
≤ε
X̃m,`
m
+ Cmk−1 1{X̃m,` ≤ x}.
i
i=2
By the inductive hypothesis (3.67) and Lyapunov’s inequality, we have that
for all i ∈ {2, 3, . . . , 2k} and all ` ≤ m,
2k−i
2k−i
2k−2 2k−2
]
E[X̃m,`
] ≤ E[X̃m,`
≤ C`k−(i/2) ≤ Cmk−(i/2) ,
162
Chapter 3. Lamperti’s problem
for some C ∈ R+ (depending on k but not on m). Thus, for all ` ≤ m,
2k X
2k
i
i=2
2k−i
E[X̃m,`
]m(i−2)/2 ≤ Cmk−1 ,
where, again, C ∈ R+ depends on k but not on m. Together with the
j = k − 1 case of (3.67), it follows that for any ε > 0, for all 0 ≤ ` ≤ m with
m ≥ m0 sufficiently large,
k
Y
2k
2k
] − E[X̃m,`
] − k`k−1 (2a + (2i − b))
E[X̃m,`+1
i=1
k−1
≤ εm
k−1
+ Cm
P[X̃m,` ≤ x].
(3.71)
Hence summing from ` = 0 to ` = n − 1 in (3.71) we see that, for any ε > 0,
for all m sufficiently large,
k
Y
2k
2k
sup E[X̃m,n
] − E[X̃m,0
] − nk
(2a + (2i − b))
0≤n≤m
i=1
!
m−1
1 X
k
k
P[X̃m,` ≤ x] ≤ 2εmk ,
≤ εm + Cm
m
`=0
by (3.64). This verifies the j = k case of (3.67), and completes the induction.
We can now finish the proof of Theorem 3.11.2.
Proof of Theorem 3.11.2. We appeal to the method of moments (see e.g. [123,
§8.4]): Lemma 3.11.3 shows that, for all j ∈ N,
lim E
m→∞
m
−1
2
X̃m,m
j a
jΓ b
= (2b)
Γ
+
1
2
a
b
+
+j
= E[Y j ],
1
2
where Y is a gamma-(α, θ) random variable with α = ab + 21 and θ = 2b.
Since the law of Y is determined uniquely by its moments, it follows that
2
m−1 X̃m,m
converges in distribution to Y , and hence, as X̃m,m ≥ 0, we get
√
−1/2
m
X̃m,m converges to Y . This establishes (3.63) and hence proves the
theorem.
163
3.12. Supercritical case
3.12
Supercritical case
In this section we turn to the supercritical case where µ̄1 (x) and µ1 (x) are
positive and of order x−β for some β ∈ (0, 1), in the general setting of
Section 3.3. In this case, under mild conditions, transience is assured: our
primary interest is to quantify this transience by studying the rate of escape
and accompanying second-order behaviour.
As before, we need to assume a moments condition on the increments
∆n = Xn+1 − Xn .
(L5) Suppose that for some C ∈ R+ , E[|∆n |p | Fn ] ≤ C, a.s.
If (L5) holds for p ≥ 1, then µ̄1 and µ1 given by (3.5) and (3.6) exist as
R-valued functions. The next result gives transience of Xn .
Theorem 3.12.1. Suppose that (L0) and (L2) hold, and that there exists
β ∈ [0, 1) such that (L5) holds for some p > 1 + β, and
lim inf (xβ µ1 (x)) > 0.
x→∞
Then Xn → ∞ a.s. as n → ∞.
The main topic of this section is the quantitative asymptotic behaviour
of Xn . Our results involve the constants λ(a, β) defined by
1
λ(a, β) := (a(1 + β)) 1+β .
(3.72)
The next result gives sharp almost-sure bounds on Xn , and shows that for
β ∈ (0, 1) the transience given in Theorem 3.12.1 is super-diffusive but sub1
is greater than 12 but less than 1.
ballistic, since the exponent 1+β
Theorem 3.12.2. Suppose that (L0) and (L2) hold. Suppose that there
exist β ∈ [0, 1) and a, A with 0 < a ≤ A < ∞ such that
a = lim inf (xβ µ1 (x)) ≤ lim sup(xβ µ̄1 (x)) = A,
x→∞
(3.73)
x→∞
and (L5) holds for some p > 2 + 2β. Then, a.s.,
λ(a, β) ≤ lim inf n
n→∞
1
− 1+β
Xn ≤ lim sup n
n→∞
1
− 1+β
Xn ≤ λ(A, β).
Remark 3.12.3. The proof of the upper bound on Xn given by Theorem 3.12.2
only uses the condition on µ̄1 in (3.73), and not the condition on µ1 there.
164
Chapter 3. Lamperti’s problem
A corollary of Theorem 3.12.2, obtained on taking a = A = ρ, is the
following strong law of large numbers.
Theorem 3.12.4. Suppose that (L0) and (L2) hold. Suppose that there
exists β ∈ [0, 1) such that (L5) holds for some p > 2 + 2β, and
lim xβ µ̄1 (x) = lim xβ µ1 (x) = ρ ∈ (0, ∞).
x→∞
(3.74)
x→∞
Then
lim n
1
− 1+β
n→∞
Xn = λ(ρ, β), a.s.
(3.75)
The next result shows that there is a central limit theorem to accompany
the law of large numbers in Theorem 3.12.4, provided that we impose a
somewhat stronger version of (3.74) and an asymptotic stability condition
on the second moments of the increments. Unlike the preceding results in
this section, the case β = 0 is excluded from the following theorem.
Theorem 3.12.5. Suppose that (L0) and (L2) hold. Suppose that there
exist β ∈ (0, 1) and ρ ∈ (0, ∞) such that (L5) holds for some p > max{2 +
2β, 3+β
1+β }, and
µ1 (x) = ρx−β + o(x−β−
1−β
2
);
µ̄1 (x) = ρx−β + o(x−β−
1−β
2
).
(3.76)
Suppose also that there exists σ 2 ∈ (0, ∞) such that
E[∆2n | Fn ] → σ 2 , a.s.
(3.77)
Then, as n → ∞,
s
n
−1/2
Xn − λ(ρ, β)n
1
1+β
d
−→ Zσ
1+β
,
1 + 3β
where Z is a standard normal random variable.
Example 3.12.6. We consider a nearest-neighbour walk that generalizes
slightly the model of Section 2.2 by allowing transitions whereby the walk
stays at the same site. Specifically, suppose that there exist sequences
ax , bx , cx (x ∈ N) with ax > 0, bx ≥ 0, cx > 0 and ax + bx + cx = 1.
Define the transition law of Xn as follows: for x ∈ N,
P[Xn+1 = x + 1 | Xn = x] = ax ,
P[Xn+1 = x | Xn = x] = bx ,
165
3.12. Supercritical case
P[Xn+1 = x − 1 | Xn = x] = cx ,
and with reflection from 0 governed by P[Xn+1 = 1 | Xn = 0] = a0 ∈ (0, 1).
For x ∈ N, with µ1 (x) and µ2 (x) defined by (3.1) we have that
µ1 (x) = ax − cx , and µ2 (x) = 1 − bx > 0.
The asymptotically zero drift case is the case where limx→∞ (ax − cx ) = 0.
We are in the supercitical case if, for β ∈ (0, 1),
lim xβ (ax − cx ) = ρ ∈ (0, ∞).
x→∞
(3.78)
In this case Theorem 3.12.4 implies that (3.75) holds. Moreover, if we suppose that
ax − cx = ρx−β + o(x−β−
1−β
2
);
lim bx = b ∈ [0, 1),
x→∞
then Theorem 3.12.5 yields the central limit theorem
Xn −
λ(ρ, β)n1/(1+β)
n1/2
s
d
−→ Z
(1 − b)(1 + β)
.
1 + 3β
4
In the rest of this section we give the proofs of the theorems stated
above. We start with a technical result. Again we use the notation Eε (n) =
{|∆n | ≤ (1 + Xn )1−ε } as at (3.19). The next truncation result is a relative
of Lemma 3.4.2.
Lemma 3.12.7. Suppose that (L0) holds and that (L5) holds for p > 0. For
any q ∈ [0, p] and ε ∈ (0, 1) there exists C ∈ R+ for which, for all n ≥ 0,
E |∆n |q 1(Eεc (n)) Fn ≤ C(1 + Xn )−(p−q)(1−ε) , a.s.
Proof. For any n ≥ 0,
|∆n |q 1(Eεc (n)) = |∆n |p |∆n |q−p 1(Eεc (n))
≤ |∆n |p (1 + Xn )(q−p)(1−ε) .
Taking expectations and using (L5) gives the result.
Next we give the proof of the transience result, Theorem 3.12.1.
166
Chapter 3. Lamperti’s problem
Proof of Theorem 3.12.1. We use the Lyapunov function f (x) = (1 + x)−γ ,
γ > 0. We claim that there exist γ > 0 and x0 ∈ R+ such that for all n ≥ 0,
P[f (Xn+1 ) − f (Xn ) | Fn ] ≤ 0, on {Xn ≥ x0 }.
(3.79)
Note that our assumption on µ1 implies that for some c > 0 and x1 ∈ R+ ,
E[∆n | Fn ] ≥ cXn−β on {Xn ≥ x1 }. Let ε ∈ (0, 1) be such that pε < p−1−β.
Since f is decreasing,
f (Xn+1 ) − f (Xn ) ≤ (f (Xn+1 ) − f (Xn ))1(Eε (n))
+ (f (Xn+1 ) − f (Xn ))1{∆n ≤ −(1 + Xn )1−ε }. (3.80)
Taylor’s formula with Lagrange remainder shows that
(f (Xn+1 ) − f (Xn ))1(Eε (n))
Fn
(Xn−ε ))∆n 1(Eε (n)).
= −γ(1 + Xn )−γ−1 (1 + OX
n
(3.81)
Since (L5) holds for p > 1 + β ≥ 1, we may apply the q = 1 case of
Lemma 3.12.7 to obtain
Fn
(Xn−(p−1)+pε ),
E[∆n 1(Eε (n)) | Fn ] = E[∆n | Fn ] + OX
n
(3.82)
−β
n
and this last error term is oF
Xn (Xn ) by choice of ε. Hence taking expectations in (3.81) and using (3.82) we obtain
n
E[(f (Xn+1 ) − f (Xn ))1(Eε (n)) | Fn ] ≤ −γXn−γ−1−β (c + oF
Xn (1)).
(3.83)
On the other hand, since f (x) ∈ [0, 1], the q = 0 case of Lemma 3.12.7 yields
E[|f (Xn+1 ) − f (Xn )|1{∆n ≤ −(1 + Xn )1−ε } | Fn ]
Fn
−γ−1−β
n
≤ P[Eεc (n) | Fn ] = OX
(Xn−p(1−ε) ) = oF
),
Xn (Xn
n
(3.84)
provided we take γ > 0 with γ < p − 1 − β − pε, which is possible by choice
of ε. From (3.80) with (3.83) and (3.84), we therefore conclude that, for
γ > 0 sufficiently small,
−γ−1−β
n
E[f (Xn+1 ) − f (Xn ) | Fn ] ≤ −(cγ + oF
,
Xn (1))Xn
which yields the claim (3.79). Transience now follows from an application
of Theorem 3.5.6.
Now we turn to our main aim, and work toward the proof of Theorem 3.12.2. The following technical result will allow us to work with the
process Xn1+β .
167
3.12. Supercritical case
Lemma 3.12.8. Suppose that (L0) holds, and that there exists β ∈ [0, 1)
such that (L5) holds for some p > 1 + β.
(i) If lim supx→∞ (xβ µ̄1 (x)) < ∞, then there exists C ∈ R+ such that,
1+β
E[Xn+1
− Xn1+β | Fn ] ≤ C, a.s.
(3.85)
(ii) Suppose that for some A ∈ (0, ∞), lim supx→∞ (xβ µ̄1 (x)) ≤ A. Then
for any ε > 0 there exists x ∈ R+ such that for all n ≥ 0,
1+β
E[Xn+1
− Xn1+β | Fn ] ≤ A(1 + β) + ε, on {Xn ≥ x}.
(3.86)
(iii) Suppose that for some a ∈ (0, ∞), lim inf x→∞ (xβ µ1 (x)) ≥ a. Then for
any ε > 0 there exists x ∈ R+ such that for all n ≥ 0,
1+β
E[Xn+1
− Xn1+β | Fn ] ≥ a(1 + β) − ε, on {Xn ≥ x}.
(3.87)
p
(iv) Let r ∈ [1, 1+β
). Then there exists C ∈ R+ such that for all n ≥ 0,
1+β
− Xn1+β |r | Fn ] ≤ C(1 + Xnβr ), a.s.
E[|Xn+1
(3.88)
Proof. Let ε ∈ (0, 1) be such that pε < p − 1 − β. By a Taylor’s formula
argument that should now be familiar, we have that
1+β
Fn
(Xn−ε ))Xnβ ∆n 1(Eε (n)) + Rn ,
Xn+1
− Xn1+β = (1 + β + OX
n
(3.89)
1+β
− Xn1+β )1(Eεc (n)). Since on Eεc (n) we have Xn ≤
where Rn = (Xn+1
|∆n |1/(1−ε) , it follows that, by choice of ε,
|Rn | ≤ C|∆n |1+β+pε 1(Eεc (n)), a.s.,
(3.90)
for some absolute constant C ∈ R+ . Then taking expectations in (3.90) and
using the q = 1 + β + pε < p case of Lemma 3.12.7 we have that for n ≥ 0,
n
E[|Rn | | Fn ] ≤ C(1 + Xn )−(p−1−β−pε)(1−ε) = oF
Xn (1),
by choice of ε. Also, as at (3.82), we have that
−β
n
E[∆n 1(Eε (n)) | Fn ] = E[∆n | Fn ] + oF
Xn (Xn ),
by choice of ε. It follows from (3.89) that, a.s., for all n ≥ 0,
1+β
Fn
β
n
E[Xn+1
− Xn1+β | Fn ] = (1 + β + oF
Xn (1))Xn E[∆n | Fn ] + oXn (1).
168
Chapter 3. Lamperti’s problem
Under the conditions of part (ii) of the lemma we have that E[∆n | Fn ] ≤
−β
n
(A + oF
Xn (1))Xn , and (3.86) follows. Similarly (3.87) follows under the
conditions of part (iii) of the lemma.
Next we prove part (iv) of the lemma. From (3.89) and (3.90), for any
ε > 0 small enough,
1+β
|Xn+1
− Xn1+β |r ≤ C(1 + Xn )βr |∆n |r + C|∆n |(1+β+pε)r , a.s.,
p
for a constant C ∈ R+ . Given r < 1+β
, we may choose ε > 0 small enough
so that (1 + β + pε)r ≤ p. Then (L5) shows that both E[|∆n |r | Fn ] and
E[|∆n |(1+β+pε)r | Fn ] are uniformly bounded above. Thus (3.88) follows.
Finally, part (i) of the lemma follows on combining part (ii) with the
r = 1 case of part (iv).
Under conditions of comparable strength to those in Theorem 3.12.1, we
have the following upper bound, which, while relatively crude, will be useful
in the proofs of the sharper results of this section.
Lemma 3.12.9. Suppose that (L0) and (L2) hold, and that there exists β ∈
[0, 1) such that (L5) holds for some p > 1 + β, and lim supx→∞ (xβ µ̄1 (x)) <
∞. Then for any ε > 0, a.s., for all but finitely many n ≥ 0,
1
1
max Xk ≤ n 1+β (log n) 1+β
0≤k≤n
+ε
.
(3.91)
Proof. Lemma 3.12.8(i) shows that under the conditions of the lemma, we
may apply Theorem 2.8.1 with f (x) = x1+β and a(x) = x(log x)1+ε , which
yields the result.
For notational convenience, for the rest of this section we write
Yn := Xn1+β , and Dn := E[Yn+1 − Yn | Fn ].
The Doob decomposition for Yn (see Theorem 2.3.1) is
Y n = Mn + A n ,
(3.92)
Pn−1
where A0 := 0, An := k=0 Dk for n ≥ 1, and Mn is an Fn -adapted martingale. The next result gives conditions for An to be a good approximation
for Xn1+β .
Lemma 3.12.10. Suppose that (L0) holds, and that there exists β ∈ [0, 1)
such that (L5) holds for some p > 2 + 2β. If lim supx→∞ (xβ µ̄1 (x)) < ∞,
then
lim n−1 Xn1+β − An = 0, a.s.
n→∞
169
3.12. Supercritical case
Proof. Since Mn = Yn −
Pn−1
k=0
Dk is a martingale, we have that
2
E[Mn+1
− Mn2 | Fn ] = E[(Mn+1 − Mn )2 | Fn ]
= E[(Yn+1 − Yn − Dn )2 | Fn ]
= E[(Yn+1 − Yn )2 | Fn ] − Dn2 ,
(3.93)
where we have expanded the term (Yn+1 − Yn − Dn )2 and used the fact that
Dn is Fn -measurable. Now by (3.93) and the r = 2 case of (3.88) (which is
valid since p > 2 + 2β) we have that for all n ≥ 0,
2
2
E[Mn+1
− Mn2 | F0 ] = E E[Mn+1
− Mn2 | Fn ] F0 ≤ C + C E[Xn2β | F0 ].
Now since β ∈ [0, 1), the (conditional) Jensen inequality implies that
E[Xn2β | F0 ] ≤ E[Xn1+β | F0 ]
2β
1+β
2β
≤ (X01+β + Cn) 1+β ,
by (3.85). Thus Mn2 is a non-negative submartingale with
E[Mn2
| F0 ] ≤
≤
Y02
+
n−1
X
k=0
X02+2β
2
− Mk2 | F0 ]
E[Mk+1
2β
+ n(X01+β + Cn) 1+β .
Doob’s inequality (Theorem 2.3.14) then implies that for any ε > 0,
1+3β
+ε
− 1+3β −ε
2
1+β
≤ n 1+β E[Mn2 ] = O(n−ε ).
P max Mk > n
0≤k≤n
Hence the Borel–Cantelli lemma implies that for any ε > 0, a.s.,
1+3β
maxm |Mk | ≤ (2m ) 2+2β
0≤k≤2
+ε
,
for all but finitely many m ∈ Z+ . Since for any n ∈ N we have 2m(n) ≤ n <
2m(n)+1 for some m(n) ∈ Z+ with m(n) → ∞ as n → ∞, we have that for
any ε > 0, a.s., for all but finitely many n ≥ 0,
max |Mk | ≤
0≤k≤n
max
0≤k≤2m(n)+1
1+3β
|Mk | ≤ (2m(n)+1 ) 2+2β
+ε
1+3β
≤ Cn 2+2β
+ε
,
for some C ∈ R+ not depending on n. Since β ∈ [0, 1), we may take ε small
1+3β
enough so that 2+2β
+ ε ≤ 1 − ε. Then we have that |An − Yn | ≤ Cn1−ε for
all but finitely many n ≥ 0.
170
Chapter 3. Lamperti’s problem
Proof of Theorem 3.12.2. Under the conditions of the theorem, we have
from Lemma 3.12.8 that (3.86) and (3.87) hold. Moreover, we know from
Theorem 3.12.1 that Xn → ∞ as n → ∞, a.s. Hence for any ε > 0, a.s.,
a(1 + β) − ε ≤ Dn ≤ A(1 + β) + ε,
for all but finitely many n ≥ 0. Hence with An given in (3.92) we have that
for any ε > 0, a.s.,
a(1 + β) − ε ≤ n−1 An ≤ A(1 + β) + ε,
for all but finitely many n ≥ 0. Since ε > 0 was arbitrary, this means
a(1 + β) ≤ lim inf n−1 An ≤ lim sup n−1 An ≤ A(1 + β), a.s.
n→∞
n→∞
Together with Lemma 3.12.10, this implies
a(1 + β) ≤ lim inf n−1 Xn1+β ≤ lim sup n−1 Xn1+β ≤ A(1 + β), a.s.,
n→∞
n→∞
and the statement in the theorem follows.
The basic ingredients of the proof of Theorem 3.12.5 are already in place
in the decomposition used in the proof of Lemma 3.12.10, but we need to
revisit some of our earlier estimates and obtain sharper bounds under the
conditions of Theorem 3.12.5. First we have the following refinement of
Lemma 3.12.8 in this case.
Lemma 3.12.11. Suppose that (L0) holds, and that for β ∈ (0, 1) and
ρ ∈ (0, ∞), (3.76) holds. Suppose that (L5) holds for p > max{2 + 2β, 3+β
1+β }.
Then, as n → ∞,
1−β 1+β
n 2+2β E[Xn+1
− Xn1+β | Fn ] − ρ(1 + β) → 0, a.s.
(3.94)
If in addition (3.77) holds for some σ 2 ∈ (0, ∞), then
n
2β
− 1+β
1+β
E[(Xn+1
− Xn1+β )2 | Fn ] → σ 2 (1 + β)2 λ(ρ, β)2β , a.s.
(3.95)
Proof. We need to obtain better estimates for the error terms in (3.89) than
we did in the proof of Lemma 3.12.8. For this reason the ε there will not
be arbitrarily small, so we cannot use (3.90). Instead, from the argument
leading to (3.90) we have
1+β
|Rn | ≤ C|∆n | 1−ε 1(Eεc (n)),
171
3.12. Supercritical case
so that from the q = 1+β
1−ε case of Lemma 3.12.7, which is valid provided
pε ≤ p − 1 − β, we have
1+β
E[|Rn | | Fn ] ≤ C(1 + Xn )−(p− 1−ε )(1−ε) = C(1 + Xn )−(p−1−β−pε) .
Now choose ε >
1−β
2
such that p − 1 − β − pε >
− 1−β
2
Fn
p > 3+β
1+β . Then E[|Rn | | Fn ] = oXn (Xn
the q = 1 case of Lemma 3.12.7,
1−β
2 ;
this is possible provided
). Moreover, we have that from
− 1−β
2
n
Xnβ E[∆n 1(Eεc (n)) | Fn ] ≤ C(1 + Xn )−(p−1−β−εp) = oF
Xn (Xn
),
by choice of ε. Hence
− 1−β
2
n
Xnβ E[∆n 1(Eε (n)) | Fn ] = Xnβ E[∆n | Fn ] + oF
Xn (Xn
− 1−β
2
n
) = ρ + oF
Xn (Xn
),
by (3.76). With these sharper bounds, from (3.89) and the present choice
of ε we obtain
− 1−β
2
1+β
n
− Xn1+β | Fn ] = (1 + β)ρ + oF
E[Xn+1
Xn (Xn
).
Under the conditions of the lemma, Theorem 3.12.4 applies so that Xn ∼
1
λ(ρ, β)n 1+β . Thus we obtain (3.94). The argument for (3.95) is similar,
starting by squaring (3.89) and then taking ε > 0 small enough, so we omit
the details.
Lemma 3.12.12. Suppose that (L0) holds, and that for β ∈ (0, 1) and
ρ ∈ (0, ∞), (3.76) holds. Suppose that (L5) holds for p > max{2 + 2β, 3+β
1+β }
2
and (3.77) holds for σ ∈ (0, ∞). Then with Mn as defined at (3.92), we
have that as n → ∞,
s
(1 + β)3
d
− 1+3β
β
n 2+2β Mn −→ Zσλ(ρ, β)
,
(1 + 3β)
where Z is a standard normal random variable.
Proof. We will apply a standard martingale central limit theorem. Set
− 1+3β
Mn,k = n 2+2β (Mk −Mk−1 ) for 1 ≤ k ≤ n. For fixed n, Mn,k is a martingale
difference sequence with E[Mn,k | Fk−1 ] = 0. Moreover,
n
X
k=1
2
E[Mn,k
| Fk−1 ] = n
− 1+3β
1+β
n
X
k=1
E[(Mk − Mk−1 )2 | Fk−1 ]
172
Chapter 3. Lamperti’s problem
=n
− 1+3β
1+β
n X
k=1
1+β 2
E[(Xk1+β − Xk−1
) | Fk−1 ] + O(1) , a.s.,
by (3.93), using the fact that |Dn | = O(1), which is easily seen by combining
part (iii) of Lemma 3.12.8 with the r = 1 case of part (iv). Then by (3.95),
n
X
2
E[Mn,k
k=1
| Fk−1 ] ∼ n
− 1+3β
1+β
→ σ2
2
2
2β
σ (1 + β) λ(ρ, β)
n
X
2β
k 1+β
k=1
β)3
(1 +
λ(ρ, β)2β , a.s.
1 + 3β
(3.96)
We also need to verify a form of the conditional Lindeberg condition. We
claim that
n
X
2
1{|Mn,k | > δ} | Fk−1 ] → 0, a.s.,
(3.97)
E[Mn,k
k=1
p
for any δ > 0. To see this, take q ∈ (2, 1+β
). By the elementary inequality
2
2−q
q
|Mn,k | 1{|Mn,k | > δ} ≤ δ |Mn,k | we have that, for any δ > 0,
2
1{|Mn,k | > δ} | Fk−1 ] ≤ δ 2−q E[|Mn,k |q | Fk−1 ].
E[Mn,k
Then we have that
E[|Mn,k |q | Fk−1 ] = n
−
(1+3β)q
2+2β
=n
−
(1+3β)q
2+2β
E[(Mk − Mk−1 )q | Fk−1 ]
1+β
− Dk−1 )q | Fk−1 ],
E[(Xk1+β − Xk−1
where |Dk−1 | = O(1), and by the r = q case of (3.88),
1+β q
E[(Xk1+β − Xk−1
) | Fk−1 ] ≤ C(1 + Xk−1 )βq .
Hence by the (conditional) Minkowski inequality,
−
E[|Mn,k |q | Fk−1 ] ≤ Cn
(1+3β)p
2+2β
(1 + Xk−1 )βq .
1
Here, by Theorem 3.12.4, we have Xk−1 ≤ Cn 1+β for some (random) C < ∞
and all k ≤ n. Thus we obtain
n
X
k=1
βq
1+ 1+β
−
2
E[Mn,k
1{|Mn,k | > δ} | Fk−1 ] ≤ Cn
(1+3β)q
2+2β
, a.s.,
for some (random) C < ∞. From this last bound we verify (3.97) since p > 2.
Given (3.96) and (3.97), we can apply a standard central limit theorem for
martingale differences (e.g. [25, Theorem 35.12, p. 476]) to complete the
proof.
173
3.13. Proofs for the Markovian case
Proof of Theorem 3.12.5. Recall the decomposition at (3.92). Throughout
this proof, εn will denote a sequence such that εn → 0, a.s. Under the
conditions of the theorem, Theorem 3.12.4 and the proof of Lemma 3.12.10
imply that |Mn | ≤ εn An , a.s., so that
1
1
Xn = (An + Mn ) 1+β = An1+β +
− β
1
Mn An 1+β (1 + εn ).
1+β
1+3β
Here we have from (3.94) that, a.s., An = ρ(1 + β)n + εn n 2+2β . It follows
that, a.s.,
1
Xn = λ(ρ, β)n 1+β + n1/2 εn +
1
− β
λ(ρ, β)−β n 1+β Mn (1 + εn ).
1+β
Rearranging we obtain, a.s.,
1
1+3β
1
Xn − λ(ρ, β)n 1+β
−β − 2+2β
=
λ(ρ,
β)
n
Mn (1 + εn ) + εn .
1+β
n1/2
Now, on letting n → ∞, Lemma 3.12.12 completes the proof.
3.13
Proofs for the Markovian case
We now record the proofs of the results stated in Section 3.2; this is mostly
simply collecting results from the previous sections. We start with the recurrence classification.
Proof of Theorem 3.2.3. The transience condition in part (i) is a consequence
of Theorem 3.5.1; Theorem 3.6.8 demonstrates the equivalence of the two
notions of transience. The positive-recurrence condition in part (iii) is a
consequence of the α = 1 case of Theorem 3.8.1. In part (ii), recurrence
follows from Theorem 3.5.2; Theorem 3.6.8 demonstrates the equivalence of
the two notions of recurrence. Finally, the null property in part (ii) follows
from Theorem 3.7.3.
Next we give the proof of the result on moments of return times.
Proof of Theorem 3.2.6. The existence of moments statement in part (i)
follows from Theorem 3.8.1, while the non-existence of moments in part
(ii) follows from Theorem 3.8.2.
Finally, the almost-sure bounds on trajectories.
174
Chapter 3. Lamperti’s problem
Proof of Theorem 3.2.7. This is a combination of the bounds from Theorem 3.9.5 with the lower bound from Theorem 3.10.1 in the transient case
for the ‘lim inf’ half of part (i).
Bibliographical notes
Section 3.1
The foundations for the material in this chapter were laid in the early 1960s
by J. Lamperti in a seminal series of papers [190, 191, 192]. Lamperti
systematically studied conditions under which the asymptotic behaviour of
a non-negative real-valued discrete-time stochastic process is governed by
the (first two) moment functions of its increments. These three papers
have provided a rich source of material for subsequent theoretical work, as
well as providing key evidence of the power of the semimartingale approach
for studying multidimensional processes via Lyapunov functions. Some of
the chapters later in this book describe examples of the application of this
method to near-critical processes, often in cases where it is hard to find any
serious methodological competitor.
Example 3.1.1 gives a hint to the ubiquity of Lamperti-type situations.
As remarked in the text, if ξn is Markov then typically Xn = f (ξn ) is not:
see Section 1.3 and the accompanying bibliographical notes for Chapter 1.
Many dimensional zero-drift random walks satisfying the conditions of Example 3.1.1 may be either recurrent or transient, as we will see in Chapter 4;
this contrasts with the one-dimensional case where recurrence is assured under similar conditions, as shown in Theorem 2.5.7.
Section 3.2
The Markovian recurrence classification, Theorem 3.2.3, improves a little
on Lamperti’s original results from [190, 192] in the Markov case. Theorem 3.2.3 is usually sufficiently sharp for any application, but can be
sharpened near the phase boundaries; an ultimate refinement is given by
Menshikov et al. [228]. Theorem 3.2.3 parts (i) and (ii) are contained in
Theorem 3.1 of [190] and Theorem 2.1 of [192] respectively; part (iii) is
sharper than Lamperti’s results, and is contained in the results of [228].
Nearest neighbour walks in the vein of Example 3.2.5 have been extensively studied for a long time; these models also go under the name birthand-death chains. In the context, the observation that the regime where
xµ1 (x) = O(1) is critical from the point of view of the recurrence clas-
175
Bibliographical notes
sification goes back at least to Harris [127]; another early contribution is
[129]. In certain cases, nearest-neighbour models are amenable to explicit
calculation, facilitated by reversibility and associated algebraic structure.
An intricate apparatus was introduced by Karlin and McGregor [152, 153],
using orthogonal polynomials to provide a spectral representation, to give a
deep analysis of models such as
P[Xn+1 = Xn ± 1 | Xn = x] =
1
δ
∓
,
2 4x + 2δ
for x > 0 and a parameter δ, which is an instance of Example 3.2.5 with c =
−δ/2. This and closely related models were considered by many subsequent
authors, including for instance [275, 91, 92, 93, 299, 112, 310] and, more
recently [65, 64, 67, 140, 2, 182]. Alexander [2] calls such nearest-neighbour
random walks with drift O(x−1 ) at x ‘Bessel-like’. See e.g. [59] for a survey
of methods based on orthogonal polynomials.
Theorem 3.2.6 on moments of return times in the Markov case is a consequence of the more general results in Section 3.7; see the notes below on
that section for bibliographical details. Likewise, Theorem 3.2.7 is a consequence of the more general results on almost-sure bounds in Sections 3.9
and 3.10; see the notes below on the relevant sections.
One possible extension to the models of this section is to move from the
half-line R+ to a Markov process on a half strip A × Z+ , where A is a finite
set, in which the process has different Lamperti-type characteristics on each
copy of Z+ , and is close to being Markov on A: see [115].
Section 3.3
For all the reasons outlined in the text, the importance of a setting more general than the Markovian one of Section 3.2 was emphasized by Lamperti [190].
The formulation (3.5) and (3.6) builds on the original formulation of [190],
and rectifies an omission in a similar earlier version in [236].
The representation (3.7) is a very general stochastic difference equation;
stochastic recursions of this form have received much attention in various
contexts over the years, particularly when ψ1,n ≡ 0, including stochastic
growth models (see e.g. [155]) and stochastic approximation (see e.g. [255]).
The natural non-confinement condition (L2) was used by Lamperti [190],
and avoids technicalities involving irreducibility in general spaces. The uniform ellipticity assumption discussed in Example 3.3.5 is common in the
literature on multidimensional random walks.
176
Chapter 3. Lamperti’s problem
Section 3.4
The Lyapunov function computations presented in this section are variations
on a well-established theme. The closest presentation in the literature is that
of [135]. Instead of the truncation near zero in defining fγ,ν at (3.16), an
alternative is to take say (e + x)γ logν (e + x), but this makes some of the
computations a little more cumbersome.
Section 3.5
The transience criterion, Theorem 3.5.1, is the transience half of Theorem 3.2
of [190]. The recurrence criterion, Theorem 3.5.2, improves on the recurrence
half of Theorem 3.2 of [190], which admits θ = 1 but not θ ∈ (0, 1). Formally,
Theorem 3.5.2 is not contained in the results of [228], which are stated in
the Markovian case, although it is likely that these sharper results can be
extended to the more general setting.
The fact that symmetric SRW on Zd is recurrent if and only if d ≤ 2 is
a seminal result of Pólya [259]; the fact that the statement extends to nondegenerate homogeneous random walks whose increments have finite second
moments is a consequence of the equally famous Chung–Fuchs theorem (see
[47] or Chapter 9 of [150]). Pólya’s original proof is based on path-counting
and generating functions, and is still the most widely-presented proof of the
SRW result. The proof of the Chung–Fuchs theorem is by Fourier analysis.
The approach via Lyapunov functions, as presented in Examples 3.5.3 and
3.5.4, is due to Lamperti [190], and represents a significant breakthrough, not
only in clarifying the probabilistic intuition behind Pólya’s result, but also
in providing a powerful methodology that is applicable in greater generality
than combinatorial or analytical approaches.
The fundamental semimartingale results Theorems 3.5.6 and 3.5.8 build
on ideas of [190], as well as being relatives of the results of Foster presented
in Chapter 2. Lamperti [190] worked under the non-confinement assumption (L2), and his formulations are somewhat different. The formulation
here, which permits cases where (L2) does not hold, is closely related to
that contained in Kersting [159].
The analysis of branching processes via Lamperti-type methods, as in
Example 3.5.9, goes back to Lamperti himself in a different context [193].
A reference for the classical theory of Galton–Watson processes is [11].
177
Bibliographical notes
Section 3.6
The material on irreducibility and regeneration and their consequences on
countable state-spaces is standard, although no single reference covers the
presentation in this section. A useful reference for regenerative processes is
Chapter VI of [6].
The ergodic theorem for occupation times (Theorem 3.6.9) is a standard
result, and the regenerative structure and irreducibility combine to give an
easy and natural proof. The statement of the result given here is not always
apparent in the standard references, which often focus on the (deeper) result
limn→∞ P[Xn = x] = πx , which requires aperiodicity (cf. Theorem 2.1.6).
Under suitable conditions, there is a central limit theorem to accompany
Theorem 3.6.9, namely,
Ln (x) − nπx
√
n
converges in distribution to a centred normal distribution with finite positive
variance: see e.g. §6.9 of [252] or §VI.3 of [6] for further details.
More generally than Theorem 3.6.9, similar ergodicity results hold for
time-averages of other functionals f : S → R, provided they are integrable
with respect to the limiting measure πx . Specifically, if (I) and (R) hold for
a countable S with 0 ∈ S, and if E κ1 < ∞ then, a.s.,
n−1
X
1 X
1X
f (Xm ) = lim
Ln (x)f (x) =
πx f (x),
n→∞ n
n→∞ n
lim
m=0
x∈S
P
x∈S
provided x∈S πx |f (x)| < ∞; the proof is similar to that given for Theorem 3.6.9. Compare e.g. [6, Theorem VI.3.1, p. 178].
Much of the discussion in this section can be extended to more general
state-spaces, at the expense of technical complications: in that context, detailed accounts of the relevant concepts may be found in the books [247, 270,
246, 239]. Roughly speaking, to obtain analogous results to those presented
here, one needs a concept that fulfils not only the role of irreducibility, but
also that of the local finiteness of the state-space.
Although we do not discuss it in this book, it is an interesting problem to
determine the x-asymptotics of the stationary distribution πx for a positiverecurrent Lamperti process on a locally finite subset of R+ , for which the
conditions (M0), (M1) and (M2) hold with 2a < −b, say. In the case of a
Markov process on Z+ with uniformly bounded increments, Menshikov and
Popov [230] showed that
πx = x−(2a/b)+o(1) , as x → ∞;
178
Chapter 3. Lamperti’s problem
related results appear in [8].
We sketch
P how neighbouring results may be obtained for the associated
tail π̄x := y≥x πy from the ingredients in this chapter. Note that
κX
1 −1
1
1 X
E `1 (y) =
1{Xm ≥ x}.
E
π̄x =
E κ1
E κ1
m=0
y≥x
For a lower bound on π̄x , the idea of Example 2.7.8 gives, for some c > 0
and ε > 0,
2
π̄y ≥ cy P max Xm ≥ 2y ≥ cy 1+(2a/b)−ε ,
0≤m≤λx
by Lemma 3.7.6. On the other hand, an upper bound is
π̄y ≤ E κ1 1{ max Xm ≥ y} ,
0≤m≤κ1
so we have from Hölder’s inequality that
π̄y ≤ E[κp1 ]
i1/p
1/p h
P max Xm ≥ y
.
0≤m≤κ1
By Theorem 3.2.6, for example, the optimal choice is to take p < b−2a
2b as
1+(2a/b)+ε
big as possible, which will give π̄y ≤ cy
. Denisov et al. [72] obtain
sharper asymptotics for the tail under certain conditions; see also [180].
The results of Menshikov and Popov [230] also address the rate of convergence to stationarity; see also [182] in the case of a nearest-neighbour
random walk and [308, 309] for a similar approach to some queueing models. This is one sense in which Lamperti processes exhibit only polynomial
ergodicity rather than geometrical ergodicity.
Sections 3.7 and 3.8
As described in the bibliographical notes for Section 2.7, results on the
existence and non-existence of moments of passage times originate with
Lamperti [192] (for positive integer moments) and were extended in [10]
and subsequently in [7, 9].
Theorem 3.7.1 is closely related to Proposition 1 of [10], which was stated
in the Markovian case. This extended Theorem 2.2 of [192], which applied
only to integer moments, and was again stated in the Markovian case, although Lamperti indicated how it held more generally.
Bibliographical notes
179
Theorems 3.7.2 and 3.7.3 on lower tail bounds and non-existence of moments extend Proposition 2 of [10] and Theorem 3.2 of [192], which were
again stated in the Markovian case. The results of [10] did not provide a
tail bound, while [192] applied only to integer moments. The tail bound
in Theorem 3.7.2 can be derived, under more restrictive assumptions (including uniformly bounded increments), from Corollary 1 of [7]. The proofs
of the lower tail bounds given here, which use Lemma 2.7.7 and lower tail
bounds on excursions, are related to the arguments in [135], where related
excursion bounds are given under slightly different conditions.
The fact that tails of passage times for Lamperti processes satisfy upper and lower bounds of polynomial order is typical of near-critical systems, which exhbit polynomial behaviour; in the case of a positive-recurrent
Markov chain, this is polynomial ergodicity in another sense (closely related
to that described above for stationary distributions).
In the special case of nearest neighbour random walks, Fal’ [91] gives
asymptotics for excursion times and the number of excursions.
Section 3.9
Related almost-bounds in a general Lamperti-type setting are given in [232,
Section 4]; there a uniform bound on the increments was imposed. The
upper bounds in Theorems 3.9.1 and 3.9.2 are variations on Theorems 4.1(i)
and 4.3(i) from [232]; see also Section 6 of [53].
In [232], lower bounds were also given, using a version of Theorem 2.8.3,
but this required a somewhat unnatural ejection condition. The approach
to lower bounds given here, building on the excursion developments of Sections 3.7 and 3.8, follows similar ideas to [135], where slightly sharper bounds
are given under somewhat different assumptions.
In various special cases of certain nearest-neighbour random walks on
Z+ , various results on almost-sure bounds have been obtained; often, using
methods restricted to the nearest-neighbour case, sharper results are available. The diffusive case (cf Theorem 3.9.1) has received most attention, and
iterated-logarithm results are available [275, 93, 299, 112, 310, 140]; of these,
only [140] treats the sub-diffusive case (cf Theorem 3.9.2).
Section 3.10
Theorem 3.10.1 is a substantial improvement of Theorem 4.2 of [232], which
gave the first general almost-sure lower bounds for the transient Lamperti
problem. Theorem 4.2 of [232] gave a lower bound Xn ≥ n1/2 (log n)−D , for
180
Chapter 3. Lamperti’s problem
some constant D not given explicitly, under much more stringent conditions,
including the Markov assumption, a uniform bound on the increments, and
an unnatural ‘re-entry’ condition to overcome the technicality addressed here
by Lemmas 3.10.5 and 3.10.8.
The proof of Theorem 3.10.1 given here improves the proof in [232];
the connection to last exit times via the inequality (3.52) simplifies the
argument. Lemmas 3.10.5 and 3.10.6 extend the idea of Lemma 3.2 of [232]
to permit an exceptional set in the Lyapunov function condition, and to
replace a uniform increments bound by a moments condition. Related ideas
appear in Section 3 of [72].
As mentioned in the text of Example 3.10.3, the rate of escape for transient SRW on Zd (d ≥ 3) was studied by Dvoretzky and Erdős [84]. Essentially the same result holds for spatially homogeneous, zero-drift random
walks under suitable moments assumptions: see e.g. [121, 261] and references
therein.
As mentioned in the text of Example 3.10.4, in the much more restricted setting of nearest-neighbour random walks, a sharp version of Theorem 3.10.1 is given by Csáki et al. [64]. Specifically, under the conditions
of Example 3.10.4, Theorem 6.1 of [64] says that for any ε > 0, a.s., for all
but finitely many n,
1
1
Xn ≥ n 2 (log n) 1+2c −ε ,
and that this result is sharp in that it fails with ε = 0. The proof in [64] proceeds via strong approximation to an appropriate transient Bessel process,
and then appeals to results on the latter.
Another way to quantify transience is via the rate of growth of the renewal function H(x) defined at (3.58). Under conditions similar to those in
Theorem 3.10.1, Denisov et al. [72] show that H(x) is of order x2 as x → ∞.
Section 3.11
The weak convergence result Theorem 3.11.2 was first obtained by Lamperti
under stronger conditions [191]. Although already implicit in Lamperti’s
work, the fact that his results could be extended to processes with different
scaling behaviour was not appreciated more widely until much later: Klebaner [173] and Kersting [160] provided extended versions of Theorem 3.11.2,
although both [173] and [160] admit only the transient case, where Xn → ∞.
The version of Theorem 3.11.2 presented here follows Denisov et al. [72].
For other related recent work, see [24]. A version of Theorem 3.11.2 for
Lamperti-type processes on half strips is given in [115].
181
Bibliographical notes
In [191], Lamperti extended his version of the marginal weak limit result
Theorem 3.11.2 to a full invariance principle, i.e., weak convergence on
the appropriate sample-path space of the scaled Markov chain to a timehomogeneous diffusion process on R+ . The diffusions that arise in the limit
are, up to a deterministic scale factor, Bessel processes. The parameter
(‘dimension’) of the process is given by 1+ 2a
b , where a and b are the constants
in the statement of Theorem 3.11.2.
Section 3.12
The presentation and results in this section are based on [236]. Theorem 3.12.1
is Theorem 2.1 of [236]. The strong law of large numbers Theorem 3.12.4
and accompanying central limit theorem (Theorem 3.12.5) are Theorems 2.4
and 2.5 of [236], respectively. The version of Theorem 3.12.5 stated here
corrects a minor error in the moments assumption stated in Theorem 2.5
of [236]. A version of Theorem 3.12.4 in the case where the moments Markov
property (3.15) holds is contained in Keller et al. [155], among results
that permit more general growth rates. The first result in this direction
was a weak law of large numbers obtained by Lamperti [191]. Specifically,
if Xn is a time-homogeneous Markov process with limx→∞ xβ µ1 (x) = ρ
and supx |µk (x)| < ∞ for all k, where µk is given by (3.1), then Theorem 7.1 of [191] says that (3.75) holds with convergence in probability
(only). The question of a central limit theorem in this context was raised
by Lamperti [191, p. 768], and seems to have remained open until [236] even
for the case of a birth-and-death chain. The paper [155] (a reference unknown to the authors of [236] when that paper was written) builds on earlier
work of Küster [189], and includes certain second order (distributional limit)
results, but none pertinent to Theorem 3.12.5.
The application to the nearest-neighbour random walk of Example 3.12.6
is taken from Section 3.1 of [236]. In the setting of Example 3.12.6, Theorem 3.12.4 generalizes a result of Voit [311, Theorem 2.11], who imposed
the additional assumption that
lim ax and lim cx exist in (0, 1),
x→∞
x→∞
(3.98)
(in which case the two limits must take the same value). Note that there
is a misprint in the limiting constant in the statement of Theorem 2.11
of [311] (the proof there does yield the correct constant): the 1/(1 + α)
power should be applied to the entire limiting expression, not just the µ
there; this typo persists into [65, Theorem D]. The central limit theorem
182
Chapter 3. Lamperti’s problem
presented in Example 3.12.6 is from [236]. Note that the assumption that
limx→∞ bx = b implies that (3.98) holds (with limit 1−b
2 for ax and cx ), so
this central limit theorem can be seen as the natural companion to Voit’s
law of large numbers [311, Theorem 2.11].
We make some final remarks on the case, of secondary interest to us
here, where β = 0 in Example 3.12.6. Theorem 3.12.4 applies to the case
β = 0, i.e., where ax − cx → ρ ∈ (0, 1] as x → ∞, in which case the result
says that n−1 Xn → ρ a.s. as n → ∞. This particular result was obtained by
Pakes [251, Proposition 4], under some more restrictive conditions, including
ρ = 1 and bx ≡ 0, and also, for general ρ but again under conditions more
restrictive than ours, in a result of Voit [310, Corollary 2.6]. In the case
β = 0, the second-order behaviour of Xn is somewhat different; see for
instance [251, Theorem 7] and [310, Theorems 2.7–2.10].
Further remarks
In recent years, significant interest in processes with asymptotically zero
drifts has come from a community of probabilists and statistical physicists
from the point of view of modelling random polymers and interfaces, their
structure, and their interactions with a medium or boundary. In the context
of random polymers, the path of the process models the physical polymer
chain; the asymptotically zero drift indicates the presence of long-range
interaction with a boundary, which can be either attractive or repulsive. For
a random interface, the walk models the behaviour of a liquid interface on a
solid substrate (including wetting and pinning phenomena); in this context
the drift may represent affinity for the boundary. We refer to [70, 116, 307]
for recent surveys, and to [67, 140, 2] for recent works inspired by the random
polymer motivation.
Lamperti-type ideas have been employed in the context of branching
processes (cf. Example 3.5.9); in particular maximal branching processes
were studied by Laperti himself [193, 194], and much more recently in [12].
Another class of models studied by the methods of this chapter are growth
processes, see e.g. [189, 159, 155, 173].
In the context of diffusion processes, Bessel processes can be viewed as a
certain analogue of Lamperti-type processes; the analogy is rather narrow,
as special methods are available for Bessel processes, comparable to the
case of nearest-neighbour random walks. A Bessel process is a R+ -valued
diffusion (Xt , t ∈ R+ ) determined by the stochastic differential equation
dXt =
a
dt + bdBt ,
Xt
Bibliographical notes
183
where Bt is standard Brownian motion on R. Similarly one can consider
supercritical cases, where the drift is of order Xt−β , β ∈ (0, 1), or cases
where the drift is o(1/Xt ): see e.g. [68].
184
Chapter 3. Lamperti’s problem
Chapter 4
Many-dimensional random
walks
4.1
4.1.1
Introduction
Chapter overview: Anomalous recurrence
Famously, a d-dimensional, spatially homogeneous random walk whose increments are non-degenerate, have finite second moments, and have zero
mean is recurrent if d ∈ {1, 2} but transient if d ≥ 3: this is a consequence
of the classical Chung–Fuchs theorem (see Theorem 1.5.2). Once spatial
homogeneity is relaxed, this is no longer true, as discussed in Section 1.5:
there are zero-drift spatially non-homogeneous random walks which, in any
ambient dimension d ≥ 2, can be either transient or recurrent depending on
the details of the jump distributions. Such potential anomalous recurrence
behaviour is the first focus of this chapter. As described in Theorem 1.5.4,
anomalous recurrence behaviour can be excluded under mild assumptions,
provided we impose a fixed covariance structure on the increments of the
walk. In that case, by analogy with the Lamperti problem described in
Chapter 3, it is natural to subsequently relax the assumption of zero drift to
asymptotically zero drift, to probe phase transitions in recurrence behaviour.
The second focus of this chapter is the asymptotically zero drift case, and in
particular the family of examples known as centrally biased random walks.
In this chapter we explore recurrence/transience phase transitions for
spatially non-homogeneous random walks in Rd , and give proofs of the results discussed in Sections 1.5 and 1.7. Now we give an informal overview
of the models and results that we will present, and an outline of the rest of
185
186
Chapter 4. Many-dimensional random walks
this chapter.
In the zero drift case, natural examples of anomalous recurrence behaviour are provided by random walks whose increments are supported on
ellipsoids that are symmetric about the ray from the origin through the
walk’s current position. These elliptic random walks generalize the classical homogeneous Pearson–Rayleigh random walk (PRRW): recall from Example 1.4.2 that a PRRW proceeds via a sequence of unit-length steps, each
in an independent and uniformly random direction in Rd . The PRRW can
be represented via partial sums of sequences of i.i.d. random vectors that are
uniformly distributed on the unit sphere Sd−1 in Rd . Clearly the PRRW has
zero drift. It is well known (cf. Theorem 1.5.2) that the PRRW is recurrent
for d ∈ {1, 2} and transient if d ≥ 3.
Suppose that we replace the spherically symmetric increments of the
PRRW by increments that instead have some elliptical structure, while retaining the zero drift. For example, one could take the increments to be
uniformly distributed on the surface of an ellipsoid of fixed shape and orientation, as represented by the picture on the right of Figure 4.1. More
generally, one should view the ellipses in Figure 4.1 as representing the covariance structure of the increments of the walk (we will give a concrete
example later; the uniform distribution on the ellipse is actually not the
most convenient for calculations).
Figure 4.1: Pictorial representation of spatially homogeneous random walks
with increments distributed on a fixed circle (left) and a fixed ellipse (right).
A little thought shows that the walk represented by the picture on the
right of Figure 4.1 is essentially no different to the PRRW: an affine transformation of Rd will map the walk back to a walk whose increments have
the same covariance structure as the PRRW (cf. Remark 1.5.5). To obtain
4.1. Introduction
187
genuinely different behaviour, it is necessary to abandon spatial homogeneity.
In Section 4.2 we consider a family of spatially non-homogeneous elliptic
random walks with zero drift. These include generalizations of the PRRW
in which the increments are not i.i.d. but have a distribution supported on
an ellipsoid of fixed size and shape but whose orientation depends upon
the current position of the walk. Figure 4.2 gives representations of two
important types of example, in which the ellipsoid is aligned so that its
principal axes are parallel or perpendicular to the vector of the current
position of the walk, which sits at the centre of the ellipse.
Figure 4.2: Pictorial representation of spatially non-homogeneous random
walks with increments distributed on a radially-aligned ellipse with major
axis aligned in the radial sense (left) and in the transverse sense (right).
The random walks represented by Figure 4.2 are no longer sums of
i.i.d. variables. These modified walks can behave very differently to the
PRRW. For instance, one of the two-dimensional random walks represented
in Figure 4.2 is transient while the other (as in the classical case) is recurrent. The reader who has not seen this kind of example before may take a
moment to identify which is which.
The elliptic random walk models represented by Figure 4.2 require uncountably many different increment distributions in order to generate their
anomalous recurrence behaviour. A natural combinatorial question concerns
the minimal number of different increment distributions required; this is the
subject of Section 4.3. One can view this problem as one of control : the
controller is able to implement a number of different increment distributions,
and wishes to minimize this number while achieving the desired recurrence
properties; thus we address this question in the context of controlled random
188
Chapter 4. Many-dimensional random walks
walks with zero drift.
Section 4.4 moves beyond the zero drift setting to discuss walks with
asymptotically zero drift. Analogously to the one-dimensional Lamperti
problem (Chapter 3), this is the critical regime in which to investigate various phase transitions in asymptotic behaviour for the process. In higher
dimensions, one natural model is the centrally biased random walk in which
the mean drift field has magnitude that tends to zero with the distance
from the origin, and direction either away from or towards the origin: see
Figure 4.3.
Figure 4.3: Pictorial representation of centrally biased random walks with
asymptotically drift aligned in the radial sense, outward (left) and inward
(right).
The final section of this chapter, Section 4.5, addresses the different
(but related) topic of the range of a many-dimensional random walk, which
means, roughly speaking, the volume of the state-space visited by the walk
up to a given time.
4.1.2
Notation for many-dimensional random walks
Our notation in this chapter coincides, whenever possible, with that in
Sections 1.4 and 1.5. We work in Rd , d ≥ 1. Our main interest is in
d ≥ 2, as we shall explain in the following sections. Write e1 , . . . , ed for
the standard orthonormal basis vectors in Rd . Write 0 for the origin and
Sd−1 = {u ∈ Rd : kuk = 1} for the unit sphere in Rd , and let k · k denote the Euclidean norm and h · , · i the Euclidean inner product on Rd . For
x ∈ Rd \ {0}, x̂ = x/kxk is the unit vector parallel to x; we use the convention 0̂ := 0. Vectors x ∈ Rd are viewed as column vectors throughout.
189
4.1. Introduction
In most (but not all) of this chapter, we will consider the following basic
setting for our many-dimensional non-homogeneous random walk.
(MD0) Let d ≥ 1. Suppose that (ξn , n ≥ 0) is a discrete-time, timehomogeneous Markov process on an unbounded subset Σ of Rd , with
0 ∈ Σ.
For the increments of the random walk, we use the notation
θn := ξn+1 − ξn , n ∈ Z+ .
By assumption (MD0), given ξ0 , . . . , ξn , the distribution of θn depends only
on the position ξn ∈ Σ and not on n.
We use the shorthand Px [ · ] = P[ · | ξ0 = x] for probabilities when
the walk is started from x ∈ Σ; similarly we use Ex for the corresponding
expectations.
We will typically impose a moments assumption on θn of the form,
sup Ex [kθ0 kp ] < ∞,
(4.1)
x∈Σ
for appropriate p > 0. If (4.1) holds for p ≥ 1, then the one-step mean drift
vector is well defined, and will be denoted
µ(x) := Ex [θ0 ], x ∈ Σ.
(4.2)
If (4.1) holds for p ≥ 2, then the increment covariance matrix is also well
defined, and will be denoted
M (x) := Ex [θ0 θ0> ], x ∈ Σ.
(4.3)
A little linear algebra shows that M (x) encodes various information about
second moments, including, for y ∈ Rd ,
Ex [hy, θ0 i2 ] = Ex [y> θ0 θ0> y] = y> M (x)y,
2
and Ex [kθ0 k ] = tr M (x).
4.1.3
(4.4)
(4.5)
Beyond the Chung–Fuchs theorem
The starting point for much of the discussion in this chapter is the question raised in Section 1.5: to what extent does the recurrence classification for spatially homogeneous random walks extend to the spatially nonhomogeneous case? Our main interest is in the case where the increments
190
Chapter 4. Many-dimensional random walks
of the walk have uniformly bounded second moments, and then it is a
consequence of the Chung–Fuchs theorem that the conclusion of Pólya’s
theorem (Theorem 1.2.1) holds for any zero drift, spatially homogeneous
random walk: see Theorem 1.5.2. We now start to consider in detail the
non-homogeneous case.
We use the notation of Section 4.1.2 and consider a non-homogeneous
random walk ξn on Σ ⊆ Rd , d ≥ 1. We suppose for this section that the
increment moments condition (4.1) holds for some p > 2, so that the mean
drift vector µ(x) given by (4.2) is well defined, and that the random walk
has zero drift:
(MD1) Suppose that µ(x) = 0 for all x ∈ Σ.
Our moments assumption also ensures that the increment covariance matrix
M (x) given by (4.3) is well defined. To rule out pathological cases, we
impose the following non-degeneracy condition on the increments.
(MD2) There exists v > 0 such that tr M (x) ≥ v for all x ∈ Σ.
Note that assumption (MD2) is weaker than uniform ellipticity such as (3.13).
First, we state the following basic non-confinement result.
Lemma 4.1.1. Suppose that (MD0), (MD1), and (MD2) hold, and that
(4.1) holds for some p > 2. Then
lim sup kξn k = +∞, a.s.
(4.6)
n→∞
Lemma 4.1.1, which we prove at the end of this section, ensures that
questions of the escape of trajectories to infinity are non-trivial. We are
then interested in classifying ξn as either recurrent or transient, in the sense
of Definition 1.5.1. If ξn is an irreducible time-homogeneous Markov chain
on a locally finite state-space, these definitions reduce to the usual notions
of recurrence and transience.
In dimension d = 1, it is a consequence of the Chung–Fuchs theorem (see
Chapter 9 of [150]) that a spatially homogeneous random walk with zero
drift is necessarily recurrent. For a spatially non-homogeneous random walk
this is not true in general, and there is a counterexample with increments
of infinite variance that is transient [273]; see the bibliographical notes to
Section 2.5. Our conditions exclude such heavy-tailed phenomena, so that in
d = 1 recurrence is assured in our setting. The following result is a variant
of Theorem 2.5.7, which was stated for a locally finite state space.
191
4.1. Introduction
Theorem 4.1.2. Suppose that (MD0), (MD1), and (MD2) hold, and that
(4.1) holds for some p > 2. Suppose that d = 1. Then ξn is recurrent.
In dimensions d ≥ 2, we first seek an analogue of Theorem 1.5.2. The
next result shows that one recovers the classical recurrence classification for
zero-drift random walk as soon as one imposes a fixed increment covariance
structure across all of space.
Theorem 4.1.3. Suppose that (MD0) and (MD1) hold, and that (4.1) holds
for some p > 2. Suppose that for all x ∈ Σ, M (x) = M for some positivedefinite matrix M . Then ξn is recurrent if d ≤ 2 and transient if d ≥ 3.
A weaker version of Theorem 4.1.3, assuming a uniform bound on the
increments, was stated as Theorem 1.5.4 and established in Example 3.5.4.
Importantly, the zero drift assumption can be relaxed: for example, for
the conclusion of Theorem 4.1.3 one may replace µ(x) = 0 by (essentially)
kµ(x)k = o(kxk−1 ), as we shall see later. This fact leads naturally to the
question of the extent to which one can ‘perturb’ a zero-drift walk and preserve its recurrence or transience. Lamperti showed that the critical scale for
perturbations is O(kxk−1 ). Roughly speaking, for any walk satisfying (4.1)
for p > 2, regardless of its covariance structure, one can arrange µ with
kµ(x)k = O(kxk−1 ) for which the walk is recurrent or transient, as desired:
see Section 4.1.5 for details.
We prove Theorems 4.1.2 and 4.1.3 in Section 4.1.5, after presenting a
more general result (Theorem 4.1.7). The rest of this section is devoted to
the proof of Lemma 4.1.1. We first present a general result for martingales
on Rd satisfying a non-degeneracy condition; the result can be viewed as
a d-dimensional martingale version of Kolmogorov’s other inequality (Theorem 2.4.12). Theorem 4.1.4 extends the one-dimensional martingale result Theorem 2.4.12 to higher dimensions, and also replaces the uniformly
bounded jumps assumption by a moments condition. The proof here uses a
different idea from that of Theorem 2.4.12.
Theorem 4.1.4. Let d ∈ N. Suppose that (Yn , n ≥ 0) is an Rd -valued
process adapted to a filtration (Fn , n ≥ 0), with P[Y0 = 0 | F0 ] = 1. Suppose
that there exist a stopping time η and constants p > 2, v > 0, B < ∞ such
that for all n ≥ 0, a.s.,
E[kYn+1 − Yn kp | Fn ] ≤ B;
(4.7)
2
(4.8)
E[Yn+1 − Yn | Fn ] = 0.
(4.9)
E[kYn+1 − Yn k | Fn ] ≥ v1{n < η};
192
Chapter 4. Many-dimensional random walks
Then there exists D < ∞, depending only on B, p, and v, such that for all
n ∈ Z+ and all x ∈ R+ ,
i
h
D(1 + x)2
P max kYm k ≥ x F0 ≥ 1 −
− P[η ≤ n | F0 ], a.s.
0≤m≤n
n
Proof. Let x ∈ R+ and set σ = min{n ≥ 0 : kYn k ≥ x}. For the increments
of the process write Dn = Yn+1 − Yn , and let
(
Yn
if kYn k ≤ A(1 + x),
Wn =
b
Yn−1 + Dn−1 (A − 1)(1 + x) if kYn k > A(1 + x),
where A > 1 is a constant to be specified later. Note that Wn is Fn measurable. Now, on {kYn k ≤ x}, Wn = Yn and
E[Wn+1 − Wn | Fn ] = E[Dn | Fn ]
b n ((A − 1)(1 + x) − kDn k) 1{kYn+1 k > A(1 + x)} | Fn ].
+ E[D
But {kYn+1 k > A(1 + x)} ∩ {kYn k ≤ x} implies that kDn k > (A − 1)(1 + x),
and by (4.9), E[Dn | Fn ] = 0. Hence, on {kYn k ≤ x},
E[Wn+1 − Wn | Fn ] ≤ E[kDn k1{kDn k > (A − 1)(1 + x)} | Fn ]
≤ (A − 1)−1 (1 + x)−1 E[kDn k2 | Fn ]
≤ B 0 (A − 1)−1 (1 + x)−1 , a.s.,
where, by (4.7) and Lyapunov’s inequality, B 0 < ∞ depends only on B and
p. Hence we can choose A ≥ A0 for some A0 = A0 (B, p, v) large enough so
E[Wn+1 − Wn | Fn ] ≤ (v/8)(1 + x)−1 , on {kYn k ≤ x}.
(4.10)
Also, on {kYn k ≤ x}, by a similar argument, E[kWn+1 − Wn k2 | Fn ] equals
E[kDn k2 | Fn ]
+ E[ (A − 1)2 (1 + x)2 − kDn k2 1{kYn+1 k > A(1 + x)} | Fn ]
≥ E[kDn k2 | Fn ] − E[kDn k2 1{kDn k > (A − 1)(1 + x)} | Fn ]
≥ v1{n < η} − (A − 1)2−p (1 + x)2−p E[kDn kp | Fn ]
≥ (v/2)1{n < η},
(4.11)
for all x ≥ 0 and A ≥ A1 for some sufficiently large A1 = A1 (B, p, v), using
(4.7) and (4.8).
193
4.1. Introduction
Now, set Zn = kWn∧σ∧η k2 . Then, on {n < σ ∧ η}, by (4.10) and (4.11),
E[Zn+1 − Zn | Fn ] = E[kWn+1 k2 − kWn k2 | Fn ]
= E[kWn+1 − Wn k2 | Fn ] + 2 Wn , E[Wn+1 − Wn | Fn ]
v
2kWn kv
v
vx
v
≥ −
≥ −
≥ .
2 8(1 + x)
2 4(1 + x)
4
Hence Zn −
Pn−1
k=0
vk =
vk is an Fn -adapted submartingale, where
v
v
1{k < σ ∧ η} ≥ 1{n < σ ∧ η}, for 0 ≤ k < n.
4
4
By construction, 0 ≤ Zn ≤ A2 (1 + x)2 , so
0 = E[Z0 | F0 ] ≤ E[Zn | F0 ] −
n−1
X
E[vk | F0 ]
k=0
n−1
X
≤ A2 (1 + x)2 −
k=0
v
P[n < σ ∧ η | F0 ],
4
which implies that n(v/4) P[n < σ ∧ η | F0 ] ≤ A2 (1 + x)2 . Hence, since
i
h
P max kYm k < x F0 = P[n < σ | F0 ]
0≤m≤n
= P[n < σ, n < η | F0 ] + P[n < σ, n ≥ η | F0 ]
≤ P[n < σ ∧ η | F0 ] + P[η ≤ n | F0 ],
the statement of the theorem follows.
Now we can give the proof of Lemma 4.1.1.
Proof of Lemma 4.1.1. Let Xn = kξn k and Fn = σ(ξ0 , . . . , ξn ). Then the
assumption (MD0) on ξn shows that Xn satisfies (L0) with S = {kxk :
x ∈ Σ} ⊆ R+ ; note that inf S = 0 since 0 ∈ Σ and sup S = ∞ since Σ is
unbounded.
To prove the non-confinement statement, we show that Xn satisfies (L3)
and then apply Proposition 3.3.4. Thus we must verify (3.10). By the
Markov property and time-homogeneity, it suffices to check that for any
x ∈ R+ there exist rx ∈ N and δx > 0 such that
i
h
P max Xm ≥ x F0 ≥ δx , on {X0 ≤ x}.
(4.12)
0≤m≤rx
194
Chapter 4. Many-dimensional random walks
To verify (4.12) we apply Theorem 4.1.4 with Yn = ξn − ξ0 ; the assumptions (4.1), (MD2), and (MD1) imply that the conditions (4.7), (4.8), and
(4.9) are satisfied (with η = ∞). Hence the triangle inequality and Theorem 4.1.4 show that, on {kξ0 k ≤ x},
i
i
h
h
P max Xm ≥ x F0 ≥ P max kξm − ξ0 k ≥ 2x F0
0≤m≤n
0≤m≤n
≥1−
D(1 + 2x)2
, a.s.
n
Thus for any x we may choose n = rx big enough so that this last probability
is at least 1/2, say, which verifies (4.12) and completes the proof.
4.1.4
Relation to Lamperti’s problem
The basic idea behind the proof of Theorem 4.1.3 is the same as that described in Section 1.5 and carried through in Example 3.5.4, namely to
consider the one-dimensional process Xn = kξn k, which can be studied by
the methods of Chapter 3. The additional technical difficulty introduced by
assuming (4.1) rather than uniformly bounded increments is overcome using truncation ideas similar to those employed in Theorems 2.5.7 and 2.5.18,
and repeatedly in Chapter 3.
Let Fn = σ(ξ0 , . . . , ξn ). The assumption (MD0) on ξn shows that Xn
satisfies (L0) with S = {kxk : x ∈ Σ} ⊆ R+ ; note that inf S = 0 since
0 ∈ Σ and sup S = ∞ since Σ is unbounded. As in Chapter 3, we write
∆n = Xn+1 − Xn . To apply the results of Chapter 3, we must study the
increments ∆n given ξn = x ∈ Σ; in general, Xn is not itself a Markov
process.
We make an important comment on notation. When we write O(kxk−1−δ ),
and similar expressions, these are understood to be uniform in x. That is,
if f : Rd → R and g : R+ → R+ , we write f (x) = O(g(kxk)) to mean that
there exist C ∈ R+ and r ∈ R+ such that
|f (x)| ≤ Cg(kxk) for all x ∈ Σ with kxk ≥ r.
(4.13)
Lemma 4.1.5. Suppose that (MD0) holds, and that (4.1) holds for some
p > 0. Then, for Xn := kξn k, we have
sup E[|∆n |p | ξn = x] < ∞.
(4.14)
x∈Σ
Moreover, the radial increment moment functions satisfy the following.
195
4.1. Introduction
(i) If p > 1, then, for any r > 0 with r < (p ∧ 2) − 1, as kxk → ∞,
E[∆n | ξn = x] = hx̂, µ(x)i + O(kxk−r ).
(4.15)
(ii) If p > 2, then, for some δ = δ(p) > 0, as kxk → ∞,
E[∆n | ξn = x] = hx̂, µ(x)i +
tr M (x) − x̂> M (x)x̂
+ O(kxk−1−δ );
2kxk
(4.16)
E[∆2n | ξn = x] = x̂> M (x)x̂ + O(kxk−δ ).
(4.17)
Lemma 4.1.5 is central to our analysis of the recurrence and transience
of ξn , by applying the results of Chapter 3 to Xn = kξn k. For example, if
µ(x) = 0 then (4.16) shows that the drift of Xn is of order 1/Xn , the critical
regime for Lamperti processes: see Section 4.1.5.
The rest of this section is devoted to the proof of Lemma 4.1.5. The
following notation will be useful. Given x 6= 0 and y ∈ Rd , write
yx :=
hx, yi
= hx̂, yi,
kxk
(4.18)
so that yx is the component of y in the x̂ direction, and y − yx x̂ is a vector
orthogonal to x̂. For convenience, we write simply θ for θ0 = ξ1 − ξ0 , and let
θx be the radial component of θ at ξ0 = x in accordance with the notation
defined at (4.18); no confusion should arise with our notation θn defined
previously.
Given ξ0 = x, the increment ∆0 = X1 − X0 is given in terms of x and
the increment θ = ξ1 − ξ0 by
p
kx + θk − kxk = hx + θ, x + θi − kxk
#
"
1/2
kθk2
2θx
(4.19)
= kxk 1 +
+
−1 .
kxk kxk2
We would like to use Taylor’s theorem here, as in Example 3.5.4, but this is
only legitimate provided θ is not too big compared to x. Thus let
Ax := {kθk ≤ kxkγ },
(4.20)
for some γ ∈ (0, 1) to be chosen appropriately. On the event Ax we approximate (4.19) using Taylor’s formula for (1 + y)1/2 , and on the event Acx
we bound (4.19) using (4.1). For various truncation estimates, we need the
following analogue of Lemma 3.4.2.
196
Chapter 4. Many-dimensional random walks
Lemma 4.1.6. Suppose that (4.1) holds for some p > 0, and Ax is as
defined at (4.20) for some γ ∈ (0, 1). Then, for any q ∈ [0, p],
Ex kθkq 1(Acx ) = O(kxk−γ(p−q) ).
Proof. By definition of Ax , we have for q ∈ [0, p],
kθkq 1(Acx ) = kθkp kθkq−p 1(Acx ) ≤ kθkp kxkγ(q−p) ,
and the result follows on taking expectations, by (4.1).
Now we can complete the proof of Lemma 4.1.5 by carrying through the
scheme outlined above.
Proof of Lemma 4.1.5. By time-homogeneity, it suffices to consider
the case
n = 0. By the triangle inequality, |X1 − X0 | = kξ0 + θk − kξ0 k ≤ kθk, so
that (4.14) follows from (4.1).
Define the event Ax as at (4.20), for some γ ∈ (0, 1) to be specified later.
We prove part (ii); so suppose that (4.1) holds for p > 2. For all y > −1,
Taylor’s formula with Lagrange remainder shows that
1
1
(1 + y)1/2 = 1 + y − y 2 (1 + ϕy)−3/2 ,
2
8
for some ϕ = ϕ(y) ∈ [0, 1], so on the event Ax , kx + θk − kxk is equal to
!
2
θx
kθk2
1 2θx
kθk2
γ−1
kxk
+
−
+
1 + O(kxk ) ,
kxk 2kxk2 8 kxk kxk2
where θx = hx̂, θi as defined at (4.18). Here, on the event Ax ,
2θx
kθk2
+
kxk kxk2
2
=
4θx2
+ kθk2 O(kxkγ−3 ),
kxk2
using the fact that |θx | ≤ kθk ≤ kxkγ for γ < 1. Hence, on the event Ax ,
kx + θk − kxk = θx +
kθk2 − θx2
+ kθk2 O(kxkγ−3 ).
2kxk
It follows from (4.21) and the triangle inequality that
2
2 (kx + θk − kxk) 1(Ax ) − θx + kθk − θx 2kxk
(4.21)
197
4.1. Introduction
kθk2 − θx2 1(Acx ) + kθk2 O(kxkγ−3 )
≤ θx +
2kxk kθk2
≤ kθk1(Acx ) +
1(Acx ) + kθk2 O(kxkγ−3 ).
2kxk
(4.22)
Here, by the q = 1 and q = 2 cases of Lemma 4.1.6, we have
Ex [kθk1(Acx )] = O(kxk−γ(p−1) ), and Ex [kθk2 1(Acx )] = O(kxk−γ(p−2) ),
so that, taking expectations in (4.22), we obtain
kθk2 − θx2 Ex (kx + θk − kxk) 1(Ax ) − θx +
2kxk
= O(kxk−γ(p−1) ) + O(kxk−γ(p−2)−1 ) + O(kxkγ−3 ).
1+δ
< γ < 1. Then
Let δ ∈ (0, 1) with δ < p − 2. Choose γ such that p−1
−γ(p − 2) − 1 < −γ(p − 1) < −1 − δ and γ − 3 < −1 − δ, so it follows that
Ex [kθk2 − θx2 ]
Ex (kx + θk − kxk) 1(Ax ) = Ex θx +
+ O(kxk−1−δ )
2kxk
tr M (x) − x̂> M (x)x̂
= hx̂, µ(x)i +
+ O(kxk−1−δ ),
(4.23)
2kxk
using (4.4), (4.5), and the observation that, by linearity,
Ex θx = Ex hx̂, θi = hx̂, Ex θi = hx̂, µ(x)i.
On the other hand, by the triangle inequality,
Ex kx + θk − kxk1(Acx ) ≤ Ex kθk1(Acx ) = O(kxk−γ(p−1) ),
(4.24)
by the q = 1 case of Lemma 4.1.6; again, by choice of γ, we have that the
right-hand side of (4.24) is O(kxk−1−δ ). Since
kx + θk − kxk = kx + θk − kxk 1(Ax ) + kx + θk − kxk 1(Acx ),
we can combine (4.23) and (4.24) to obtain (4.16). For the second moment,
we use the identity
(kx + θk − kxk)2 = kx + θk2 − kxk2 − 2kxk(kx + θk − kxk)
= 2kxkθx + kθk2 − 2kxk(kx + θk − kxk),
198
Chapter 4. Many-dimensional random walks
so that
Ex [∆20 ] = 2kxk Ex θx + Ex [kθk2 ] − 2kxk Ex ∆0 = Ex [θx2 ] + O(kxk−δ ),
by (4.16), which gives (4.17). This completes the proof of part (ii).
The proof of part (i) is similar, now supposing that (4.1) holds for p > 1
only. In this case, Taylor’s theorem shows that the analogue of (4.21) is
kx + θk − kxk 1(Ax ) = hx̂, θi1(Ax ) + kθk2 1(Ax )O(kxk−1 ).
(4.25)
Here, by definiton of Ax ,
+
+
Ex [kθk2 1(Ax )] ≤ Ex [kθkp kθk(2−p) 1(Ax )] ≤ kxkγ(2−p) Ex [kθkp ],
+
which is O(kxkγ(2−p) ) by (4.1), and, as shown above, Ex [kθk1(Acx )] =
O(kxk−γ(p−1) ) , so
Ex [hx̂, θi1(Ax )] = hx̂, Ex θi + O(kxk−γ(p−1) ).
Thus taking expectations in (4.25) we obtain
Ex
+
kx + θk − kxk 1(Ax ) = hx̂, Ex θi + O(kxk−γ(p−1) ) + O(kxkγ(2−p) −1 ).
On the other hand, (4.24) still applies, which contributes another O(kxk−γ(p−1) )
term, so that
+ −γ
Ex [kx + θk − kxk] = hx̂, µ(x)i + O(kxkγ(2−p)
),
since γ(1 − p) = γ(2 − p) − γ ≤ γ(2 − p)+ − γ. Since γ ∈ (0, 1) was arbitrary,
and (2 − p)+ − 1 = 1 − (p ∧ 2) < 0, we obtain (4.15).
4.1.5
Sufficient conditions for recurrence and transience
In this section we present sufficient conditions for recurrence and transience
of the non-homogeneous random walk ξn , assuming (MD0) and that (4.1)
holds for some p > 2. In the zero drift case, the main result of this section
(Theorem 4.1.7) will allow us to deduce Theorems 4.1.2 and 4.1.3, but the
result is also valid for the non-zero drift case, and will later enable us to
deduce results on elliptic random walks (Section 4.2) and centrally biased
random walks (Section 4.4).
For this section, since we do not impose a zero drift condition, we cannot
rely on Lemma 4.1.1; instead we must assume a non-confinement condition.
199
4.1. Introduction
(MD3) Suppose that lim supn→∞ kξn k = +∞, a.s.
In the case where Σ is locally finite and ξn is irreducible, (MD3) is satisfied,
as shown in Corollary 2.1.10.
For some of our results, we also need to assume a non-degeneracy condition on the increment covariance matrix M (x) as defined at (4.3), to ensure
that the walk is genuinely d-dimensional.
(MD4) Suppose that there exists v0 > 0 such that,
inf e> M (x)e ≥ v0 , for all x ∈ Σ.
e∈Sd
(4.26)
The assumption (MD4) is stronger than (MD2), in that it imposes a
uniform lower bound not just on the trace of M (x) but on all its eigenvalues,
but weaker than uniform ellipticity, since (3.13) implies (4.26).
Theorem 4.1.7. Suppose that (MD0) and (MD3) hold, and that (4.1) holds
for some p > 2.
(i) A sufficient condition for ξn to be transient is
lim
inf
2kxkhx̂, µ(x)i + tr M (x) − 2x̂> M (x)x̂ > 0. (4.27)
r→∞ x∈Σ:kxk≥r
(ii) If (MD4) also holds, then a sufficient condition for ξn to be recurrent
is, as r → ∞,
2kxkhx̂, µ(x)i+tr M (x)−2x̂> M (x)x̂ ≤ o(log−1 r). (4.28)
sup
x∈Σ:kxk≥r
(iii) Let λr = min{n ≥ 0 : kξn k ≤ r}, and suppose that
lim
sup
2kxkhx̂, µ(x)i + tr M (x) < 0.
r→∞ x∈Σ:kxk≥r
(4.29)
Then there exists r0 ∈ R+ such that, for any r > r0 , E λr < ∞.
One interpretation of Theorem 4.1.7 is as follows: for fixed dimension
d ∈ N and increment covariance matrix function M (x), there is a constant
ctra such that a sufficient condition to ensure that ξn is transient is
lim inf kxkhµ(x), x̂i > ctra ;
n→∞
200
Chapter 4. Many-dimensional random walks
specifically, one may take
ctra = lim sup x̂> M (x)x̂ − 21 tr M (x) .
r→∞ kxk≥r
Analogously, given (MD4), a sufficient for recurrence is the condition
lim sup kxkhµ(x), x̂i < crec ,
n→∞
where
crec = lim inf
r→∞ kxk≥r
x̂> M (x)x̂ − 21 tr M (x) .
In other words, given M (x) one can always arrange a drift field µ(x), satisfying kµ(x)k = O(kxk−1 ), such that the walk is transient; or recurrent; or
positive-recurrent. We give some concrete examples in Section 4.4. In Section 4.2 we will apply Theorem 4.1.7 to show that in any dimension d ≥ 2,
even if µ(x) = 0 everywhere, by suitably adjusting M (x) one can achieve
transience or recurrence as desired (but not positive-recurrence).
One useful corollary to Theorem 4.1.7 is the following sufficient condition
for transience in the zero-drift setting. Recall that λmax (M ) denotes the
maximum eigenvalue of matrix M .
Corollary 4.1.8. Suppose that (MD0), (MD1) and (MD3) hold, and that (4.1)
holds for some p > 2. A sufficient condition for ξn to be transient is
lim
inf
tr M (x) − 2 λmax (M (x)) > 0.
(4.30)
r→∞ x∈Σ:kxk≥r
Proof. We apply Theorem 4.1.7(i). We have from (MD1) that
2kxkhx̂, µ(x)i + tr M (x) − 2x̂> M (x)x̂ ≥ tr M (x) − 2 sup
u> M (x)u
u∈Sd−1
= tr M (x) − 2 λmax (M (x)),
by the variational characterization of the maximal eigenvalue. Under condition (4.30), we thus have that (4.27) holds, and then Theorem 4.1.7(i)
completes the proof.
Here is a different example of the application of Theorem 4.1.7.
Example 4.1.9. Suppose that ξn satisfying (MD0) is an irreducible Markov
chain on Zd , d ≥ 1, with transition probabilities given for x 6= 0 by
1
ε
1
ε
Px [θ0 = ei ] =
1−
and Px [θ0 = −ei ] =
1+
,
2d
hx, ei i
2d
hx, ei i
201
4.1. Introduction
for all i ∈ {1, . . . , d}, where ε ∈ (−1, 1) is a fixed parameter, and
P0 [θ0 = ei ] = P0 [θ0 = −ei ] =
1
.
2d
Then we calculate M (x) = d1 Id and, for x 6= 0,
d
ε X ei
µ(x) = −
.
d
hx, ei i
i=1
Thus for x 6= 0,
2kxkhx̂, µ(x)i + tr M (x) − 2x̂> M (x)x̂ = −2ε + 1 − (2/d),
which is strictly positive if and only if ε < d−2
2d . Thus Theorem 4.1.7 parts
(i) and (ii) show that this random walk is transient if ε < d−2
2d and recurrent
d−2
if ε ≥ 2d . Similarly, part (iii) shows that the walk is positive-recurrent if
ε > 21 .
4
Proof of Theorem 4.1.7. We consider the process of norms Xn = kξn k and
use Lemma 4.1.5 to apply the results of Chapter 3. The condition (MD3)
shows that Xn = kξn k satisfies (L2). Given that (4.1) holds for some p > 2,
Lemma 4.1.5 shows that Xn = kξn k satisfies (L1) for the same p > 2. In the
notation of Section 3.3, we have for k ∈ {1, 2} that
µk (x) ≥
inf
x∈Σ:kxk≥x
E[∆kn | ξn = x],
and similarly for µ̄k (x). Lemma 4.1.5(ii) shows that
2kxk E[∆n | ξn = x] − E[∆2n | ξn = x]
= 2kxkhx̂, µ(x)i + tr M (x) − 2x̂> M (x)x̂ + O(kxk−δ ).
Suppose that (4.27) holds. Then,
lim inf 2xµ1 (x) − µ̄2 (x) > 0,
x→∞
and Theorem 3.5.1 yields transience, proving part (i). Now consider part
(ii). It follows from (MD4) and (4.17) that
lim inf µ2 (x) ≥ lim inf x> M (x)x̂ ≥ v0 .
(4.31)
x→∞
r→∞ x∈Σ
Moreover, it follows from (4.16) and (4.17) that
2xµ̄1 (x) − µ2 (x)
202
Chapter 4. Many-dimensional random walks
≤
sup
x∈Σ:kxk≥x
2kxkhx̂, µ(x)i + tr M (x) − 2x̂> M (x)x̂ + O(x−δ ),
which is o(log−1 x) by (4.28). Together with (4.31), this shows that
1−ψ
1−ψ
2xµ̄1 (x) − 1 +
µ2 (x) ≤ −
µ (x) + o(log−1 x)
log x
log x 2
≤ − (1 − ψ)v0 + o(1) log−1 x,
which is negative for ψ ∈ (0, 1) and all x sufficiently large, so Theorem 3.5.2
yields recurrence. This proves part (ii).
Finally, a similar argument shows that (4.29) implies that
lim sup 2xµ̄1 (x) + µ̄2 (x) < 0,
x→∞
from which the α = 1 case of Theorem 3.7.1 gives part (iii).
To conclude this section, we apply Theorem 4.1.7 to give the proofs of
Theorems 4.1.2 and 4.1.3.
Proof of Theorem 4.1.2. Suppose that d = 1. In one dimension, M (x) is the
scalar Ex [θ02 ], and the condition (MD2) is equivalent to (MD4). Lemma 4.1.1
shows that (MD3) holds. Moreover, the zero drift condition (MD1) then
shows that
2kxkhx̂, µ(x)i + tr M (x) − 2x̂> M (x)x̂ = − Ex [θ02 ] ≤ −v,
by (MD2). Hence the conditions of Theorem 4.1.7(ii) are satisfied, and this
establishes recurrence.
Proof of Theorem 4.1.3. First, the argument in Remark 1.5.5 shows that,
by considering an affine transformation of Rd , it suffices to consider the case
where M = Id . Again, Lemma 4.1.1 shows that (MD3) holds. Given that
M (x) = Id , we have tr M (x) = d and, for any u ∈ Sd−1 , u> M (x)u = 1;
thus (MD4) holds. Now, we have
2kxkhx̂, µ(x)i + tr M (x) − 2x̂> M (x)x̂ = d − 2,
which is strictly positive if d ≥ 3, in which case transience follows from
Theorem 4.1.7(i), and non-positive if d ≤ 2, in which case recurrence follows
from Theorem 4.1.7(ii).
203
4.2. Elliptic random walks
4.2
Elliptic random walks
4.2.1
Recurrence classification
Theorem 4.1.2 shows that in d = 1, under mild conditions, the classical
Chung–Fuchs recurrence classification for homogeneous zero-drift random
walks extends to zero-drift non-homogeneous random walks. This extension
fails for dimension d ≥ 2; in this section we demonstrate this, and hence
prove Theorem 1.5.3. To do so, we introduce a family of non-homogeneous
random walks that we call elliptic random walks, that demonstrate the possible anomalous behaviour of non-homogeneous random walks when compared to classical homogeneous random walks.
The following assumption on the asymptotic stability of the covariance
structure of the process along rays is central; recall that k · kop denotes the
matrix (operator) norm.
(E1) Suppose that there exists a positive-definite matrix function σ 2 with
domain Sd−1 such that, as r → ∞,
ε(r) :=
sup
x∈Σ:kxk≥r
kM (x) − σ 2 (x̂)kop → 0.
A little informally, (E1) says that M (x) → σ 2 (x̂) as kxk → ∞; in what
follows, we will often make similar statements, formal versions of which may
be cast as in (E1).
Note that (MD2) and (E1) together imply that tr σ 2 (u) ≥ v > 0; next we
impose a key assumption on the form of σ 2 that is considerably stronger. To
describe this, it is convenient to introduce the notation h · , · iu that defines,
for each u ∈ Sd−1 , an inner product on Rd via
hy, ziu := y> σ 2 (u)z = hy, σ 2 (u)zi, for y, z ∈ Rd .
(E2) Suppose that there exist constants U and V with 0 < U ≤ V < ∞
such that, for all u ∈ Sd−1 ,
hu, uiu = U, and tr σ 2 (u) = V.
Informally, V quantifies the total variance of the increments, while U quantifies the variance in the radial direction; necessarily U ≤ V . The assumption
that U > 0 excludes some degenerate cases. As we will see, one possible
way to satisfy condition (E2) is to suppose that the eigenvectors of σ 2 (u)
are all parallel or perpendicular to the vector u, and that the corresponding
204
Chapter 4. Many-dimensional random walks
eigenvalues are all constant as u varies; the level sets of the corresponding
quadratic forms qu (x) := hx, xiu for u ∈ Sd−1 are then ellipsoids like those
depicted in Figure 4.2.
Our main result on elliptic random walks is the following, which shows
that both transience and recurrence are possible for any d ≥ 2, depending on
parameter choices; as seen in Theorem 4.1.2, this possibility of anomalous
recurrence behaviour is a genuinely multidimensional phenomenon under
our regularity conditions.
Theorem 4.2.1. Suppose that (MD0), (MD1), and (MD2) hold, and that
(4.1) holds for some p > 2. Suppose also that (E1) and (E2) hold, with
constants 0 < U ≤ V < ∞ as defined in (E2). Then the following recurrence
classification is valid.
(i) If 2U < V , then ξn is transient.
(ii) If 2U > V , then ξn is recurrent.
(iii) If 2U = V and (E1) holds with ε(r) = o(log−1 r), then ξn is recurrent.
An explicit family of examples satisfying conditions (E1) and (E2) is
presented in Section 4.2.2. We deduce Theorem 4.2.1 by another application
of Theorem 4.1.7.
Proof of Theorem 4.2.1. Lemma 4.1.1 shows that (MD3) holds. By definition of ε(r) at (E1) we have kM (x) − σ 2 (x̂)kop = O(ε(kxk)) as kxk → ∞.
Then (E2) implies that
tr M (x) = tr σ 2 (x̂) + O(ε(kxk)) = V + O(ε(kxk)); and
x̂> M (x)x̂ = hx̂, σ 2 (x̂)x̂i + O(ε(kxk)) = U + O(ε(kxk)).
These expressions, together with (MD1), show that
sup
x∈Σ:kxk≥r
2kxkhx̂, µ(x)i + tr M (x) − 2x̂> M (x)x̂ − (V − 2U ) = O(ε(r)).
Given that ε(r) → 0, if V − 2U > 0, then Theorem 4.1.7(i) yields transience,
while if V − 2U < 0, then Theorem 4.1.7(ii) yields recurrence. In the boundary case V − 2U = 0, Theorem 4.1.7(ii) again yields recurrence, provided
that ε(r) = o(log−1 r).
205
4.2. Elliptic random walks
4.2.2
Example: Increments supported on ellipsoids
Let d ≥ 2. We describe a specific model on Σ = Rd where the jump distribution at x ∈ Rd is supported on an ellipsoid having one distinguished axis
aligned with the vector x. The model is specified by two constants a, b > 0.
Construct θ as follows. Let Qx̂ be an orthogonal matrix
representing a
√
transformation of Rd mapping e1 to x̂, and write D = d diag (a, b, . . . , b).
Given X0 = x, take ζ uniform on Sd−1 and set
(
Qx̂ Dζ if x 6= 0;
θ=
(4.32)
ζ
if x = 0.
See Figure 4.4 for an illustration of this construction.
Dζ
θ
ζ
x
0
Figure 4.4: Definition of θ = Qx̂ Dζ.
Thus θ is a random
√ point on an ellipsoid that has one distinguished
semi-axis, of
√ length a d, aligned in the x̂ direction, and all other semi-axes
of length b d. Note that the law of θ is well defined, since the distribution
of Qx̂ Dζ is the same for any orthogonal matrix Qx̂ satisfying Qx̂ e1 = x̂.
Concretely, observe that for x 6= 0, θ can be rewritten as
θ = Qx̂ DQ>x̂ Qx̂ ζ = Qx̂ DQx̂> ζ̃,
where ζ̃ = Qx̂ ζ is also uniform on Sd−1 , by the spherical symmetry of the
uniform distribution. Moreover, the symmetric matrix Hx̂ := Qx̂ DQ>x̂ is
determined explicitly in terms of x̂ via
√
√
Hx̂ = Qx̂ (b dId + (a − b) de1 e>1 )Qx̂>
√
√
= b dId + (a − b) dx̂x̂> .
206
Chapter 4. Many-dimensional random walks
Thus a more concrete specification of θ is
√
√
θ = Hx̂ ζ̃ = b dζ̃ + (a − b) dx̂hx̂, ζ̃i,
with ζ̃ taken to be uniform on Sd−1 .
As a corollary to Theorem 4.2.1, we get the following recurrence classification for this example.
Corollary 4.2.2. Let d ≥ 1 and a, b ∈ (0, ∞). Let ξn be a random walk
on Rd with distribution determined by (4.32) for parameters a, b ∈ (0, ∞).
Then ξn is transient if a2 < (d − 1)b2 and recurrent if a2 ≥ (d − 1)b2 .
√
Proof. Since kθk is bounded above by d max{a, b}, assumption (4.1) holds
for all p > 2. Moreover, by (4.32), we compute
M (x) = Ex [θθ> ] = E[Qx̂ Dζζ > DQ>x̂ ] = Qx̂ D E[ζζ > ]DQ>x̂ =
1
Qx̂ D2 Q>x̂ ,
d
by linearity of expectation, and using the fact that E[ζζ > ] =
uniformly distributed on Sd−1 . Hence (E1) holds with
σ 2 (u) =
1
d Id
for ζ
1
Qu D2 Q>u , for u ∈ Sd−1 ,
d
and with ε(r) identically zero. Also, by a similar calculation (or a symmetry argument) we have µ(x) = 0 for all x ∈ Rd , since E ζ = 0; thus
(MD1) holds. Finally, to verify that (MD2) and (E2) hold, note that the
matrix σ 2 (u) represented in coordinates for the orthonormal basis {Qu e1 =
u, Qu e2 , . . . , Qu ed } is diagonal with entries a2 , b2 , . . . , b2 . Indeed,
σ 2 (u) = Qu [b2 Id + (a2 − b2 )e1 e>1 ]Q>u
= a2 uu> + b2 (Id − uu> ),
and therefore hu, uiu = hu, σ 2 (u)ui = a2 > 0 for all u ∈ Sd−1 , and
tr M (x) = tr σ 2 (x̂) = a2 + (d − 1)b2 > 0 for all x ∈ Rd . Thus we verify
all the assumptions of Theorem 4.2.1, with U = a2 and V = a2 + (d − 1)b2 ,
and the claimed recurrence classification follows.
In two dimensions we can explicitly describe the random walk as follows.
For x ∈ R2 , x 6= 0 with x = (x1 , x2 ) in Cartesian components, set x⊥ :=
(−x2 , x1 ). Fix a, b ∈ (0, ∞). Let Ex (a, b) denote the ellipse with
√ centre
√x
and principal axes aligned in the x, x⊥ directions, with lengths 2 2a, 2 2b
respectively, given in parametrized form by
√
√ x⊥
x
cos ϕ + 2b
sin ϕ : ϕ ∈ (−π, π] , (4.33)
Ex (a, b) := x + 2a
kxk
kxk
4.3. Controlled driftless random walks
207
and for x = 0 set
E0 (a, b) := {e1 cos ϕ + e2 sin ϕ : ϕ ∈ (−π, π]} ,
the unit circle. The parameter ϕ in the parametrization (4.33) should be
interpreted with caution: it is not, in general, the central angle of the parametrized point on the ellipse.
Given ξn = x ∈ R2 , ξn+1 is taken to be distributed on Ex (a, b), ‘uniformly’ with respect to the parametrization (4.33). Precisely, let ϕ0 , ϕ1 , . . .
be a sequence of independent random variables uniformly distributed on
(−π, π]. Then, on {ξn 6= 0},
√
√
ξn
ξ⊥
ξn+1 = ξn + 2a
cos ϕn + 2b n sin ϕn ,
kξn k
kξn k
while, on {ξn = 0}, ξn+1 = (cos ϕn , sin ϕn ).
Figure 4.5 shows two simulated sample paths of this two-dimensional
elliptic random walk, for different choices of a and b, one recurrent and the
other transient, corresponding roughly to the two case pictures in Figure 4.2.
Figure 4.5: Simulation of 105 steps of the elliptic random walk in R2 for the
recurrent case a > b (left) and the transient case a < b (right).
4.3
Controlled driftless random walks
Consider k spatially homogeneous random walks in Rd , d ≥ 3, with zero
drift and with increment covariance matrices M1 , . . . , Mk . Assume also that
208
Chapter 4. Many-dimensional random walks
M1 , . . . , Mk are full-rank, so that these walks are truly d-dimensional, and
therefore transient.
Now, assume that at each discrete moment of time, we are able to choose
(according to some rule that does not depend on the future) among the k
available jump laws; in other words, we control the distribution of the next
step. The main question of this section is: Is it possible to make the walk
recurrent by choosing a suitable control policy? Below, we are going to show
that having k < d walks is not enough to achieve recurrence, but there are
recurrent examples with k = d.
We now give the precise definition of the walks we will be considering.
Definition 4.3.1. Let π1 , . . . , πk be k probability measures in Rd and let
(θnj , j = 1, . . . , k, n = 1, 2, 3, . . .) be independent random variables with θnj ∼
πj for all n and all j = 1, . . . , k. Define an adapted rule ` = (`(n), n ≥ 0) with
respect to a filtration (Fn , n ≥ 0) to be a process such that, for all n ∈ Z+ ,
`(n) ∈ {1, . . . , k} is Fn -measurable. We say that the walk (ξn , n ≥ 0) with
ξ0 = 0 is generated by the measures π1 , . . . , πk and the rule `, if ξn is Fn measurable and
`(n)
ξn+1 = ξn + θn+1 .
In particular, observe that any Markov chain with at most k different jump distributions fits into this framework. As announced, we assume
that the jump distributions have mean zero, the corresponding covariance
matrices are M1 , . . . , Mk , and also that they satisfy the moment condition (4.1) for some p > 2.
Our first result is as follows.
Theorem 4.3.2. Let π1 , . . . , πk be d-dimensional measures in Rd , d ≥ 3,
with zero mean and p moments, for some p > 2 and k < d. If ξn is a random
walk generated by these measures and an arbitrary adapted rule `, then ξn
is transient.
Proof. Clearly, without loss of generality we may assume that k = d−1. The
key to the proof is the following fact, which is a consequence of Theorem 1
of [87]: there exists a matrix A such that
tr(AMi A> ) > 2λmax (AMi A> ), for all i ∈ {1, . . . , d − 1}.
(4.34)
With (4.34) to hand, the rest of the proof is quite straightforward. Define
ξˆn = Aξn . Clearly, ξˆn is also a controlled walk with zero drift and covariance
matrices AMi A> , i = 1, . . . , d−1. As we saw on the route to Corollary 4.1.8,
the Lyapunov function kxk−α with small enough α > 0 “works” for each of
4.3. Controlled driftless random walks
209
ˆ so, this function also shows transience
the d − 1 measures belonging to ξ;
ˆ
of ξ itself.
We now construct an example that shows that k = d may be already
enough for recurrence. Let us consider a nearest-neighbour random walk
(ξn , n ≥ 0) on Zd , d ≥ 3, defined in the following way. Fix a parameter
γ > 0, and for x = (x0 , . . . , xd−1 ) ∈ Zd define %(x) = min{k : |xk | =
maxj=0,...,d−1 |xj |}. Then
ξn+1 = ξn + θn+1 ,
γ
where θn+1 = ±e%(ξn ) with probabilities 2(γ+d−1)
and δn+1 = ±ek for k 6=
1
%(ξn ) with probabilities 2(γ+d−1) . In words, we choose the maximal (in
absolute value) coordinate of ξn with weight γ and all the other coordinates
with weight 1, and then add 1 or −1 to the chosen coordinate with equal
probabilities.
Theorem 4.3.3. For any dimension d ≥ 3 there exists large enough γ =
γ(d) such that the above random walk is recurrent.
The proof of this result is somewhat different from the arguments used
in the rest of the book. Instead of taking a particular Lyapunov function
(maybe depending on a few parameters), we work with an “unknown” function, write down the conditions that it must satisfy, and prove that such a
function exists.
Proof of Theorem 4.3.3. By Theorem 2.5.2, to prove recurrence it is enough
to find a nonnegative function f such that f (x) → ∞ as kxk → ∞, and
E[f (ξn+1 ) − f (ξn ) | ξn = x] ≤ 0, for all x with kxk large enough. (4.35)
Before presenting the explicit construction of such a function, let us
informally explain the intuition behind this construction. First of all, recall
from (1.4) and (1.5) that if Sn is a simple random walk in Zd , then
d−1 1
+ O(kxk−2 );
2d kxk
1
E[(kSn+1 k − kSn k)2 | Sn = x] = + O(kxk−1 ).
d
E[kSn+1 k − kSn k | Sn = x] =
One can observe that the ratio of the drift to the second moment behaves
d−1
as 2kxk
; combined with the well-known fact that the SRW is recurrent for
d = 2 and transient for d ≥ 3, this suggests that, to obtain recurrence, the
210
Chapter 4. Many-dimensional random walks
case 1:
case 3:
case 2:
case 4:
Figure 4.6: Looking at the level sets: how large is the drift? We have very
small drift in case 1, very large drift in case 2, and moderate drifts in cases
3 and 4.
constant in this ratio should not be too large (in fact, at most 21 ). Then,
the second moment depends essentially on the dimension, and thus it is
crucial to look at the drift. So, consider a (smooth in Rd \ {0}) function
g(x) = Θ(kxk); we shall try to figure out how the level sets of g should be
so that the “drift outside” with respect to g “behaves well” (i.e., the drift
multiplied by kxk is uniformly bounded above by a not-so-large constant).
For that, let us look at Figure 4.6: level sets of g are indicated by solid lines,
vectors’ sizes correspond to transition probabilities. Then, it is intuitively
clear that the case of “moderate” drift corresponds to the following:
• the “preferred” direction is radial, the curvature of level lines is large,
or
• the “preferred” direction is transversal and the curvature of level lines
4.3. Controlled driftless random walks
211
0
Figure 4.7: How the level sets of f should look like?
is small;
also, it is clear that “very flat” level lines always generate small drift. However, one cannot hope to make the level lines very flat everywhere, as they
should go around the origin. So, the idea is to find in which places one can
afford “more curved” level lines.
Observe that, for the random walk we are considering now, the preferred
direction near the axes is the radial one, while in the “diagonal” regions it is
in some intermediate position between transversal and radial. This indicates
that the level sets of the Lyapunov function should look as depicted on
Figure 4.7: more curved near the axes, and more flat off the axes.
We are going to use the Lyapunov function
f (x) = kxkα ϕ(x̂),
where α is a positive constant and ϕ : Sd−1 7→ R is a positive continuous
function, symmetric in the sense that for any (u0 , . . . , ud−1 ) ∈ Sd−1 we have
ϕ(u0 , . . . , ud−1 ) = ϕ(τ0 uσ(0) , . . . , τd−1 uσ(d−1) ) for any permutation σ and
any τ ∈ {−1, 1}d . By the previous discussion, to have the level sets as on
212
Chapter 4. Many-dimensional random walks
Figure 4.7, we are aiming at constructing ϕ with values close to 1 near the
“diagonals” and less than 1 near the axes.
By symmetry, it is enough to define the function ϕ for u ∈ Sd−1 such
that u0 ≥ u1,...,d−1 ≥ 0 (clearly, it then holds that u0 > 0), and, again by
symmetry, it is enough to prove (4.35) for all large enough x ∈ Zd of the
same kind. For such u ∈ Sd−1 abbreviate sj = uj /u0 , j = 1, . . . , d − 1;
observe that, if u = x̂, then sj = xj /x0 . We are going to look for the
function (for u as above) ϕ(u) = 1 − αψ(s1 , . . . , sd−1 ), where ψ is a function
with continuous third partial derivatives on [0, 1]d−1 (in fact, it will become
clear that the function ψ extended by means of symmetry on [−1, 1]d has
continuous third derivatives on [−1, 1]d .
Next, we proceed in the following way: we do calculations in order to
figure out, which conditions the function ψ should satisfy in order to guarantee that (4.35) holds, and then try to construct a concrete example of ψ
that satisfies these conditions.
In the computations below, we will use the abbreviations
∂ψ(s1 , . . . , sd−1 )
, j = 1, . . . , d − 1,
∂sj
∂ 2 ψ(s1 , . . . , sd−1 )
00
ψij
:=
, i, j = 1, . . . , d − 1.
∂si ∂sj
ψj0 :=
Let us now consider x ∈ Zd . From now on we will refer to the situation
when x0 > x1,...,d−1 ≥ 0 as the “non-boundary case” and x0 = x1 = · · · =
xm > xm+1 ≥ . . . ≥ xd−1 ≥ 0 for some m ≥ 1 as the “boundary case”.
Observe for the boundary case the corresponding s will be of the form s =
(1, . . . , (1)m , sm+1 , . . . , sd−1 ); here and in the sequel we indicate the position
of the symbol in a row by placing parentheses and putting a subscript.
First we deal with the non-boundary case.
Let us consider x ∈ Zd such that x0 > x1,...,d−1 ≥ 0. After some extensive
but straightforward computations (see (3.6) in [257] for details) it can be
obtained that
E[f (ξn+1 ) − f (ξn ) | ξn = x]
= −αkxkα−2 Φ(x, ψ) + αkxkα−2 γ −1 Φ1 (x, ψ, γ, α) + αΦ2 (x, ψ, γ, α) ,
(4.36)
where Φ1 and Φ2 are uniformly bounded for large enough x, and
Φ(x, ψ) =
d−1
d−1
x20
1 kxk2 X
1 X
0
00
−
+
s
ψ
+
s
s
ψ
j j
i j ij .
kxk2 2
2
x20 j=1
i,j=1
(4.37)
213
4.3. Controlled driftless random walks
The idea is then to prove that, with a suitable choice for ψ, the quantity
Φ(x, ψ) will be uniformly positive for all large enough x, and then the second
term in the right-hand side of (4.36) can be controlled by choosing large γ
and small α. This will make (4.36) negative for all large x.
Now, in order to obtain a simplified form for (4.37), we pass to the
(hyper)spherical coordinates:
s1 = r cos β1 ,
s2 = r sin β1 cos β2 ,
...
sd−2 = r sin β1 . . . sin βd−3 cos βd−2 ,
sd−1 = r sin β1 . . . sin βd−3 sin βd−2 .
Since
kxk2
x20
= 1 + r2 , and (abbreviating ψr0 =
d−1
1X
sj ψj0 ,
r
ψr0 =
00
ψrr
=
j=1
∂ψ
∂r
∂2ψ
)
∂r2
00 =
and ψrr
d−1
1 X
00
si sj ψij
,
r2
i,j=1
we have
1
1
r2 00 2
0
−
+
(1
+
r
)
rψ
+
ψ
r
1 + r2 2
2 rr
1 + r2 1 − r2
2 0 0
+
r
ψ
.
=
r
r
2
(1 + r2 )2
Φ(x, ψ) =
(4.38)
Now, we define the function ψ (it will depend on r only, not on β1 , . .√. , βd−2 )
in the following way. First, clearly, we need to define ψ(r) for r ∈ [0, d − 1].
Then, observe that
√
Z
0
d−1
1 − r2
dr =
(1 + r2 )2
√
d−1
> 0,
d
(4.39)
so, for a suitable (small enough) ε0 we can construct a smooth function h
with the following properties (on the Cartesian plane with coordinates (r, y),
r2
think of going from the origin along y = 4ε
2 until it intersects with y =
0
1−r2
(1+r2 )2
and then modify a little bit the curve around the intersection point
to make it smooth, see Figure 4.8):
(i) 0 ≤ h(r) ≤
1−r2
(1+r2 )2
for all r < 2ε0 and h(r) =
1−r2
(1+r2 )2
for r ≥ 2ε0 ;
214
Chapter 4. Many-dimensional random walks
1−r 2
(1+r 2 )2
1
h(r)
1
4
1
0
√
ε0 2ε0
d−1
r
Figure 4.8: On the construction of h
(ii) h(0) = 0 and h(r) ∼
(ii)
1−r2
− h(r) > 12
(1+r2 )2
R √d−1
h(r) dr
b := 0
r2
4ε20
as r → 0;
for r ≤ ε0 ;
> 0 (by (4.39) it holds in fact that b ∈ (0, 1));
√
Rr
br3
(v) 0 h(u) du > 3(d−1)
d − 1].
3/2 for all r ∈ (0,
√
Rr
Denote H(r)
√ = 0 h(u) du, so that we have H( d − 1) = b. Then, define
for r ∈ [0, d − 1]
Z √d−1 bv
H(v)
ψ(r) =
−
dv.
(4.40)
v2
3(d − 1)3/2
r
(iv)
For the function ψ defined in this way, we have r2 ψ 0 (r) =
br3
3(d−1)3/2
− H(r),
so h(r) + (r2 ψ 0 (r))0 = b(d − 1)−3/2 r2 . By construction, it then holds that
1 − r2
0 1
2 0
≥ b(d − 1)−3/2 ε20 ∧ ,
inf
+
r
ψ
(r)
(4.41)
√
2
2
2
r∈[0, d−1] (1 + r )
and this (recall (4.37) and (4.38)) shows that, if γ is large enough and α
is small enough then the right-hand side of (4.36) is negative for all large
enough x ∈ Zd .
215
4.4. Centrally biased random walks
To complete the proof of the theorem, it remains to deal with the boundary case. Let x0 = x1 = · · · = xm > xm+1 ≥ . . . ≥ xd−1 ≥ 0 for some m ≥ 1.
We have (again, we omit some long computations; see (3.12) in [257] for more
details)
E[f (ξn+1 ) − f (ξn ) | ξn = x]
"
#
d−1
m−1
X
X
γ
+
m
1
0
= αkxkα
ψk0 + 2ψm
+
sk ψk0 + o(kxk−1 ) .
2(γ + d − 1) x0
k=1
k=m+1
(4.42)
Now√simply note that by the property (v), we have ψ 0 (r) < 0 for all
that for some positive constant δ0 it holds
r ∈ (0, d − 1]. Observe also √
0
that ψ (r) ≤ −δ0 for all r ∈ [1, d − 1]. Then (recall that in the boundary
case s1 = 1 and sj ≥ 0 for all j = 2, . . . , d − 1) we have
ψj0 =
sj 0
ψ ≤ 0 for all j = 1, . . . , d − 1
r r
and
ψ10 (s) ≤ − √
δ0
.
d−1
This implies that the right-hand side of (4.42) is negative for all large
enough x ∈ Zd and thus concludes the proof of Theorem 4.3.3.
4.4
4.4.1
Centrally biased random walks
Model and notation
The results of Sections 4.2 and 4.3 demonstrate that in the spatially nonhomogeneous setting, even assuming zero drift is no guarantee of predictable
behaviour for a many-dimensional random walk, at least in terms of recurrence versus transience. However, we saw in Theorem 4.1.3 that one regains
some control given suitable assumptions on the covariance structure of the
increments of the walk: then the classical recurrence theory for homogeneous
random walk has a natural analogue for non-homogeneous random walk. In
this setting, it turns out that the zero drift assumption can be relaxed to
an assumption of asymptotically zero drift, which also turns out to be the
correct regime in which to probe the recurrence/transience phase transition,
in much the same way as in the one-dimensional setting of Chapter 3. The
most natural framework in which to investigate these questions, and more,
is that of the centrally biased random walk, which is the focus for this part
of the present chapter.
216
Chapter 4. Many-dimensional random walks
Once more we use the notation of Section 4.1.2 and consider a nonhomogeneous random walk ξn on Σ ⊆ Rd , d ≥ 1. In this section, our basic
assumption will again be (MD0).
As mentioned in Section 4.1.3, under mild conditions, a non-homogeneous
random walk with zero drift and fixed increment covariance matrix is recurrent if and only if d ≤ 2. We will see in Section 4.4.2 that the assumption of
µ(x) = 0 may be relaxed to (essentially) kµ(x)k = o(kxk−1 ). On the other
hand, it is possible to achieve either transience or recurrence (even positiverecurrence) in any dimension under the condition kµ(x)k = O(kxk−1 ), as
described in Section 4.1.5. In view of these phenomena, and others that
we will see later, we call the case where kµ(x)k = O(kxk−1 ) the critical
case. Before considering in detail that case, we examine in turn the subcritical case in which kxkkµ(x)k → 0 and the supercritical case in which
kxkkµ(x)k → ∞.
Let us comment briefly on our use of the terminology ‘centrally biased’
random walk (see the bibliographical notes at the end of this chapter for
historical remarks): in its broadest use we mean any random walk whose
drift field µ is perturbed around the origin, so, typically, µ(x) is asymptotically zero as kxk → ∞. In some, but certainly not all, of the results in this
section, the ‘central bias’ takes on the additional sense that the dominant
component of the drift field is in the radial direction, towards or away from
the origin.
4.4.2
Subcritical case
The following result extends Theorem 4.1.3 to admit non-zero drifts that
decay sufficiently fast.
Theorem 4.4.1. Suppose that (MD0), (MD3), and (MD4) hold, and that (4.1)
holds for some p > 2. Suppose that for some positive-definite M ,
kµ(x)k = o(kxk−1 ) and kM (x) − M kop → 0, as kxk → ∞,
(4.43)
Then ξn is recurrent if d = 1 and transient if d ≥ 3.
If d = 2 and, as kxk → ∞,
kµ(x)k = o(kxk−1 log−1 kxk) and kM (x) − M kop = o(log−1 kxk), (4.44)
then ξn is recurrent.
Proof. The idea of Remark 1.5.5 shows that it suffices to suppose that M =
Id . We again apply Theorem 4.1.7. First suppose that (4.43) holds. Then
217
4.4. Centrally biased random walks
tr M (x) = d + o(1) and, for any u ∈ Sd−1 , u> M (x)u = 1 + o(1); thus
2kxkhx̂, µ(x)i + tr M (x) − 2x̂> M (x)x̂ = d − 2 + o(1),
which, for kxk sufficiently large, is strictly positive for if d ≥ 3, in which case
transience follows from Theorem 4.1.7(i), and negative if d ≤ 1, in which
case recurrence follows from Theorem 4.1.7(ii).
Finally, suppose that d = 2 and (4.44) holds. Then, similarly,
2kxkhx̂, µ(x)i + tr M (x) − 2x̂> M (x)x̂ = o(log−1 kxk),
which with another application of Theorem 4.1.7(ii) yields recurrence.
4.4.3
Supercritical case
In this section we suppose that the moments condition (4.1) holds for p > 1
(at least), so that µ as given by (4.2) is well defined, and we will study
models in which kxkkµ(x)k is unbounded. The model that we focus on
in this section has dominant drift in the (outwards) radial direction. In
this case, to conclude about asymptotic behaviour one typically needs no
assumptions on the covariance structure of the walk; indeed, without further
assumptions the covariance matrix (4.3) may not exist.
Specifically, we suppose that for some ρ ∈ R+ and β ≥ 0,
µ(x) = ρkxk−β x̂ + O(kxk−β log−2 kxk),
(4.45)
as kxk → ∞. In equation (4.45) the notational convention described at (4.13)
is in force, i.e., (4.45) is to be understood as
sup
x∈Σ:kxk≥r
kµ(x) − ρkxk−β x̂k = O(krk−β log−2 krk), as r → ∞,
in the usual scalar sense of O( · ). We will assume the following.
(CB1) Suppose that for some β ∈ [0, 1), ρ ∈ R+ , and p > 2 + 2β, the
conditions (4.1) and (4.45) hold.
Our main result on the supercritical case is the following, which shows
that ξn satisfies a strong law of large numbers with a super-diffusive rate of
escape and has a limiting direction; this result implies Theorem 1.7.3.
Theorem 4.4.2. Suppose that (MD0), (MD3), and (CB1) hold. Then there
exists a random u ∈ Sd−1 such that, as n → ∞,
n
1
− 1+β
1
ξn → (ρ(1 + β)) 1+β u, a.s.
In particular, limn→∞ ξˆn = u, a.s.
218
Chapter 4. Many-dimensional random walks
π
0
500
1000
2
+
0
−1000
−500
π
3π
2
−1000
−500
0
500
1000
Figure 4.9: Simulation of 105 steps of a centrally biased random walk satisfying (CB1) with β = 3/4, showing the trajectory (left) and a histogram of
the corresponding angles (right).
The first step in the proof of Theorem 4.4.2 is to appeal to Lemma 4.1.5(i),
which shows that the process Xn = kξn k is of supercritical Lamperti type,
so that we may apply the results of Section 3.12. In particular, we establish
the following law of large numbers for Xn .
Lemma 4.4.3. Suppose that (MD0), (MD3), and (CB1) hold. Then,
n
1
− 1+β
1
kξn k → (ρ(1 + β)) 1+β , a.s., as n → ∞.
Proof. We take Xn = kξn k. Since β < 1, (4.15) shows that, for some δ > 0,
E[∆n | ξn = x] = hx̂, µ(x)i + O(kxk−β−δ )
= ρkxk−β + O(kxk−β log−2 kxk),
by (4.45). Then Theorem 3.12.4 applied to Xn gives the result.
The second step in the proof of Theorem 4.4.2 is to show that the process ξn has a limiting direction. To this end, the next result concerns the
angular process ξˆn on Sd−1 ∪ {0}.
Lemma 4.4.4. Suppose that (MD0) holds. Let β ∈ [0, 1). Suppose that
(4.1) holds for some p > 1 + β, and that, as kxk → ∞,
hy, µ(x)i = O(kxk−β log−2 kxk).
sup
(4.46)
y∈Sd−1 :hy,xi=0
219
4.4. Centrally biased random walks
Then, as kxk → ∞,
E ξˆn+1 − ξˆn ξn = x = O(kxk−1−β log−2 kxk);
E kξˆn+1 − ξˆn k2 ξn = x = O(kxk−1−β log−2 kxk).
(4.47)
(4.48)
Proof. Without loss of generality, suppose that p ∈ (1 + β, 2]. For the
duration of this proof, condition on ξn = x for x ∈ Σ \ {0}. Recall the
definition of Ax from (4.20), and let γ ∈ (0, 1) with γ > 1+β
p , which is
possible since p > 1 + β. Also, write
θn⊥ := θn − hθn , x̂ix̂;
note that hθn⊥ , x̂i = 0 and kθn⊥ k2 = kθn k2 − hθn , x̂i2 .
Now, a computation shows, given ξn = x with kxk > 0, for kθn k < kxk,
θn
kx + θn k − kxk
ˆ
ˆ
ξn+1 − ξn =
− x̂
kx + θn k
kx + θn k
⊥
θn
x̂
2hθn , xi + kθn k2 hθn , xi
=
−
−
.
kx + θn k kx + θn k kxk + kx + θn k
kxk
Note that, here,
2hθn , xi
hθn , xi
−
= hθn , xi
kxk + kx + θn k
kxk
kxk − kx + θn k
kxk(kxk + kx + θn k)
,
and, by the triangle inequality,
kxk − kx + θn k kθn k
kxk(kxk + kx + θn k) ≤ kxk2 .
Hence
2hθn , xi + kθn k2 hθn , xi 2kθn k2
kxk + kx + θn k − kxk ≤ kxk2 .
By definition of Ax , we may choose kxk large enough so that kθn k < kxk/2,
and then
x̂
2hθn , xi + kθn k2 hθn , xi 1(Ax ) ≤ 4kθn k2 kxk−2 1(Ax )
−
kx + θn k kxk + kx + θn k
kxk ≤ 4kθn kp kxkγ(2−p)−2 ,
which has an expectation of O(kxkγ(2−p)−2 ), by (4.1). In particular,
E[(ξˆn+1 − ξˆn )1(Ax ) | ξn = x] = kxk−1 (1 + o(1)) E[θn⊥ 1(Ax ) | ξn = x]
220
Chapter 4. Many-dimensional random walks
+ O(kxkγ(2−p)−2 ).
(4.49)
Expressed in an orthonormal basis containing x̂, the (magnitudes of the)
non-zero components of θn⊥ are of the form hy, θn⊥ i for y with hy, x̂i = 0,
where
E[hy, θn⊥ i | ξn = x] = E[hy, θn i | ξn = x] = hy, µ(x)i = O(kxk−β log−2 kxk),
by assumption (4.46). Hence k E[θn⊥ | ξn = x]k = O(kxk−β log−2 kxk). Also,
by the q = 1 case of Lemma 4.1.6,
k E[θn⊥ 1(Acx ) | ξn = x]k ≤ E[kθn k1(Ax ) | ξn = x] = O(kxk−γ(p−1) ).
Hence, using the fact that γ >
1+β
p
>
β
p−1 ,
E[θn⊥ 1(Ax ) | ξn = x] = O(kxk−β log−2 kxk),
We also have that 2(γ − 1) − pγ < −pγ < −1 − β since γ >
obtain from (4.49) and (4.50) that
(4.50)
1+β
p ,
so we
E[(ξˆn+1 − ξˆn )1(Ax ) | ξn = x] = O(kxk−1−β log−2 kxk).
On the other hand,
E[kξˆn+1 − ξˆn k1(Acx ) | ξn = x] ≤ 2 P[kθn kp > kxkpγ | ξn = x] = O(kxk−pγ ),
β
, we obtain (4.47).
by Markov’s inequality and (4.1). Since γ > p−1
Now note that, given ξn = x with kxk > 0, for kθn k < kxk,
x(kxk − kx + θn k) + θn kxk ≤ 2kθn k ,
kξˆn+1 − ξˆn k = kx + θn k
kxkkx + θn k
by the triangle inequality. Hence, for all kxk sufficiently large,
E[kξˆn+1 − ξˆn k2 1(Ax ) | ξn = x] ≤ 8kxk−2 E[kθn k2 1(Ax ) | ξn = x]
≤ 8kxkγ(2−p)−2 E[kθn kp | ξn = x],
which is O(kxk−1−β log−2 kxk), by (4.1). On the other hand,
E[kξˆn+1 − ξˆn k2 1(Acx ) | ξn = x] ≤ 4 P[Acx | ξn = x] = O(kxk−pγ ),
by Markov’s inequality and (4.1). Since γ >
1+β
p ,
we obtain (4.48).
Now we can complete the proof of Theorem 4.4.2.
4.4. Centrally biased random walks
221
Proof of Theorem 4.4.2. Set Fn := σ(ξ0 , . . . , ξn ). Then (ξˆn , n ≥ 0) is an
(Fn , n ≥ 0)-adapted process taking values in Sd−1 ∪ {0}, and using the
vector-valued version of Doob’s decomposition we may write
ξˆn = An + Mn ,
(4.51)
where M0 = ξˆ0 , Mn is a d-dimensional P
martingale, and An is the previsible
ˆ
ˆ
sequence defined by A0 = 0 and An = n−1
m=0 E[ξm+1 − ξm | Fm ] for n ≥ 1.
ˆ
To show that ξn converges, we show that both An and Mn converge.
We have from (4.47) that, for finite positive constants C1 and C2 ,
kAn+1 − An k ≤ C1 kξn k−1−β log−2 kξn k, on kξn k ≥ C2 .
By Lemma 4.4.3, there is a positive constant c such that kξn k ∼ cn1/(1+β) ,
a.s. It follows that
lim sup
n→∞
kAn+1 − An k
< ∞, a.s.,
n−1 (log n)−2
P
so n≥0 kAn+1 − An k < ∞, a.s., implying that An converges almost surely
to some limit A∞ ∈ Rd .
Taking expectations in the vector identity kMn+1 − Mn k2 = kMn+1 k2 −
kMn k2 − 2Mn · (Mn+1 − Mn ) and using the martingale property, we have
E[kMn+1 k2 − kMn k2 | Fn ] = E[kMn+1 − Mn k2 | Fn ]
= E kξˆn+1 − ξˆn − E[ξˆn+1 − ξˆn | Fn ]k2 Fn .
Expanding out the expression in the latter expectation, we obtain
E[kMn+1 k2 − kMn k2 | Fn ] = E[kξˆn+1 − ξˆn k2 | Fn ] − (E[ξˆn+1 − ξˆn | Fn ])2
≤ E[kξˆn+1 − ξˆn k2 | Fn ].
Then by (4.48) and the fact that kξn k ∼ cn1/(1+β) , by Lemma 4.4.3, it
follows that
lim sup
n→∞
E[kMn+1 k2 − kMn k2 | Fn ]
< ∞, a.s.
n−1 (log n)−2
Hence n≥0 E[kMn+1 k2 − kMn k2 | Fn ] < ∞, a.s., which implies that Mn
has an almost-sure limit M∞ ∈ Rd , by e.g. the d-dimensional version of [83,
Theorem 5.4.9, p. 254].
Taking n → ∞ in (4.51), we conclude that, a.s., ξˆn → A∞ + M∞ =: u,
for some random vector u ∈ Sd−1 ∪ {0}. Finally, note that (MD3) implies
P
222
Chapter 4. Many-dimensional random walks
P[u = 0] = 0, since, any sequence Sd−1 ∪ {0} with 0 as a limit must be
eventually equal to 0, so u = 0 if and only if ξn = 0 for all but finitely
many n, which contradicts (MD3); hence we may assume that u ∈ Sd−1 .
Combining the convergence of ξˆn with the asymptotics for kξn k given in
Lemma 4.4.3 completes the proof of the theorem.
4.4.4
Critical case: Angular asymptotics
Throughout this section, we assume d ≥ 2. The main result of the previous
section, Theorem 4.4.2, showed that the angular process ξˆn has a limit in
the supercritical case, where kxkkµ(x)k → ∞. We will see that the case in
which kxkkµ(x)k remains bounded is fundamentally different: under mild
assumptions, ξˆn does not converge. This angular wandering is a classical
property of zero-drift homogeneous random walk in dimensions d ≥ 2 (see
Theorem 1.7.2), and in this section we show the extent to which this property
carries across to more general models. We emphasize that the phenomena
exhibited in this section are genuinely many dimensional (d ≥ 2 is essential),
as we explain below.
In addition to taking d ≥ 2, for this section we will make the following
assumption on the moments and drift of the increments.
(CB2) Suppose that the moments condition (4.1) holds for some p > 2,
and that there exists C < ∞ such that
kµ(x)k ≤ C(1 + kxk)−1 , for all x ∈ Σ.
(4.52)
The main result of this section is the following.
Theorem 4.4.5. Suppose that, for d ≥ 2, (MD0), (MD3), (CB2), and
(MD4) hold. Then P[limn→∞ ξˆn exists] = 0.
The remainder of this section is devoted to the proof of Theorem 4.4.5.
First we state an important consequence of Theorem 2.4.8.
Lemma 4.4.6. Suppose that (4.1) holds for some p ≥ 2 and that (4.52)
holds. Then for any δ > 0 there exists ε > 0 such that
h
i
P
max
kξn − ξ0 k ≤ 12 kξ0 k ≥ 1 − δ.
0≤n≤bεkξ0 k2 c
Proof. Again let Fn = σ(ξ0 , . . . , ξn ). Set Xn = kξn − ξ0 k2 , so X0 = 0 and
Xn ≥ 0. Let
η = min n ≥ 0 : kξn − ξ0 k ≥ 21 kξ0 k = min n ≥ 0 : Xn ≥ 41 kξ0 k2 .
223
4.4. Centrally biased random walks
π
0
200
400
2
+
0
−400
−200
π
3π
2
−400
−200
0
200
400
Figure 4.10: Simulation of 105 steps of a centrally biased random walk
satisfying (CB2), showing the trajectory (left) and a histogram of the corresponding angles (right).
Then Xn+1 − Xn = kθn k2 + 2hθn , ξn − ξ0 i, so that
E[Xn+1 − Xn | Fn ] ≤ E[kθn k2 | Fn ] + 2kξn − ξ0 kkµ(ξn )k
2Ckξn − ξ0 k
≤B+
,
1 + kξn k
for some constants B < ∞ and C < ∞, by the p = 2 case of (4.1) and (4.52).
On {n < η}, we have kξn k ≥ kξ0 k − kξn − ξ0 k ≥ 12 kξ0 k, so there is some
constant B 0 < ∞ for which
E[Xn+1 − Xn | Fn ] ≤ B +
Ckξ0 k
≤ B 0 , on {n < η}.
1 + 12 kξ0 k
Hence we may apply Theorem 2.4.8 with the stopping time τ = η = σx ,
where x = 14 kξ0 k2 and, as usual, σx = min{n ≥ 0 : Xn ≥ x}, to obtain
x P[σx ≤ n] ≤ E[Xn∧σx ] ≤ B 0 n.
In other words, P[σx > n] is equal to
h
i
h
i
4B 0 n
P max kξm − ξ0 k ≤ 12 kξ0 k = P max Xm ≤ 14 kξ0 k2 ≥ 1 −
.
0≤m≤n
0≤m≤n
kξ0 k2
Taking n = bεkξ0 k2 c gives the result, on choosing ε > 0 small enough so
that 4B 0 ε < δ.
224
Chapter 4. Many-dimensional random walks
The next ingredient is the following estimate for the probability that the
walk makes a significant deviation in an orthogonal direction.
Lemma 4.4.7. Suppose that (CB2) and (MD4) hold. There exists y0 < ∞
such that, for any δ > 0, we can choose ε > 0 and h > 0 such that, for all
x ∈ Σ with kxk ≥ y0 ,
h
i
inf
P
max
|hξn , ei| ≥ hkxk ξ0 = x ≥ 1 − δ.
e∈Sd−1 :he,xi=0
0≤n≤bεkxk2 c
Proof. Fix δ > 0. Suppose that ξ0 = x ∈ Σ with kxk = y > 0. Fix e ∈ Sd−1
such that he, xi = 0. Denote the Doob decomposition of hξn , ei as
hξn , ei = hξ0 , ei + Mn + An = Mn + An ,
where Mn is a martingale with M0 = 0, and
An =
n−1
X
m=0
E[hξm+1 − ξm , ei | Fm ] =
n−1
X
he, µ(ξm )i.
m=0
Again let η = min{n ≥ 0 : kξn − ξ0 k ≥ 21 kξ0 k}. By Lemma 4.4.6, there
exists ε1 = ε1 (δ) > 0 such that, for all ε ∈ (0, ε1 ), for any y > 0,
P[η ≥ εy 2 | ξ0 = x] ≥ 1 − δ.
Moreover, for ξ0 = x and n < η we have kξn k ≥ y/2 so that, on {n < η},
|he, µ(ξm )i| ≤ kµ(ξm )k ≤
C
2C
≤
, a.s.,
1 + (y/2)
y
by (4.52). Hence |An∧η | ≤ 2Cn/y, a.s. In particular,
h
i
P max |An | ≥ 4Cεy ξ0 = x ≤ P[η ≤ εy 2 | ξ0 = x] ≤ δ.
0≤n≤bεy 2 c
(4.53)
Next we show that the martingale Mn satisfies the hypotheses of the
d = 1 case of Theorem 4.1.4. Indeed,
Mn+1 − Mn = hθn − µ(ξn ), ei,
so that, a.s.,
E[(Mn+1 − Mn )2 | Fn ] = E[he, θn i2 | Fn ] − he, µ(ξn )i2
≥ v0 − kµ(ξn )k2 ,
(4.54)
225
4.4. Centrally biased random walks
for v0 > 0 as given by (MD4). Hence, on {n < η}, a.s., by (4.52),
E[(Mn+1 − Mn )2 | Fn ] ≥ v0 − (2C/y)2 ≥ v0 /2,
for all y ≥ y0 sufficiently large; note that y0 depends only on C and v0 ,
and not on δ. Hence (4.8) is satisfied for Yn = Mn for this choice of η and
v = v0 /2. Also, (4.54) shows that
E[|Mn+1 − Mn |p | Fn ] ≤ E[kθn − µ(ξn )kp | Fn ].
Here, by (4.1), E[kθn kp | Fn ] ≤ B a.s. for some finite constant B, while
kµ(ξn )kp ≤ sup kµ(x)kp ≤ sup E[kθn kp | ξn = x],
x∈Σ
x∈Σ
by Jensen’s inequality. Hence, by Minkowski’s inequality, (4.7) is also satisfied for Yn = Mn . So applying Theorem 4.1.4 we obtain, for some constant
D < ∞ depending on v0 and C, but not on ε or δ,
h
P
i
D(2hy)2
|Mn | ≥ 2hy ξ0 = x ≥ 1 −
− P[η ≤ εy 2 | ξ0 = x]
εy 2
0≤n≤bεy 2 c
max
≥1−
4Dh2
− δ,
ε
p
for ε ∈ (0, ε1 ). Now take h = h(ε) = 21 δε/D, so that
h
i
P max |Mn | ≥ 2hy ξ0 = x ≥ 1 − 2δ,
0≤n≤bεy 2 c
(4.55)
for all y ≥ y0 . Thus, from (4.53) and (4.55), we see that with probability at
least 1 − 3δ, the event
n
o n
o
max |Mn | ≥ 2hy ∩
max |An | ≤ 4Cεy
(4.56)
0≤n≤bεy 2 c
0≤n≤bεy 2 c
occurs. Since h/ε → ∞ as ε ↓ 0, there exists a constant ε0 = ε0 (δ) ∈ (0, ε1 )
such that h ≥ 4Cε for all ε ∈ (0, ε0 ). Then, for such ε, on the event (4.56),
max
0≤n≤bεy 2 c
|hξn , ei| ≥
max
0≤n≤bεy 2 c
|Mn | −
max
0≤n≤bεy 2 c
|An |
≥ 2hy − 4Cεy ≥ hy,
for all y ≥ y0 . This completes the proof, after a relabelling of the arbitrary
positive constant δ.
226
Chapter 4. Many-dimensional random walks
Now we can complete the proof of Theorem 4.4.5.
Proof of Theorem 4.4.5. Again let Fn = σ(ξ0 , . . . , ξn ). Suppose that ξ0 = x
with kxk ≥ y0 , the constant in the statement of Lemma 4.4.7, and take
e ∈ Sd−1 with hx, ei = 0. Note
|hξn , ei|
kξˆn − ξˆ0 k ≥ |hξˆn − ξˆ0 , ei| =
,
kξn k
(4.57)
since hξ0 , ei = 0. By the δ = 1/4 cases of Lemmas 4.4.6 and 4.4.7, we can
choose ε > 0 and h > 0 such that, with probability at least 1 − 2δ = 1/2,
the event
n
o n
o
max
kξn − ξ0 k ≤ 21 kxk ∩
max
|hξn , ei| ≥ hkxk
0≤n≤bεkxk2 c
0≤n≤bεkxk2 c
occurs. On this event, we have from (4.57) that
max
0≤n≤bεkxk2 c
kξˆn − ξˆ0 k ≥
2h
hkxk
=
.
3kxk/2
3
With the strong Markov property, we have thus shown that, for any finite
stopping time τ , P[E(τ ) | Fτ ] ≥ 21 1{kξτ k ≥ y0 }, where
E(τ ) :=
2h o
.
kξˆτ +n − ξˆτ k ≥
3
0≤n≤bεkξτ k2 c
n
max
Define τ0 = min{n ≥ 0 : kξn k ≥ y0 }, and recursively, for k ∈ Z+ ,
τk+1 = min{n > τk + εkξτk k2 : kξn k ≥ y0 }.
Then, by (MD3), the stopping times (τk , k ≥ 0) are a.s. finite, and, by
construction,
1
E(τk ) ∈ Fτk+1 , and P[E(τk ) | Fτk ] ≥ .
2
Hence, by Lévy’s extension of the Borel–Cantelli lemma (Theorem 2.3.19),
the events E(τk ) occur for infinitely many k, a.s. But this implies that
lim inf sup kξˆm+n − ξˆm k > 0, a.s.,
m→∞ n≥0
and so ξˆn does not converge to a limit.
4.4. Centrally biased random walks
4.4.5
227
Critical case: Recurrence classification and excursions
Having established some angular properties in Section 4.4.4, we now return
to the question of (compact set) recurrence. Again, we use the same basic
notation as in Section 4.4.1; in particular, recall the definitions of the increment mean drift µ(x) and increment covariance matrix M (x) from (4.2)
and (4.3) respectively.
The sufficient conditions in Theorem 4.1.7 are not, in general, sharp. For
the rest of this section we impose additional regularity in order to obtain
a spectrum of fine results on the behaviour of the walk. We will impose
the following many-dimensional analogue of the assumption (M2) for the
one-dimensional Lamperti problem.
(CB3) Suppose that there exist ρ ∈ R and σ 2 ∈ (0, ∞) for which, as
kxk → ∞,
µ(x) = ρx̂kxk−1 + o(kxk−1 log−1 kxk); and
kM (x) − σ 2 Id kop = o(log−1 kxk).
For convenience, the following results also assume that the state space
Σ is locally finite and that ξn is irreducible; versions of all of the following
results still hold in the case of more general state spaces, under appropriate
conditions. First we have the following recurrence classification.
Theorem 4.4.8. Suppose that (MD0) holds for Σ ⊆ Rd locally finite with
0 ∈ Σ, and that ξn is irreducible. Suppose also that (4.1) holds for some
p > 2, and that (CB3) holds. Then ξn is
(i) transient if 2ρ/σ 2 > 2 − d;
(ii) null-recurrent if −d ≤ 2ρ/σ 2 ≤ 2 − d;
(iii) positive-recurrent if 2ρ/σ 2 < −d.
Next we give a result on existence of moments of return times.
Theorem 4.4.9. Suppose that (MD0) holds for Σ ⊆ Rd locally finite with
0 ∈ Σ, and that ξn is irreducible. Suppose also that (CB3) holds, and
that (4.1) holds with p > 2. Let s ∈ (0, p/2) and write s0 := 1 − (d/2) −
(ρ/σ 2 ). Then E[(τ0+ )s ] < ∞ if s < s0 and E[(τ0+ )s ] = ∞ if s > s0 .
The next result gives almost-sure scaling behaviour for kξn k.
228
Chapter 4. Many-dimensional random walks
Theorem 4.4.10. Suppose that (MD0) holds for Σ ⊆ Rd locally finite with
0 ∈ Σ, and that ξn is irreducible. Suppose also that (CB3) holds.
(i) Suppose that 2ρ/σ 2 > 2 − d and (4.1) holds with p > 2. Then
log kξn k
1
= , a.s.
n→∞ log n
2
lim
(ii) Suppose that −d ≤ 2ρ/σ 2 ≤ 2 − d and (4.1) holds with p > 2. Then
lim sup
n→∞
log kξn k
1
= , a.s.
log n
2
(iii) Suppose that 2ρ/σ 2 < −d and (4.1) holds with p > 2 − d − (2ρ/σ 2 ).
Then
log kξn k
1
=
, a.s.
lim sup
log n
2 − d − (2ρ/σ 2 )
n→∞
The rest of this section is devoted to the proofs of the preceding theorems.
The following consequence of Lemma 4.1.5 shows the relation to Lamperti’s
problem in this case.
Lemma 4.4.11. Suppose that (MD0) holds for Σ ⊆ Rd locally finite with
0 ∈ Σ, and that ξn is irreducible. Suppose also that (4.1) holds for some
p > 2, and that (CB3) holds. Then, for Xn := kξn k, we have that (4.14)
holds for the given p > 2, and
2xµ1 (x) = 2ρ + σ 2 (d − 1) + o(log−1 x) = 2xµ̄1 (x);
µ2 (x) = σ 2 + o(log−1 x) = µ̄2 (x).
Proof. It follows from (CB3) that
hx̂, µ(x)i = ρkxk−1 + o(kxk−1 log−1 kxk);
tr M (x) = dσ 2 + o(log−1 kxk);
>
sup e M (x)e − σ 2 = o(log−1 kxk).
e∈Sd−1
Lemma 4.1.5(ii) then yields the result.
Now we can complete the proofs of the theorems in this section.
4.5. Range and local time
229
Proof of Theorem 4.4.8. As in the proof of Lemma 4.4.11, (CB3) shows that
2kxkhx̂, µ(x)i + tr M (x) − 2x̂> M (x)x̂ = 2ρ + (d − 2)σ 2 + o(log−1 kxk).
It follows from Theorem 4.1.7(i) that ξn is transient if 2ρ/σ 2 > 2 − d, and
it follows from Theorem 4.1.7(ii) that ξn is recurrent if 2ρ/σ 2 ≤ 2 − d.
It remains to distinguish between positive- and null-recurrence; we apply
the results of Section 3.8 to the process Xn = kξn k and use Lemma 4.4.11.
Then, for the process Xn ,
lim sup 2xµ̄1 (x) + (2α − 1)µ̄2 (x) ≤ 2ρ + σ 2 (d + 2α − 2).
(4.58)
x→∞
If α = 1 and 2ρ/σ 2 < −d, then (4.58) is strictly negative, and so Theorem 3.8.1(ii) yields positive-recurrence. On the other hand, for any θ ∈
(0, 1),
1−θ
1−θ
2
2xµ1 (x) + 1 +
µ2 (x) ≥ 2ρ + σ d +
(σ 2 + o(1)),
log x
log x
which is positive for all x sufficiently large if σ 2 > 0 and 2ρ/σ 2 ≥ −d. In
this case Theorem 3.8.3 shows that E[τ0+ ] = ∞, and hence the random walk,
if recurrent, is null-recurrent.
Proof of Theorem 4.4.9. Consider Xn = kξn k. Lemma 4.4.11 shows that we
may apply Theorems 3.8.1 and 3.8.2 to obtain, respectively, the existence
and non-existence of moments results.
Proof of Theorem 4.4.10. Consider Xn = kξn k. Lemma 4.4.11 shows that
we may apply Theorem 3.10.1 to obtain part (i) of the theorem and Theorem 3.9.5 to obtain parts (ii) and (iii) of the theorem.
4.5
Range and local time of many-dimensional martingales
Throughout this section we assume that d ≥ 2 and consider a discrete-time
process (ξn , n ≥ 0) with values in Σ ⊆ Rd , adapted to a filtration (Fn , n ≥ 0).
We suppose that Σ is a set of infinite cardinality; the elements of Σ will be
called sites. Without restriction of generality, we assume that 0 ∈ Σ. For
the process ξn , we suppose that it is uniformly elliptic (can advance in any
given direction with uniformly positive probability), has uniformly bounded
jumps, and is a martingale in each coordinate (see Definition 4.5.1 below for
230
Chapter 4. Many-dimensional random walks
a precise statement). In principle, we do not assume homogeneity in space
or time, or even that the process is Markovian.
In this section we study two related questions:
• How many different sites can be visited by the process ξn by time n?
• How big can be the number of visits to a given site?
Of course, in the absence of space/time homogeneity one cannot hope to be
able to characterize the precise behavior of the quantities of interest; in this
section we content ourselves in proving that with probability 1 − exp(−nε )
1
the number of visits to any fixed site by time n is less than n 2 −δ for some
δ > 0. This in its turn implies that the number is different sites visited by
1
the process by time n with very high probability will be at least n 2 +δ .
Although it is not important for the formulation of our results, while
reading this section one may always assume that Σ is the vertex set of the
integer lattice Zd , the vertex set of some other mosaic, or just any locally
finite (in particular, countable) set. This makes sense in light of the discrete
nature of our questions of interest raised above; however, it is not hard to
formulate ‘continuous’ versions of these questions: see the bibliographical
notes at the end of this chapter.
We give formal definitions and state the results of this section. Let
Ln (x) :=
n
X
1{ξk = x}
k=0
be the local time of the process in x ∈ Σ up to time n, and denote by
Ξn := {ξ0 , ξ1 , . . . , ξn }
the set of visited sites up to time n. The range is #Ξn , the number of
visited sites up to time n. Define the random variable Dn ∈ Rd to be the
(conditional) drift of the process at time n:
Dn := E[ξn+1 − ξn | Fn ] = E[ξn+1 | Fn ] − ξn .
Definition 4.5.1. We say that the Fn -adapted process ξn in Σ ⊆ Rd , d ≥ 2,
(a) has uniformly bounded jumps if there exists K > 0 such that kξn+1 −
ξn k ≤ K a.s. for all n (we assume without restriction of generality that
K ≥ 1);
4.5. Range and local time
231
(b) is uniformly elliptic if there exist h, r > 0 such that for all e ∈ Sd−1 we
have P[(ξn+1 − ξn ) · e > r | Fn ] > h a.s. (we assume without restriction
of generality that r ≤ 1);
(c) is a d-dimensional martingale, if Dn = 0 a.s. for all n.
The main result of this section is the following.
Theorem 4.5.2. Let d ≥ 2. Suppose that ξn is a uniformly elliptic, ddimensional
martingale with uniformly bounded jumps. Then, there exist
1
γ ∈ 0, 2 , C1 , C2 , δ > 0 such that, for any x ∈ Σ, for all n ≥ 0,
P[Ln (x) > nγ ] ≤ C1 exp{−C2 nδ };
and P[#Ξn < n1−γ ] ≤ C1 n exp{−C2 nδ }.
Proof. So, suppose that ξn is a martingale in dimension d ≥ 2, with bounded
jumps and uniform ellipticity. To begin, we show that there exist b ∈ (0, 1)
close enough to 1 and γ 0 > 0 (depending only on the constants K, h, r in
Definition 4.5.1(a) and (b)) such that
E[kξn+1 kb | Fn ] ≥ kξn kb 1{kξn k > γ 0 }.
To see that this inequality holds, first observe that for a fixed y ∈ Rd ,
b/2
kx + ykb = kxk2 + 2x · y + kyk2
x·y
bkyk2
1
(x · y)2
−2
= kxkb 1 + b
+
−
b(2
−
b)
+
o(kxk
)
,
kxk2 2kxk2 2
kxk4
as kxk → ∞. Taking x = ξn and y = θn := ξn+1 − ξn , denoting by ϕn the
angle between ξn and θn , and using the martingale property,
E[kξn+1 kb − kξn kb | Fn ]
b
Fn
2
2
2
−b
=
E[kθ
k
|
F
]
−
(2
−
b)
E[kθ
k
cos
ϕ
|
F
]
+
o
(kξ
k
)
.
n
n
n
n
n
n
kξn k
2kξn k2−b
(4.59)
Using the uniform ellipticity and the boundedness of jumps, one can
obtain that
E[kθn k2 cos2 ϕn | Fn ] < (1 − ε0 ) E[kθn k2 | Fn ]
for some ε0 > 0. It follows that if b < 1 is close enough to 1, the right-hand
side of (4.59) is positive for all large enough kξn k, since E[kθn k2 | Fn ] ≥
232
Chapter 4. Many-dimensional random walks
hr2 > 0 by uniform ellipticity. (More details can be found in the proof of
Lemma 5.2 in [222].)
Recall that B(x; s) is the closed ball of radius s centred at x. The proof
of Theorem 4.5.2 is now quite straightforward; we give the following sketch.
• The Optional Stopping Theorem implies that, starting from x0 ∈ Rd ,
the process will reach Rd \ B(x0 ; α) without coming back to 0 with
probability at least O(α−b ) (to apply the Optional Stopping Theorem,
one has first to force the process a bit away from the origin, which
happens with positive probability by uniform ellipticity).
• Doob’s inequality implies that from any place in Rd \ B(x0 ; α) with
probability bounded away from 0 the process will not return to x0
after additional c1 α2 steps, where c1 > 0 is a small enough constant.
• Then, consider α = c2 n1/2 for large enough c2 . By the previous discussion, each time the process is in x0 , independently of the past it
has probability at least of order n−b/2 of not returning to x0 during
next n steps.
• Take ε > 0 such that b + ε < 1. Then, by an obvious coin-tossing
b+ε
argument, the number of visits to x0 by time n will not exceed n 2
ε/2
with probability at least 1 − c3 e−c4 n .
ε/2
• So, with probability at least 1 − c3 ne−c4 n
b+ε
visit at least n1− 2 different sites.
the process will have to
Theorem 4.5.2 can be extended to a more general class of processes,
namely, the so-called strongly directed submartingales. To give a formal
definition, let us denote by PL the operator of projection on the linear
subspace L ⊂ Rd . Assuming that L is a two-dimensional subspace of Rd ,
` ∈ Sd−1 ∩ L and u ∈ R, define
n
o
PL x · `
u
H`,L
= x ∈ Rd : PL x = 0 or
≥u ,
kPL xk
see Figure 4.11.
Definition 4.5.3. We say that the Fn -adapted process ξn is a (u, `, L)u | F ] = 1 a.s.
strongly directed submartingale, if u > 0 and P[Dn ∈ H`,L
n
Informally, the drift of a strongly directed submartingale should always
belong to the “interior of the open book” as in Figure 4.11. Observe that
233
4.5. Range and local time
L
0
u
`
u
H`,L
u ; observe that H u and H −u “comFigure 4.11: On the definition of H`,L
`,L
−`,L
plement” each other (i.e., their union is Rd and they intersect on a set of
measure 0).
234
Chapter 4. Many-dimensional random walks
for (u, `, L)-strongly directed submartingale it holds that the (conditional on
the history) expected projection of the drift to ` is always nonnegative (so
that it is what one would naturally call a “submartingale in direction `”).
Then, we have the following generalization of Theorem 4.5.2:
Theorem 4.5.4. Let d ≥ 2. Suppose that ξn is a (u, `, L)-strongly directed
submartingale, which is uniformly elliptic and has uniformly bounded jumps.
is a uniformly elliptic, d-dimensional martingale with uniformly bounded
jumps. Then, there exist constants γ ∈ (0, 21 ), C1 , C2 , δ > 0 (apart from
the dimension, depending only on u and on K, r, h from Definition 4.5.1
(a)–(b)) such that for any x ∈ Σ, for all n ≥ 0,
P[Ln (x) > nγ ] ≤ C1 exp{−C2 nδ };
and P[#Ξn < n1−γ ] ≤ C1 n exp{−C2 nδ }.
We do not give the proof of this theorem here, since it is quite technical,
does not rely on Lyapunov functions, and is not as instructive as the proof
of Theorem 4.5.2.
Bibliographical notes
Section 4.1
The non-confinement result for zero-drift random walks, Lemma 4.1.1, can
be found as Proposition 2.1 in [114]; the d-dimensional martingale version of Kolmogorov’s other inequality, Theorem 4.1.4, on which the nonconfinement result rests, is a variation of Lemma 4.1 of [114].
As mentioned in the text, Theorem 4.1.2, which states that in a onedimensional zero-drift random walk is recurrent under mild second moment
conditions, is a relative of Theorem 2.5.7: see the bibliographical notes to
Section 2.5 for further discussion.
The recurrence classification for a zero-drift random walk with fixed
increment covariance matrix, Theorem 4.1.3, is contained in Theorem 4.1 of
Lamperti [190], under a uniform bound on the increments of the walk. The
version stated here is contained, for example, in Theorem 2.2 of [114].
The analysis of many-dimensional random walks by considering the process of norms, as in Section 4.1.4, was an important contribution of Lamperti’s
seminal work [190, 192]. Early versions of the computations for Lemma 4.1.5,
assuming uniformly bounded increments, can be found for example in [190,
Bibliographical notes
235
§4]. The sufficient conditions for recurrence and transience given in Theorem 4.1.7 extend Theorem 4.1 of Lamperti [190], which, as well as assuming uniformly bounded increments, applies only to the case where the
increment covariance matrix M (x) is (asymptotically) constant. The approach here unifies the proofs not only of Theorems 4.1.2 and 4.1.3, but also
of Theorems 4.2.1, 4.4.2, and 4.4.8.
The model in Example 4.1.9 is due to Gillis [117], who calls it a ‘centrally
biased discrete random walk’. Gillis established recurrence for (only) ε >
d−2
d−2
2d and transience for ε < 2d ; his method used generating functions.
Building on Gillis’s argument, Flatto [104] established recurrence for ε =
d−2
2d .
Section 4.2
The material on elliptic random walks is based on [114], although the germ
of the model goes back to earlier work of two of the authors of the present
book (see the Acknowledgements in [114]).
Consider the example with increments supported on ellipsoids discussed
in Section 4.2.2. It follows from (4.32) that
kξ1 k2 = kξ0 k2 + 2kξ0 khξb0 , θi + kθk2
= kξ0 k2 + 2kξ0 khe1 , Dζi + hζ, D2 ζi
√
= kξ0 k2 + 2a dkξ0 khe1 , ζi + (a2 − b2 )dhe1 , ζi2 + b2 d.
(4.60)
In particular, for this family of models (kξn k, n ≥ 0) is itself a Markov
process, since the distribution of kξn+1 k depends only on kξn k and not ξn ;
however, in the general setting of Section 4.2.1, this need not be the case.
One-dimensional processes with evolutions reminiscent to that given by
(4.60) have been studied previously by Kingman [172] and Bingham [26].
Those processes can be viewed, respectively, as the distance from its start
point of a random walk in Euclidean space, and the geodesic distance from
its start point of a random walk on the surface of a sphere, but in both cases
the increments of the random walk have the property that the distribution
of the jump vector is a product of the independent marginal distributions
of the length and direction of the jump vector. In contrast, for the elliptic
random walk the laws of kθn k and hξbn , θbn i are not independent (except when
a = b).
236
Chapter 4. Many-dimensional random walks
Section 4.3
As mentioned before, the results in this section are from [257] and [87]. In
fact, the main result of [87] is a generalization of (4.34), and there are also
some bounds on hitting probabilities for the controlled walks.
The work [257] originated from the following question, posed in [20].
Let π1 and π2 be two zero mean measures in R4 with finite supports that
span the whole space. On the first visit to a site the jump of the process has
law π1 and at further visits it has law π2 . Is the resulting walk transient?
More generally, one can consider any adapted rule (a rule depending
on the history of the process) for choosing between π1 and π2 , and ask the
same question. It turns out that the answer to this question is positive, even
in R3 , as proved in Theorem 4.3.2.
This naturally fits into the wider context of random walks that are not
Markovian, namely where the next step the walk takes also depends on
the past. Recently there has been a lot of interest in random walks of this
kind. A large class of such walks are the so-called vertex (or edge) reinforced
random walks, where the walker chooses the next vertex to jump to with
weight proportional to the number of visits to that vertex up to that time;
see e.g. [18, 237, 256, 262, 301, 312]. Another class of such walks is the
so-called excited random walks, when the transition probabilities depend on
whether it is the first visit to a site or not, see e.g. [21, 22, 181, 222, 306, 321].
Section 4.4
Many-dimensional Markov processes with asymptotically zero drift, such as
those in this section, were studied by Lamperti [190, 192] under the name
centrally biased random walks, due to the nature of the drift field; the name
had been used earlier by Gillis [117] for the model in Example 4.1.9.
The results on the supercritical case given in Section 4.4.3 are adapted
from [236, 211, 53]. In (4.45) the exponent −2 on the logarithm is chosen for
simplicity; it could be replaced with any exponent strictly less than −1. The
law of large numbers for the process of norms, Lemma 4.4.3, is a variation
on Theorem 3.2 of [236]. The limiting direction part of Theorem 4.4.2 is
a variation on Theorem 2.2 of [211]. The method of proof of the limiting
direction, via Lemma 4.4.4, is different from (and more transparent than)
the method in [211], and extends the approach of [53, §6.2], which (in a
slightly different context) was restricted to two dimensions and assumed a
uniform bound on the increments. A particular case of the model with β = 0
is studied in [109], who also obtain a central limit theorem to accompany
237
Bibliographical notes
the law of large numbers.
The conclusion of Theorem 4.4.5 that ξˆn does not have a limit can be
strengthened in many circumstances. For example, a stronger property is
angular recurrence, which says that ξˆn visits every neighbourhood on Sd−1
infinitely often; see [211, 212] for some approaches to studying angular recurrence for non-homogeneous random walks. Stronger still is angular ergodicity, such as
lim inf n
n→∞
−1
n−1
X
m=0
1{ku − ξˆm k < ε} > 0.
The proof of Theorem 4.4.5 given here uses some ideas similar to those in
[211, 212], but is considerably more transparent.
The recurrence classification in Section 4.4.5 is similar to that given
in [135], and builds on work going back to Lamperti. Specifically, Theorem 4.4.8 extends results of Lamperti [190, 192], who assumed uniformly
bounded increments and a stronger version of (CB3) with the error term
log−1 kxk replaced by kxk−δ for δ > 0: see Theorem 4.1 of [190, p. 324] and
Theorem 5.1 of [192, p. 142]; the latter result is not sharp enough to decide
between null- or positive-recurrence at the boundary case 2ρ/σ 2 = −d. The
result on moments of return times, Theorem 4.4.9, also extends Theorem 5.1
of [192]; Lamperti’s result only covers integer s.
The almost-sure bounds in Theorem 4.4.10 are a variation on similar
results from [135]. Upper bounds similar to those in Theorem 4.4.10 can be
derived from [232, §3]: see Theorem 2.4 of [53] for a similar application of
such results, albeit under more restrictive assumptions.
Section 4.5
The range (i.e., the cardinality of the set visited sites, or sometimes this set
itself) and the local time (i.e., the number of visits to a given site) for spacehomogeneous discrete-time random walks have been extensively studied in
the literature. It is a classical result that the expected range of the simple
random walk is O( logn n ) for d = 2 and O(n) for d ≥ 3, see e.g. Section 6.1
of [139]. It is not difficult to obtain from this fact (using an independence
argument as e.g. in Lemma 3.1 of [3]) that with very high probability the
walk visits at least n1−δ distinct sites by time n. Finer results for the range
of homogeneous random walks can be found in a number of papers; see e.g.
in [19, 77, 124] and references therein. For nonhomogeneous random walks
these questions, of course, are more difficult; we mention [263] that contains
238
Chapter 4. Many-dimensional random walks
results on the range of simple random walk on a supercritical percolation
cluster.
The behaviour of the local time (i.e., the number of visits) in a fixed
site, or the field of local times in all sites, was much studied in the literature
as well. It is quite elementary to obtain that the expected number of visits
to the origin by time n for the simple random walk is O(log n) for d = 2
and O(1) for d ≥ 3. Also, one can easily obtain for the simple random
walk in dimension 2 (using e.g. E1 of Section III.16 of [293]) that, with
stretched-exponentially small probability, the number of visits to the origin
is less than nδ for any fixed δ > 0. Of course, finer results (for more general
random walks as well) are available; see e.g. [36, 62, 63, 66, 217].
Theorem 4.5.2 is taken from [222], although the exposition basically
follows [221]; the last paper also contains the proof of more general Theorem 4.5.4.
Theorem 4.5.2 shows that the range of a many-dimensional martingale
is of order at least n(1/2)+ε with high probability. As discussed above, for
simple random walk the range is in fact of order at least n1−ε . A natural
question is thus:
Open problem 4.5.5. Under the conditions of Theorem 4.5.2, either prove
that #Ξn is at least n1−ε with high probability for any ε > 0, or produce a
counterexample.
In general state-spaces Σ ⊆ Rd , one natural definition of the range is
Rn := ∪x∈Ξn B(x; r), for fixed r > 0,
(4.61)
the volume of the r-radius ‘sausage’ around the path. If Σ is locally finite,
then, as r ↓ 0,
r−d Rn → vd #Ξn ,
where vd = |B(0; 1)| is the volume of the unit-radius d-ball. So for locally
finite Σ, it is usual to work simply with #Ξn , the number of sites visited,
and also call this the range of the walk. This is the approach taken in
Section 4.5.
Open problem 4.5.6. Study the behaviour of Rn as defined by (4.61) for
the elliptic random walks of Section 4.2.
Chapter 5
Heavy tails
5.1
Chapter overview
So far, the processes that we have considered in this book have typically had
increments whose first (and second) moments have been uniformly bounded.
The focus of this chapter is on the case where first (or second) moments do
not exist, i.e., the increments are heavy-tailed. We consider Markov processes
on R, and study questions of asymptotic behaviour, such as recurrence or
transience.
This chapter deals with two different types of process. First, in Section 5.2, we study processes whose increments have, in some sense, heavier
tails in one direction than the other. We investigate sufficient conditions for
transience in the direction of the heavier tail, and quantify the transience
by deriving results on the rate of escape and on moments of first passage
and last exit times.
Second, in Section 5.3, we study processes whose increments are of two
different types, depending on whether the current position is to the left or to
the right of the origin. Such processes are known as oscillating random walks,
and their study (via classical methods) goes back to Kemperman [157]. We
consider a version of the model in which the increment distribution is signed,
i.e., only jumps towards the origin are allowed. For these models we study
recurrence and transience.
In keeping with the theme of this book, the methods of this chapter are
based on the semimartingale ideas of Chapter 2: for appropriate choices of
Lyapunov function, we verify Foster–Lyapunov style drift conditions. Verification of drift conditions usually entails some Taylor’s formula expansions
(as in Chapter 3) as well as some careful truncation ideas to deal with the
239
240
Chapter 5. Heavy tails
heavy tails. The resulting proofs are relatively short, and based on some
intuitively appealing ideas.
5.2
5.2.1
Directional transience
Introduction
In this section we assume the following.
(H0) Let Xn be a time-homogeneous Markov chain on Σ ⊆ R with 0 ∈ Σ,
inf Σ = −∞ and sup Σ = +∞. Suppose that X0 = 0.
For n ∈ Z+ , we write ∆n := Xn+1 − Xn for the increments of Xn . In
most of our results, we impose ‘heavy tail’ conditions on either ∆+
n (posit−
ive jumps) or ∆n (negative jumps) or both; typically these conditions are
one-sided (i.e., inequalities). Two of the most common assumptions are as
follows.
(H1) There exist α > 0, c > 0, and y0 < ∞ for which, for all x ∈ Σ and all
−α .
y ≥ y0 , P[∆+
n > y | Xn = x] ≥ cy
(H2) There exist α ∈ (0, 1), c > 0, and y0 < ∞ for which, for all x ∈ Σ and
+
1−α .
all y ≥ y0 , E[∆+
n 1{∆n ≤ y} | Xn = x] ≥ cy
The following non-confinement result shows that, under the conditions of
most of the results in this section, the process Xn has non-trivial asymptotic
behaviour.
Proposition 5.2.1. Suppose that (H0) holds. Suppose that either (H1) or
+
(H2) holds, or either holds with ∆−
n instead of ∆n . Then
lim sup |Xn | = ∞, a.s.
(5.1)
n→∞
Proof. We claim that under any of the conditions in the proposition, it is
the case that for any y ≥ 0 sufficiently large, there exists δ(y) > 0 for which,
P[|∆n | > y | Xn = x] ≥ δ(y), for all x ∈ Σ.
(5.2)
Consider the events An := {|Xn | > B} and let Fn = σ(X0 , . . . , Xn ). Given
(5.2), for any B < ∞ large enough,
P[An+1 ∪ An | Fn ] = P[An+1 | Fn ]1(Acn ) + 1(An ),
241
5.2. Directional transience
P
and on Acn we have P[An+1 | Fn ] ≥ δ(2B) > 0. Thus n≥0 P[An+1 ∪ An |
Fn ] = ∞ a.s., and An+1 ∪ An ∈ Fn+1 , so Lévy’s extension of the Borel–
Cantelli lemma (Theorem 2.3.19) shows that lim supn→∞ |Xn | ≥ B, a.s.
Since B was arbitrarily large, (5.1) follows.
−
It remains to verify (5.2). Since |∆n | = ∆+
n + ∆n , it suffices to verify
−
(5.2) with one of ∆+
n or ∆n in place of |∆n |. In the case where (H1) holds,
the claim is immediate. If (H2) holds, then, for any y ≥ y0 , for z > y,
+
+
+
E[∆+
t 1{y ≤ ∆t ≤ z} | Xn = x] ≥ E[∆n 1{∆n ≤ z} | Xn = x] − y > 1,
provided z > y0 + ((1 + y)/c)1/(1−α) , say. Then,
+
+
1 < E[∆+
n 1{y ≤ ∆n ≤ z} | Xn = x] ≤ z P[y ≤ ∆n ≤ z | Xn = x]
≤ z P[∆+
n ≥ y | Xn = x],
which implies (5.2) in this case also.
5.2.2
A condition for directional transience
Our first main result, which gives conditions under which limn→∞ Xn = +∞,
imposes a moments condition on the negative jumps of the process:
β
(H3) There exist β > 0 and C < ∞ for which E[(∆−
n ) | Xn = x] ≤ C for
all x ∈ Σ.
Theorem 5.2.2. Suppose that (H0) holds, and that (H2) and (H3) hold for
α ∈ (0, 1) and β > α. Then Xn → +∞ a.s. as n → ∞.
To prove Theorem 5.2.2, we use the Lyapunov function fz,δ : R → [0, 1]
defined for z ∈ R and δ > 0 by
(
1
if y ≤ z;
fz,δ (y) :=
(5.3)
−δ
(1 + y − z)
if y > z.
Note that fz,δ is non-increasing on R.
Lemma 5.2.3. Suppose that (H0) holds, and that (H2) and (H3) hold for
α ∈ (0, 1) and β > α. Then for any δ ∈ (0, β − α) and some A > 0
sufficiently large, for any z ∈ R,
E[fz,δ (Xn+1 ) − fz,δ (Xn ) | Xn = x] ≤ 0, for all x > z + A.
242
Chapter 5. Heavy tails
Proof. It suffices to suppose that n = 0 and z = 1. Let δ > 0, and let
fδ := f1,δ be as defined at (5.3). Let γ ∈ (0, 1); we will specify δ and γ later.
Since fδ is non-increasing and [0, 1]-valued, we have for x > 1 sufficiently
large that
h
i
−δ
−δ
γ
fδ (x + ∆0 ) − fδ (x) ≤ (x + ∆+
)
−
x
1{∆+
0
0 ≤x }
h
i
−
−δ
−δ
γ
γ
+ (x − ∆−
)
−
x
1{∆−
(5.4)
0
0 ≤ x } + 1{∆0 > x }.
We will take expectations on both sides of (5.4), conditioning on X0 = x.
The final term in (5.4) then becomes, by Markov’s inequality and (H3),
− β
γβ
−γβ
γ
.
Px [∆−
0 > x ] = Px [(∆0 ) > x ] ≤ Cx
(5.5)
For the second term on the right-hand side of (5.4), since γ < 1, Taylor’s
formula implies that, as x → ∞,
i
h
γ
γ
−1−δ −
−δ
−δ
∆0 1{∆−
1{∆−
)
−
x
(x − ∆−
0 ≤ x }, (5.6)
0 ≤ x } = δ(1+o(1))x
0
where the o(1) term is non-random. Here we have for the product of the
final two terms in (5.6) that
+
− β∧1
−
γ
(1−β)
γ
1{∆−
(∆−
∆−
0 ≤x }
0)
0 1{∆0 ≤ x } = (∆0 )
+
β∧1 γ(1−β)
x
.
≤ (∆−
0)
Combining (5.6) and (5.7), and using (H3), we obtain
i
i
hh
γ
−1−δ+γ(1−β)+
−δ
).
− x−δ 1{∆−
Ex (x − ∆−
0 ≤ x } = O(x
0)
(5.7)
(5.8)
For the first term on the right-hand side of (5.4), another application of
Taylor’s formula implies that, as x → ∞,
h
i
−δ
−δ
γ
−1−δ +
γ
(x + ∆+
)
−
x
1{∆+
∆0 1{∆+
0
0 ≤ x } = −δ(1 + o(1))x
0 ≤ x }.
Taking expectations and using (H2) we obtain, for x sufficiently large,
hh
i
i
+
−δ
−δ
γ
Ex (x + ∆+
)
−
x
1{∆
≤
x
}
≤ −(cδ/2)x−1−δ+γ(1−α) .
(5.9)
0
0
Thus from (5.4), using the estimates (5.5), (5.8) and (5.9), we verify that
Ex [fδ (X1 ) − fδ (X0 )] ≤ 0, for x > A with some A sufficiently large, provided
that the negative term arising from (5.9) dominates, i.e.,
−1 − δ + γ(1 − α) > −γβ,
and
− 1 − δ + γ(1 − α) > −1 − δ + γ(1 − β)+ .
243
5.2. Directional transience
The second inequality holds since α < β ∧ 1. The first inequality holds
provided we choose δ ∈ (0, β − α), which we may do since α < β, and then
1+δ
choose γ ∈ ( 1+β−α
, 1).
Now we can give the proof of Theorem 5.2.2.
Proof of Theorem 5.2.2. The argument is similar to Theorem 3.5.6. First
we show that, under the conditions of the theorem,
h
i
P lim inf Xn = −∞ = 0.
(5.10)
n→∞
Let a > 0, to be chosen later. For x ∈ R, set
νx := min{n ≥ 0 : Xn > x + a};
ηx := min{n ≥ νx : Xn ≤ x}.
In particular, since X0 = 0, we have that νx = 0 for all x < −a.
Let δ ∈ (0, β − α) and write Fn = σ(X0 , . . . , Xn ). Then Lemma 5.2.3
shows that, on {νx < ∞}, (fx−A,δ (Xn∧ηx ), n ≥ νx ) is a non-negative supermartingale, and so converges a.s. to a finite limit, Lx , say. On {νx < ∞},
we have by the supermartingale property that
E[fx−A,δ (Xn∧νx ) | Fνx ] ≤ fx−A,δ (Xνx ) ≤ (1 + A + a)−δ , a.s.,
while by Fatou’s lemma, also on {νx < ∞},
lim E[fx−A,δ (Xn∧ηx ) | Fνx ] ≥ E[Lx | Fνx ]
n→∞
≥ E[Lx 1{ηx < ∞} | Fνx ]
≥ (1 + A)−δ P[ηx < ∞ | Fνx ],
since, on {ηx < ∞}, Xn∧ηx ≤ x for all n sufficiently large. So on {νx < ∞}
we have, a.s.,
1 + A + a −δ
.
P[ηx < ∞ | Fνx ] ≤
1+A
Let ε > 0. Then we can take a sufficiently large so that P[ηx = ∞ |
Fνx ] ≥ 1 − ε, a.s., on {νx < ∞}. For such a choice of a, suppose that
x < −a; then νx < ∞ a.s. (indeed, since X0 = 0, νx = 0 a.s.). Hence for
such an x,
h
i
P lim inf Xn > x = P[ηx = ∞] ≥ E [P[ηx = ∞ | Fνx ]1{νx < ∞}]
n→∞
≥ 1 − ε.
(5.11)
244
Chapter 5. Heavy tails
It follows from (5.11) that
h
i
h
i
P lim inf Xn = −∞ ≤ P lim inf Xn ≤ −a − 1 ≤ ε.
n→∞
n→∞
Since ε > 0 was arbitrary, (5.10) follows.
Proposition 5.2.1 applies under condition (H2). Hence, together with
(5.1), (5.10) implies that, a.s., lim supn→∞ Xn = ∞; in other words, for any
a > 0 and any x ∈ R, νx < ∞ a.s. Hence the argument for (5.11) extends
to any x ∈ R, which implies that for any x ∈ R, a.s., lim inf n→∞ Xn > x, so
Xn → ∞ a.s.
5.2.3
Almost-sure bounds
Our next two results deal with the growth rate of Xn . First we have the
following upper bounds.
Theorem 5.2.4. Suppose that there exist θ ∈ (0, 1], ϕ ∈ R, y0 < ∞ and
C < ∞ such that, for all y ≥ y0 ,
−θ
ϕ
P[∆+
n ≥ y | Xn = x] ≤ Cy (log y) , for all x ∈ Σ.
(5.12)
(i) If θ ∈ (0, 1), then, for any ε > 0, a.s., for all but finitely many n ≥ 0,
Xn ≤ n1/θ (log n)
ϕ+2
+ε
θ
.
(ii) If θ = 1, then, for any ε > 0, a.s., for all but finitely many n ≥ 0,
+ +1+ε
Xn ≤ n(log n)(1+ϕ)
.
The next result shows that if we impose condition (H1) instead of the
condition (H2) in Theorem 5.2.2, not only does Xn → +∞, a.s., but it does
so at a particular rate of escape.
Theorem 5.2.5. Suppose that (H0) holds, and that (H1) and (H3) hold for
α ∈ (0, 1) and β > α. Then for any ε > 0, a.s., for all but finitely many
n ≥ 0,
Xn ≥ n1/α (log n)−(1/α)−ε .
Theorems 5.2.4 and 5.2.5 have the following corollary.
245
5.2. Directional transience
Corollary 5.2.6. Suppose that (H0) holds, and that (H3) holds. Suppose
also that for α ∈ (0, 1) with α < β,
log P[∆+
n > y | Xn = x]
lim sup + α = 0.
y→∞ x∈Σ
log y
Then
1
log Xn
= , a.s.
n→∞ log n
α
lim
Proof. The condition in the corollary ensures that for any ε > 0 there exists
y0 < ∞ such that, for all y ≥ y0 and all x ∈ Σ,
−α+ε
y −α−ε ≤ P[∆+
, a.s.
n > y | Xn = x] ≤ y
Theorem 5.2.4 with the upper bound in the last display then shows that for
any ε > 0, a.s., Xn ≤ n(1/α)+ε for all but finitely many n. On the other
hand, Theorem 5.2.5 with the lower bound in the last display and (H3)
shows that for any ε > 0, a.s., Xn ≥ n(1/α)−ε for all but finitely many n.
Since ε > 0 was arbitrary, the result follows.
The rest of this section is devoted to the proofs of Theorems 5.2.4
and 5.2.5. First we give s a general condition for obtaining almost-sure
upper bounds.
Lemma 5.2.7. Let h : R+ → R+ be increasing and concave. Suppose that
there exists C < ∞ such that E[h(∆+
n ) | Xn = x] ≤ C, for all x ∈ Σ. Then
for any ε > 0, a.s., for all but finitely many n ≥ 0,
Xn ≤
n−1
X
m=0
−1
1+ε
∆+
).
m ≤ h (n(log n)
P
+
Proof. Set Y0 := 0 and for n ∈ N let Yn := n−1
m=0 ∆m . Then Yn ≥ 0 is nondecreasing and Xn ≤ X0 + Yn = Yn , since X0 = 0. Since h is non-negative
and concave, it is subadditive, i.e., h(a + b) ≤ h(a) + h(b) for a, b ∈ R+ .
Hence
+
E[h(Yn + ∆+
n ) − h(Yn ) | Xn = x] ≤ E[h(∆n ) | Xn = x] ≤ C,
(5.13)
by hypothesis. The almost-sure upper bound in Theorem 2.8.1 applied to
h(Yn ) implies that, for any ε > 0, a.s., h(Yn ) ≤ n(log n)1+ε , for all but
finitely many n. Since h is increasing and Xn ≤ Yn , it follows that for any
ε > 0, a.s., h(Xn ) ≤ n(log n)1+ε .
246
Chapter 5. Heavy tails
Next we need a result on the maxima of the increments of Xn .
Lemma 5.2.8. Suppose that (H1) holds for α ∈ (0, ∞). Then for any ε > 0,
a.s., for all but finitely many n ≥ 0,
1/α
max ∆+
(log n)−(1/α)−ε .
m ≥n
0≤m≤n
Proof. By repeated applications of the Markov property and (H1),
n
Y
+
P max ∆m < y ≤
(1 − cy −α ) ≤ (1 − cy −α )n ,
0≤m≤n
(5.14)
m=0
for y ≥ y0 . Taking y = n1/α (log n)q in (5.14) we obtain, for n sufficiently
large,
n
+
1/α
q
P max ∆m < n (log n) ≤ 1 − cn−1 (log n)−αq
0≤m≤n
= O exp −c(log n)−αq ,
which is summable over sufficiently large n provided q < −1/α. Hence the
Borel–Cantelli lemma completes the proof.
Now we can complete the proofs of Theorems 5.2.4 and 5.2.5.
Proof of Theorem 5.2.4. First we prove part (i), so let θ ∈ (0, 1). For ε > 0,
take h(x) = (K+x)θ (log(K+x))−ϕ−1−ε . For a large enough choice of K ≥ 1,
h is non-negative, increasing, P
and concave. Moreover, E[h(∆+
n ) | Xn = x] is
0 (k) P[∆+ > k | X = x] is uniformly
h
uniformly bounded provided ∞
n
n
k=1
bounded; see e.g. [123, p. 76]. This is indeed the case under the hypothesis of
the theorem, by (5.12), since h0 (x) = O(xθ−1 (log x)−ϕ−1−ε ). Now (i) follows
ϕ+1
from Lemma 5.2.7, noting that h−1 (x) = O(x1/θ (log x) θ +ε ). The proof of
+
(ii) is similar, this time taking h(x) = (K + x)(log(K + x))−(ϕ+1) −ε .
Proof of Theorem 5.2.5. Let ε > 0. Lemma 5.2.7 applied to −Xn with
h(x) = xβ∧1 , using (H3), shows that for all but finitely many n,
n−1
X
m=0
1/(β∧1)
∆−
(log n)(1/(β∧1))+ε , a.s.
m ≤n
On the other hand, Lemma 5.2.8 implies that for all but finitely many n,
n−1
X
m=0
∆+
m ≥
max
0≤m≤n−1
1/α
∆+
(log n)−(1/α)−ε , a.s.
m ≥n
Combining these bounds and using the fact that α < β ∧ 1 completes the
proof.
5.2. Directional transience
5.2.4
247
Moments of passage times
Recall that ρx = min{n ≥ 0 : Xn ≥ x} denotes the first passage time into
the half-line [x, ∞). Under the conditions of Theorem 5.2.2, Xn → +∞,
a.s., so that ρx < ∞ a.s., for all x ∈ R. It is natural to study the tails or
moments of the random variable ρx in order to quantify the transience.
Theorem 5.2.9. Suppose that (H0) holds, and that (H2) and (H3) hold for
α ∈ (0, 1) and β > α. Then for any x ∈ R and any s ∈ [0, β/α), E[ρsx ] < ∞.
Theorem 5.2.10. Let α ∈ (0, 1] and β > 0. Suppose that, for some C < ∞,
α
− β
E[(∆+
n ) | Xn = x] ≤ C for all x ∈ Σ, and E[(∆n ) | Xn = x] = ∞ for all
β/α
x ∈ Σ. Then, for any x > 0, E[ρx ] = ∞.
Note that in Theorem 5.2.9, β/α > 1, so in particular E ρx < ∞ for any
x ∈ R. The proof of Theorem 5.2.9 will use another Lyapunov function,
described in the next result.
Lemma 5.2.11. Suppose that (H0) holds, and that (H2) and (H3) hold
for α ∈ (0, 1) and β > α. For γ ∈ (α, β) and y ∈ R let Wn := (y −
Xn )γ 1{Xn < y}. Then the following hold.
(i) Take γ = β − ε. Then for any ε ∈ (0, β(β−α)
1+β−α ) there exists a finite
constant K such that, for all n,
E[Wn+1 − Wn | Xn = x] ≤ K, for all x ∈ Σ.
(ii) For any η ∈ (0, 1 − (α/β)), we can choose z < y and γ ∈ (α, β) such
that, for some c > 0, for all x < z,
E[Wn+1 − Wn | Xn = x] ≤ −cWnη .
Proof. Fix y ∈ R. Also take γ ∈ (α, β) and θ ∈ (0, 1); we will make more
restrictive specifications for these parameters later. It suffices to suppose
that n = 0. If X0 = x < y − 1, we have (y − X0 )θ < y − X0 and so
W1 − W0 = (y − x − ∆0 )γ 1{X1 < y} − (y − x)γ
+
γ
γ
θ
≤ (y − x − ∆+
0 ) − (y − x) 1{∆0 ≤ (y − x) }
−
γ
γ
θ
+ (y − x + ∆−
0 ) − (y − x) 1{∆0 ≤ (y − x) }
−
γ
θ
+ (y − x + ∆−
0 ) 1{∆0 ≥ (y − x) }.
(5.15)
248
Chapter 5. Heavy tails
We bound the three terms on the right-hand side of (5.15) in turn. For the
first term, we have from Taylor’s formula that
+
γ
γ
θ
(y − x − ∆+
0 ) − (y − x) 1{∆0 ≤ (y − x) }
γ−1
θ
= −γ∆+
(1 + o(1))1{∆+
0 (y − x)
0 ≤ (y − x) },
where the o(1) is non-random and as y −x → ∞. Hence, taking expectations
and using (H2), it follows that for a fixed y and any x for which y − x is
large enough,
h
i
+
γ
γ
θ
Ex (y − x − ∆+
)
−
(y
−
x)
1{∆
≤
(y
−
x)
}
0
0
≤ −(cγ/2)(y − x)γ−1+θ(1−α) .
For the second term on the right-hand side of (5.15), a similar application
of Taylor’s formula yields, for y − x sufficiently large,
−
θ
γ
γ
(y − x + ∆−
0 ) − (y − x) 1{∆0 ≤ (y − x) }
+
θ
(1−β)
β∧1
1{∆−
(∆−
≤ 2γ(y − x)γ−1 (∆−
0 ≤ (y − x) }
0)
0)
+
β∧1
.
≤ 2γ(y − x)γ−1+θ(1−β) (∆−
0)
Taking expectations and using (H3), we obtain
h
i
+
−
θ
γ
γ
≤
(y
−
x)
}
≤ K(y − x)γ−1+θ(1−β) ,
Ex (y − x + ∆−
)
−
(y
−
x)
1{∆
0
0
for a constant K < ∞. For the final term in (5.15),
− γ/θ
− 1/θ
−
γ
γ
θ
γ
(y − x + ∆−
.
+ ∆−
0 ) ≤ 2 (∆0 )
0 ) 1{∆0 ≥ (y − x) } ≤ ((∆0 )
Taking γ = θβ, which requires θ ∈ (α/β, 1), and using (H3), we see that
h
i
−
γ
θ
Ex (y − x + ∆−
)
1{∆
≥
(y
−
x)
}
≤ 2γ C.
0
0
Combining these estimates and taking expectations in (5.15) we see that the
negative term dominates asymptotically provided
γ − 1 + θ(1 − α) > 0 and γ − 1 + θ(1 − α) > γ − 1 + θ(1 − β)+ .
The first inequality requires θ > 1/(1 + β − α), which is a stronger condition
than θ > α/β that we had already imposed, but which can be achieved with
θ ∈ (α/β, 1) since α < β. The second inequality reduces to 1 − α > (1 − β)+
which is satisfied since α < β ∧ 1. Part (i) follows. Moreover, for γ = θβ,
249
5.2. Directional transience
1/(1 + β − α) < θ < 1, we can take y − x large enough so that, for some
ε > 0,
Ex [W1 − W0 ] ≤ −ε(y − x)θβ−1+θ(1−α) = −εW0η ,
where η = (θβ − 1 + θ(1 − α))/(θβ) can be anywhere in (0, 1 − (α/β)), by
appropriate choice of θ, which proves part (ii).
Lemma 5.2.11 has as a consequence the following tail bound.
Lemma 5.2.12. Suppose that (H0) holds, and that (H2) and (H3) hold for
α ∈ (0, 1) and β > α. Then for any ϕ > 0 and any ε > 0, as n → ∞,
ϕ
P min Xm ≤ −n = O(n1−βϕ+ε ).
0≤m≤n
Proof. As in Lemma 5.2.11, choosing y = 0 there, let Wn = (−Xn )γ 1{Xn < 0}.
Take γ = β − ε for ε ∈ (0, β(β−α)
1+β−α ). For n > 0,
ϕ
ϕγ
P min Xm ≤ −n ≤ P max Wm ≥ n
= O(n1−ϕγ ),
0≤m≤n
0≤m≤n
by Lemma 5.2.11(i) and Theorem 2.4.8. The result follows.
Now we can give the proof of our result on existence of moments for ρx .
Proof of Theorem 5.2.9. Define Wn = (y−Xn )γ 1{Xn < y} as in Lemma 5.2.11.
For z > 0, let τz = min{n ≥ 0 : Wn ≤ z}. Since {Wn ≤ z} = {Xn ≥
y − z 1/γ }, we have that ρx = τ(y−x)γ for x ≤ y. Now fix x ∈ R. Under the conditions of the theorem, Lemma 5.2.11(ii) implies that, for any
η ∈ (0, 1 − (α/β)), for y > x sufficiently large, on {n < σ(y−x)γ }, a.s.,
E[Wn+1 − Wn | Fn ] ≤ −εWnη .
s
Then Corollary 2.7.3 shows that for any x ∈ R, E[ρsx ] = E[τ(y−x)
γ ] < ∞, for
any s < β/α.
Next we prove our non-existence of moments result for ρx . We use an
idea based on Theorem 2.4.8: roughly speaking, we show that with good
probability Xn travels a long way in the negative direction with a single
heavy-tailed jump, and then must take a long time to come back.
i
For processes with heavy-tailed increments, it is often a
single big jump that dominates asymptotics; technically, this
idea often gives estimates in one direction that are close to
sharp.
250
Chapter 5. Heavy tails
Proof of Theorem 5.2.10. Fix x > 0 and let y < x. Let Wn0 = (Xn −
0
α
y)α 1{Xn > y}. Then on {Xn ≤ y}, Wn+1
− Wn0 ≤ (∆+
n ) . On the other
hand, on {Xn > y},
0
α
α
+ α
Wn+1
− Wn0 ≤ (Xn + ∆+
n − y) − (Xn − y) ≤ (∆n ) ,
by concavity since α ∈ (0, 1]. Hence for C < ∞ (not depending on y),
0
E[Wn+1
− Wn0 | Fn ] ≤ C, a.s., so the maximal inequality Theorem 2.4.8
implies that, for any y < x,
Cm + Wn0
0
α
P max Wn+k ≥ (x − y) | Fn ≤
, a.s.
0≤k≤m
(x − y)α
In particular, on {Xn ≤ y}, Wn0 = 0 and so
α
0
P max Xn+k ≥ x | Fn ≤ P max Wn+k ≥ (x − y) | Fn ≤
0≤k≤m
0≤k≤m
Cm
, a.s.
(x − y)α
Setting m = (x − y)α /(2C) in the last display, we obtain that for some ε > 0
(not depending on x or y), on {n < ρx } ∩ {Xn ≤ y}, for any y < x,
P[ρx ≥ ε(x − y)α | Fn ] ≥ 1/2, a.s.
(5.16)
−
Since X0 = 0 and x > 0, we have that {∆−
0 > y } implies {ρx > 1} and
{X1 ≤ y}. So applying (5.16) at n = 1 we have that
1
−
α
−
P[ρx ≥ ε(x − y)α ] ≥ E 1{∆−
P[∆−
0 > y } P[ρx ≥ ε(x − y) | F1 ] ≥
0 > y ].
2
Taking y = −ε−1/α z 1/α < 0, we have that for any z > 0,
P[ρx ≥ z] ≥ P[ρx ≥ ε(x − y)α ] ≥
1
−1/α 1/α
z ].
P[∆−
0 >ε
2
Hence for any γ > 0,
Z ∞
Z
1 ∞
γ
1/γ
−1/α 1/(αγ)
E[ρx ] =
P[ρx > z ]dz ≥
P[∆−
z
]dz.
0 >ε
2
0
0
Using the substitution w = ε−γ z we obtain
Z ∞
1
1
αγ
1/(αγ)
E[ργx ] ≥ εγ
P[∆−
]dw = εγ E[(∆−
0 >w
0 ) ],
2
2
0
which is infinite provided αγ ≥ β, i.e., γ ≥ β/α.
5.2. Directional transience
5.2.5
251
Moments of last exit times
Similarly to (3.51), for x ∈ R we denote the last exit time from (−∞, x] by
ηx := max{n ≥ 0 : Xn ≤ x}.
If Xn → +∞ a.s. (such as under the conditions of Theorem 5.2.2) then
ηx < ∞ a.s. for all x ∈ R, and the moments of the random variables ηx
provide a quantitative characterization of the transience.
Theorem 5.2.13. Suppose that (H0) holds, and that (H2) and (H3) hold
for α ∈ (0, 1) and β > α. Then for any x ∈ R and any s ∈ [0, (β/α) − 1),
E[ηxs ] < ∞.
Theorem 5.2.14. Suppose that (H0) holds. Let α ∈ (0, 1] and β > α.
α
Suppose that there exist c > 0, C < ∞, and y0 < ∞ such that E[(∆+
n) |
−
−β
Xn = x] ≤ C for all x ∈ Σ, and P[∆n > y | Xn = x] ≥ cy
for all y ≥ y0
and all x ∈ Σ. Then for any x ∈ R and any s > (β/α) − 1, E[ηxs ] = ∞.
We will again use the Lyapunov function fz,δ defined at (5.3).
Lemma 5.2.15. Let α ∈ (0, 1) and β > α. Suppose that there exist C < ∞,
α
c > 0, and x0 < ∞ for which E[(∆+
n ) | Xn = x] ≤ C for all x ∈ Σ and
−β
−
for all y ≥ y0 and all x ∈ Σ. Then for any
P[∆n ≥ y | Xn = x] ≥ cy
δ > β − α and some A > 0 sufficiently large, for any z ∈ R,
E[fz,δ (Xn+1 ) − fz,δ (Xn ) | Xn = x] ≥ 0, for all x > z + A.
Proof. As in the proof of Lemma 5.2.3, it suffices to take n = 0 and z = 1.
Let δ > 0, and let fδ := f1,δ be as defined at (5.3). Let γ ∈ (0, 1); we will
specify δ and γ later. For x > 1 we have
−δ
γ
− x−δ ]1{∆+
fδ (x + ∆0 ) − fδ (x) ≥ [(x + ∆+
0)
0 ≤x }
+
−δ
γ
+ (1 − x−δ )1{∆−
0 ≥ x} − x 1{∆0 > x }.
(5.17)
We bound the three terms on the right-hand side of (5.17). For the first
term, we have that by Taylor’s formula, as x → ∞, since γ < 1,
h
i
−δ
−δ
γ
−1−δ +
γ
(x + ∆+
)
−
x
1{∆+
∆0 1{∆+
0
0 ≤ x } = −δ(1 + o(1))x
0 ≤ x }, a.s.,
where, as usual, the o(1) term is non-random. Similarly to (5.7), we have
+
+ α (1−α)γ
γ
that ∆+
, so that
n 1{∆0 ≤ x } ≤ (∆0 ) x
+ α (1−α)γ−1−δ
−δ
γ
− x−δ 1{∆+
).
(x + ∆+
0)
0 ≤ x } = O((∆0 ) x
252
Chapter 5. Heavy tails
It follows that
h
i
+
−δ
−δ
γ
α
Ex |(x + ∆+
)
−
x
|1{∆
≤
x
}
= O(x(1−α)γ−1−δ Ex [(∆+
0
0
0 ) ])
= O(x(1−α)γ−1−δ ).
(5.18)
For the second term on the right-hand side of (5.17), we have that for some
A > 1 sufficiently large,
−
−β
Ex [(1 − x−δ )1{∆−
.
0 ≥ x}] ≥ (1/2) Px [∆0 ≥ x] ≥ (c/2)x
(5.19)
For the third term on the right-hand side of (5.17), we have that, by
Markov’s inequality,
α
−δ−αγ
γ
−δ −αγ
).
Ex [(∆+
Ex [x−δ 1{∆+
0 ) ] = O(x
0 >x } ]≤x x
(5.20)
Combining (5.17) with (5.18), (5.19) and (5.20) we have that
Ex [fδ (X1 ) − fδ (X0 )] ≥ (c/2)x−β + O(x−δ−αγ ) + O(x(1−α)γ−1−δ ).
The positive x−β term here dominates for A large enough provided that
−β > −δ − αγ and − β > (1 − α)γ − 1 − δ.
For any δ > β − α, the second inequality holds since α ∈ (0, 1) and γ <
1. Given any such δ, the first inequality holds provided we choose γ ∈
( β−δ
α , 1).
Finally, we can complete the proofs of our results on last exit times.
Proof of Theorem 5.2.13. Fix x ∈ R and let y > x, to be specified later. For
this proof, define the stopping time λy,x := min{n ≥ ρy : Xn ≤ x}, the time
of reaching (−∞, x] after having first reached [y, ∞). To prove our result
on finiteness of moments for λx , we prove an upper tail bound for ηx . For
y > x, {ρy ≤ n} ∩ {λy,x = ∞} implies {ηx ≤ n}, so
P[ηx > n] ≤ P[λy,x < ∞] + P[ρy > n].
(5.21)
We obtain an upper bound for P[λy,x < ∞] using our usual supermartingale
argument. Under the conditions of the theorem, Lemma 5.2.3 applies. It
follows that for δ ∈ (0, β − α), on {ρy < ∞}, (fx−A,δ (Xn∧ηy,x ), n ≥ ρy ) is a
non-negative supermartingale adapted to (Fn , n ≥ ρy ), and hence converges
a.s. as n → ∞ to a limit, Ly,x , say. Then, on {ρy < ∞}, by Fatou’s lemma,
fx−A,δ (Xρy ) ≥ E[Ly,x | Fρy ] ≥ E[fx−A,δ (Xλy,x )1{λy,x < ∞} | Fρy ]
253
5.2. Directional transience
≥ (1 + A)−δ P[λy,x < ∞ | Fρy ].
By definition, on {ρy < ∞}, Xρy ≥ y, so fx−A,δ (Xρy ) ≤ (1 + A + y − x)−δ .
Hence,
P[λy,x < ∞] = E[P[λy,x < ∞ | Fρy ]1{ρy < ∞}] = O(y −δ ).
(5.22)
For the final term in (5.21), for y > 0, P[ρy > n] = P [max0≤m≤n Xm < y],
where
P max Xm < y ≤ P max Xm ≤ y, min Xs ≥ −y + P min Xm ≤ −y
0≤m≤n
0≤m≤n
0≤m≤n
0≤m≤n
+
≤P
max ∆m ≤ 2y + P min Xs ≤ −y .
0≤m≤n−1
0≤m≤n
(5.23)
We choose y = n(1/α)−ε , for ε ∈ (0, 1/α). Then we have from (5.14) that for
c0 > 0,
+
(1/α)−ε
P
max ∆m ≤ 2n
= O(exp{−c0 nαε }).
(5.24)
0≤m≤n−1
On the other hand, the ϕ = (1/α) − ε case of Lemma 5.2.12 implies that
(1/α)−ε
P min Xm ≤ −n
= O(n1−(β/α)+ε ).
(5.25)
0≤m≤n
Using the bounds (5.24) and (5.25) in the y = n(1/α)−ε case of (5.23), we
obtain
(1/α)−ε
P max Xm < n
= O(n1−(β/α)+(β+1)ε ).
(5.26)
0≤m≤n
Thus taking y = n(1/α)−ε in (5.21) and δ as close as we wish to β − α, and
combining (5.22) with (5.26), we conclude that, for any ε > 0, P[ηx > n] =
O(n1−(β/α)+ε ), which yields the claimed moment bounds.
Proof of Theorem 5.2.14. Fix x ∈ R and let y > x. For this proof, define
λn,x := min{m ≥ n : Xm ≤ x}, the first time of reaching (−∞, x] after time
n. Similarly, set ρn,y := min{m ≥ n : Xm ≥ y}. We have that, for r > 0,
P[λx > n] ≥ E [1{Xn ≤ r} P[λn,x < ∞ | Fn ]] .
(5.27)
Under the conditions of the theorem, Lemma 5.2.15 applies. It follows
that for δ > β − α, (fx−A,δ (Xm∧λn,x ∧ρn,y ), m ≥ n) is a non-negative submartingale adapted to (Fm , m ≥ n); moreover, it is uniformly bounded and
254
Chapter 5. Heavy tails
so converges a.s. and in L1 , as m → ∞, to the limit fx−A,δ (Xλn,x ∧ρn,y ),
since νn,x ∧ ρn,y < ∞ a.s., by (5.1), which is available since Proposition 5.2.1
applies under the conditions of the theorem. Hence, a.s.,
fx−A,δ (Xn ) ≤ E[fx−A,δ (Xλn,x ∧ρn,y ) | Fn ] ≤ P[λn,x < ∞ | Fn ] + fx−A,δ (y).
Since y was arbitrary, and fx−A,δ (y) → 0 as y → ∞, it follows that, a.s.,
P[νn,x < ∞ | Fn ] ≥ fx−A,δ (Xn ) ≥ fx−A,δ (r),
on {Xn ≤ r}. Hence from (5.27) we obtain for r ≥ x,
P[λx > n] ≥ fx−A,δ (r) P[Xn ≤ r] ≥ (1 + A + r − x)−δ P[Xn ≤ r].
(5.28)
It remains
a lower bound for P[Xn ≤ r], for a suitable choice of
Pto obtain
+ . Following the argument for (5.13), with h(y) = y α ,
∆
r. Let Yn = n−1
m=0 m
α ∈ (0, 1], we may apply Theorem 2.4.8 to Zn = Ynα to obtain
P max Ymα ≥ x = P[Yn ≥ x1/α ] ≤ Cnx−1 ,
0≤m≤n
for some C < ∞ and all n ≥ 0, x > 0, which implies that
i
i
h
h
P Xn ≤ (2Cn)1/α ≥ P Yn ≤ (2Cn)1/α ≥ 1/2,
since Xn ≤ X0 + Yn = Yn . Thus taking r = (2Cn)1/α , we have P[Xn ≤ r] ≥
1/2, and with this choice of r in (5.28) we obtain P[λx > n] ≥ εn−δ/α , for
some ε > 0 and all n sufficiently large. Since δ > β − α was arbitrary, the
result follows.
5.3
5.3.1
Oscillating random walk
Introduction
We consider (Xn , n ∈ Z+ ) a discrete-time, time-homogeneous Markov process on R, with transition kernel p(x, dz) = wx (z − x)dz so that
Z
Z
P[Xn+1 ∈ A | Xn = x] =
p(x, dz) =
wx (z − x)dz,
(5.29)
A
A
for all Borel sets A ⊆ R. Here the local transition densities wx : R → R+
are Borel functions. We will assume one of several ‘heavy-tailed’ conditions
for the wx .
We are interested in the recurrence classification of the non-homogeneous
random walk Xn , where we say Xn is
5.3. Oscillating random walk
255
• recurrent if lim inf n→∞ |Xn | = 0, a.s.;
• transient if limn→∞ |Xn | = ∞, a.s.
For a probability density (Borel) function v : R → R+ and an exponent
α ∈ (0, ∞), we write v ∈ Dα to mean that there exists c : R+ → (0, ∞) with
supy≥0 c(y) < ∞ and limy→∞ c(y) = c ∈ (0, ∞) for which
(
c(y)y −1−α if y > 0,
v(y) :=
(5.30)
0
if y ≤ 0.
Our main interest concerns models in which wx depends on x only
through sgn x, the sign of x. Such models are known as oscillating random walks. The first example of this type is as follows.
(Os1) Let vα ∈ Dα and vβ ∈ Dβ , for some α, β > 0. For x, y ∈ R, let
(
vα (−y) if x ≥ 0,
wx (y) :=
vβ (y)
if x < 0.
This case is known as the one-sided oscillating random walk; in words, the
walk always jumps in the direction of (and possibly over) the origin, with
tail exponent α from the positive half-line and exponent β from the negative
half-line. The following recurrence classification applies.
Theorem 5.3.1. Suppose that (Os1) holds. Then the one-sided oscillating
random walk is transient if α + β < 1 and recurrent if α + β > 1.
Remarks 5.3.2. (a) Theorem 5.3.1 does not cover the critical case α+β = 1,
which cannot be decided upon without further assumptions on the rate of
convergence of c(y) in (5.30).
(b) An analogous result to Theorem 5.3.1 holds in the discrete-space case,
where instead of a density function for the increments we have a probability
mass function.
A special case in which α = β and vα = vβ in (Os1) is the antisymmetric
case in which
wx (y) = vα (−y sgn(x)), for x, y ∈ R.
In this case, Theorem 5.3.1 shows that the walk is transient for α < 1/2
and recurrent for α > 1/2. The rest of this section is devoted to the proof
of Theorem 5.3.1. First in Section 5.3.2 we present some comments on
neighbouring models.
256
5.3.2
Chapter 5. Heavy tails
Relation to some two-dimensional random walks
(1)
(2)
Example 5.3.3. Consider ξn = (ξn , ξn ), n ∈ Z+ , a nearest-neighbour
random walk on Z2 with transition probabilities
P[ξn+1 = (y1 , y2 ) | ξn = (x1 , x2 )] = p(x1 , x2 ; y1 , y2 ).
Suppose that the probabilities are given for x2 6= 0 by,
1
p(x1 , x2 ; x1 , x2 + 1) = p(x1 , x2 ; x1 , x2 − 1) = ;
3
1
p(x1 , x2 ; x1 + 1, x2 ) = 1{x2 < 0};
3
1
p(x1 , x2 ; x1 − 1, x2 ) = 1{x2 > 0};
3
(5.31)
(the rest being zero) and for x2 = 0 by p(x1 , 0; x1 , 1) = 1 for all x1 > 0,
p(x1 , 0; x1 , −1) = 1 for all x1 < 0, and p(0, 0; 0, 1) = p(0, 0; 0, −1) = 1/2.
See Figure 5.1 for an illustration.
(2)
Set τ0 := 0 and define recursively τk+1 = min{n > τk : ξn = 0} for
(1)
k ≥ 0; consider the embedded Markov chain Xn = ξτn . We show that Xn is
a discrete version of the oscillating random walk described in Section 5.3.1.
(2)
Indeed, ξn is a random walk on Z with increments taking values −1, 0, +1
each with probability 1/3. We then have that for some constant c ∈ (0, ∞),
P[τ1 > r] = (c + o(1))r−1/2 , as r → ∞;
(1)
see e.g. [99, p. 415]. Suppose that ξ0
(1)
times τ0 and τ1 , ξn is monotone,
= x > 0. Note that since between
(1)
P[ξτ(1)
− ξτ(1)
< −r] ≥ P[τ1 > 3r + r3/4 ] − P[ξ3r+r3/4 ≥ −r].
1
0
Here, by the Azuma–Hoeffding inequality (Theorem 2.4.14),
(1)
P[ξ3r+r3/4 ≥ −r] ≤ exp{−εr1/2 },
for some ε > 0. Similarly,
(1)
P[ξτ(1)
− ξτ(1)
< −r] ≤ P[τ1 > 3r − r3/4 ] + P[ξ3r−r3/4 ≤ −r],
1
0
where
(1)
P[ξ3r−r3/4 ≤ −r] ≤ exp{−εr1/2 }.
257
5.3. Oscillating random walk
(1)
Combining these bounds, and using the symmetric argument for {ξτ1 > r}
(1)
when ξ0 = x < 0, we see that
P[Xn+1 − Xn < −r | Xn = x] = u(r), if x > 0, and
P[Xn+1 − Xn > r | Xn = x] = u(r), if x < 0,
(5.32)
−60
−40
−20
0
20
40
60
where u(r) = (c + o(1))r−1/2 . Thus the process Xn satisfies a discrete-space
analogue of (Os1) with α = β = 1/2. This is the critical case identified in
Theorem 5.3.1, and a finer analysis is required to determine the behaviour;
we conjecture that the walk is recurrent.
4
−100
−50
0
50
100
Figure 5.1: Pictorial representation of the non-homogeneous nearestneighbour random walk on Z2 of Example 5.3.4, plus a simulated trajectory
of 5000 steps of the walk. We conjecture that the walk is recurrent.
Example 5.3.4. We present two variations on the previous example, which
are superficially similar but turn out to be less delicate. First, modify the
random walk of the previous example by supposing that (5.31) holds but
replacing the behaviour at x2 = 0 by p(x1 , 0; x1 , 1) = p(x1 , 0; x1 , −1) = 1/2
for all x1 ∈ Z. See the left-hand part of Figure 5.2 for an illustration.
As in Example 5.3.3, consider the embedded process Xn . In this case,
for all x ∈ Z, for r ≥ 0,
P[Xn+1 − Xn < −r | Xn = x] = P[Xn+1 − Xn > r | Xn = x] = u(r), (5.33)
where u(r) = (c/2)(1 + o(1))r−1/2 . Thus Xn is a random walk with symmetric increments, and a result of Shepp [283] implies that the walk is transient.
258
Chapter 5. Heavy tails
Next, modify the random walk of Example 5.3.3 by supposing that (5.31)
holds but replacing the behaviour at x2 = 0 by p(x1 , 0; x1 , 1) = p(x1 , 0; x1 , −1) =
1/2 if x1 ≥ 0, and p(x1 , 0; x1 , 1) = 1 for x1 < 0. See the right-hand part of
Figure 5.2 for an illustration.
This time the walk takes a symmetric increment as at (5.33) when x < 0
but a one-sided increment as at (5.32) when x ≥ 0. In this case a result of
Rogozin and Foss [273] shows that the walk is transient.
4
Figure 5.2: Pictorial representation of the two non-homogeneous nearestneighbour random walks on Z2 of Example 5.3.4. Each of these walks is
transient.
5.3.3
Complexes of half-lines
As an aside, we describe an extension of the model of Section 5.3.1 to a
Markov process on a complex of half-lines R+ × S, where S is finite. On
a given half-line, a particle performs a random walk with a heavy-tailed
increment distribution, until the point at which it would exit the half-line,
when it moves (in general, at random) through the origin to another halfline.
We consider (Yn , ξn ), n ∈ Z+ , a discrete-time, time-homogeneous Markov
process with state space R+ × S, where S is a finite non-empty set. The
state space is equipped with the appropriate Borel sets, namely, sets of the
form B × A where B ∈ B(R+ ) is a Borel set in R+ , and A ⊆ S. The Markov
process will be described by:
• an irreducible stochastic matrix P = (p(i, j), i, j ∈ S); and
259
5.3. Oscillating random walk
• a collection (wi , i ∈ S) of Rprobability density functions, so wi : R → R+
is a Borel function with R wi (y)dy = 1.
We view R+ × S as a complex of half-lines R+ × {k}, or branches, connected
at a central origin O := {0} × S; at time n, the coordinate ξn describes
which branch the process is on, and Yn describes the distance along that
branch at which the process sits.
The transition kernel of the process is given for (y, i) ∈ R+ × S by
P [(Yn+1 , ξn+1 ) ∈ B × {j} | (Yn , ξn ) = (y, i)] =
Z
Z
wi (z − y)dz,
wi (−z − y)dz + 1{i = j}
p(i, j)
B
(5.34)
B
for all Borel sets B ⊆ R+ and all j ∈ S.
The dynamics of the process represented by (5.34) can be described
algorithmically as follows. Given (Yn , ξn ) = (y, i) ∈ R+ × S, generate (independently) a spatial increment ϕn+1 from the distribution given by wi and
a random index ηn+1 ∈ S according to the distribution p(i, · ). Then,
• if y + ϕn+1 ≥ 0, set (Yn+1 , ξn+1 ) = (y + ϕn+1 , i); or
• if y + ϕn+1 < 0, set (Yn+1 , ξn+1 ) = (|y + ϕn+1 |, ηn+1 ).
In words, the walk takes a wξn -distributed step. If this step would bring
the walk beyond the origin, it passes through the origin and switches onto
branch ηn+1 (or, if ηn+1 happens to be equal to ξn , it reflects back along the
same branch).
The finite irreducible stochastic matrix P is associated with a (unique)
positive invariant probability distribution (µk , k ∈ S) satisfying
X
µj p(j, k) − µk = 0, for all k ∈ S.
(5.35)
j∈S
For future reference, we state the following.
(Os2) Let P = (p(i, j), i, j ∈ S) be an irreducible stochastic matrix, and
let (µk , k ∈ S) denote the corresponding invariant distribution.
The wk are then described by a collection of positive parameters (αk , k ∈ S).
(Os3) Suppose that, for each k ∈ S, we have an exponent αk ∈ (0, ∞) and
a density function vk ∈ Dαk . Then suppose that, for all y ∈ R, wk is
given by wk (y) = vk (−y) .
260
Chapter 5. Heavy tails
The following recurrence classification, whose proof we omit, is contained
in the main result of [229], and contains Theorem 5.3.1 as a special case
where S = {−1, +1}, and R+ × S is identified with R in the natural way.
Theorem 5.3.5. Suppose that (Os2) and (Os3) hold.
(a) Suppose that maxk∈S αk ≥ 1. Then the process is recurrent.
(b) Suppose that maxk∈S αk < 1.
P
(i) If k∈S µk cot(παk ) < 0, then the process is recurrent.
P
(ii) If k∈S µk cot(παk ) > 0, then the process is transient.
5.3.4
Lyapunov functions
The proofs of the results in Section 5.3.1 are based on the Lyapunov function
approach; our function is roughly of the form x 7→ |x|ν , with two modifications: one is a minor adjustment near x = 0 to control the case ν < 0,
and the other is the introduction of an extra parameter, λ, to give different weights to the different sides of the origin, which is a crucial technical
tool. Precisely, for parameters ν ∈ R and λ ∈ R+ , we consider the function
fν ( · ; λ) : R → R+ given by

ν

if x ≥ 1,
|x|
1+x
fν (x; λ) := λ + (1 − λ) 2
(5.36)
if |x| < 1,


ν
λ|x|
if x ≤ −1.
Then x 7→ fν (x; λ) is continuous. The form given at (5.36) ensures that fν
is bounded as a function of x for ν < 0. Note that for λ > 0 and any x ∈ R,
fν (x; λ) = λfν (−x; 1/λ).
(5.37)
We consider the process fν (Xn ; λ) for appropriately chosen ν and λ. By
(5.29), the one-step mean increment of fν (Xn ; λ) is given by
E [fν (Xn+1 ; λ) − fν (Xn , λ) | Xn = x]
Z ∞
=
(fν (x + y; λ) − fν (x; λ)) wx (y)dy =: Fν (x; λ),
−∞
say. Suppose that (Os1) holds. Let x > 0. Then some manipulation using (5.37) shows that
Z ∞
one
Fν (x; λ) =
(fν (x − y; λ) − fν (x; λ)) vα (y)dy =: Fν,α
(x; λ), and
0
261
5.3. Oscillating random walk
Z
Fν (−x; λ) = λ
0
∞
(fν (x − y; 1/λ) − fν (x; 1/λ)) vβ (y)dy.
In other words, (Os1) implies that
(
one (|x|; λ)
Fν,α
Fν (x; λ) =
one (|x|; 1/λ)
λFν,β
if x > 0
if x < 0.
(5.38)
one (x; λ) in Section 5.3.6 below; in our analysis the following
We will study Fν,α
integrals will appear.
Z 1
(1 − u)ν − 1
iν,α
:=
du;
(5.39)
1
u1+α
0
Z ∞
(u − 1)ν
ν,α
i2 :=
du.
(5.40)
u1+α
1
as ν → 0; this is the purpose of the
and iν,α
We will need to estimate iν,α
2
1
next subsection.
5.3.5
Technical preliminaries
We collect some facts from [1]. The Euler gamma function Γ satisfies the
functional equation zΓ(z) = Γ(z + 1), as well as the following ‘reflection
formula’, equation 6.1.17 from [1, p. 256]:
Γ(z)Γ(1 − z) = −zΓ(−z)Γ(z) = π cosec πz,
(5.41)
valid for z ∈ (0, 1); one consequence of the functional equation is that
Γ(z) ∼ 1/z,
as z ↓ 0.
(5.42)
Moreover, for x > 0, by Taylor’s formula,
Γ(x + h) = Γ(x)(1 + hψ(x) + O(h2 )),
(5.43)
d
as h → 0, where ψ(z) = dγ
log Γ(z) = Γ0 (z)/Γ(z) is the digamma function, and ψ(1) = −γ, where γ ≈ 0.5772 is Euler’s constant. The digamma
function also satisfies a reflection formula, namely equation 6.3.7 from [1, p.
259]:
ψ(1 − z) − ψ(z) = π cot πz.
(5.44)
The beta function is given for a > 0, b > 0 by
Z 1
Γ(a)Γ(b)
.
B(a, b) =
ua−1 (1 − u)b−1 du =
Γ(a + b)
0
(5.45)
262
Chapter 5. Heavy tails
The Gauss hypergeometric function is defined via the series
2 F1 (a, b; c; z)
=
∞
X
Γ(a + n)Γ(b + n)Γ(c)
Γ(a)Γ(b)Γ(c + n)
n=0
zn,
(5.46)
which is convergent for |z| < 1. A consequence of (5.46) is that 2 F1 (a, b; c; z) =
1 + O(z) as z → 0. For c > b > 0 there is the integral representation
2 F1 (a, b; c; z) =
Γ(c)
Γ(b)Γ(c − b)
1
Z
0
ub−1 (1 − u)c−b−1 (1 − zu)−a du.
(5.47)
We will need the following linear transformation formula, equation 15.3.6
from [1, p. 559],
Γ(c)Γ(c − a − b)
2 F1 (a, b; a + b − c + 1; 1 − z)
Γ(c − a)Γ(c − b)
Γ(c)Γ(a + b − c)
+ (1 − z)c−a−b
2 F1 (c − a, c − b; c − a − b + 1; 1 − z).
Γ(a)Γ(b)
(5.48)
2 F1 (a, b; c; z)
=
Lemma 5.3.6. The following hold.
(i) For α ∈ (0, 1) and ν ∈ (−1, α),
iν,α
1 =
1
α
Γ(1 + ν)Γ(1 − α)
Γ(1 + ν − α)
1−
.
(5.49)
(ii) For α > 0 and ν ∈ (−1, α),
iν,α
2 =
Γ(1 + ν)Γ(α − ν)
.
Γ(1 + α)
(5.50)
Proof. For part (i), we compute the integral via
1
Z
−1−α
ν
0
((1 − u) − 1)u
Z
du = lim
t↓0
t
1
((1 − u)ν − 1)u−1−α du.
For the last integral, we get, for α > 0 and t > 0,
Z
t
1
ν
−1−α
((1 − u) − 1)u
Z
du =
t
1
(1 − u)ν u−1−α du +
1 − t−α
.
α
263
5.3. Oscillating random walk
With the substitution v =
1
Z
t
ν −1−α
(1 − u) u
1−u
1−t ,
the last integral becomes
du = (1 − t)
=
1+ν
Z
0
1
v ν (1 − (1 − t)v)−α−1 dv
(1 − t)1+ν
2 F1 (1 + α, 1 + ν; 2 + ν; 1 − t),
1+ν
provided ν > −1, by (5.47). Now, by (5.48),
2 F1 (1
Γ(2 + ν)Γ(−α)
2 F1 (1 + α, 1 + ν; 1 + α; t)
Γ(1 − α + ν)
1+ν
+ t−α
2 F1 (1 − α + ν, 1; 1 + α; t)
α
Γ(2 + ν)Γ(−α) 1 + ν −α
=
+
t + O(t1−α ),
Γ(1 − α + ν)
α
+ α, 1 + ν; 2 + ν; 1 − t) =
as t ↓ 0, provided α ∈ (0, 1). Combining these results we get
Z 1
(1 − t)1+ν Γ(2 + ν)Γ(−α) 1 + ν −α
ν
−1−α
1−α
((1 − u) − 1)u
du =
+
t + O(t
)
1+ν
Γ(1 − α + ν)
α
t
1 − t−α
+
α
Γ(1 + ν)Γ(−α)
1
+ O(t1−α ).
= +
α
Γ(1 − α + ν)
Hence
Z
0
1
1
Γ(1 + ν)Γ(−α)
+
α
Γ(1 − α + ν)
Γ(1 + ν)Γ(1 − α)
1
1−
,
=
α
Γ(1 − α + ν)
((1 − u)ν − 1)u−1−α du =
using (5.41). Thus we obtain (5.49).
For part (ii),
Z ∞
Z 1
Γ(1 + ν)Γ(α − ν)
ν −1−α
(u − 1) u
du =
(1 − v)ν v α−ν−1 dv =
,
Γ(1 + α)
1
0
by (5.45), provided ν > −1 and α − ν > 0. Hence we obtain (5.50).
Lemma 5.3.7.
(i) For α ∈ (0, 1), as ν → 0,
iν,α
1 =
ν
(ψ(1 − α) − ψ(1)) + O(ν 2 ).
α
(5.51)
264
Chapter 5. Heavy tails
(ii) For α > 0, as ν → 0,
1
ν
+ (ψ(1) − ψ(α)) + O(ν 2 ).
α α
iν,α
3 =
(5.52)
Proof. For part (i), we have from (5.49) with (5.43) that
iν,α
1 =
1
1 − (1 + νψ(1) + O(ν 2 ))(1 − νψ(1 − α) + O(ν 2 )) ,
α
which gives (5.51).
For part (ii), we have from (5.50) with (5.43) that
iν,α
3 =
Γ(α)
(1 + νψ(1) + O(ν 2 ))(1 − νψ(α) + O(ν 2 )),
Γ(1 + α)
which gives (5.52).
5.3.6
Increment estimates
one (x; λ) for α ∈ (0, 1).
The following result gives an estimate for Fν,α
Lemma 5.3.8. Suppose that vα ∈ Dα with α ∈ (0, 1). Then for any ν ∈
(−1, α) and λ ∈ R+ , as x → +∞,
one
Fν,α
(x; λ) = cxν−α Rν,α (λ) + o(xν−α ),
(5.53)
where
1
α
Moreover, suppose that for θ ∈ R, λ(θ, ν) > 0 is such that
ν,α
Rν,α (λ) = iν,α
1 + λi2 −
1 − λ(θ, ν)
= θ.
ν→0
ν
(5.55)
ν
Rν,α (λ(θ, ν))) = − r(θ, α) + o(ν),
α
(5.56)
lim
Then, as ν → 0,
(5.54)
where r(θ, α) = θ − π cot πα.
For α ≥ 1 we have the following result.
Lemma 5.3.9. Suppose that vα ∈ Dα with α ≥ 1. Then for any ν ∈ (0, α)
and λ ∈ R+ , there exists ε > 0 such that, as x → ∞,
one
Fν,α
(x; λ) ≤ −εxν−1 + o(xν−1 ).
265
5.3. Oscillating random walk
Before giving the proofs of these two lemmas, we introduce the notation
I1ν,α (x) :=
I2ν,α (x; λ)
Z
x−1
Z0 ∞
:=
I3ν,α (x; λ) :=
((x − y)ν − xν ) vα (y)dy;
(λ(y − x)ν − xν ) vα (y)dy;
x+1
Z x+1 x−1
ν
λ + (1 − λ) 1+x−y
−
x
vα (y)dy.
2
From (5.36), we obtain the decomposition
one
Fν,α
(x; λ) = I1ν,α (x) + I2ν,α (x; λ) + I3ν,α (x; λ).
(5.57)
We estimate the integrals on the right-hand side of (5.57) individually via
a series of lemmas.
Lemma 5.3.10. Suppose that vα ∈ Dα with α > 0.
(i) If α ∈ (0, 1) then, for any ν ∈ (−1, α), as x → ∞,
ν−α
I1ν,α (x) = cxν−α iν,α
).
1 + o(x
(ii) If α = 1 then, for any ν > 0, there exist ε > 0 and x0 ∈ R+ such that
I1ν,α (x) ≤ −εxν−1 log x, for all x ≥ x0 .
(iii) If α > 1 then, for any ν > 0, there exist ε > 0 and x0 ∈ R+ such that
I1ν,α (x) ≤ −εxν−1 , for all x ≥ x0 .
Proof. First suppose that α ∈ (0, 1). Then
I1ν,α (x)
=x
ν−α
Z
1−(1/x)
g(u)c(ux)du,
0
where g(u) = ((1 − u)ν − 1)u−1−α . Note that g(u) ∼ νu−α as u ↓ 0, so
that g(u) is integrable over (0, 1) provided α < 1 and ν > −1. Then for any
ε > 0,
Z
Z 1−(1/x)
1−(1/x)
Z 1
g(u)c(ux)du − c
g(u)du ≤ sup |c(y) − c|
|g(u)|du,
ε
y≥εx
ε
0
266
Chapter 5. Heavy tails
which tends to 0 as x → ∞, since c(y) → c. Moreover,
Z ε
Z ε
|g(u)|du,
|g(u)|c(ux)du ≤ sup c(y)
y≥0
0
0
which tends to 0 as ε ↓ 0. It follows from (5.39) that
Z 1−(1/x)
g(u)c(ux)du = ciν,α
1 + o(1),
0
yielding part (i).
Now suppose that α ≥ 1 and ν > 0. For any ν > 0, there exists δ ∈ (0, 1)
such that (1 − u)ν − 1 ≤ −(ν/2)u for all u ∈ [0, δ]. Moreover, c(y) ≥ c/2 > 0
for all y ≥ y0 sufficiently large. Hence, since the integrand is non-positive,
for x > y0 /δ,
Z x−1
I1ν,α (x) = xν
(1 − xy )ν − 1 c(y)y −1−α dy
0
Z δx
c
ν
≤x
(1 − xy )ν − 1 c(y)y −1−α dy
2 y0
Z δx
cν
y −α dy,
≤ − xν−1
4
y0
which yields parts (ii) and (iii) of the lemma.
Lemma 5.3.11. Suppose that vα ∈ Dα with α > 0. Then for any ν ∈
(−1, α) and any λ ∈ R+ , as x → ∞,
1
ν,α
ν,α
ν−α
λi2 −
+ o(xν−α ).
I2 (x; λ) = cx
α
Proof. The substitution u = y/x shows that
Z ∞
I2ν,α (x; λ) = xν−α
g(u)c(ux)du,
1+(1/x)
where g(u) = (λ(u − 1)ν − 1)u−1−α . Here
Z
Z ∞
∞
Z ∞
g(u)c(ux)du
−
c
g(u)du
≤
sup
|c(y)
−
c|
|g(u)|du,
1+(1/x)
y≥x
1
1+(1/x)
which tends to 0 as x → ∞ since the last integral is finite provided ν ∈
(−1, α). The statement in the lemma follows from (5.40).
267
5.3. Oscillating random walk
Lemma 5.3.12. Suppose that α > 0, ν ∈ R, and λ ∈ R. Then I3ν,α (x; λ) =
o(xν−α ) as x → ∞.
Proof. Note that
|I3ν,α (x; λ)|
Z
≤ sup c(y)
y≥0
x+1
x−1
(|λ| + |1 − λ| + xν ) (x − 1)−1−α dy,
which gives the result.
Now we can complete the proofs of Lemmas 5.3.8 and 5.3.9.
Proof of Lemma 5.3.8. From (5.57) with the expressions for the integrals
given in Lemmas 5.3.10(i), 5.3.11 and 5.3.12, we obtain (5.53).
ν,α
From (5.53) with the estimates for iν,α
1 and i2 given at (5.51) and (5.52)
respectively, we obtain
Rν,α (λ) =
1
λ − 1 − ν (λψ(α) − ψ(1 − α) + (λ − 1)ψ(1)) + O(ν 2 ) ,
α
as ν → 0, uniformly for |λ| bounded away from infinity. In particular, if
λ = λ(θ, ν) = 1 − θν + o(ν) we obtain
Rν,α (λ(θ, ν)) = −
ν
(θ + ψ(α) − ψ(1 − α)) + o(ν),
α
and then (5.56) follows from the digamma reflection formula (5.44).
Proof of Lemma 5.3.9. Recall the decomposition (5.57). Suppose ν ∈ (0, α).
If α = 1, then Lemmas 5.3.10(ii), 5.3.11 and 5.3.12 show that
one
Fν,α
(x) ≤ −εxν−1 log x + O(xν−1 ) ≤ −εxν−1 ,
for all x sufficiently large. If α > 1, then Lemmas 5.3.10(iii), 5.3.11 and 5.3.12
show that
one
Fν,α
(x) ≤ −εxν−1 + O(xν−α ),
as required.
5.3.7
Proof of recurrence classification
In this section we complete the proof of Theorem 5.3.1. First we show the
following non-confinement result.
Lemma 5.3.13. Suppose that (Os1) holds. Then lim supn→∞ |Xn | = ∞,
a.s.
268
Chapter 5. Heavy tails
Proof. We claim that for each x ∈ R+ , there exists εx > 0 such that
P[|Xn+1 | − |Xn | ≥ 1 | Xn = y] ≥ εx , for all y ∈ [−x, x].
(5.58)
Indeed, given x ∈ R+ , we may choose z0 ≥ x sufficiently large so that, for
some ε > 0, vα (z) ≥ εz −1−α and vα (z) ≥ εz −1−β for all z ≥ z0 . Then if
y ∈ [0, x],
Z ∞
P[Xn+1 < −y − 1 | Xn = y] ≥
εz −1−α dz,
2z0 +1
which implies (5.58); a similar argument applies when y ∈ [−x, 0). The
property (5.58) implies the result by an application of Proposition 3.3.4.
Lemma 5.3.14. Suppose that (Os1) holds.
(i) If α + β > 1, then there exist ν > 0, λ ∈ R, and a bounded set A ⊆ R,
such that
Fν (x; λ) < 0, for all x ∈
/ A.
(5.59)
(ii) If α + β < 1, then there exist ν < 0, λ ∈ R, and a bounded set A ⊆ R,
such that
Fν (x; λ) < 0, for all x ∈
/ A.
(5.60)
Proof. First we prove part (i). First suppose that β ≥ 1. If α ≥ 1 as well,
then Lemma 5.3.9 applied twice shows that, for all x sufficiently large,
one
one
(x; 1/λ) < 0,
Fν,α
(x; λ) < 0, and λFν,β
(5.61)
regardless of choice of ν > 0 and λ, giving (5.59). If β ≥ 1 but α ∈ (0, 1), we
choose λ = λ(θ, ν) = 1 − θν satisfying (5.55); since α ∈ (0, 1) we may choose
θ ∈ R so that r(θ, α) > 0. Thus we may choose ν > 0 sufficiently small and
λ = λ(θ, ν) such that Rν,α (λ) < 0, giving the first inequality in (5.61). The
second inequality in (5.61) follows from another application of Lemma 5.3.9.
A similar argument applies if α ≥ 1.
It remains to suppose that α + β > 1 with α, β ∈ (0, 1). We choose
λ = λ(θ, ν) = 1 − θν satisfying (5.55), so that 1/λ = 1 + θν + o(ν) as ν → 0.
We aim to choose θ so that, for ν > 0 sufficiently small, both Rν,α (λ) < 0
and Rν,β (1/λ) < 0. Thus we require to choose θ ∈ R so that r(θ, α) > 0 and
r(−θ, β) > 0, i.e.,
π cot (πα) − θ < 0, and π cot (πβ) + θ < 0.
269
5.3. Oscillating random walk
The desired θ exists provided cot πα + cot πβ < 0, which using the identity
cot πα + cot πβ =
sin π(α + β)
,
sin πα · sin πβ
is equivalent to α + β > 1.
Next we prove part (ii). Suppose α, β ∈ (0, 1). We again choose λ =
λ(θ, ν) = 1 − θν satisfying (5.55), so that 1/λ = 1 + θν + o(ν) as ν → 0. We
aim to choose θ so that, for ν < 0 sufficiently close to 0, both Rν,α (λ) < 0
and Rν,β (1/λ) < 0. Thus we require to choose θ ∈ R so that r(θ, α) < 0 and
r(−θ, β) < 0, i.e.,
π cot (πα) − θ > 0, and π cot (πβ) + θ > 0.
The desired θ exists provided α + β < 1.
To establish recurrence, we use the following analogue of Theorem 3.5.8.
Lemma 5.3.15. Let Xn be an Fn -adapted process taking values in R. Let
f : ¶ → R+ be such that limx→+∞ f (x) = limx→−∞ = ∞, and E f (X0 ) < ∞.
Suppose that there exist x0 ∈ R+ and C < ∞ for which, for all n ≥ 0,
E[f (Xn+1 ) − f (Xn ) | Fn ] ≤ 0, on {|Xn | > x0 }, a.s.;
E[f (Xn+1 ) − f (Xn ) | Fn ] ≤ C, on {|Xn | ≤ x0 }, a.s.
Then
h
i
P lim sup |Xn | < ∞ ∪ lim inf |Xn | ≤ x0 = 1.
n→∞
n→∞
Proof. The proof is analogous to the proof of Theorem 3.5.8, using the nonnegative supermartingale Ym = f (Xm∧λ ) where λ = min{m ≥ n : |Xm | ≤
x0 }. Writing Y∞ = limm→∞ Ym , this means that
lim sup f (Xm ) ≤ Y∞ , on {λ = ∞}.
m→∞
Setting ζ+ = sup{x ≥ 0 : f (x) ≤ Y∞ } and ζ− = sup{x ≥ 0 : f (−x) ≤ Y∞ },
it follows that lim supm→∞ |Xm | ≤ ζ+ ∨ ζ− < ∞ on {λ = ∞}. Hence
h
i
P lim sup |Xn | < ∞ ∪ inf |Xm | ≤ x0 = 1.
n→∞
m≥n
Since n ∈ Z+ was arbitrary, it follows that
h
i
\
P lim sup |Xn | < ∞ ∪
inf |Xm | ≤ x0 = 1,
n→∞
which gives the result.
n≥0
m≥n
270
Chapter 5. Heavy tails
To establish transience, we use the following analogue of Lemma 3.5.7.
Lemma 5.3.16. Let Xn be an Fn -adapted process taking values in R. Let
f : R → R+ be such that supx f (x) < ∞ and limx→+∞ f (x) = limx→−∞ = 0.
Suppose that there exists x1 ∈ R+ for which inf |y|≤x1 f (y) > 0 and, for all
n ≥ 0,
E[f (Xn+1 ) − f (Xn ) | Fn ] ≤ 0, on {|Xn | > x1 }.
Then for any ε > 0 there exists x ∈ (x1 , ∞) for which, for all n ≥ 0,
i
h
P inf |Xm | ≥ x1 Fn ≥ 1 − ε, on {|Xn | > x}.
m≥n
Proof. The proof is analogous to the proof of Lemma 3.5.7, using the nonnegative supermartingale Ym = f (Xm∧λ ) where λ = min{m ≥ n : |Xm | ≤
x1 }.
Now we can complete the proof of Theorem 5.3.1.
Proof of Theorem 5.3.1. First suppose that α + β > 1. Then Lemma 5.3.14
shows that there exist ν > 0, λ ∈ R, and x0 ∈ R+ , such that
E[fν (Xn+1 ; λ) − fν (Xn ; λ) | Xn = x] ≤ 0, for |x| > x0 .
Thus we may apply Lemma 5.3.15, which together with Lemma 5.3.13 shows
that lim inf n→∞ |Xn | ≤ x0 , a.s. Thus there exists an interval I ⊆ [0, x0 + 1]
such that |Xn | ∈ I i.o.; let τ0 := 0 and for k ∈ N define τk = min{n >
τk−1 + 1 : |Xn | ∈ I}. Fix ε ∈ (0, 1) and z0 ∈ R+ such that vα (z) > εz −1−α
and vβ (z) > εz −1−β for all z ≥ z0 . Then
P[|Xτk +2 | < ε | Xτk ] ≥ P[|Xτk +2 | < ε | Xτk +1 ∈ − sgn(Xτk )[z0 + 1, z0 + 2]]
× P[Xτk +1 ∈ − sgn(Xτk )[z0 + 1, z0 + 2] | Xτk ]
≥ 2ε(z0 + 3)−1−α∨β (x0 + z0 + 3)−1−α∨β ,
uniformly in k. Thus an application of Lévy’s extension of the Borel–Cantelli
lemma (Theorem 2.3.19) shows that |Xn | < ε, i.o. Since ε ∈ (0, 1) was
arbitrary, we obtain lim inf n→∞ |Xn | = 0, a.s.
Next suppose that α + β < 1. Then Lemma 5.3.14 shows that there exist
ν < 0, λ ∈ R, and x1 ∈ R+ , such that
E[fν (Xn+1 ; λ) − fν (Xn ; λ) | Xn = x] ≤ 0, for |x| > x1 .
271
Bibliographical notes
Thus we may apply Lemma 5.3.16, which shows that for any ε > 0 there
exists x ∈ (x1 , ∞) for which, for all n ≥ 0,
i
h
P inf |Xm | ≥ x1 Fn ≥ 1 − ε, on {|Xn | > x}.
m≥n
As usual, let σx = min{n ≥ 0 : |Xn | ≥ x}. Then, on {σx < ∞},
i
h
P inf |Xm | > x1 Fσx ≥ 1 − ε, a.s.
m≥σx
But on {σx < ∞} ∩ {inf m≥σx |Xm | > x1 } we have lim inf m→∞ |Xm | ≥ x1 , so
h
h
i
i
P lim inf |Xm | ≥ x1 ≥ E P inf |Xm | > x1 Fσx 1{σx < ∞}
m→∞
m≥σx
≥ (1 − ε) P[σx < ∞] = (1 − ε),
by Lemma 5.3.13. Since ε > 0 was arbitrary, we get lim inf m→∞ |Xm | ≥ x1 ,
a.s., and since x1 was arbitrary we get limm→∞ |Xm | = ∞, a.s.
Bibliographical notes
Section 5.2
There is an extensive and rich theory of sums of independent, identically distributed (i.i.d.) random variables (classical ‘random walks’): see for instance
the books of Kallenberg [150, Chapter 9], Loève [201, §26.2], or Stout [295,
§3.2]. When the summands are integrable, the (first-order) asymptotic behaviour is governed by the mean. Completely different phenomena occur
when the mean does not exist: see for example the book of Foss et al. [105].
To compare with the Markov chain setting of Section 5.2, it is helpful
Pnto keep in mind the classical independent-increments case, where Sn :=
m=1 ζm for a sequence of independent (often, i.i.d.) R-valued random
variables ζ1 , ζ2 , . . .. Thus we give a brief summary of some known results
in that setting. Many of the results that we discuss for random walks have
analogues for suitable Lévy processes: see e.g. the book of Sato [278], particularly Sections 37 and 48.
A classical result of Kesten [163, Corollary 3] states that if ζ1 , ζ2 , . . . are
i.i.d. random variables with E |ζ1 | = ∞, then as n → ∞, n−1 Sn either: (i)
tends to +∞ a.s.; (ii) tends to −∞ a.s.; or (iii) satisfies
− ∞ = lim inf n−1 Sn < lim sup n−1 Sn = +∞, a.s.
n→∞
n→∞
(5.62)
272
Chapter 5. Heavy tails
Erickson [89] gives criteria for classifying such behaviour. Note that Kesten’s
result implies the non-confinement result (5.1) in the i.i.d. case. Other classical results deal with the growth rate of the upper envelope of Sn , i.e.,
determining sequences an for which |Sn | ≥ an infinitely often (or not), or
Sn ≥ an infinitely often; here we mention the work of Feller [97], as well
as results related to the Marcinkiewicz–Zygmund strong law of large numbers (see e.g. [166, Theorem 1]). The lower envelope behaviour, i.e., when
|Sn | ≥ an all but finitely often, is considered by Griffin [121] (particularly
Theorem 3.5); see also Pru90 [261].
Note that (5.62) can hold and Sn be transient (with respect to bounded
sets); Loève [201, §26.2] gives the example of a symmetric stable random
walk without a mean. The general criterion for deciding between transience and recurrence is due to Chung and Fuchs (see e.g. [150, Theorem 9.4]
or [201, §26.2]), and is rather subtle: Shepp showed [284] that there exist
distributions for ζ1 with arbitrarily heavy tails but for which Sn is still recurrent. By assuming additional regularity for the distribution of ζ1 , one can
obtain more tractable criteria for recurrence; Shepp gives a criterion when
the distribution of ζ1 is symmetric [283, Theorem 5].
The results of Section 5.2 are based on [133], which considered the more
general setting where Xn is adapted to a filtration Fn , and not necessarily
Markov (i.e., the analogue of the general Lamperti setting of Chapter 3).
Theorem 5.2.2 is contained in Theorem 2.2 of [133]. In Theorem 5.2.2,
γ
γ−1 ∆+ 1{∆+ ≤ x} for any
condition (H2) is natural. For γ ≤ 1, (∆+
n
n
n) ≥ x
+
γ
x > 0, so (H2) implies that E[(∆n ) | Xn = x] = ∞ for any γ > α. A
counterexample due to K.L. Chung (see the Mathematical Reviews entry
for [74]; also Baum [16]) shows that (H2) cannot be replaced by a condition
on the moments of the increments, even in the case of a sum of i.i.d. random
variables. Chung’s example has, for α ∈ (0, 1) and β > α, E[(ζ1− )β ] < ∞ and
E[(ζ1+ )α ] = ∞, but E[ζ1+ 1{ζ1+ ≤ x}] = o(x1−α ) along a subsequence, so (H2)
does not hold. For Xn = Sn as in Chung’s example, lim inf n→∞ Xn = −∞,
a.s.
Theorems 5.2.4 and 5.2.5 are contained in Theorems 2.4 and 2.6 of [133],
respectively. In the case of a sum of independent random variables, Theorem 5.2.4 is slightly weaker than optimal. Suppose that ζ1 , ζ2 , . . . are independent, and that for some θ ∈ (0, 1) and ϕ ∈ R,
sup lim sup(xθ (log x)−ϕ P[|ζk | ≥ x]) < ∞.
k∈N x→∞
Then, with Sn =
Pn
k=1 ζk ,
for any ε > 0, a.s., for all but finitely many
273
Bibliographical notes
n ≥ 0,
|Sn | ≤ n1/θ (log n)
ϕ+1
+ε
θ
.
(5.63)
The bound (5.63) belongs to a family of classical results with a long history;
the case ϕ = 0 is due to Lévy and Marcinkiewicz (quoted by Feller [97,
p. 257]), and the general case of (5.63) follows for example from a result of
Loève [201, p. 253]. Under the additional condition that the summands are
identically distributed, sharp results are given by Feller [97, Theorem 2]; for
a recent reference, see [185]. Related results in the i.i.d. case are also given
by Chow and Zhang [42] (see also [166, Theorem 2]).
Theorems 5.2.2 and 5.2.5 together can be viewed as an analogue of Erickson’s [89] result in the case of a sum of i.i.d. random variables; in the
i.i.d. case the conclusion of Theorem 5.2.2 follows from [89, Corollary 1].
The results of [89] show that the conditions in Theorem 5.2.2 are close to
optimal.
Note that (H1) implies that, a.s.,
Z ∞
Z ∞
α
+
1/α
E[(∆+
)
|
X
=
x]
=
P[∆
>
y
|
X
=
x]dy
≥
c
y −1 dy = ∞.
n
n
n
n
0
x0
Conditions (H2) and (H1) are closely related, but neither implies the other.
However, if one replaces the inequalities by equalities, the former implies
the latter. In the case where Xn = Sn is a sum of i.i.d. random variables,
a weaker version of Theorem 5.2.5 was obtained by Derman and Robbins
[74] and stated in a stronger form by Stout [295, Theorem 3.2.6]; although
Stout’s statement is still weaker than our Theorem 5.2.5, his proof gives
essentially the same result (in the i.i.d. case). Also relevant in the i.i.d. case
is a result of Chow and Zhang [42, Theorem 1]. Chung’s counterexample
(see above) shows that the condition (H1) cannot be replaced by a moments
condition, for instance.
The results on first passage times, Theorems 5.2.9 and 5.2.10 are contained in Theorems 2.9 and 2.10 of [133], respectively. The conditions in
Theorems 5.2.9 and 5.2.10 are not far from optimal; in the i.i.d. case, sharp
results on the existence or non-existence of moments for ρx are given by
Kesten and Maller [165, Theorem 2.1]; see [165] for references to earlier
work. Lemma 5.2.12 is essentially a large deviations result of the same kind
as (but more general than) those obtained in [137] for the case Xn = Sn , a
sum of i.i.d. nonnegative random variables; indeed, the results in [137] show
that Lemma 5.2.12 is close to best possible.
The results on last exit times, Theorems 5.2.13 and 5.2.14, are contained
in Theorems 2.11 and 2.12 of [133], respectively. Again, in the i.i.d. case
sharp results are given by Kesten and Maller [165, Theorem 2.1].
274
Chapter 5. Heavy tails
Section 5.3
The material in this section is based on [229]. The oscillating random walk
was studied in the case of a discrete state-space by Kemperman [157], to
whom the model was suggested in 1960 by Anatole Joffe and Peter Ney
(see [157, p. 29]). The model was studied subsequently by Rogozin and
Foss [273] and in a series of papers by Sandrić [276, 277]. Kemperman [157]
and Rogozin and Foss [273] obtain their results using some sophisticated
classical methods for the analysis of random walks, primarily Wiener–Hopf
theory together with an appeal to various deep results from renewal theory. The Lyapunov function approach presented here follows [229], and has
similarities to the approach of Sandrić [].
Theorem 5.3.1 on the one-sided oscillating random walk was obtained in
the case of a discrete state-space by Kemperman [157, p. 21].
The first two-dimensional random walk of Example 5.3.4 walk was studied by Campanino and Petritis [34], who showed that it is transient using
completely different methods to those presented here.
Chapter 6
Further applications
6.1
6.1.1
Random walk in random environment
Introduction
The first two sections of this chapter give some applications to processes
in random environments. The law of a time-homogeneous Markov chain is
determined by its collection of one-step transition probabilities, which describe how the process moves around the state space. Viewing the process
as describing the dynamics of a particle, the collection of transition probabilities can be seen as describing the environment in which the particle
moves. Motivated by dynamics in disordered systems, the environment can
itself be random, i.e., the transition probabilities are chosen according to
some probabilistic rule, before the process starts. The prototypical model
of this type is the random walk in random environment, which we discuss in
this section.
Given an infinite sequence ω = (p0 , p1 , p2 , . . .) such that px ∈ (0, 1) for
all x ∈ Z+ , we consider (Xn , n ≥ 0) a nearest-neighbour random walk on
Z+ defined as follows. Set X0 = 0, and for x ≥ 1,
Pω [Xn+1 = x − 1 | Xn = x] = px ,
Pω [Xn+1 = x + 1 | Xn = x] = 1 − px =: qx ,
(6.1)
and Pω [Xn+1 = 0 | Xn = 0] = p0 , Pω [Xn+1 = 1 | Xn = 0] = 1 − p0 =:
q0 . The assumption that each px be in (0, 1), with the given form for the
reflection at the origin, ensures that Xn is an irreducible, aperiodic Markov
chain on Z+ under the quenched law Pω .
The sequence of jump probabilities ω specifies the environment for the
random walk. We take ω itself to be random. Specifically, p0 , p1 , . . . will
275
276
Chapter 6. Further applications
be a sequence of i.i.d. (0, 1)-valued random variables on a probability space
(Ω, F, P). We use E to denote expectation with respect to the (environment)
probability measure P, and Eω to denote expectation with respect to the
(quenched) probability measure Pω .
In Section 6.2 we consider random strings in random environment. The
random strings model includes the random walk in random environment
model given here as a special case, and the proofs in Sections 6.1 and 6.2
have several common strands. As we shall see, studying recurrence of the
random walk requires analysis of products of independent random variables
(compare Section 2.2), while in the case of strings one must study products
of random matrices. Via a simple transformation, one can study products
of random variables by instead studying sums, and so a wealth of classical
theory is available; for matrices, we must appeal to some more sophisticated
theory. For these reasons, it is recommended to read Section 6.1 before
Section 6.2.
6.1.2
Recurrence classification
For any realization of the environment ω, the irreducible Markov chain Xn
is either recurrent or transient under Pω . The following recurrence classification, which is essentially due to Solomon [291], shows that the crucial
quantity is ζx := log(px /qx ). In this and subsequent results, statements are
made that hold true ‘for P-a.e. ω’, that is, for almost every environment.
Theorem 6.1.1. Suppose that E |ζ0 | < ∞.
(i) If E ζ0 < 0, then Xn is transient for P-a.e. ω.
(ii) If E ζ0 = 0, then Xn is null recurrent for P-a.e. ω.
(iii) If E ζ0 > 0, then Xn is positive recurrent for P-a.e. ω.
For fixed ω, Xn is the nearest-neighbour random walk of Section 2.2.
Similarly to that section, we use the notation


x−1
x−1 
X
Y py
dx := dx (ω) :=
(6.2)
= exp
ζy .


qy
y=0
y=0
Proof of Theorem 6.1.1.
P∞ For fixed ω, Theorem 2.2.5 shows that Xn is recurrent if and only if P
of large
x=1 dx = ∞. If E ζ0 < 0, then the strong
Plaw
∞
numbers shows that x−1
ζ
→
−∞
as
x
→
∞,
and
hence
d
y=0 y
x=1 x < ∞
for P-a.e. ω, proving part (i).
277
6.1. Random walk in random environment
For part
Theorem 2.2.5 shows that Xn is positive-recurrent if and
P∞ (iii),
−1 < ∞. If E ζ > 0, then the strong law of large numd
only if
0
x=1 x P
P∞ −1
x−1
=
ζ
→
+∞
as x → ∞, and hence
bers shows that
x=1 dx
y=0 y
Px−1
P∞
y=0 ζy } < ∞ for P-a.e. ω, proving part (iii).
x=1 exp{−
Finally, for part (ii), suppose that E ζ0 = 0. Then either P[ζ0 = 0] = 1
(the degenerate case of symmetric simple random walk with reflection at 0)
and dx = d−1
x = 1 for all x, or else P[ζ0 = 0] < 1 and then
lim sup
x→∞
x−1
X
y=0
ζy = lim sup
x→∞
x−1
X
(−ζy ) = +∞,
y=0
for P-a.e. ω (see e.g. [150, p. 167]). In particular, in either case we have
dx ≥ 1 for infinitely many x for P-a.e. ω, and we obtain recurrence, and
d−1
≥ 1 for infinitely many x for P-a.e. ω, so the walk is not positivex
recurrent. This proves part (ii).
6.1.3
Almost-sure upper bounds
The null-recurrent case in which E[ζ02 ] ∈ (0, ∞) and E ζ0 = 0 is known
as Sinai’s regime; it is this case that we study in detail for the rest of
this section. First we give an almost-sure upper bound which demonstrates
the very slow movement of the random walk. See Figure 6.1 below for a
simulation.
Theorem 6.1.2. Suppose that E[ζ02 ] ∈ (0, ∞) and E ζ0 = 0. Then for any
ε > 0, for P-a.e. ω, Pω -a.s., for all but finitely many n ≥ 0,
Xn ≤ (log n)2 (log log n)2+ε .
We prove this result using the Lyapunov function t(x) as defined at (2.8),
which for the present purposes is most usefully expressed as


y
y
x−1 X
 X

X
−1
t(x) := t(x, ω) :=
qy−z exp
ζw .
(6.3)


y=0 z=0
w=y−z+1
We will use the following lower bound on t(x).
Lemma 6.1.3. Let t be as defined at (6.3). For any ω and any x ∈ Z+ ,
(
)
y−1
X
t(x) ≥ exp max
ζz .
(6.4)
0≤y≤x
z=0
278
Chapter 6. Further applications
−1
Proof. Since qy−z
= 1 + (py−z /qy−z ), we have from (6.3) that
t(x) =
y
x−1 X
X
y=0 z=0
exp


y
X

ζw
w=y−z+1



+
y
x−1 X
X
exp
w=y−z
y
X
)
ζw
.
(6.5)
w=y−z
y=0 z=0
For the lower bound, (6.5) shows that
)
( y
X
ζw ≥
t(x) ≥ max max exp
0≤y≤x−1 0≤z≤y
(
(
max exp
0≤y≤x−1
y
X
)
ζw
,
w=0
which yields the result in (6.4).
The next result, which is a consequence of Theorem 2.8.1, shows how to
deduce from lower bounds for t(x) an upper bound for Xn .
Lemma 6.1.4. Let w : Z+ → R+ be increasing. Suppose that for P-a.e. ω,
t(x) ≥ w(x) for all but finitely many x ∈ Z+ . Then for any ε > 0, for
P-a.e. ω, Pω -a.s., for all but finitely many n ≥ 0,
Xn ≤ w−1 (n1+ε ).
Proof. Let ε > 0. For a fixed ω, Lemma 2.2.4 shows that we may apply
Theorem 2.8.1 with f = t (and B = 1) to obtain, Pω -a.s., for all but finitely
many n ≥ 0, Xn ≤ t−1 ((2n)1+ε ).
Under the assumptions of the lemma, for P-a.e. ω there exists xω ∈ Z+
such that t(x) ≥ w(x) for all x ≥ xω , so that t−1 (y) ≤ w−1 (y) for all y ≥ yω ,
where yω = t(xω ). For such ω, Pω -a.s.,
Xn ≤ t−1 ((2n)1+ε ) ≤ w−1 ((2n)1+ε ),
for all n sufficiently large, and the result follows.
The final ingredient is to obtain lower bounds for the right-hand side
of (6.4). We will use the following result, due to Csáki [61], which extended
a result of Hirsch [128].
Lemma 6.1.5. Suppose that ζ0 , ζ1 , . . . are i.i.d. with E[ζ02 ] ∈ (0, ∞) and
E ζ0 = 0. Then, for any ε > 0, P-a.s., for all but finitely many n ≥ 0,
max
0≤i≤n
i
X
j=0
ζj ≥ n1/2 (log n)−1−ε .
279
6.1. Random walk in random environment
Now we can complete the proof of Theorem 6.1.2.
Proof of Theorem 6.1.2. Let ε > 0. We have from (6.4) and Lemma 6.1.5
that for P-a.e. ω, for all but finitely many x ∈ Z+ ,
n
o
t(x) ≥ exp x1/2 (log x)−1−ε .
Thus we may apply Lemma 6.1.4 with
n
o
w(x) = exp x1/2 (log x)−1−ε , so that w−1 (y) ≤ (log y)2 (log log y)2+3ε ,
for y sufficiently large, to obtain that for P-a.e. ω, Pω -a.s.,
Xn ≤ w−1 (n1+ε ) ≤ (log n)2 (log log n)2+4ε ,
for all but finitely many n ≥ 0, which yields the result.
6.1.4
Almost-sure lower bounds
Next we give an almost-sure lower bound that is attained infinitely often.
Theorem 6.1.6. Suppose that E[ζ02 ] = σ 2 ∈ (0, ∞) and E ζ0 = 0. Then for
any ε > 0, for P-a.e. ω, Pω -a.s., for infinitely many n ≥ 0,
Xn ≥ (1 − ε)
2
(log n)2 log log log n.
π2σ2
The following upper bound on t(x) is a companion to Lemma 6.1.3
Lemma 6.1.7. Let t be as defined at (6.3). For any ω and any x ∈ Z+ ,
y
)
(
X t(x) ≤ 2x2 exp 2 max ζz .
(6.6)
0≤y≤x−1 z=0
Proof. For the first term on the right-hand side of (6.5) we have




y
y
y
x−1 X
x−1
 X
 X
 X

X
exp
ζw ≤
(y + 1) max exp
ζw




0≤z≤y
y=0 z=0
w=y−z+1
y=0
w=y−z+1


y


X
2
≤ x exp
max max
ζw .
0≤y≤x−1 0≤z≤y

w=y−z+1
280
Chapter 6. Further applications
Here we have that
max
y
X
max
0≤y≤x−1 0≤z≤y
ζw =
w=y−z+1
≤
y
X
max
0≤y≤x−1
max
0≤y≤x−1
ζw + max
w=0
y
X
0≤z≤y
ζw +
w=0
max
y−z
X
!
(−ζw )
w=0
y
X
0≤y≤x−1
(−ζw ).
w=0
Hence
y
x−1 X
X
y=0 z=0
exp


y
X

w=y−z+1
ζw



(
2
≤ x exp
max
0≤y≤x−1
y
X
z=0
ζz +
max
0≤y≤x−1
y
X
)
(−ζz ) .
z=0
A similar calculation yields the same upper bound for the second term on
the right-hand side of (6.5), and (6.6) follows.
The key to the proof of Theorem 6.1.6 is the following technical result.
Lemma 6.1.8. Suppose that there are non-negative, increasing functions u
and v such that
(i) for P-a.e. ω, t(x) ≤ u(x) for all but finitely many x ∈ N;
(ii) for P-a.e. ω, t(x) ≤ v(x) for infinitely many x ∈ N.
Suppose also that
∞
X
u(x)
< ∞, and
v(x3/2 )
x=1
lim v(x3/4 )/v(x) = 0.
x→∞
(6.7)
Then for any ε > 0, for P-a.e. ω, Pω -a.s., for infinitely many n ≥ 0,
Xn ≥ v −1 ((1 − ε)n).
Proof. Recall that τx = min{n ≥ 0 : Xn = x}, and, from Lemma 2.2.3, that
Eω τx = t(x) given that X0 = 0. First we obtain a rough upper bound on
τx (equation (6.8) below). Condition (i) and Markov’s inequality yield, for
P-a.e. ω,
t(x)
u(x)
Pω [τx > v(x3/2 )] ≤
≤
,
3/2
v(x )
v(x3/2 )
for all x ≥ x0 , where x0 := x0 (ω) < ∞. Hence, for P-a.e. ω,
X
x≥1
Pω [τx > v(x3/2 )] ≤ x0 +
X u(x)
< ∞,
3/2 )
v(x
x≥1
281
6.1. Random walk in random environment
by the first condition in (6.7). By the Borel–Cantelli lemma, for P-a.e. ω,
τx ≤ v(x3/2 ),
(6.8)
for all x ≥ N where N := N (ω) (a random variable for each ω) has
Pω [N (ω) < ∞] = 1 for P-a.e. ω.
Now we use (6.8) and condition (ii) to show that τx is in fact much
smaller than the bound in (6.8) for infinitely many x. Let ε > 0. Condition
(ii) implies that for P-a.e. ω there exist xi := xi (ω), i ∈ N, such that
xi+1 > x2i for all i and t(xi ) ≤ v(xi ) for all i (the xi are a subsequence of
that specified by (ii) chosen so as to have very large spacings). By Markov’s
inequality and the fact that Eω τxi+1 = t(xi+1 ) ≤ v(xi+1 ),
Pω [τxi+1 − τxi > (1 + ε)v(xi+1 )] ≤ Pω [τxi+1 > (1 + ε)v(xi+1 )] ≤ (1 + ε)−1 .
It follows that, for all i ∈ N,
Pω [τxi+1 − τxi ≤ (1 + ε)v(xi+1 )] ≥ ε0 ,
(6.9)
where ε0 > 0 depends only on ε. So by (6.9), for P-a.e. ω, for any ε > 0,
X
Pω [τxi+1 − τxi ≤ (1 + ε)v(xi+1 )] = ∞.
(6.10)
i∈N
Under Pω , the random variables τxi+1 − τxi , i ∈ N, are independent, by the
strong Markov property. Hence (6.10) and the Borel–Cantelli lemma imply
that, for P-a.e. ω, Pω -a.s., for infinitely many i,
τxi+1 − τxi ≤ (1 + ε)v(xi+1 ).
With (6.8) this implies that, for P-a.e. ω, Pω -a.s., for infinitely many i,
3/2
3/4
τxi+1 ≤ (1 + ε)v(xi+1 ) + v(xi ) ≤ (1 + ε)v(xi+1 ) + v(xi+1 ),
1/2
since xi < xi+1 and v is increasing. Hence by the second condition in (6.7),
we obtain that, for any ε > 0, for P-a.e. ω, Pω -a.s., for infinitely many x,
τx ≤ (1 + 2ε)v(x) = (1 + 2ε)v(Xτx ).
Since τx 6= τy for any x 6= y, it follows that, for any ε > 0, for P-a.e. ω,
Pω -a.s., n ≤ (1 + 2ε)v(Xn ) for infinitely many n, which gives the result.
We will use the so-called ‘other’ law of the iterated logarithm due to
Chung [44] (under a 3rd moments condition) and Jain and Pruitt [141]:
282
Chapter 6. Further applications
Lemma 6.1.9. Suppose that ζ0 , ζ1 , . . . are i.i.d. with E[ζ02 ] = σ 2 ∈ (0, ∞)
and E ζ0 = 0. Then, P-a.s.,
i
X πσ
−1/2
1/2
lim inf n
(log log n)
max ζj = √ .
n→∞
0≤i≤n
8
j=0
Proof of Theorem 6.1.6. For ε ∈ (0, 1/4) set
n
o
u(x) = 2x2 exp 2x(1/2)+ε , and
n
o
v(x) = 2x2 exp (1 + ε)2−1/2 πσx1/2 (log log x)−1/2 .
Consider the upper bound for t(x) given in (6.6). Then for P-a.e. ω, t(x) ≤
u(x) for all but finitely many x, as follows easily from the law of the iterated
logarithm, or from Theorem 2.8.1. Moreover, Lemma 6.1.9 shows that, for
P-a.e. ω, t(x) ≤ v(x) for infinitely many x.
It is straightforward to verify that u and v also satisfy (6.7). Thus we
may apply Lemma 6.1.8 with this choice of u and v to obtain the result.
6.1.5
Perturbation of Sinai’s regime
We finish this section by studying a non-i.i.d. environment. Again suppose
that Xn has quenched transition probabilities as defined at (6.1), but now
suppose that there exist constants x0 ∈ N, a ∈ R, and β > 0 such that
px ∈ (0, 1) for all x ≥ 0, and
px = ξx + ax−β , and qx = 1 − ξx − ax−β , for all x ≥ x0 ,
(6.11)
where ξ0 , ξ1 , . . . are i.i.d. random variables on probability space (Ω, F, P).
We assume that there exists δ > 0 for which
P[δ ≤ ξ0 ≤ 1 − δ] = 1;
(6.12)
this is a uniform ellipticity condition. In this section we define
ξx
ζx := log
.
1 − ξx
We assume that E[ζ02 ] > 0 and E ζ0 = 0. In the case a = 0, this model
reduces to the random walk in i.i.d. random environment in Sinai’s regime,
as considered earlier in this section, and in that case the present definition
of ζx reduces to the usage in Section 6.1.2. For a 6= 0, this model represents
6.1. Random walk in random environment
283
a perturbation of Sinai’s regime, in which the px are independent but no
longer i.i.d.
This perturbation of the homogeneous (i.i.d.) environment is motivated
by analogy with the problems studied in Chapters 3 and 4 in which the homogeneous (zero-drift) regime is perturbed by an asymptotically zero drift. Our
first result for this model is the following recurrence classification; whereas
in the context of asymptotically zero-drift random walks, a perturbation of
order x−1 is required to effect a phase transition, the random environment is
seen to be more stable to perturbations, with x−1/2 being the critical order.
Theorem 6.1.10. Suppose that for x0 ∈ N, a ∈ R, δ > 0 and β > 0 the
conditions (6.11) and (6.12) hold, and that E[ζ02 ] > 0 and E ζ0 = 0.
(i) If a < 0 and β ∈ (0, 1/2), then for P-a.e. ω, Xn is transient.
(ii) If a > 0 and β ∈ (0, 1/2), then for P-a.e. ω, Xn is positive-recurrent.
(iii) If β ≥ 1/2, then for P-a.e. ω, Xn is null-recurrent.
We also present almost-sure bounds giving the rate of escape in the
transient case.
Theorem 6.1.11. Suppose that for x0 ∈ N, a < 0, δ > 0 and β ∈ (0, 1/2)
the conditions (6.11) and (6.12) hold, and that E[ζ02 ] > 0 and E ζ0 = 0.
Then for any ε > 0, for P-a.e. ω, Pω -a.s., for all but finitely many n ≥ 0,
(log log n)−(1/β)−ε ≤
Xn
≤ (log log n)(2/β)+ε .
1/β
(log n)
See Figure 6.1 for a simulation.
The rest of this section is devoted to the proofs of these two theorems.
First we use a similar idea to the proof of Theorem 6.1.1 to give the proof
of Theorem 6.1.10.
Proof of Theorem 6.1.10. From Taylor’s theorem for log(1+x), we have that
log py = log ξy + log(1 + ay −β ξy−1 )
= log ξy + ay −β ξy−1 + O(y −2β ), as y → ∞,
where we have used (6.12) to obtain the non-random error term. Similarly,
log qy = log(1 − ξy ) − ay −β (1 − ξy )−1 + O(y −2β ).
284
Chapter 6. Further applications
Figure 6.1: Simulations of 105 steps of random walk in random environment
in Sinai’s regime (left) and the perturbation with a < 0 and β = 1/5 (right).
Also shown are curves proportional to (log n)2 (left) and (log n)5 (right).
Hence, as y → ∞,
log(py /qy ) = ζy + ay −β ξy−1 (1 − ξy )−1 + O(y −2β ).
(6.13)
It follows from (6.13) that, with the definition of dx = dx (ω) from (6.2),
X
x−1
−β −1
−1
−2β
ζy + ay ξy (1 − ξy ) + O(y
) .
dx = exp
(6.14)
y=1
For fixed ω, Theorem 2.2.5 shows that Xn is recurrent if and only if
∞. Suppose that a < 0 and β ∈ (0, 1/2). Then, from (6.14),
log dx ≤
x−1
X
y=1
P∞
x=1 dx
=
ζy − Cx1−β + O(x1−2β ),
where C = C(δ, β) is a positive constant that does not depend on
Here,
Pω.
x−1
by the law of the iterated logarithm (or Theorem 2.8.1), P-a.s., y=1 ζy ≤
x1/2 log x for all but finitely many x. Thus, since 1 − β > 1/2, we have
log dx → −∞, P-a.s., and hence Xn is transient, proving part (i).
For part
Theorem 2.2.5 shows that Xn is positive recurrent if and
P∞ (ii),
−1
only if
< ∞. Suppose that a > 0 and β ∈ (0, 1/2). Then,
x=1 dx
6.1. Random walk in random environment
285
from (6.14),
log d−1
x
≤
x−1
X
y=1
(−ζy ) − Cx1−β + O(x1−2β ),
where C = C(δ, β) is a positive constant that does not depend on ω. Hence,
similarly to part (i), log d−1
x → −∞, P-a.s., and Xn is positive recurrent,
proving part (ii).
Finally, for part (iii), suppose that β ≥ 1/2. Then, from (6.14),
log dx ≥
x−1
X
y=1
ζy − Cx1/2 + O(1),
and, by the law of the iterated logarithm,
there is a constant c > 0 depending
P
1/2 (log log x)1/2 for infinitely
ζ
only on E[ζ02 ] such that, P-a.s., x−1
y=1 y ≥ cx
many x. It follows that lim supx→∞ log dx = +∞ for P-a.e. ω, so that Xn
is recurrent. A similar argument shows that lim supx→∞ log d−1
x = +∞ for
P-a.e. ω, so that Xn is not positive recurrent.
Now we turn to the proof of Theorem 6.1.11. As in Section 6.1.3, we will
use the Lyapunov function t defined at (6.3).
Lemma 6.1.12. Suppose that for x0 ∈ N, a < 0, δ > 0 and β ∈ (0, 1/2)
the conditions (6.11) and (6.12) hold, and that E[ζ02 ] > 0 and E ζ0 = 0. For
any ε > 0, for P-a.e. ω, for all but finitely many x,
exp(xβ (log x)−2−ε ) ≤ t(x) ≤ exp(xβ (log x)1+ε ).
(6.15)
To obtain the bounds in Lemma 6.1.12 we need two technical results.
Lemma 6.1.13. Let Y0 , Y1 , . . . be independent, uniformly bounded random
variables with E Yn = 0 and E[Yn2 ] = σ 2 ∈ (0, ∞) for all n.
(i) There exists C ∈ R+ such that, a.s., for all but finitely many i,
X
i
Yk ≤ Cj 1/2 (log i)1/2 , for all j ∈ {1, 2, . . . , i}.
k=i−j+1 (ii) Let b : [1, ∞) → R+ be a non-decreasing function such that for some
γ > 0 and x0 ∈ R+ , xγ ≤ b(x) ≤ x for all x ≥ x0 . Then for any ε > 0,
a.s., for all but finitely many n,
max
max
1≤i≤n 1≤j≤b(i)
i
X
k=i−j+1
Yk ≥ (b(n/2))1/2 (log n)−1−ε .
286
Chapter 6. Further applications
P
i
Proof. First we prove part (i). Set Zji = ik=i−j+1 Yk . Then Zj+1
− Zji =
i
Yi−j ; so for fixed i, Zj is a martingale over j ∈ {1, 2, . . . , i} with uniformly bounded increments. Hence the Azuma–Hoeffding inequality (Theorem 2.4.14) implies that for some c ∈ R+ , for all j ∈ {1, . . . , i}, for t > 0,
P[|Zji | ≥ t] ≤ 2 exp(−cj −1 t2 ).
Thus for a suitable C ∈ R+ , for j ≤ i, P[|Zji | ≥ Cj 1/2 (log i)1/2 ] ≤ i−3 . Then
∞ X
i
X
i=1 j=1
P[|Zji |
≥ Cj
1/2
(log i)
1/2
]≤
∞
X
i=1
i−2 < ∞,
and the Borel–Cantelli lemma implies that, a.s., there are only finitely many
pairs (i, j) (with j ≤ i) for which |Zji | ≥ Cj 1/2 (log i)1/2 . This proves (i).
Next we prove part (ii). For fixed i, note that
i
X
max
1≤j≤b(i)
d
Yk = max
1≤j≤b(i)
k=i−j+1
j
X
Wk ,
k=1
d
where W1 , W2 , . . . are independent random variables with Wk = Yi+1−k for
each k. Fix ε > 0. Define the event
i
X
1/2
−1−ε
Ei :=
max
Yk ≤ (b(i)) (log b(i))
.
1≤j≤b(i)
k=i−j+1
Then Corollary 2 of Hirsch [128, p. 118] implies that there are absolute
constants C, C 0 ∈ (0, ∞) such that for all i ≥ x0 ,
P[Ei ] ≤ C(log b(i))−1−ε ≤ C 0 (log i)−1−ε ,
since b(i) ≥ iγ . Consider the subsequence i = 2m for m = 1, 2, . . .. Then
∞
X
m=1
P[E2m ] ≤ C
∞
X
m=1
m−1−ε < ∞.
Hence by the Borel–Cantelli lemma, E2m occurs for only finitely many m,
a.s. Each n ≥ 2 satisfies 2m(n) ≤ n < 2m(n)+1 with m(n) ∈ N such that
m(n) → ∞ as n → ∞. Then,
max
max
1≤i≤n 1≤j≤b(i)
i
X
k=i−j+1
Yk ≥
max
max
1≤i≤2m(n) 1≤j≤b(i)
i
X
k=i−j+1
Yk
287
6.1. Random walk in random environment
≥
m(n)
2X
max
1≤j≤b(2m(n) )
k=2m(n) −j+1
Yk ≥ (b(2m(n) ))1/2 (log(2m(n) ))−1−ε ,
a.s., for all n large enough. Since n ≥ 2m(n) > n/2, part (ii) follows.
Proof of Lemma 6.1.12. First we prove the upper bound in (6.15). Throughout this proof, C and y0 will denote finite positive constants that depend
only on δ and β, and whose value may change from line to line. Since a < 0
and E ζ0 = 0, we have from (6.13) with (6.12) that
E log(py /qy ) ≤ −Cy −β , for all y ≥ y0 .
Using the simple inequality
y
X
w=y−z+1
w−β ≥ z
min
y−z+1≤w≤y
w−β = zy −β ,
(6.16)
we obtain that for all y, z with y − z ≥ y0 ,
y
X
E
w=y−z+1
log(pw /qw ) ≤ −Czy −β .
(6.17)
By Lemma 6.1.13(i) we have that, for some C ∈ R+ , for P-a.e. ω, all but
finitely many y, and all z ∈ {1, 2, . . . , y},
y
X
w=y−z+1
(log(pw /qw ) − E log(pw /qw )) ≤ Cz 1/2 (log y)1/2 .
(6.18)
For ε > 0 set r(y) := dy 2β (log y)1+ε e. Then from (6.18) with (6.17), for
P-a.e. ω, for all but finitely many y ∈ N and all z with r(y) ≤ z ≤ y,
y
X
w=y−z+1
log(pw /qw ) ≤ −Czy −β + Cz 1/2 (log y)1/2 ≤ −(C/2)zy −β . (6.19)
On the other hand, for all but finitely many y ∈ N and all z with z ≤ r(y),
y
X
w=y−z+1
log(pw /qw ) ≤ Cz 1/2 (log y)1/2 .
(6.20)
288
Chapter 6. Further applications
So from (6.3) with (6.19) and (6.20) we obtain, for P-a.e. ω, for any ε > 0,
for all but finitely many y,
t(x) ≤
≤
r(y)
x−1 X
X
exp(Cz 1/2 (log y)1/2 ) +
y=0 z=0
x−1
X
y
x−1 X
X
exp(−Czy −β )
y=0 z=r(y)
exp(Cy β (log y)1+ε ),
y=0
which gives the upper bound for t(x) in (6.15).
We now prove the lower bound for t(x) in (6.15). For ε > 0 define
s(y) := by 2α (log y)−2−ε c; since β < 1/2 we have s(y) < y. Then,
t(x) ≥
max
max exp
1≤y≤x−1 1≤z≤s(y)
y
X
log(pw /qw ).
(6.21)
w=y−z+1
Now we have from (6.13) with (6.12) that, for P-a.e. ω,
log(py /qy ) ≥ ζy − Cy −β , for all y ≥ y0 .
Hence, for y − z ≥ y0 ,
y
X
w=y−z+1
log(pw /qw ) ≥ −C(y 1−β − (y − z)1−β ).
(6.22)
Taylor’s theorem implies that for β ∈ (0, 1),
y 1−β − (y − z)1−β ≤ Czy −β , for all y − z ≥ y0 .
It follows from (6.21) with (6.22) and (6.23) that for all x ≥ 1,
y
X
−β
t(x) ≥ exp
max
max
ζw − Cs(x)x
,
1≤y≤x−1 1≤z≤s(y)
(6.23)
(6.24)
w=y−z+1
using the fact that y −β s(y) is eventually increasing for β > 0. We have by
Lemma 6.1.13(ii) that for any ε > 0, for P-a.e. ω,
max
max
1≤y≤x−1 1≤z≤s(y)
y
X
w=y−z+1
ζw ≥ (s(x/2))1/2 (log x)−1−(ε/4)
≥ Cxβ (log x)−2−(3ε/4) ,
for all but finitely many x, while s(x)x−β ≤ xβ (log x)−2−ε . Hence (6.24)
implies the lower bound in (6.15).
289
6.1. Random walk in random environment
Now we can complete the proof of Theorem 6.1.11. For the upper bound
in the theorem, we once more use Lemma 6.1.4. The lower bound needs an
additional idea. The idea of Lemma 6.1.8 was to obtain an upper bound
on hitting times, which could be translated to a lower bound on Xn that
was valid infinitely often. In order to extend this technique to the transient
case, and obtain a lower bound for Xn valid all but finitely often, we show
in addition that (roughly speaking), in the present case, the time of the
last visit of the random walk to a site is not too much greater than the
first hitting time; a similar approach was used in Theorem 3.10.1 for the
transient Lamperti problem.
Proof of Theorem 6.1.11. The lower bound in (6.15) implies that for any
ε > 0, for P-a.e. ω, for all x sufficiently large,
t(x) ≥ exp(xβ (log x)−2−ε ) =: w(x).
Hence Lemma 6.1.4 implies that for any ε > 0, for P-a.e. ω, Pω -a.s.,
Xn ≤ w−1 (n1+ε ) ≤ (log n)1/β (log log n)(2+2ε)/β ,
for all but finitely many n. This proves the claimed upper bound.
It remains to prove the lower bound in the theorem. For fixed ω, let ax
denote the probability that the random walk Xn hits x in finite time, given
that it starts at 2x. For x ≥ 1 define
y
y
∞ Y
∞
X
X
X
px+z
px+z
Mx := 1 +
=1+
exp
log
.
(6.25)
qx+z
qx+z
y=1 z=1
y=1
z=1
Standard hitting probability arguments yield a0 = 1, and, if Mx < ∞,
y
y
∞ Y
∞
X
X
X
px+z
px+z
ax = Mx−1
= Mx−1
exp
log
, x ≥ 1. (6.26)
q
q
x+z
x+z
y=x
y=x
z=1
z=1
Since a < 0 and E ζ0 = 0, we have from (6.13) with (6.12) that
E
y
X
z=1
y
X
log(px+z /qx+z ) ≤ −c
(x + z)−β ≤ −c0 y 1−β ,
(6.27)
z=1
for all y ≥ x ≥ x0 , for positive constants c, c0 , and x0 . Also, by the
Azuma–Hoeffding inequality (Theorem 2.4.14) and an argument similar to
Lemma 6.1.13, we have that, for P-a.e. ω,
y
X
z=1
(log(px+z /qx+z ) − E[log(px+z /qx+z )]) ≤ Cy 1/2 (log y)1/2 ,
290
Chapter 6. Further applications
for all but finitely many (x, y) with y ≥ x. Together with (6.27) this shows
that for P-a.e. ω, for all but finitely many (x, y) with y ≥ x,
y
X
z=1
log(px+z /qx+z ) ≤ −Cy 1−β + Cy 1/2 (log y)1/2 ≤ −(C/2)y 1−β ,
(6.28)
since β ∈ (0, 1/2). Hence, for P-a.e. ω, from (6.25) with (6.12) and (6.28),
for x ≥ 1 and a constant Cδ depending only on δ,
Mx ≤ 1 + (Cδ )x +
∞
X
y=x
exp(−Cy 1−β ) < ∞.
Further, since Mx ≥ 1 for all x, (6.26) and (6.28) imply, for P-a.e. ω, for all
but finitely many x,
ax ≤
∞
X
y=x
exp(−Cy 1−β ) ≤ exp(−C 0 x1−β ),
P
for some C 0 ∈ (0, ∞). Thus, for P-a.e. ω, x ax < ∞.
The Borel–Cantelli lemma then implies that, for P-a.e. ω, Pω -a.s., for
only finitely many sites x does Xn return to x after visiting 2x. Denoting by
ηx the time of the last visit of Xn to x, we then have that ηx ≤ τ2x , Pω -a.s. for
all but finitely many x. Suppose P
t(x) ≤ u(x) for all but finitely
many x.
P
Then since Eω τx = t(x) we have x≥1 Pω [τx ≥ x2 t(x)] ≤ x≥1 x−2 < ∞,
so another application of the Borel–Cantelli lemma shows that for P-a.e. ω,
Pω -a.s., for all x ≥ x0 with a finite (random) x0 ,
ηx ≤ τ2x ≤ 4x2 u(2x).
(6.29)
Moreover, since, for P-a.e. ω, Xn is transient, we have that Xn ≥ x0 for all
n sufficiently large. Hence from (6.29), using the fact that ηXn ≥ n for all
n, we have that for P-a.e. ω, Pω -a.s., for all but finitely many n,
n ≤ ηXn ≤ 4Xn2 u(2Xn ).
Then, with the upper bound in (6.15), we obtain, for any ε > 0, for P-a.e. ω,
Pω -a.s., n < exp(Xnβ (log n)1+ε ), for all but finitely many n, using the fact
that Xn ≤ n to bound the logarithmic term. This gives the claimed lower
bound.
6.2. Random strings in random environment
6.2
6.2.1
291
Random strings in random environment
Introduction
Consider a finite alphabet A = {1, . . . , d}. A string is a finite sequence
of symbols from A. We write |s| for the length of the string s, i.e., if
s = s1 . . . s` , with each si ∈ A, then |s| = `.
In this section we define a time-homogeneous Markov chain (Xn , n ≥ 0)
whose state space is the set of all finite strings over A. The transitions of
this Markov chain are as follows. If the current state is the string Xn = s =
s1 . . . s` , whose rightmost symbol is s` = i ∈ A, then a transition to Xn+1 is
one of the following three kinds:
• erase the rightmost symbol of s with probability ri` ;
` ;
• substitute the rightmost symbol i by j with probability qij
• append the symbol j to the right end of the string with probability p`ij .
Of course we assume that
`
ri` ≥ 0, qij
≥ 0, p`ij ≥ 0, for all i, j ∈ A and all ` ∈ N;
X
X
`
ri` +
qij
+
p`ij = 1, for all i ∈ A and all ` ∈ N.
j∈A
(6.30)
(6.31)
j∈A
The specification of the Markov chain is completed by describing the possible transitions from the empty string ∅ (the string with length zero): for
definiteness, we suppose that an empty string can transition only to a string
of length 1 with probabilities given by p0j , j ∈ A.
The asymptotic behaviour of the string will then be determined by the
` , p` , over ` ∈ N, i ∈ A, j ∈ A. These
collection of parameters ri` , qij
ij
parameters constitute the environment for the string dynamics.
The environment will itself be random. Let PA denote the set of all
` , p` ) , i.e., for fixed ` the set of all parameter choices
admissible (ri` , qij
ij i,j
satisfying (6.30) and (6.31). On a probability space (Ω, F, P), let ξ1 , ξ2 , . . .
be i.i.d. random elements of PA . Then for each `, ξ` specifies the parameters
` , p` for all i, j ∈ A; we will denote the realization of the environment
ri` , qij
ij
by ω = (ξ1 , ξ2 , . . .). This defines the Markov chain describing the evolution
of the string, for each fixed environment ω.
Similarly to Section 6.1, we will study this model by choosing a suitable
Lyapunov function f (s) = f (s, ω) of the state s and the environment ω.
292
Chapter 6. Further applications
Remark 6.2.1. The strings model includes the random walk in random environment of Section 6.1 as a special case. Indeed, if d = 1, then the process
tracking the length of the string |Xn | becomes a nearest-neighbour random
walk on Z+ with quenched transition probabilities given for ` ∈ N by
Pω [|Xn+1 | = ` + 1 | |Xn | = `] = p`11 ;
`
Pω [|Xn+1 | = ` | |Xn | = `] = q11
;
Pω [|Xn+1 | = ` − 1 | |Xn | = `] = r1` .
Note that for general d ≥ 2, the process |Xn | is not Markov under Pω .
6.2.2
Lyapunov exponents and products of random matrices
In this section we review some properties of products of random matrices
that we need below. Recall that for a matrix A we denote its transpose by
A> and its largest eigenvalue by λmax (A), and that for a d × d real matrix
A, its operator norm is
1/2
kAkop = sup kAuk = λmax (A> A)
.
u∈Sd−1
On probability space (Ω, F, P), we say that a d × d random real matrix
A satisfies condition (N) if:
(N) E log+ kAkop < ∞, where log+ x = max{log x, 0}.
Let A, A1 , A2 , . . . be a sequence of i.i.d. matrices satisfying condition (N).
Let a1 (n) ≥ a2 (n) ≥ · · · ≥ ad (n) ≥ 0 be the square roots of the (random)
eigenvalues of (An · · · A1 )> (An · · · A1 ). Then there exist constants γj (A),
j ∈ {1, . . . , d}, with
−∞ ≤ γd (A) ≤ · · · ≤ γ1 (A) < ∞,
depending on the distribution of A, such that for each j ∈ {1, . . . , d},
1
(6.32)
P lim log aj (n) = γj (A) = 1;
n→∞ n
see Proposition 5.6 of [31] (note that A does not need to be invertible). The
numbers γj (A) are called the Lyapunov exponents of the sequence of random
matrices An . In particular,
γ1 (A) = lim
n→∞
1
1
log a1 (n) = lim log kAn · · · A1 kop , P -a.s.,
n→∞
n
n
is the top Lyapunov exponent.
(6.33)
6.2. Random strings in random environment
6.2.3
293
Recurrence classification
To describe our results on recurrence and transience, we need some more
` and r ` as defined in Section 6.2.1, we write P :=
notation. With p`ij , qij
`
i
` for i 6= j and
(p`ij )i,j∈A and D` := (d`ij )i,j∈A , where d`ij = −qij
X
`
qij
.
d`ii = ri` +
j∈A\{i}
Thus P1 , P2 , . . . and D1 , D2 , . . . are each i.i.d. sequences of random d × d
matrices on (Ω, F, P). We will impose the following assumption.
(D) E log(1/ri` ) < ∞, for all i ∈ A.
We will show shortly (in Proposition 6.2.2) that condition (D) ensures
that D` is P-a.s. invertible; then we define the sequence of i.i.d. random
matrices A` , given by
(6.34)
A` := D`−1 P` ,
and we denote the associated Lyapunov exponents by γ1 , . . . , γd . The relevance of condition (D) is displayed by the following result.
Proposition 6.2.2. If (D) holds, then the matrices D` are P-a.s. invertible,
with E log+ kD`−1 kop < ∞, and the associated matrices A` defined by (6.34)
satisfy condition (N).
Proof. For v = (v1 , . . . , vd )> and i ∈ A, we have
X
X
`
(D` v)i =
d`ij vj` = ri` vi +
qij
(vi − vj ).
j
(6.35)
j6=i
Suppose that (D) holds. Then, P-a.s., ri` > 0 for all i ∈ A. Fix v 6= 0, and
let vi denote the component of v with greatest absolute value. If vi > 0
then (6.35) shows that (D` v)i ≥ ri` vi > 0; if vi < 0 then (6.35) shows that
(D` v)i ≤ ri` vi < 0. In particular, any non-zero vector v ∈ Rd has a non-zero
image D` v; thus the kernel of the map v 7→ D` v is trivial, and hence D`−1
exists and the linear map is bijective. Moreover
kD` vk ≥ |(D` v)i | ≥ ri` |vi | ≥ d−1/2 ri` kvk.
Thus if u = D` v we have kD`−1 uk ≤ d1/2 (1/ri` )kuk and hence kD`−1 kop ≤
d1/2 maxi∈A (1/ri` ). Therefore
X
E log+ kD`−1 kop ≤ d1/2
E log(1/ri` ) < ∞.
i∈A
294
Chapter 6. Further applications
The final claim in the proposition follows from this, together with the fact
that kA` kop = kD`−1 P` kop ≤ kD`−1 kop kP` kop and the fact that elements of
P` are uniformly bounded.
Now we are ready to formulate our results. Roughly speaking, the top
Lyapunov exponent γ1 determines the recurrence or transience of the process. First we give a sufficient condition for transience. Note that (by
Proposition 6.2.2), under the hypotheses of Theorem 6.2.3 both D` and P`
are P-a.s. invertible, and hence so is A` as defined by (6.34).
Theorem 6.2.3. Suppose that (D) holds. Suppose also that P` is P-a.s. invertible, and that A−1
satisfies condition (N). If γ1 > 0, then for P-a.e. ω
`
the Markov chain Xn is transient.
Next we give a sufficient condition for positive-recurrence.
Theorem 6.2.4. Suppose that (D) holds. If γ1 < 0, then for P-a.e. ω the
Markov chain Xn is positive-recurrent.
The final result covers the case γ1 = 0.
Theorem 6.2.5. Suppose that (D) holds. Suppose also that A` is P-a.s. invertible, and that no finite union of proper subspaces of Rd is a.s. stabilized
by A` . If γ1 = 0, then for P-a.e. ω the Markov chain Xn is recurrent.
6.2.4
Proofs
We will need the following simple lemma, which establishes relations between
the Lyapunov exponents of A and A−1 .
Lemma 6.2.6. Suppose that A is P-a.s. invertible, and that both A and A−1
satisfy condition (N). Then for j ∈ {1, . . . , d},
γj (A−1 ) = −γd−j+1 (A).
(6.36)
Proof. Let b1 (n) ≥ · · · ≥ bd (n) be the square roots of the eigenvalues of
−1 >
−1
−1
> −1
(A−1
.
n · · · A1 ) (An · · · A1 ) = (A1 · · · An )(A1 · · · An )
Since U V has the same eigenvalues as V U , (b1 (n))−1 , . . . , (bd (n))−1 are
the square roots of the eigenvalues of (A1 · · · An )> (A1 · · · An ). But the
product A1 · · · An has the same law as the product An · · · A1 . So for all j ∈
{1, . . . , d}, we have (bj (n))−1 has the same law as ad−j+1 (n), and consequently, γj (A−1 ) = −γd−j+1 (A).
295
6.2. Random strings in random environment
We will also need the classical multiplicative ergodic theorem of Oseledets [249].
Theorem 6.2.7. Let A, A1 , A2 , . . . be a sequence of i.i.d. random matrices
satisfying condition (N), and let γ1 ≥ γ2 ≥ · · · ≥ γd be the associated
Lyapunov exponents. Let t := #{γ1 , . . . , γd } denote the number of distinct
exponents, define i1 := 1, it+1 := d + 1, and
is+1 := min{i > is : γi < γis }, for 1 ≤ s ≤ t − 1.
Then, P-a.s.:
(i) for every v ∈ Rd , limn→∞ n−1 log kAn . . . A1 vk exists or is −∞;
(ii) for s ∈ {1, 2, . . . , t, t + 1},
o
n
V (s, ω) = v ∈ Rd : lim n−1 log kAn . . . A1 vk ≤ γis
n→∞
is a random linear subspace of Rd with dimension d − is + 1 (we have
V (1, ω) = Rd and V (t + 1, ω) = {0});
(iii) for s ∈ {1, 2, . . . , t}, v ∈ V (s, ω) V (s + 1, ω) implies that
lim n−1 log kAn . . . A1 vk = γis .
n→∞
Note that in Theorem 6.2.7 the non-random integers is mark the points
of decrease of the γi , and γ1 = γi1 > γi2 > · · · > γit = γd .
Now we can give the proof of our transience result.
Proof of Theorem 6.2.3. Suppose that (D) holds, that P` is P-a.s. invertible,
and that A−1
` satisfies condition (N). We construct a Lyapunov function h(s)
and apply Corollary 2.5.12 to deduce transience. We show that there exists
a sequence of column vectors v` = (v1` , . . . , vd` )> , depending on the environment ω, such that the function h(s) of the string s = s1 . . . s|s| can be
defined as
|s|
X
h(s) =
vsjj .
(6.37)
j=1
Indeed, for h(s) defined by (6.37) we have for s with |s| = ` and s` = i ∈ A,
X
X
`
Eω [h(Xn+1 ) − h(Xn ) | Xn = s] = −ri` vi` +
qij
(−vi` + vj` ) +
p`ij vj`+1
j∈A
j∈A
296
Chapter 6. Further applications
= −D` v` + P` v`+1 i ,
(6.38)
where the matrices P` and D` are as introduced in Section 6.2.3.
If h(s) is to satisfy the conditions of Corollary 2.5.12, we require the
right-hand side of (6.38) to vanish for all non-empty strings s and all i.
Thus we require the following relation between v` and v`+1 :
`
v`+1 = D`−1 P` v` = A−1
` v .
(6.39)
From (6.39) it follows that we may choose
−1 1
v`+1 = A−1
` · · · A1 v , for all ` ≥ 1,
for some vector v1 .
According to Theorem 6.2.7 (in particular, the s = t case of part (iii))
−1
applied to the sequence A−1
1 , A2 , . . ., for P-a.e. environment ω there exists
a non-zero random vector v1 = v1 (ω) such that
1
1
−1 1
−1
log kv`+1 k = lim log kA−1
` · · · A1 v k = γd (A ) = −γ1 , (6.40)
`→∞ `
`→∞ `
lim
using Lemma 6.2.6 for the final equality. From (6.40), if γ1 > 0 it follows that there exists a positive constant C1 = C1 (ω) such that kv` k ≤
C1 exp{−γ1 `/2} for all ` ≥ 1. Thus from (6.37), we find that
sup |h(s)| ≤
s
∞
X
j=1
kvj k ≤
C1 exp{−γ1 /2}
< ∞.
1 − exp{−γ1 /2}
So this function satisfies all the conditions of Corollary 2.5.12, which yields
transience.
Remark 6.2.8. The string Lyapunov function h defined at (6.37) bears a
close analogy to the nearest-neighbour random walk function of Section 2.2
as defined at (2.7). The string function is a sum of products of the matrices
A−1
n , while the walk function is a sum of products of the scalars pn /qn .
To prove Theorem 6.2.4, we need one supplementary fact.
Lemma 6.2.9. Suppose that (D) holds. P-a.s., all entries of matrix D`−1
are non-negative.
Proof. As shown in Proposition 6.2.2, condition (D) ensures that D`−1 exists
and the map v 7→ D` v is bijective. It suffices to prove that all components
of D`−1 v are non-negative whenever all components of v are non-negative.
6.2. Random strings in random environment
297
Thus it suffices to prove that any vector v = (v1 , . . . , vd )> with at least one
negative component has an image D` v with at least one negative component.
Choose i such that vi = minj vj ; then, by (6.35), (D` v)i ≤ ri` vi < 0, since
ri` > 0 under (D).
Proof of Theorem 6.2.4. In contrast to the proof of Theorem 6.2.3, where
h(s) was constructed using ingredients provided only implicitly by Theorem 6.2.7, here we explicitly construct a Lyapunov function t(s), which is
an analogue of the random walk Lyapunov function given by (6.3).
Recall that Id denotes the d × d identity matrix, and denote by 1 =
(1, . . . , 1)> the column vector with all d components equal to 1. Similarly
to (6.37), we define
|s|
X
t(s) =
vsjj ,
j=1
but now we define v` = (v1` , . . . , vd` )> by
X
−1
1
v` = D`−1 1 +
A` · · · Am Dm+1
m≥`
=
(D`−1
−1
−1
+ A` D`+1
+ A` A`+1 D`+2
+ · · · )1.
We need to show that v` is P
finite and hence t(s) is well-defined. First, since
−1
+
for any x ∈ R+ we have x ≥ ∞
m=1 1{x ≥ m}, the assumption E log kD1 kop <
∞ shows that, for any C ∈ (0, ∞),
∞
X
m=1
−1
P C −1 log kDm
kop > m < ∞.
Thus, by the Borel–Cantelli lemma, P-a.s., for all but finitely many m,
−1 k
kDm
op < exp(Cm). Since γ1 < 0, we may choose C ∈ (0, −γ1 ). Then we
obtain, using (6.33), P-a.s.,
−1
−1
kv` k ≤ (kD`−1 kop + kA` kop kD`+1
kop + kA` A`+1 kop kD`+2
kop + · · · )k1k < ∞.
In addition, we have that v` has all components non-negative, by Lemma 6.2.9.
Analogously to (6.38), we have for s with |s| = ` and s` = i ∈ A,
Eω [t(Xn+1 ) − t(Xn ) | Xn = s] = −D` v` + P` v`+1 i .
With the present choice of v` , the vector whose ith component is on the
right-hand side of the last display is equal to
−1
−1
(−Id − D` A` D`+1
− D` A` A`+1 D`+2
− · · · )1
298
Chapter 6. Further applications
−1
−1
−1
+ (P` D`+1
+ P` A`+1 D`+2
+ P` A`+1 A`+2 D`+3
+ · · · )1
−1
−1
= −Id 1 − D` (A` D`+1
− A` A`+1 D`+2
− · · · )1
−1
−1
+ D` (A` D`+1
+ A` A`+1 D`+2
+ · · · )1,
recalling from (6.34) that A` = D`−1 P` . Hence
Eω [t(Xn+1 ) − t(Xn ) | Xn = s] = −1, for all s 6= ∅.
Therefore the function t(s) satisfies all the conditions of Theorem 2.6.4 and
positive-recurrence follows.
Finally, we complete the proof of Theorem 6.2.5.
Proof of Theorem 6.2.5. From a theorem in [30] the assumptions imply
lim inf kA1 . . . A` kop = 0, P -a.s.,
`→∞
and therefore
P1 A2 · · · A` 1 ≤ D1 1, for infinitely many `.
(6.41)
Define recursively the random times τ0 = 1, and, for k ≥ 0,
τk+1 = min n > τk : −Dτk 1 + Pτk Aτk +1 · · · An 1 ≤ 0 .
From (6.41) the times τk are P-a.s. finite. Similarly to (6.37), we define
f (s) =
|s|
X
vsjj ,
j=1
but where now vτk = 1 and
v` = A` A`+1 · · · Aτk+1 1, for τk < ` ≤ τk+1 .
(6.42)
Hence we have v` = A` v`+1 when τk < ` < τk+1 , k ≥ 0, and
−Dτk vτk + Pτk vτk +1 = −Dτk 1 + Pτk Aτk +1 · · · Aτk+1 1 ≤ 0,
by definition of τk+1 . Similarly to (6.38), this means that f (Xn ) is a supermartingale except at s = ∅. From Lemma 6.2.9 and from the definition (6.42)
the vector v` is non-negative, and therefore f is also non-negative. Finally,
since the τk are a.s. finite and vτk = 1, f (s) → ∞ as |s| → ∞. Theorem 2.5.2
applies, and the random string is recurrent.
6.3. Stochastic billiards
6.3
6.3.1
299
Stochastic billiards
Introduction
In this section we consider a model for the dynamics of a particle in a
planar domain moving at constant speed and undergoing random reflections
on collision with the boundary. The domain is of the form
D := D(γ, A) := {(x, y) ∈ R2 : x ≥ A, |y| ≤ xγ },
(6.43)
where γ < 1 is a fixed parameter and A > 0 is a constant to be fixed later.
The dynamics are described roughly as follows. A particle moves at unit
speed in D; while in the interior D \ ∂D, the direction also remains fixed.
When the particle hits the boundary ∂D, it is (independently) reflected at a
random angle to the inwards-pointing normal vector at the boundary. The
law of the process is determined by specifying the distribution of the random
reflections. See Figure 6.2 for some simulations of the process.
Suppose that the random variable α for the angle of reflection satisfies,
for some α0 ∈ (0, π/2),
P[0 < |α| < α0 ] = 1 and E tan α = 0.
(6.44)
Thus the distribution of α is bounded strictly away from ±π/2 and does
not have an atom at 0; a sufficient condition for E tan α = 0 is that the
distribution of α be symmetric about 0.
The process that we have described informally above is a continuous-time
process on D. It is more convenient to work with a discrete-time process,
obtained by recording the locations of the successive hits of the particle on
the boundary ∂D. This is a Markov chain with state-space ∂D that we
(1) (2)
denote by (ξn , n ≥ 0), where we write in coordinates ξn = (ξn , ξn ). Thus
(1)
(2)
(1)
when ξn > A, ξn = ±(ξn )γ . We call ξn the collisions process.
To construct ξn formally, suppose that ξ0 ∈ ∂D. We perform a step of
our process as follows.
• Take an independent draw of an angle α satisfying (6.44). The realization of α specifies a ray Γ starting at ξn with angle α to the interior
normal to ∂D at ξn . We adopt the convention that positive values
of the angle correspond to the right of the normal; negative values
correspond to the left.
• Let ξn+1 be the first point of intersection of the ray Γ with ∂D \ {ξn }.
300
Chapter 6. Further applications
It is not a priori clear that ξn is well-defined, as the ray Γ might never
intersect the boundary; however, this does not occur assuming that the A
in (6.43) is chosen sufficiently large, as the following result shows.
Lemma 6.3.1. Suppose that α satisfies (6.44) for some α0 ∈ (0, π/2), and
that γ < 1. Then there exists A0 ∈ R+ (depending on the distribution
of α as well as on α0 and γ) such that for any A ≥ A0 the process ξn
is well-defined on ∂D where D = D(γ, A) is given by (6.43). Moreover,
(1)
lim supn→∞ ξn = ∞, a.s.
Thus our standing assumption for this section will be the following.
(SB) Suppose that α satisfies (6.44) for some α0 ∈ (0, π/2), and that D is
given by (6.43) for some γ < 1 and A ≥ A0 as in Lemma 6.3.1.
Our first result covers the recurrence/transience classification for ξn . It
turns out that a key quantity is E[tan2 α]; note that under (6.44) we have
E[tan2 α] ≤ E[tan2 α0 ] < ∞ and E[tan2 α] > 0 by the fact that α does not
degenerate to 0. Define
γc :=
E[tan2 α]
,
1 + 2 E[tan2 α]
(6.45)
so that γc ∈ (0, 1/2) under (6.44).
Theorem 6.3.2. Suppose that (SB) holds. Then:
(1)
(i) If γ > γc the process is transient, i.e., limn→∞ ξn = ∞, a.s.;
(1)
(ii) If γ ≤ γc the process is recurrent, i.e., lim inf n→∞ ξn < ∞, a.s.
6.3.2
Reduction to Lamperti’s problem
The key to our analysis of the stochastic billiards process is to consider
a rescaled version of the process to obtain an instance of the Lamperti
problem. As at (3.1), we write µk (x) = E[(Xn+1 − Xn )k | Xn = x] for the
increment moment functions of Xn .
(1)
Theorem 6.3.3. Suppose that (SB) holds. Set Xn := (ξn )1−γ . Then Xn
is a time-homogeneous Markov process on R+ satisfying, for some B ∈ R+ ,
P[|Xn+1 − Xn | ≤ B], and, as x → ∞,
2γ(1 − γ)(1 + E[tan2 α])
+ O(x−2 );
x
µ2 (x) = 4(1 − γ)2 E[tan2 α] + O(x−1 ).
µ1 (x) =
(6.46)
(6.47)
301
6.3. Stochastic billiards
●
●
●●●
●●
●
●●
● ●●
●●●● ●
●●●● ●●
●●●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
● ●
●
●●
● ●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●●
● ●
● ●●
●●
●
●
● ●●
● ●● ●
●●
● ●
●● ● ●●●
● ●
●
●●
●
●
●
● ●
●
●
●
Figure 6.2: Simulations of part of the trajectory of the billiards process
with α uniform on [−π/4, π/4] in domains with γ = 1/10 (left) and γ = 1/2
4−π
(right). Here E[tan2 α] = 4−π
π and γc = 8−π ≈ 0.177, so the first case is
recurrent and the second transient.
i
If a near-critical process on R+ is transformed to find a
scale on which the increments of the process have uniformly bounded second moments, one often ends up with
a Lamperti process with asymptotically zero drift.
We defer the proof of Theorem 6.3.3 to Section 6.3.3; here we use the
theorem and the results of Chapter 3 to establish our recurrence classification.
Proof of Theorem 6.3.2. Consider the process Xn given in Theorem 6.3.3;
(1)
then Xn is recurrent if and only if ξn is recurrent. From (6.46) and (6.47)
we have that
2xµ1 (x) − µ2 (x) = 4(1 − γ) γ(1 + 2 E[tan2 α]) − E[tan2 α] + O(x−1 )
= 4(1 − γ) (γ − γc )(1 + 2 E[tan2 α]) + O(x−1 ) ,
where γc is given by (6.45). Thus if γ > γc we have lim inf x→∞ (2xµ1 (x) −
µ2 (x)) > 0 which yields transience by Theorem 3.5.1.
Similarly, for θ ∈ (0, 1), by (6.46) and (6.47),
1−θ
2xµ1 (x) − 1 +
µ2 (x)
log x
302
Chapter 6. Further applications
2
= 4(1 − γ)(γ − γc )(1 + 2 E[tan α]) −
1 − θ + o(1)
log x
4(1 − γ)2 E[tan2 α],
which is strictly negative for all x sufficiently large provided γ ≤ γc ; then
Theorem 3.5.2 establishes recurrence.
6.3.3
Increment estimates
(1)
(2)
Write g(x) := xγ . Suppose that at time n = 0 we have ξ0 = (ξ0 , ξ0 ) =
(x, ±g(x)) for x > A, and then ξ1 is obtained after reflecting at angle α
(1)
(1)
to the normal. Denote D(x, α) := ξ1 − ξ0 , the resulting increment of
the horizontal component of the process. Also set θ := arctan g 0 (x), so
tan θ = g 0 (x) = γxγ−1 .
We now proceed to obtain estimates for D(x, α) and its moments. The
next lemma gives an upper bound on D(x, α) that follows from the fact that
for large enough x our domain is almost flat, while α is bounded strictly away
from ±π/2.
Lemma 6.3.4. Suppose that (SB) holds. Then there exist x0 ∈ R+ and
C ∈ R+ , both depending only on α0 and γ, such that for all x ≥ x0 and all
α ∈ (−α0 , α0 ),
|D(x, α)| ≤ Cxγ .
Proof. We use a geometrical argument depicted in Figure 6.3. Since γ < 1,
we have g 0 (x) → 0 and θ → 0 as x → ∞. Thus we may choose x ≥ x0 large
enough so that |θ| < min{α0 , (π/2) − α0 } and then tan(α0 + θ) = c0 for
c0 < ∞; suppose that x0 is also chosen sufficiently large that |g 0 (x)| < 1/c0
for all x ≥ x0 .
By symmetry, it suffices to suppose that we start on the positive half
of the curve, i.e., at ξn = (x, g(x)). First suppose that γ ≥ 0. Then g
is non-decreasing, so θ ≥ 0. Consider D(x, α), α ≥ 0, as represented in
Figure 6.3.
In the case α ≥ 0, |D(x, α)| ≤ |D(x, α0 )|. The reflected ray at angle α0
to the normal from (x, g(x)) has equation in (u, v) given by
u − x = −(v − g(x)) tan(α0 + θ), u ≥ x.
(6.48)
Let a = 1/c0 ; by choice of x0 , we have |g 0 (u)| < a for all u ≥ x ≥ x0 .
Consider the line
v + g(x) = −a(u − x), u ≥ x.
(6.49)
303
6.3. Stochastic billiards
(x, xγ )
θ α
N
(x, 0)
D(x, α)
D0
α+θ
(x, −xγ )
α+θ
v + xγ = −a(u − x)
(1)
(1)
Figure 6.3: The increment D(x, α) = ξn+1 − ξn when the particle at ξn =
(x, xγ ) reflects at angle α to the inwards normal (indicated by ‘N ’ in the
diagram). Here tan θ = γxγ−1 . The quantity D0 is an upper bound for
D(x, α), obtained by extending a line of slope −a for 0 < a < γxγ−1 from
(x, −xγ ).
This line intersects ∂D at (x, −g(x)) and it intersects the reflected ray at
angle α0 whose equation is (6.48) at (u, v) with
u−x=
2g(x) tan(α0 + θ)
.
1 − a tan(α0 + θ)
(6.50)
Since |g 0 (u)| < a for u ≥ x, the curve v = −g(u) remains above the line (6.49)
for all u ≥ x. Thus |D(x, α0 )| is bounded by u − x as given by (6.50); this
is D0 in Figure 6.3.
304
Chapter 6. Further applications
Thus, for some C ∈ R+ , |D(x, α0 )| ≤ Cg(x), for all x ≥ x0 large enough.
A similar argument applies when α < 0 and we use the bound |D(x, α)| ≤
|∆(x, −α0 )|. With minor modifications, the same argument also works for
γ < 0.
Proof of Lemma 6.3.1. Lemma 6.3.4 shows that we may choose x0 sufficiently large, depending on α0 and γ, so that for all x ≥ x0 , ξn ∈ ∂D with
(1)
ξn = (x, ±xγ ) implies that ξn+1 ∈ ∂D, a.s. If ξn = A, then since α has no
atom at 0 it is also not hard to see that ξn+1 ∈ ∂D, a.s. Thus if we take
A > x0 , we ensure that ξn ∈ ∂D is well defined for all n ≥ 0.
(1)
To show the non-confinement result lim supn→∞ ξn = ∞, a.s., we apply
Proposition 3.3.4. It is sufficient to show that for any admissible distribution
for α, for all y ≥ A, there exists εy > 0 for which
(1)
P[ξn+1 − ξn(1) > εy | ξn(1) = x] > εy , for all x ∈ [A, y].
(6.51)
If x = A, the result is clear. So suppose x > A. Since α is not identically
0 and E tan α = 0, there exists ν > 0 for which P[α > ν] > ν. Thus with
positive probability D(x, α) ≥ D(x, ν). Then (see Figure 6.3), D(x, ν) ≥ D1
where D1 = g(x) tan(θ + ν). If γ ≥ 0, then θ ≥ 0 and (6.51) follows easily,
since then for x ∈ [A, y] we have D(x, α) ≥ g(A) tan(ν) with probability at
least ν. (So in the case γ ≥ 0, εy need not depend on y.)
If γ < 0, we may choose A big enough so that the angle θ to the normal
satisfies |θ| < ν for all x ≥ A, and so tan(θ + ν) is still strictly positive, so
that for x ∈ [A, y] we have D(x, α) ≥ g(y) tan(θ + ν) with probability at
least ν, and we verify (6.51) in that case too. Note that the choice of A for
this last argument depended on ν and hence on the distribution of α, as well
as on α0 and γ.
The next lemma gives crucial estimates for the first two moments of
D(x, α).
Lemma 6.3.5. Suppose that (SB) holds. Then as x → ∞
E[D(x, α)] = 2γx2γ−1 (1 + 2 E[tan2 α]) + O(x3γ−2 );
2
and E[D(x, α) ] = 4x
2γ
2
E[tan α] + O(x
3γ−1
).
(6.52)
(6.53)
Proof. To ease notation we write D = D(x, α). Recall that x ≥ A is large
enough so that if ξn = (x, xγ ) is on the positive part of ∂D then the next
collision ξn+1 = (x + D, −(x + D)γ ) is on the negative part. See Figure 6.3.
305
6.3. Stochastic billiards
We have, repeatedly using the facts that | tan(α + θ)| and | tan α| are
uniformly bounded, and |D| = O(xγ ) by Lemma 6.3.4,
D = (g(x) + g(x + D)) tan(α + θ)
= xγ (1 + (1 + (D/x))γ ) tan(α + θ)
= xγ (2 + Dγx−1 ) tan(α + θ) + O(x2γ−2 )
tan α + tan θ
= xγ (2 + Dγx−1 )
+ O(x2γ−2 )
1 − tan α tan θ
= xγ (2 + Dγx−1 )(tan α + γxγ−1 )(1 + γxγ−1 tan α) + O(x3γ−2 ),
where, as always, the implicit constants in the O terms are non-random.
Collecting terms we have
D = 2γx2γ−1 + (tan α) xγ (2 + Dγx−1 ) + 2γx2γ−1
+ (tan2 α) 2γx2γ−1 + O(x3γ−2 ).
Re-arranging, we obtain
(2xγ + 2γx2γ−1 ) tan α + 2γx2γ−1 tan2 α
+ O(x3γ−2 )
1 − γxγ−1 tan α
= 2γx2γ−1 + (2xγ + 2γx2γ−1 ) tan α + 4γx2γ−1 tan2 α + O(x3γ−2 ). (6.54)
D = 2γx2γ−1 +
Taking expectations in (6.54) and using the fact that E tan α = 0 we obtain (6.52). Similarly, squaring both sides of (6.54) gives
D2 = 4x2γ tan2 α + O(x3γ−1 ),
which on taking expectations yields (6.53).
Now we can complete the proof of Theorem 6.3.3.
(1)
Proof of Theorem 6.3.3. Set Xn = (ξn )1−γ . Then, Xn is a Markov chain
and, by Lemma 6.3.4,
|Xn+1 − Xn | ≤ max |(x + Cxγ )1−γ − x1−γ |, |(x − Cxγ )1−γ − x1−γ | .
Here
(x ± Cxγ )1−γ − x1−γ = x1−γ
1 ± Cxγ−1
1−γ
−1 ,
which is uniformly bounded. Thus |Xn+1 − Xn | ≤ B, a.s., for some B ∈ R+ .
(1)
Given ξn = x we have Xn = x1−γ , and, by Taylor’s formula,
Xn+1 − Xn = (x + D(x, α))1−γ − x1−γ
306
Chapter 6. Further applications
= (1 − γ)x−γ D(x, α) −
γ(1 − γ) −γ−1
x
D(x, α)2 + O(x2γ−2 ),
2
(6.55)
using Lemma 6.3.4 for the error term. Taking expectations in (6.55) and
using (6.52) and (6.53) we obtain
E[Xn+1 − Xn | Xn = x1−γ ] = 2(1 − γ)γ(1 + E[tan2 α])xγ−1 + O(x2γ−2 ),
which yields (6.46) after a change of variable. Similarly we obtain (6.47)
after squaring both sides of (6.55).
6.4
6.4.1
Exclusion and voter models
Introduction
In this section, we apply the method of Lyapunov functions to some models
of interacting particle systems, in which instead of a single particle on a
countable state space, there many particles each of which can influence the
others’ dynamics.
The processes that we will consider are Markov processes on configurations of particles on Z. Thus we take as our state space {0, 1}Z ; we call
s ∈ {0, 1}Z a configuration, and we interpret a coordinate value s(x) = 1
as the presence of a particle at the site x ∈ Z in the configuration s, and
s(x) = 0 as the absence of a particle at x.
The two processes that we will consider are the exclusion process and
the voter model, and we will also study a more general process that is a
mixture of the two. The dynamics of these processes may be interpreted
in terms of particles that hop on Z (the case of the exclusion process) or
appear and disappear at the sites of Z (the case of the voter model). In
either case, the dynamics are driven by the presence of discrepancies 01
or 10 in the configuration. In order to obtain well-defined processes, we
consider dynamics on configurations with finitely many discrepancies.
At each time step, the exclusion process selects uniformly at random
from among all discrepancies. If the chosen discrepancy is 01, it flips to 10
with probability p (else there is no change); if the pair is 10, it flips to 01
with probability 1−p. On the other hand, at each time step the voter model
selects uniformly at random from all discrepancies and then flips the chosen
pair to either 00 or 11, with equal chance of each. The exclusion-voter
process is a hybrid of these two processes, whereby at each time step we
determine independently at random whether to perform a voter-type move
(with probability β) or an exclusion-type move (probability 1 − β).
6.4. Exclusion and voter models
307
Individually, the exclusion process and voter model exhibit very different
behaviour. For instance, in the exclusion process there is local conservation
of 1s: the number of 1s in a bounded interval can change only through the
boundary. There is no such conservation in the voter model. In the hybrid
process, voter moves and exclusion moves interact in a highly non-trivial
way.
Consider the Heaviside configuration defined by 1{x ≤ 0}, which consists
of a single pair 10 abutted by infinite strings of 1s and 0s to the left and
right, respectively:
. . . 11110000 . . .
If the voter model starts from the Heaviside configuration, then at any future
time it is a random translate of the same configuration. Indeed, the position
of the rightmost particle performs a symmetric simple random walk. If the
voter model starts from a perturbation of the Heaviside configuration, it
is natural to study the time it takes to reach a translate of the Heaviside
configuration. This example motivates the following notation.
Let S 0 ⊂ {0, 1}Z denote the set of configurations with a finite number
of 0s to the left of the origin and 1s to the right, i.e., s0 ∈ {0, 1}Z for which
there exist `, r ∈ Z with ` < r such that s0 (x) = 1 for all x ≤ ` and s0 (x) = 0
for all x ≥ r. In other words, S 0 contains those configurations of {0, 1}Z
in which there is only a finite number of discrepancies, and the number of
discrepancies of type 10 minus the number of discrepancies of type 01 is
equal to 1; note that S 0 is countable.
Let ‘∼’ denote the equivalence relation on S 0 such that for s01 , s02 ∈ S 0 ,
0
s1 ∼ s02 if and only if s01 and s02 are translates of each other, i.e., there exists
y ∈ Z such that s01 (x) = s02 (x + y) for all x ∈ Z. Then set S := S 0 / ∼.
In other words, S is the set of configurations of the form infinite string of
1s—finite number of 0s and 1s—infinite string of 0s, modulo translations.
For example, one s ∈ S is
s = . . . 1110000000011100001001001000000001111000 . . .
(6.56)
We now formally define our processes. Fix β ∈ [0, 1] (the mixing parameter) and p ∈ [0, 1] (the exclusion parameter). The exclusion-voter process
(ξn , n ≥ 0) with parameters (β, p) is a time-homogeneous Markov chain
on the countable state-space S. The one-step transition probabilities are
determined by the following mechanism.
• At each time step we decide independently at random whether to
perform a voter move or an exclusion move. We choose a voter move
with probability β and an exclusion move with probability 1 − β.
308
Chapter 6. Further applications
• Having decided this, choose a discrepancy (i.e. 01 or 10) uniformly at
random from the finite number of available discrepancies.
• Then execute the chosen move. The voter move is such that the chosen
pair (01 or 10) flips to 00 or 11 each with probability 1/2. The exclusion move is such that a chosen pair 01 flips to 10 with probability p
(otherwise no move) and a chosen pair 10 flips to 01 with probability
q := 1 − p (otherwise no move).
The case β = 0 is the (pure) exclusion process with parameter p, which we
abbreviate as EP(p), the case β = 1 is the (pure) voter model, VM, and
the general case is the hybrid process, HP(β, p). We use Pep , Pv , and Phβ,p to
denote probabilities for EP(p), VM, and HP(β, p) respectively; we use Eep ,
Ev , and Ehβ,p for the corresponding expectations.
The following basic result gives some elementary properties of the statespace S under Phβ,p . In particular, Proposition 6.4.1 says that for (β, p) ∈
(0, 1)2 ξn is irreducible and aperiodic. Let sH ∈ S denote the equivalence
class of the Heaviside configuration 1{x ≤ 0}.
Proposition 6.4.1. For any β ∈ [0, 1], sH is an absorbing state under Phβ,1 .
Suppose β 6= 1 and (β, p) ∈
/ {(0, 0), (0, 1)}. Then all states in S \ {sH }
communicate under Phβ,p . Suppose β 6= 1, p < 1, and (β, p) 6= (0, 0). Then
all states in S communicate under Phβ,p , and ξn is irreducible and aperiodic.
Define the relaxation time for the process ξn as
τ := min{n ≥ 0 : ξn = sH }.
We introduce some convenient terminology. If Phβ,p [τ = ∞ | ξ0 = s] > 0 for
s ∈ S, we say that ξn is transient started from s; if Phβ,p [τ < ∞ | ξ0 = s] = 1
for s ∈ S, we say that ξ is recurrent started from s. In the latter case,
if in addition Ehβ,p [τ | ξ0 = s] < ∞ for s ∈ S, we say that ξ is positiverecurrent started from s. When ξ is irreducible (see Proposition 6.4.1), this
terminology coincides with the standard usage for countable Markov chains.
In the remaining part of this section, we prove a number of results.
Namely, we prove
• The exclusion process is positive recurrent for p > 1/2 and transient
for p ≤ 1/2 (in particular, there is no null-recurrence regime).
• The voter model is positive recurrent; moreover, all moments for τ
below 3/2 exist while all moment above 3/2 do not exist.
309
6.4. Exclusion and voter models
• The exclusion-voter process is positive recurrent for p ≥ 1/2 and any β,
or for β ≥ β0 and any p, where β0 is a constant sufficiently close to 1. In
particular, note that any proportion of voter moves (β > 0) added to
the transient symmetric exclusion process (p = 1/2) makes it positive
recurrent.
6.4.2
Configurations and exclusion-voter dynamics
We introduce some more notation and terminology. A 1-block (0-block) is a
maximal string of consecutive 1s (0s). Configurations in S consist of a finite
number of such blocks. (From this point onwards we describe elements of S
simply as ‘configurations’ rather than ‘equivalence classes of configurations’.)
For s ∈ S, let N = N (s) ≥ 0 denote the number of 1-blocks not including
the infinite 1-block to the left (this is the same as number of 0-blocks not
including the infinite 0-block to the right). Enumerating left to right, let
ni = ni (s) denote the size of the i-th 0-block, and mi = mi (s) the size of
the i-th 1-block: for example, for the configuration from (6.56),
n1
m1
n2
m2
n3
m3
n4
m4
n5
m5
z }| { z}|{ z}|{ z}|{ z}|{ z}|{ z}|{ z}|{ z }| { z}|{
s = . . . 111 00000000 111 0000 1 00 1 00 1 00000000 1111 000 . . .
We may thus represent configuration s ∈ S \{sH } by the vector of block sizes
(n1 , m1 , . . . , nN , mN ). For example, the configuration s of (6.56), which has
N (s) = 5, has the representation (8, 3, 4, 1, 2, 1, 2, 1, 8, 4).
For s ∈ S \ {sH } and i ∈ {1, . . . , N } let
Ri := Ri (s) :=
i
X
j=1
nj , and Ti := Ti (s) :=
N
X
mj ;
j=i
we adopt the
PNconvention R0 = TN +1 = 0. Set |sH | := 0 and for s ∈ S \ {sH }
let |s| := i=1 (ni + mi ) = RN + T1 , the length of the string of 0s and 1s
between the infinite string of 1s to the left and the infinite string of 0s to
the right.
It is convenient to represent a configuration s ∈ S \ {sH } diagrammatically as a right-down path in the quarter-lattice Z2+ : starting from (0, T1 ),
construct a path by reading left-to-right the configuration s and for each 0
(1) taking a unit step in the right (down) direction. Thus the path starts
with a step to the right, and ends at (RN , 0) after |s| steps. The lattice
squares of Z2+ bounded by the right-down path determined by s constitute
a polygonal region in the plane that we call the staircase corresponding to
s. See Figure 6.4 for a representation of the staircase for s given by (6.56).
310
Chapter 6. Further applications
n1
m1
T1
T2
Figure 6.4: An example staircase configuration s with N (s) = 5. The filled
square is absent from s but present in s→
2 , the configuration obtained after
performing the exclusion move 10 7→ 01 on the rightmost (in this case, only)
1 of the 2nd 1-block.
With this representation of the configuration-space, the exclusion-voter
model can be viewed as a growth/depletion process on staircases. For instance, exclusion moves are particularly simple in this context, corresponding to adding or removing a square at a corner.
We now introduce notation for the changes in configuration brought
about by voter and exclusion moves. Given the staircase of s, there are 2N +
1 ‘corners’ representing 10s and 01s alternately, of which N + 1 are 10s and
N are 01s. In the staircase representation, these corners have coordinates
(Ri , Ti+1 ), i ∈ {0, . . . , N } (for 10s) and (Ri , Ti ), i ∈ {1, . . . , N } (for 01s),
where R0 = TN +1 = 0. Enumerate the 10s left-to-right in the configuration
S by 0, 1, . . . , N , and similarly the 01s by 1, . . . , N .
Give a configuration s ∈ S, we define the configurations to which it can
←
transition under either a voter move or an exclusion move, denoted s→
k , sk ,
+r
+`
−r
−`
sk , sk , sk , sk , as follows:
• s→
k is the configuration obtained from s by moving the rightmost 1 of
the kth 1-block by 1 unit to the right, k ∈ {0, . . . , N };
• s←
k is the configuration obtained from S by moving the leftmost 1 of
the kth 1-block by 1 unit to the left, k ∈ {1, . . . , N };
• s+r
k is the configuration obtained from S by adding an extra 1 to the
right of the kth 1-block, k ∈ {0, . . . , N };
6.4. Exclusion and voter models
311
• s+`
k is the configuration obtained from S by adding an extra 1 to the
left of the kth 1-block, k ∈ {1, . . . , N };
• s−r
k is the configuration obtained from S by removing the rightmost 1
from the kth 1-block, k ∈ {0, . . . , N };
• s−`
k is the configuration obtained from S by removing the leftmost 1
from the kth 1-block, k ∈ {1, . . . , N }.
←
With this notation, EP(p) can transform s to s→
k or sk , while using VM we
±r
±`
can get sk or sk . Note that in the above, the 0th 1-block is the infinite
block of 1s. See Figure 6.4 for a representation of one possible move as an
operation on staircases.
To conclude this section, we sketch the (elementary) proof of Proposition 6.4.1. As well as sH , we introduce special notation for one more configuration. Set
sD := . . . 11101000 . . . ,
the single-discrepancy configuration with N (s1 ) = 1 and vector representation (1, 1).
Proof of Proposition 6.4.1. It is not hard to see that sH is an absorbing state
for the pure voter model (β = 1) and for the left-moving totally asymmetric
exclusion process (β = 0, p = 1), and hence also for the mixture model under
Phβ,1 for any β ∈ [0, 1].
To show that all states within S communicate, it suffices to show that
Phβ,p [ξn = s1 | ξ0 = s0 ] > 0 for some n ∈ N for each of the following:
(i) s0 = sH , s1 = sD ;
(ii) s0 = sD , s1 = sH ;
(iii) any s0 with |s0 | ≥ 2 and some s1 with |s1 | = |s0 | + 1;
(iv) any s0 with |s0 | ≥ 3 and some s1 with |s1 | ≤ |s0 | − 1;
(v) any s0 with |s0 | ≥ 3 and any s1 where s1 is identical to s0 apart from
in a single position j ∈ {2, 3, . . . , |s0 | − 1}.
In other words, given that moves of types (i)–(v) can occur, it is possible
(with positive probability) to step, in a finite number of moves, between any
two configurations in S by first adjusting the length of the configuration via
moves of types (i)–(iv) and then flipping the states in the interior of the
configuration via moves of type (v). Similarly, to show that all states in
312
Chapter 6. Further applications
S \ {sH } communicate, it suffices to show that all moves of types (iii)–(v)
have positive probability.
It is not hard to see that voter moves can perform moves of types (ii), (iii)
and (iv) in a single step (i.e., with n = 1). Similarly exclusion moves with
p < 1 can perform moves of types (i) and (iii) in one step, while exclusion
moves with p > 0 can perform moves of types (ii) and (iv), possibly needing
multiple steps. We claim that moves of type (v) can be performed provided:
(a) β ∈ (0, 1); or (b) β = 0 and p ∈ (0, 1).
In case (a), suppose we need to replace a 0 by a 1 in the interior of a given
configuration. If p < 1, we may perform a voter move on the first 10 to the
left of the position to be changed, and then, if necessary, perform successive
10 7→ 01 exclusion moves to ‘step’ the 1 into the desired position. If p > 0,
an analogous procedure works, starting from the first 01 to the right. On
the other hand, if we need to replace a 1 by a 0, a similar argument applies.
In case (b) we cannot use voter moves, but both types of exclusion move
are permitted, so we can ‘bring in’ any 0 (1) from outside the disordered
region, rearrange as necessary, and ‘take out’ the excess 1 (0) to the other
boundary.
It follows that moves of type (ii)–(v) are possible provided β 6= 1 and
(β, p) ∈
/ {(0, 0), (0, 1)}, and all (i)–(v) are possible if we additionally impose
p < 1.
To complete the proof we need to demonstrate aperiodicity in the case
where β 6= 1, p < 1 and (β, p) 6= (0, 0), where all states communicate. Since
β 6= 1, exclusion moves may occur. Moreover, every configuration other
than sH contains at least one pair of each type (01 and 10). Hence there is a
positive probability that a configuration other than sH remains unchanged
at a given step (when a proposed exclusion move fails to occur). Thus, since
all states communicate, we have aperiodicity.
6.4.3
Lyapunov functions
We define two Lyapunov functions, f1 and f2 , from S to R+ that will play a
crucial role in our arguments. Set f1 (sH ) = f2 (sH ) = 0, and for s ∈ S \ {sH }
define
!
N
N
N
N
X
X
X
1 X
f1 (s) :=
mi Ri +
ni Ti =
mi Ri =
ni Ti ;
2
i=1
i=1
i=1
i=1
!
N
N
X
1 X
2
2
and f2 (s) :=
mi Ri +
ni Ti .
2
i=1
i=1
313
6.4. Exclusion and voter models
Note that f1 is the area under the ‘staircase’ representation of the configuration (see Figure 6.4).
The value f1 (s) is equal exactly to the number of nearestneighbour transpositions needed to pass from s to sH , that
is, f1 (s) is in some sense the ‘distance’ from s to the trivial
configuration. Unfortunately, as we will see later, the function f1 does not work well for some configurations s (namely,
for s such that N (s) is small with respect to |s|). The function f2 is the result of our attempts to modify f1 in order
to eliminate this disadvantage; we cannot give any intuitive
meaning of f2 (s).
i
The next result gives some basic relations between |s|, f1 (s), and f2 (s).
Lemma 6.4.2. For any s ∈ S we have:
(i)
1
2 |s|
(ii)
1
2
2 |s|
≤ f1 (s) ≤ 14 |s|2 ;
≤ f2 (s) ≤ 81 |s|3 ;
(iii) f1 (s) ≤ (f2 (s))3/4 .
Proof. For part (i), we have
N
f1 (s) =
f1 (s) =
N
1X
1X
1
|s|
mi Ri +
ni Ti ≥ (RN + T1 ) =
; and
2
2
2
2
i=1
N
X
i=1
N
X
i=1
i=1
mi Ri ≤ RN
mi = RN T1 ≤
|s|2
(RN + T1 )2
=
.
4
4
Similarly, for part (ii), we have
1 2
1
|s|2
f2 (s) ≥ (RN
+ T12 ) ≥ (RN + T1 )2 =
; and
2
4
4
N
N
1 2 X
1 X
1
|s|3
f2 (s) ≤ RN
mi + T12
ni = RN T1 (RN + T1 ) ≤
.
2
2
2
8
i=1
i=1
Finally we prove part (iii). We shall make use of the following simple consequence of
PJensen’s inequality: if we have n positive numbers γ1 , . . . , γn
such that ni=1 γi = 1, then for any x1 , . . . , xn ,
γ1 x1 + · · · + γn xn ≤ γ1 x21 + · · · + γn x2n
1/2
.
(6.57)
314
Chapter 6. Further applications
Denote αi = mi /|s|, βi = ni /|s|, so
part (ii), we get
PN
i=1 (αi
N
+ βi ) = 1. Using (6.57) and
N
1X
|s| X
f1 (s) =
(mi Ri + ni Ti ) =
(αi Ri + βi Ti )
2
2
i=1
i=1
!1/2 p
N
1/2
|s|
|s| X
≤
(αi Ri2 + βi Ti2 )
= √ f2 (s)
2
2
i=1
√
1/4
1/2
3/4
2 f2 (s)
√
≤
f2 (s)
= f2 (s)
,
2
thus completing the proof of Lemma 6.4.2.
We will need expressions for the expected increments of our Lyapunov
functions f1 (ξn ) and f2 (ξn ). First we deal with f1 .
Lemma 6.4.3. Let β, p ∈ [0, 1] and s ∈ S \ {sH }. Then
Ehβ,p [f1 (ξn+1 ) − f1 (ξn ) | ξn = s] = (1 − β)(1 − p) −
N
.
2N + 1
Proof. Let s ∈ S \ {sH }. As noted in Section 6.4.2, EP can transform a
←
configuration s only either to s→
k or to sk , and we have
→
f1 (s←
k ) − f1 (s) = −1, and f1 (sk ) − f1 (s) = 1.
(6.58)
Thus, summing over the discrepancies where an exclusion move can occur,
Eep [f1 (ξn+1 ) − f1 (ξn ) | ξn = s]
=
N
N
k=1
k=0
X
X
p
q
(f1 (s←
(f1 (s→
k ) − f1 (s)) +
k ) − f1 (s))
2N + 1
2N + 1
N (q − p) + q
=
.
2N + 1
(6.59)
±r
On the other hand, VM can transform s to one of s±`
k or sk . Direct computations yield
f1 (s+`
k ) − f1 (s) = Rk − Tk − 1,
f1 (s−`
k ) − f1 (s) = −Rk + Tk − 1,
f1 (s+r
k ) − f1 (s) = Rk − Tk+1 ,
315
6.4. Exclusion and voter models
f1 (s+r
k ) − f1 (s) = −Rk + Tk+1 .
±r
Pairing off the terms s±`
k and the terms sk , it follows that
Ev [f1 (ξn+1 ) − f1 (ξn ) | ξn = s] =
N
N
k=1
k=0
X1
X1
1
1
(−2) +
(0)
2N + 1
2
2N + 1
2
=−
N
.
2N + 1
(6.60)
In the hybrid process, a voter transition occurs with probability β, and an
exclusion transition occurs with probability 1 − β; then combining (6.59)
and (6.60) we obtain the result after some algebra.
Next we deal with f2 .
Lemma 6.4.4. Let β, p ∈ [0, 1] and s ∈ S \ {sH }. Then
Ehβ,p [f2 (ξn+1 )−f2 (ξn )
| ξn = s] = (1−β)
!
N
N +q
2p − 1 X
−
(Ri + Ti ) .
2N + 1 2N + 1
i=1
Proof. Let s ∈ S \ {sH }. The possible changes in f2 due to the EP are
1
2
(Rk + 1)2 − Rk2 + (Tk+1 + 1)2 − Tk+1
2
= 1 + Rk + Tk+1 ,
(6.61)
1
and f2 (s←
(Rk − 1)2 − Rk2 + (Tk − 1)2 − Tk2
k ) − f2 (s) =
2
= 1 − Rk − Tk .
(6.62)
f2 (s→
k ) − f2 (s) =
Combining (6.61) and (6.62), we obtain
Eep [f2 (ξn+1 ) − f2 (ξn ) | ξn = s]
N
N
X
X
p
q
=
(f2 (s←
(f2 (s→
k ) − f2 (s)) +
k ) − f2 (s))
2N + 1
2N + 1
k=1
=
N +q
p−q
−
2N + 1 2N + 1
k=0
N
X
(Ri + Ti ).
(6.63)
i=1
On the other hand, the possible changes in f2 due to the VM are
N
k
i=k+1
i=1
X
X
1
2
2
f2 (s+r
mi Ri +
ni Ti ; (6.64)
k ) − f2 (s) = 2 (Rk + Tk+1 + Rk − Tk+1 ) −
316
Chapter 6. Further applications
N
k
i=k+1
i=1
N
k
i=k
N
X
i=1
X
X
1
2
2
f2 (s−r
)
−
f
(s)
=
(R
+
T
−
R
+
T
)
+
m
R
−
ni Ti ; (6.65)
2
i
i
k
k+1
k
k+1
k
2
for k ∈ {0, . . . , N }, and, for k ∈ {1, . . . , N },
f2 (s+`
k )
X
X
1
mi Ri +
− f2 (s) = (−Rk − Tk + Rk2 − Tk2 ) −
ni Ti ; (6.66)
2
1
2
2
f2 (s−`
k ) − f2 (s) = 2 (−Rk − Tk − Rk + Tk ) +
i=k
mi Ri −
k
X
ni Ti . (6.67)
i=1
Summing up the contributions from (6.64)–(6.67) we obtain
Ev [f2 (ξn+1 ) − f2 (ξn ) | ξn = s] = 0.
(6.68)
Combining (6.63) and (6.68) gives the result.
6.4.4
Exclusion process
Our first result on the exclusion process is the following.
Theorem 6.4.5. If p > 1/2, then EP(p) is positive recurrent.
Proof. We use the Lyapunov function f2 . Since RN + T1 = |s|, Ri ≥ i and
Ti ≥ N − i + 1, it is straightforward to get that
N
X
i=1
(Ri + Ti ) ≥ max{|s|, N (N + 1)}.
Using this fact, we get from (6.63) that
Eep [f2 (ξn+1 ) − f2 (ξn ) | ξn = s] ≤
max{|s|, N (N + 1)}
N +1
− (2p − 1)
.
2N + 1
2N + 1
Let p > 1/2. If (2p − 1)|s| > 2N + 2, we have
Eep [f2 (ξn+1 ) − f2 (ξn ) | ξn = s] ≤ −
N +1
1
≤− ,
2N + 1
2
and for (2p − 1)N > 3, we have
Eep [f2 (ξn+1 ) − f2 (ξn ) | ξn = s] ≤ 1 −
3(N + 1)
1
≤− .
2N + 1
2
317
6.4. Exclusion and voter models
The set of s for which both (2p − 1)N ≤ 3 and (2p − 1)|s| ≤ 2N + 2 is finite,
so we have shown that
1
Eep [f2 (ξn+1 ) − f2 (ξn ) | ξn = s] ≤ − ,
2
for all but finitely many s ∈ S, and hence Theorem 2.6.2 shows that EP(p)
is positive recurrent when p > 1/2.
Theorem 6.4.6. If p ≤ 1/2, then EP(p) is transient.
Proof. First we consider the case p < 1/2, where we will use the Lyapunov
function f1 . In this case, we have from (6.58) that |f1 (ξn+1 ) − f1 (ξn )| ≤ 1,
Pep -a.s., and, from (6.59), for some ε = ε(p) > 0 and all s ∈ S \ {sH },
Eep [f1 (ξn+1 ) − f1 (ξn ) | ξn = s] ≥
N (1 − 2p)
≥ ε.
2N + 1
Then by Theorem 2.5.14, the process ξn is transient.
We now turn now to the more delicate case where p = q = 1/2. In this
case we have from (6.59) that
Ee1/2 [f1 (ξn+1 ) − f1 (ξn ) | ξn = s] =
1
,
2(2N + 1)
(6.69)
and so, since the right-hand side of (6.69) can be arbitrarily small, we cannot
apply Theorem 2.5.14. As observed in Section 2.5, this theorem provides
only a sufficient condition for transience, so there is no guarantee that it
should be applicable in all situations. Therefore, to deal with the case
p = 1/2, we seek to apply the more general result, Theorem 2.5.8.
We fix an arbitrary α > 0 and define the function ψ : S \ {sH } → R+ by
−α
ψ(s) := f1 (s)
.
Note that the definition is correct because f1 (s) > 0 for s 6= sH . (Actually,
for our proof of Theorem 6.4.6 it is sufficient to take α = 1, but, since we
will need analogous calculations for the hybrid process in Section 6.4.6, at
this point we prefer to do the calculations for arbitrary α > 0.)
For any C > 0, define the set AC ⊂ S by
AC := {s ∈ S : f1 (s) ≤ CN (s)}.
We claim that AC is finite for any C > 0. Indeed, Ri ≥ i and mi ≥ 1,
so f1 (s) ≥ N (N + 1)/2. Thus, for a configuration to belong to AC , it is
necessary that the number of 1-blocks be less than 2C − 1, so
AC ⊆ {s ∈ S : f1 (s) ≤ C(2C − 1)},
318
Chapter 6. Further applications
which is clearly finite (by e.g. Lemma 6.4.2(i)).
Now we have from (6.58) that
Ee1/2 [(f1 (ξn+1 ) − f1 (ξn ))2 | ξn = s]
1/2
=
2N + 1
N
X
(f1 (s←
k )
k=1
2
− f1 (s)) +
N
X
!
(f1 (s→
k )
k=0
2
− f1 (s))
1
= . (6.70)
2
By elementary calculus, for any α > 0 there exist positive constants C1 =
C1 (α) and C2 = C2 (α) such that for all |x| < C2 ,
(1 + x)−α − 1 ≤ −αx + C1 x2 .
Hence, using (6.69) and (6.70), we get
Ee1/2 [ψ(ξn+1 ) − ψ(ξn ) | ξn = s]
f1 (ξn+1 ) − f1 (ξn ) −α
−α e
− 1 ξn = s
= (f1 (s)) E1/2 1 +
f1 (ξn )
α
1
C1
≤ (f1 (s))−α −
·
+
,
f1 (s) 2(2N + 1) 2(f1 (s))2
provided s is such that f1 (s) > 1/C2 (where we have used the fact that
|f1 (ξn+1 ) − f1 (ξn )| ≤ 1). Thus we have
αf1 (s)
C1 , (6.71)
Ee1/2 [ψ(ξn+1 ) − ψ(ξn ) | ξn = s] = (f1 (s))−α−2 −
+
2(2N + 1)
2
which is negative for all s with f1 (s) > 1/C2 and f1 (s) > C1 (2N (s) + 1)/α.
The complementary set, with f1 (s) ≤ 1/C2 or f1 (s) ≤ C1 (2N (s) + 1)/α is
the union of two finite sets, using the fact that AC is finite. Thus (6.71)
holds for all but finitely many s ∈ S. Theorem 2.5.8 finishes the proof.
6.4.5
Voter model
Our first result on the voter model in the following.
Theorem 6.4.7. The VM is positive recurrent. Moreover, for any ε > 0
and any initial configuration s ∈ S,
Ev [τ (3/2)−ε | ξ0 = s] < ∞.
(6.72)
6.4. Exclusion and voter models
319
Proof. In the terminology introduced in Section 6.4.1, positive recurrence is
the existence of E τ ; thus we shall turn directly to the proof of (6.72). The
idea is to apply Theorem 2.7.1 to the process (f2 (ξn ))α for some α < 1.
First, elementary calculus gives us that for α ∈ (0, 1) there exists a
positive constant c1 = c1 (α) such that, for all |x| ≤ 1,
(1 + x)α − 1 ≤ αx − c1 x2 .
(6.73)
Hence using (6.73) we obtain
Ev [(f2 (ξn+1 ))α − (f2 (ξn ))α | ξn = s]
f2 (ξn+1 ) − f2 (ξn ) α
α v
= (f2 (s)) E
1+
− 1 ξn = s
f2 (ξn )
≤ −c1 (f2 (s))α−2 Ev [(f2 (ξn+1 ) − f2 (ξn ))2 | ξn = s],
(6.74)
by the fact from (6.68) that f2 (ξn ) is a martingale under Pv .
To obtain a lower bound for the final expectation in (6.74), observe
from (6.64) and (6.65) that
2
RN
T12
)
−
f
(s)|
≥
, and |f2 (s+r
.
2
N
2
2
It follows from these two inequalities that
|f2 (s−r
0 ) − f2 (s)| ≥
Ev [(f2 (ξn+1 ) − f2 (ξn ))2 | ξn = s]
1
1
2
2
≥
(f2 (s−r
(f2 (s−r
0 ) − f2 (s)) +
N ) − f2 (s))
4N + 2
4N + 2
4
T 4 + RN
c|s|4
≥ 1
≥
,
16N + 8
N
(6.75)
for all s ∈ S \ {sH }, where c > 0 is a constant. Now, a very important
observation is that the VM does not increase the number of blocks N (ξn ).
So we have from (6.75) that
Ev [(f2 (ξn+1 ) − f2 (ξn ))2 | ξn = s] ≥ c0 |s|4 , for all s ∈ S \ {sH },
(6.76)
with c0 = c0 (ξ0 ) = c/N (ξ0 ). Using (6.76) in (6.74), we obtain
Ev [(f2 (ξn+1 ))α − (f2 (ξn ))α | ξn = s] ≤ −c0 c1 (f2 (s))α−2 |s|4 .
(6.77)
Applying Lemma 6.4.2(ii) it follows from (6.77) that
Ev [(f2 (ξn+1 ))α − (f2 (ξn ))α | ξn = s] ≤ −84/3 c0 c1 (f2 (s))α−(2/3) .
We apply Corollary 2.7.3 to the process Xn = (f2 (ξn ))α taking α to be close
to 1 to finish the proof of (6.72).
320
Chapter 6. Further applications
The following theorem complements Theorem 6.4.7 by providing a result
in the other direction.
Theorem 6.4.8. For any ε > 0 and any s ∈ S \ {sH },
Ev [τ 3/2 | ξ0 = s] = ∞.
(6.78)
Proof. For any initial configuration ξ0 = s ∈ S \{sH }, since N (ξn+1 )−N (ξn )
is either 0 or −1 under the VM dynamics, to reach sH the process must pass
through a configuration s0 with N (s0 ) = 1. Thus it suffices to prove (6.78)
for initial configurations of this type.
Representing the process started with N (ξ0 ) = 1 by the vector of block
sizes, we obtain a random walk on Z2+ , for which τ is the time to reach
the boundary of the quarter-plane. Formally, write Xn for the size of the
finite 0-block in ξn , and Yn for the size of the finite 1-block, and set Fn =
σ(ξ0 , . . . , ξn ). Then (Xn , Yn ) is a nearest-neighbour random walk on Z2+
with transition probabilities as follows: from the state (x, y) with xy 6= 0,
the transition can occur to each of the six states (x+1, y), (x−1, y), (x, y+1),
(x, y − 1), (x + 1, y − 1) and (x − 1, y + 1) with probability 1/6. We stop
the random walk if xy = 0 (at this point the coupling breaks down, because
whereas the random walk can end up at any point on the boundary, the
voter model must end up at (0, 0)). In any case, the relaxation time is now
τ = min{n ≥ 0 : Xn Yn = 0}, the first hitting time of the boundary of Z2+ ;
we show that E[τ 3/2 ] = ∞.
It suffices to suppose that τ < ∞, a.s. Let Zn = Xn2 Yn + Xn Yn2 . Then
Zn = f2 (ξn ) on {n < τ }. In particular, Zn∧τ is a martingale, by (6.68), or
as can be verified by a calculation for the random walk.
Our argument is a variation on Example 2.6.6. Consider the Doob decomposition Xn3 = Mn + An , where Mn is a martingale with M0 = 0 and
An =
n−1
X
m=1
3
3
E[Xm+1
− Xm
| Fm ]
3
= E[(Xm+1 − Xm ) | Fm ] + 3
=2
n−1
X
m=1
n−1
X
m=1
Xm E[(Xm+1 − Xm )2 | Fm ]
Xm , on {n < τ }.
Hence
E An∧τ ≤ 2τ max Xm∧τ .
0≤m≤n
321
6.4. Exclusion and voter models
Suppose, for the purpose of deriving a contradiction, that E[τ 3/2 ] < ∞.
Then, by Hölder’s inequality,
E An∧τ ≤ 2 E[τ
3/2
i1/3
2/3 h
3
E max Xm∧τ
.
]
0≤m≤n
(6.79)
The quadratic variation associated with the martingale Xn satisfies
hXin ≤
n−1
X
m=0
2
E[(Xm+1 − Xm )2 | Fm ] = n.
3
It then follows from Burkholder’s inequality (see e.g. [286, p. 499]) that
h
i
3
E max Xm∧τ
≤ C E[hXiτ3/2 ] ≤ C E[τ 3/2 ],
(6.80)
0≤m≤n
for some constant C ∈ R+ . Thus if E[τ 3/2 ] < ∞ we have from (6.79)
3 ] = EA
and (6.80) that E An∧τ ≤ C for some C ∈ R+ . Since E[Xn∧τ
n∧τ ,
it follows that the martingale Xn∧τ is uniformly bounded in L3 , and hence
converges a.s. and in L3 to Xτ (see e.g. Theorem 5.4.5 of [83]). A similar
argument applies to Yn∧τ . Hence the martingale Zn∧τ converges a.s. and in
L1 . In particular,
0 < E Z0 = E Zτ = 0,
which is the desired contradiction. So E[τ 3/2 ] = ∞.
6.4.6
Hybrid process
Finally, we turn to the hybrid process that allows both exclusion and voter
moves. The complete recurrence classification here is an open problem (see
the bibliographical notes at the end of this chapter). The next result shows
that if voter moves are sufficiently prominent, one has positive recurrence:
in particular, HP(β, p) is positive recurrent for β > 2/3 and any p.
Theorem 6.4.9. If β, p ∈ [0, 1] are such that (1 − p)(1 − β) < 1/3, then
HP(β, p) is positive recurrent.
Proof. We have from Lemma 6.4.3 that for all s ∈ S \ {sH },
1
Ehβ,p [f1 (ξn+1 ) − f1 (ξn ) | ξn = s] ≤ (1 − β)(1 − p) − ;
3
provided (1 − p)(1 − β) < 1/3 this last expression is strictly negative, and
so applying Theorem 2.6.2 we finish the proof.
322
Chapter 6. Further applications
The next result is not surprising when p > 1/2, since EP(p) and VM
are (separately) positive recurrent in that case, but recall that EP(1/2) is
transient.
Theorem 6.4.10. For any β > 0 and p ≥ 1/2 the process HP(β, 1/2) is
positive recurrent.
To prove Theorem 6.4.10, we will apply Theorem 2.6.4 with the function
(f2 (s))α for some α < 1. This entails a computation similar to that in the
proof of Theorem 6.4.7 on the positive recurrence of the VM. In the proof of
Theorem 6.4.7 we used the fact that the number of blocks cannot increase
under the VM dynamics to obtain from (6.75) the bound (6.76). For the
hybrid process, this latter bound is not valid; instead, we have the following.
Lemma 6.4.11. Let β > 0. Then there exists c > 0 such that for all s ∈ S,
Ehβ,p [(f2 (ξn+1 ) − f2 (ξn ))2 | ξn = s] ≥ c|s|16/5 .
(6.81)
Proof. Since β > 0, there is positive probability of executing a voter move.
Similarly to (6.75), writing ∆k = f2 (s+r
k ) − f2 (s), we thus have
N
Ehβ,p [(f2 (ξn+1 )
X
β
− f2 (ξn )) | ξn = s] ≥
∆2k .
4N + 2
2
(6.82)
k=1
By simple algebraic calculations, one gets from (6.64) that
+r
∆k+1 − ∆k = f2 (s+r
k+1 ) − f2 (sk ) ≥ mk+1 Rk+1 + nk+1 Tk+1 ≥ N,
(6.83)
for k ∈ {0, . . . , N − 1}. From (6.64) one gets also that ∆0 < 0 and ∆N > 0,
so denoting L = min{k ≥ 0 : ∆k ≥ 0} we have 1 < L ≤ N , ∆k < 0 for
k < L, and ∆k ≥ 0 for k ≥ L. Then using (6.83), we get
N
X
k=1
∆2k ≥
L−1
X
k=0
= N2
(N (k − L + 1))2 +
L−1
X
k=0
k2 + N 2
N
−L
X
k=0
N
X
k=L
(N (k − L))2
k 2 ≥ c1 N 5 ,
for some c1 > 0. It then follows from (6.82) that
Ehβ,p [(f2 (ξn+1 ) − f2 (ξn ))2 | ξn = s] ≥ c2 N 4 ,
323
6.4. Exclusion and voter models
for some c2 > 0. Combining this with (6.75), we get
n
|s|4 o
Ehβ,p [(f2 (ξn+1 ) − f2 (ξn ))2 | ξn = s] ≥ c3 max N 4 ,
≥ c3 |s|16/5 ,
N
thus proving the result.
Remark 6.4.12. The exponent 16/5 in Lemma 6.4.11 is the best possible;
to see this, one may take a configuration s with n1 = mN = N 5/4 and
n2 = · · · = nN = m1 = · · · = mN −1 = 1 and compute the left-hand side
of (6.81).
Proof of Theorem 6.4.10. It suffices to take β ∈ (0, 1), since we already
know from Theorem 6.4.7 that VM is positive recurrent. First suppose that
p > 1/2. Then, by Lemma 6.4.4,
Ehβ,p [f2 (ξn+1 ) − f2 (ξn ) | ξn = s] = (1 − β)Eep [f2 (ξn+1 ) − f2 (ξn ) | ξn = s],
which is bounded above by −(1 − β)/2 for all but finitely many s ∈ S, as
shown in the proof of Theorem 6.4.5. Thus positive recurrence follows from
Theorem 2.6.2.
Finally, we consider the more delicate case where p = 1/2 and β ∈ (0, 1).
This time, by the case p = q = 1/2 of Lemma 6.4.4,
1−β
.
2
Let α ∈ (0, 1). Similarly to (6.74) we obtain from (6.73) that
Ehβ,1/2 [f2 (ξn+1 ) − f2 (ξn ) | ξn = s] =
(6.84)
Ehβ,1/2 [(f2 (ξn+1 ))α − (f2 (ξn ))α | ξn = s]
f2 (ξn+1 ) − f2 (ξn ) α
α h
= (f2 (s)) Eβ,1/2 1 +
− 1 ξn = s
f2 (ξn )
α(1 − β)
≤
(f2 (s))α−1 − c(f2 (s))α−2 |s|16/5 ,
2
for some c > 0, where we have used (6.84) and (6.81). Since |s|16/5 ≥
216/5 (f2 (s))16/15 by Lemma 6.4.2(ii), we obtain, for 14/15 < α < 1,
Ehβ,1/2 [(f2 (ξn+1 ))α − (f2 (ξn ))α | ξn = s]
α(1 − β)
(f2 (s))α−1 − 216/5 c(f2 (s))α−(14/15)
2
h
i
α(1 − β)
α−(14/15) 16/5
−1/15
= −(f2 (s))
2
c−
(f2 (s))
< −1,
2
for all but finitely many s ∈ S, as follows from Lemma 6.4.2(ii). Applying
Theorem 2.6.2, we finish the proof of Theorem 6.4.10.
≤
324
Chapter 6. Further applications
Bibliographical notes
Section 6.1
Random walk in random environment (RWRE) was first considered by
M.V. Kozlov [183] (following up on “a hypothesis by A. N. Kolmogorov”)
and F. Solomon [291] (a student of F.L. Spitzer); subsequently these prototypical models of random processes in random media have been investigated
by many authors, see e.g. [269, 319, 320, 300].
The recurrence classification, Theorem 6.1.1, is due to Solomon [291] in
the case of RWRE on Z. Many of the papers on one-dimensional RWRE
work on Z rather than Z+ . Obvious differences include the fact that on Z
one may have Xn → −∞, while in the i.i.d. case this regime corresponds
to the positive-recurrent regime for Xn on Z+ . Taking such differences into
account, for many questions of interest the distinction is inessential, and the
analysis is more-or-less unchanged.
The null-recurrent case in which E[ζ02 ] < ∞ and E ζ0 = 0 is known as
Sinai’s regime after Sinai’s remarkable result [288] that (log n)−2 Xn converges weakly to a non-degenerate limit under the annealed probability
measure P◦ given by
Z
P◦ [ · ] =
Ω
Pω [ · ] P[dω].
Golosov [119] and Kesten [164] identified the limiting distribution in Sinai’s
result.
The almost-sure upper bound in Theorem 6.1.2 is due to Deheuvels
and Révész (Theorem 4 of [69]), with a different proof. The (shorter) approach presented here follows [58, 235]. The almost-sure lower bound in
Theorem 6.1.6 improves on a result of [235] which in turn improved on Theorem 4 of [69]. The proof of Theorem 6.1.6 given here uses the method
of [136], which improved on the method of [235]; the key Lemma 6.1.8 is
Lemma 4.3 in [136].
Theorems 6.1.2 and 6.1.6 are not optimal; sharp results on the almostsure behaviour of the RWRE in Sinai’s regime were provided by Hu and
Shi [138], making use of some delicate technical estimates involving the
potential associated with the random environment:
lim sup
n→∞
Xn
2
(log n) log log log n
=
8
π2σ2
, Pω -a.s. for P -a.e. ω.
Thus the lower bound in Theorem 6.1.6 is close to sharp, while the upper bound in Theorem 6.1.2 is rougher. The Lyapunov function methods
325
Bibliographical notes
presented here are not as sharp as those say of [138], but they are simpler
and more robust, and can be employed in more general settings where the
random environment is not i.i.d., such as the model of Section 6.1.5.
The model and results of Section 6.1.5 are based on [234, 235]. The recurrence classification for the perturbation of Sinai’s regime, Theorem 6.1.10,
is contained in Theorems 6 and 7 of [234], which dealt with more general
perturbations as well as a random environment constructed by a random
perturbation of symmetric simple random walk. The almost-sure bounds
in Theorem 6.1.11 are from Theorem 3 of [235]. In the setting of Theorem 6.1.11, using a more delicate analysis, [113] obtained the precise result
that for some constant c depending on a, β, and E[ζ02 ],
lim
n→∞
Xn
1/β
(log n) (log log n)−1/β
= c, Pω -a.s. for P -a.e. ω.
Thus it is the lower bound in Theorem 6.1.11 that is closest to being sharp.
Different behaviour is observed when E[ζ02 ] = ∞. The natural model
in this heavy-tailed setting takes ζ0 to be in the domain of attraction of a
stable law. Singh [289] gave analogues of the almost-sure results of Hu and
Shi [138] in the stable law setting. Analogues of Sinai’s weak convergence
result were obtained by Schumacher [279, 280] and Kawazu et al. [154]: now
(log n)−α Xn has an annealed weak limit, where α ∈ (0, 2] is the index of the
stable law. In the heavy-tailed setting, Lyapunov function methods were
used to study some perturbations of the i.i.d. environment in [136].
Several techniques are available for studying nearest-neighbour RWRE
on Z or Z+ ; since it is reversible, explicit calculations are often possible.
In this setting the Lyapunov function method often boils down to performing the ‘classical’ calculations. However, as emphasised elsewhere in this
book, the Lyapunov function method is particularly useful in non-reversible
situations, such as in the context of random strings in random environment
discussed in Section 6.2, or for many-dimensional random walks with rare
defects [231].
Section 6.2
The presentation and results in this section follow [58]. Gajrat, Malyshev,
Menshikov and Pelikh in [110] studied a version of the model in which the
` and p` do not depend on `, but the maximal jump size for
quantities ri` , qij
ij
the string length may be greater than one. All these models may be interpreted in terms of LIFO (last in, first out) queueing systems, or as random
walks on trees: see [110]. We consider here only the one-sided evolution of
326
Chapter 6. Further applications
the string; two-sided homogeneous strings were studied by Gajrat, Malyshev
and Zamyatin in [111].
In addition to Theorem 6.2.3, [58] gave a second sufficient condition for
transience: if (N) holds for An , and E log(1/pii ) < ∞ for all i ∈ A, then
γ1 > 0 implies transience.
Section 6.3
Billiards models are dynamical systems describing the motion of a particle in
a region with reflection rules at the boundaries. Physical motivation originates with dynamics of an ideal gas in the low-density (Knudsen) regime [179].
The random reflection rule in stochastic billiards models is motivated by
the fact that the particle is small and the surface off which it reflects has a
complicated (rough) microscopic structure. Stochastic billiards models were
studied in [56, 90, 232, 57]; we refer to those papers for further references.
The results of this section on stochastic billiards in unbounded planar
domains are based on [232]. In [232], in addition to recurrence and transience, almost-sure bounds on the trajectories of the collisions process and
the associated continuous-time trajectory were obtained; in [232] it was assumed that the distribution of α be symmetric around 0, which is stronger
than our condition E tan α = 0.
As mentioned in [232], it would be of interest to relax the assumption
that α is bounded away from ±π/2; there are several natural distributions
that are supported on the full interval [56, 90, 179]. We expect that if α has
distribution on (−π/2, π/2) with sufficiently light tails at the endpoints (such
that in particular E[tan2 α] < ∞, say), then the results of Section 6.3 should
carry through (technically, one would obtain a Lamperti process whose increments were not uniformly bounded but satisfied a moments condition
that should still admit the methods of Chapter 3).
Open problem 6.4.13. How does the stochastic billiards model behave
when E[tan2 α] = ∞? This is the case, for example, when α has the natural
Lambertian or cosine law of reflection, with a density proportional to cos α
on (−π/2, π/2). We conjecture that the process is transient if and only if
γ > 1/2 in this case.
Section 6.4
The results of Section 6.4 are based on [17], while some of the exposition
is based on [210]. For background on the exclusion process and the voter
model, and interacting particle systems in general, see the books [198, 199].
Bibliographical notes
327
Liggett [197] described the set of invariant measures for the exclusion
process on Z. If p = 1/2, the invariant measures are convex combinations
of the translation-invariant product measures parametrized by the density
of particles. If p > 1/2, the set of invariant measures contains also measures with support in the set of configurations with a finite number of empty
sites to the left of the origin and a finite number of particles to the right
of it; this fact can be used to establish positive-recurrence when p > 1/2
(Theorem 6.4.5). Such measures are sometimes called blocking measures because, due to the exclusion rule and the accumulation of particles to the
left of the origin, the flux of particles is null. There are also blocking measures for p < 1/2; they are obtained by the reflection of those mentioned
above. When an asymmetric exclusion process (p 6= 1/2) is considered from
a random position determined by a so-called second-class particle, a new
set of invariant measures arises. They are called shock measures, they have
support on configurations with different asymptotic densities to the left and
right of the origin; see [102, 100, 75]. See the review paper [101] and the
book [199] for an account of properties of these measures and the asymptotic
behaviour of the second class particle.
As mentioned above, Theorem 6.4.5 can be deduced from results of Liggett [197]; for p ≤ 1/2 those results say only that EP(p) is not positiverecurrent. The transience result Theorem 6.4.6 is from [17].
In the voter model, only one site changes its value at any given time,
so the voter model is an example of a spin-flip model. There are only two
invariant measures for the voter model on Z: one supported on the ‘all-0s’
configuration and the other supported on the ‘all-1s’ configuration. A basic
tool to prove such results is duality between the voter model and a dual
process obtained when one runs the voter model backwards in time. There
are two dual processes for the voter model: coalescing random walks and
annihilating random walks. See [82] and Chapter V of [198] for accounts on
these and many other properties of the voter model.
Theorem 6.4.7 is from [17]. Theorem 6.4.8 is due to [17], where it was
shown that Ev [τ (3/2)+ε ] = ∞, and [186], where it was established that the
boundary case has Ev [τ 3/2 ] = ∞ using a generating function argument. The
martingale argument based on Example 2.6.6 used in the proof given here
is shorter and more in keeping with the spirit of this book.
The hybrid exclusion-voter process was introduced in [17], and Theorems 6.4.9 and 6.4.10 are taken from that paper; some further developments
can be found in [144, 297, 210, 186]. The process can also be viewd as a
particular case of a model of random grammars [215]. The continuous-time
exclusion-voter model can be defined via its infinitesimal generator and con-
328
Chapter 6. Further applications
structed via a Harris-type graphical construction. The discrete-time process
studied in Section 6.4 is naturally embedded in the continuous-time process. We rather study these discrete versions in depth (after all, this book
is mainly about discrete-time Markov chains), and then only note that the
corresponding results for the continuous-time versions of the processes are
either straightforward or may be obtained in an elementary way, as shown
in Section 8 of [17].
The voter model has been used to model the spread of opinion through a
static population via nearest-neighbour interactions; see, for example [132].
The hybrid model studied here is a natural extension of this model whereby
individuals do not have to remain static, but may move by switching places.
Alternative motivations, such as from the point of view of competition of
species (see e.g. [48]) can also be adapted to the hybrid process. Models
consisting of a mixture of a spin-flip dynamics and exclusion dynamics come
under the title of reaction-diffusion processes; see e.g. [17] for more discussion on such processes.
The main open problem for the exclusion-voter process is:
Open problem 6.4.14. Give the complete recurrence classification for
HP(β, p) for β, p ∈ [0, 1].
It is known from Theorems 6.4.9, 6.4.10 and Theorem 4 of [210] (which
is obtained using another Lyapunov function) that the process is recurrent
if one of the following holds:
• β > 0 and p ≥ 1/2;
• (1 − p)(1 − β) < 1/3;
• p < 1/2 and β ≥ 4/7.
On the other hand, transience is established only for β = 0 and p ≤ 1/2
(Theorem 6.4.6). See [210] for some other open problems related to this
process.
Further remarks
To conclude this chapter we give bibliographic details for some other applications of Lyapunov function ideas that did not make it into this book.
Bibliographical notes
329
Queueing theory. The process that tracks the lengths of a system of d
queues often gives rise to a random walk in a positive orthant Zd+ . Depending on the details of the queueing model, the random walk may possess
varying degrees of spatial homogeneity. In the book [96] basic applications of
the Lyapunov function method to non-critical queueing systems were given.
Subsequently, near-critical cases have been treated; for example, polling systems are studied in [225, 208]. Polynomial rates of convergence to stationarity are treated in [308, 309]. There are also applications to some more exotic
queueing models, such as the supermarket model [207] or polling systems
with randomized service-regime regenerations [209, 206].
Branching random walks. A particle performs a random walk while at
the same time producing offspring as in a classical branching process. The
natural asymptotic question regards local survival and extinction, i.e., is the
origin visited infinitely often by some member of the population? We refer
to [233, 203, 227] for approaches to this question via Lyapunov functions.
The model can be extended to consider a branching random walk in random
environment, and this too has received attention [52, 71, 204, 205, 54, 55].
Self-interacting random walks. There are several models currently popular concerning random walks whose dynamics depends on their past trajectory in some way. For example, reinforced random walks or excited random
walks where transition probabilities are functions of nearby occupation times
for the process (see e.g. [255]); in this context ideas related to Lamperti’s
problem were recently applied in [184]. Other possibilities are non-local
interactions with the occupation time distribution, such as interaction mediated by the centre of mass of the previous trajectory [53]; analysis of this
model leads to a time-inhomogeneous version of Lamperti’s problem on R+
in which the mean drift at x at time n is of order x−β − (x/n).
Growth and deposition models. Various stochastic growth models have
been studied via Lyapunov function and related methods. We mention
classes of state-dependent or controlled branching processes [159, 155, 120],
cooperative growth processes on graphs [282], and models associated with
the growth of crystals [5] or biomolecules [134].
Urn models and related random walks. Urn models are prototypical
reinforced processes [255]. In [60] an urn model that can be phrased as a
non-homogeneous random walk model on Z2 was studied by making use of
330
Chapter 6. Further applications
the connection to Lamperti’s problem. In [223] generalizations of Pólya’s
urn model give rise to another time-inhomogeneous version of Lamperti’s
problem on R+ in which the mean drift at x at time n is of order nα x−β .
Chapter 7
Markov chains in continuous
time
7.1
Introduction and notation
So far, this book has been devoted to discrete-time stochastic processes.
For this final chapter, we switch to continuous time, and present Lyapunov
function methods for studying the asymptotic behaviour of continuous-time
Markov chains on countable state spaces. We establish general theorems
for the recurrence classification of continuous-time Markov chains, and for
quantifying recurrence by studying which moments of the passage time exist,
analogously to the corresponding results for discrete-time processes from
Chapter 2.
In keeping with the theme of this book, we are particularly interested
in the near-critical case, when even the discrete-time jump chain embedded
in the continuous-time chain has non-trivial behaviour. Continuous-time
Markov chains have the additional feature, compared to discrete-time ones,
that on each visit to a state they spend a random (exponentially distributed)
holding time; in the case where the jumps rates are spatially inhomogeneous
and not bounded away from 0 and ∞, the continuous-time chain can exhibit
the phenomena of explosion (whereby a transient chain makes an infinite
number of steps in finite time) or implosion (see Definition 7.1.3 below).
These phenomena can also be explored with the help of Lyapunov functions.
In particular, we present locally verifiable conditions for explosion or nonexplosion.
We adapt the notational conventions introduced in Chapter 2 as follows.
Now (ξt , t ∈ R+ ) will be a time-homogeneous continuous-time Markov chain
331
332
Chapter 7. Markov chains in continuous time
taking values in a countably infinite state-space Σ, equipped with its full
σ-algebra E = 2Σ .
Denote by Γ = (Γxy , x, y ∈ Σ) the infinitesimal generator of the Markov
chain,P
that is, the matrix satisfying Γxy ≥ 0 if y 6= x and Γxx = −γx , where
γx := y∈Σ\{x} Γxy . Assume that
0 < γx < ∞ for all x ∈ Σ.
(7.1)
P
(The assumption that y Γxy = 0 and γx < ∞ for all x is to say that the
chain is conservative.) We construct a stochastic matrix P = (Pxy , x, y ∈ Σ)
out of Γ by defining
(Γ
xy
if γx 6= 0;
Pxy := γx
0
if γx = 0;
and Pxx := 0. Then P is the transition matrix of a discrete-time Markov
chain ξ˜ = (ξ˜n , n ∈ Z+ ) on Σ called the jump chain. Throughout this chapter,
we assume the following:
(CT) Suppose that (7.1) holds, and that the jump chain is irreducible.
˜
Define a sequence (σn , n ∈ N) of random holding times distributed, given ξ,
according to an exponential law with parameter γξ̃n−1 : for n ≥ 1,
˜ = exp(−sγ
P[σn > s | ξ]
ξ̃n−1 ), for s ≥ 0,
˜ = γ −1 . The sequence (Jn , n ∈ Z+ ) of random jump times
so that E[σn | ξ]
ξ̃n−1
P
is defined accordingly by J0 := 0 and for n ≥ 1 by Jn := nk=1 σk . The life
time of the process is
∞
X
ζ := lim Jn =
σk ,
n→∞
k=1
and we say that explosion occurs if {ζ < ∞}.
Remark 7.1.1. The parameter γx must be interpreted as the proper frequency of the internal clock of the Markov chain multiplicatively modulating the local speed of the chain. If supx∈Σ γx < ∞ then it is not hard to
see that P[ζ = ∞] = 1 so that the chain is a.s. non-explosive (also called
regular ). Interesting phenomena are observed if
• supx γx = ∞: the internal clock ticks unboundedly fast (leading to an
unbounded local speed of the chain), or
7.1. Introduction and notation
333
• inf x γx = 0: the internal clock ticks arbitrarily slowly (leading to a
local speed that can be arbitrarily close to 0).
To have a unified description of both explosive and non-explosive processes, we extend the state space to Σ̂ = Σ ∪ {∂} by adjoining a special
absorbing state ∂. The continuous-time Markov chain is then the process
(ξt , t ∈ R+ ) defined by ξ0 = ξ˜0 , and
X

ξ˜n 1{t ∈ [Jn , Jn+1 )} for 0 < t < ζ,


ξt = n∈Z+


∂
for t ≥ ζ.
Note that although no algebraic structure is imposed on the set Σ, the
above sum is well-defined since for every fixed t only one term survives.
Under condition (CT), we say that the continuous-time Markov chain ξt is
irreducible.
If A ∈ E, we denote by τA := inf{t ≥ 0 : ξt ∈ A} the first hitting time of
A. We use the notation Px [ · ] = Px [ · | ξ0 = x] for probabilities conditional
on starting the chain at x ∈ Σ; we use Ex for the corresponding expectation.
Analogously to the discrete-time case, we define the first passage time of a
set A ⊂ Σ by
τA+ := inf{t ≥ J1 : ξt ∈ A},
+
and for x ∈ Σ we set τx+ := τ{x}
.
Definition 7.1.2. Suppose that (CT) holds. The irreducible Markov chain
ξt is called
• recurrent if Px [τx+ < ∞] = 1 for all x ∈ Σ;
• transient if Px [τx+ < ∞] < 1 for all x ∈ Σ.
A recurrent Markov chain is classified further as
• positive recurrent if Ex τx+ < ∞ for all x ∈ Σ;
• null recurrent if Ex τx+ = ∞ for all x ∈ Σ.
A dual notion to explosion is that of implosion.
Definition 7.1.3. Let (ξt , t ∈ R+ ) be a continuous-time Markov chain on Σ
and let A ⊂ Σ be a proper subset of Σ. We say that the Markov chain
implodes towards A if there exists K > 0 such that Ex τA ≤ K for all
x ∈ Σ \ A.
334
Chapter 7. Markov chains in continuous time
Remark 7.1.4. It will be shown in Proposition 7.4.11 that if A is finite
and the chain is irreducible, implosion towards A is equivalent to implosion
towards any state. In this situation, we speak about implosion of the chain.
We denote the measurable functions on (Σ, E) by
mE := {f : Σ → R, f is E-measurable},
and similarly we use bE to denote bounded measurable functions, mE+ to
denote non-negative measurable functions, etc. For f ∈ mE+ and α > 0, we
denote by Sα (f ) the sublevel set of f of height α defined by
Sα (f ) := {x ∈ Σ : f (x) ≤ α}.
We recall that a function f ∈ mE+ is unbounded if supx∈Σ f (x) = +∞,
while f → ∞ means that for every n ∈ N the sublevel set Sn (f ) is finite.
Measurable functions f defined on Σ can be extended to functions fˆ, defined
on Σ̂, by fˆ(x) = f (x) for all x ∈ Σ and fˆ(∂) := 0 (with obvious extension
of the σ-algebra).
We denote the domain of the generator by
n
X
Dom(Γ) := f ∈ mE :
Γxy |f (y)| < ∞, for all x ∈ Σ ,
(7.2)
y∈Σ\{x}
and by Dom+ (Γ) we denote the set of non-negative functions in the domain.
The action of the generator Γ on f ∈ Dom(Γ) then reads
X
Γf (x) :=
Γxy f (y).
(7.3)
y∈Σ
7.2
Recurrence and transience
We have the following criteria for recurrence and transience.
Theorem 7.2.1. Suppose that (CT) holds. The following are equivalent.
(a) The chain is recurrent.
(b) There exist a finite non-empty F ⊂ Σ and f ∈ Dom+ (Γ) satisfying
Γf (x) ≤ 0 for all x ∈
/ F , and f → ∞.
Theorem 7.2.2. Suppose that (CT) holds. The following are equivalent.
335
7.2. Recurrence and transience
(a) The chain is transient.
(b) There exist a (finite or infinite) non-empty F ⊂ Σ and f ∈ Dom+ (Γ)
satisfying Γf (x) ≤ 0 for all x ∈
/ F , and f (y) < inf x∈F f (x) for some
y∈
/ F.
After introducing convenient notation, we will see that these theorems
are immediate translations of the discrete-time Foster–Lyapunov criteria of
Section 2.5. For p > 0, we denote the p-integrable functions by
o
n
X
Γxy |f (y) − f (x)|p < ∞, for all x ∈ Σ ;
`p (Γ) = f ∈ mE :
y∈Σ
by `p+ (Γ) we denote the positive p-integrable functions. Note that `1 (Γ) =
Dom(Γ) as defined at (7.2). For f ∈ `1 (Γ), we denote the f -increment of
the jump chain by
∆fn+1 := ∆f (ξ˜n+1 ) := f (ξ˜n+1 ) − f (ξ˜n ),
the local mean f -drift by
mf (x) := E[∆fn+1 | ξ˜n = x] =
X
y∈Σ
Pxy (f (y) − f (x)) = Ex ∆f1 ,
and for p ≥ 1 and f ∈ `p (Γ) the pth-moment of the f -increment by
X
Pxy |f (y) − f (x)|p = Ex [|∆f1 |p ].
vfp (x) := E |∆fn+1 |p ξ˜n = x =
y∈Σ
In terms of the local mean f -drift, the action of the generator Γ on f ∈ `1 (Γ)
given by (7.3) reads
X
Γf (x) = Γxx f (x) +
Γxy f (y)
y∈Σ\{x}
= −γx f (x) +
= γx
X
y∈Σ
X
γx Pxy f (y)
y∈Σ\{x}
Pxy (f (y) − f (x)) = γx mf (x).
(7.4)
Proof of Theorems 7.2.1 and 7.2.2. Armed with (7.4), we see that the two
theorems for the continuous-time Markov chain ξt follow from Theorems 2.5.2
and 2.5.8, respectively, applied to the jump chain ξ˜n .
336
Chapter 7. Markov chains in continuous time
7.3
Existence and non-existence of moments of passage times
The next result is the following Lyapunov function condition for existence
of moments of passage times.
Theorem 7.3.1. Suppose that (CT) holds. Let f ∈ Dom+ (Γ) be such that
f → ∞. Suppose that there exist constants a > 0, c > 0 and p > 0 such that
f p ∈ Dom+ (Γ) and
Γf p (x) ≤ −cf p−2 (x), for all x 6∈ Sa (f ).
Then Ex [τSqa (f ) ] < ∞ for all q < p/2 and all x ∈ Σ. Moreover, if p ≤ 2,
p/2
then Ex [τSa (f ) ] < ∞ as well.
We also have the following result in the other direction.
Theorem 7.3.2. Suppose that (CT) holds. Let f ∈ Dom+ (Γ) be such that
f → ∞. Suppose that there exist
(a) constants a > 0 and c1 > 0 such that Γf (x) ≥ −c1 for x ∈
/ Sa (f );
(b) constants c2 > 0 and r > 1 such that f r ∈ Dom(Γ) and Γf r (x) ≤
c2 f r−1 (x) for x ∈
/ Sa (f ); and
(c) a constant p > 0 such that f p ∈ Dom(Γ) and Γf p ≥ 0 for x ∈
/ Sa (f ).
Then Ex [τSqa (f ) ] = +∞ for all q > p and all x 6∈ Sa (f ).
Remark 7.3.3. The conditions in Theorem 7.3.1 guarantee recurrence of the
chain, as well as providing existence of polynomial moments of the hitting
time. When τA is integrable, the chain is positive recurrent. In the null
recurrent case, τA is almost surely finite but not integrable; nevertheless,
some fractional moments E[τAq ] with q < 1 can exist. Similarly, in the
positive recurrent case, some higher moments E[τAq ] with q > 1 may fail to
exist.
When p = 2, Theorem 7.3.1 says that if Γf (x) ≤ −ε for some ε > 0
and for x outside a finite set F , then Ex τF < ∞ for all x ∈ Σ. In this
situation, we have the following stronger result, which is the analogue of
Foster’s criterion Theorem 2.6.4.
Theorem 7.3.4. Suppose that (CT) holds. Suppose that the chain is nonexplosive, i.e., P[ζ < ∞] = 0. The following are equivalent.
7.3. Existence and non-existence of moments of passage times
337
(a) The chain is positive recurrent.
(b) There exist a triple (ε, F, f ), with ε > 0, F a finite non-empty subset of
Σ and f a function in Dom+ (Γ) satisfying Γf (x) ≤ −ε for all x ∈
/ F.
Remarks 7.3.5. (a) To check that the chain is non-explosive, it is sufficient
to the show that the chain is recurrent, for which, by Theorem 7.2.1, it is
sufficient that the f in part (b) satisfies f → ∞.
(b) It is clear that the triple (ε, F, f ) in Theorem 7.3.4 is not uniquely
determined. Mostly, it will be possible to choose a function f → ∞ and F
as the sublevel set of f at height a, for some a > 0. Sometimes it will be
possible to choose the function f uniformly bounded; this case will be further
considered in Theorem 7.4.12 and leads to implosion. It is also immediate
that if f satisfies the condition Γf (x) ≤ −ε for x ∈
/ F then the modified
function f + c, where c is an arbitrary positive constant, also satisfies the
same condition. Further, if a function f satisfies this condition, the function
g(x) = f (x)1{x ∈
/ F } satisfies a fortiori the same condition.
The rest of this section is devoted to the proofs of the preceding results.
First we introduce some more notation that we use throughout the rest of
this chapter. The natural right-continuous filtration (Ft , t ∈ R+ ) is defined
by Ft := σ(ξs , s ≤ t); similarly Ft− := σ(ξs , s < t), and F̃n := σ(ξ˜m , m ≤ n)
for n ∈ N. A (possibly infinite) τ ∈ Z+ is a stopping time relative to
(Ft , t ∈ R+ ) if {τ ≤ t} ∈ Ft for all t ∈ R+ . Given a stopping time τ , we
define Fτ to be the set of all A ∈ F∞ such that A ∩ {τ ≤ t} ∈ Ft for all
t ∈ R+ . Similarly,
[
Fτ − := σ F0 ∪ {A ∩ {t < τ } : A ∈ Ft } .
t≥0
Since it is not hard to show that τ is Fτ − -measurable, the only information
contained in FJn+1 but not in FJn+1 − is conveyed by the random variable
ξ˜n+1 , i.e., the position to which the chain jumps at the moment Jn+1 .
Given f ∈ Dom(Γ), the process (Xt , t ∈ R+ ) defined by Xt = f (ξt ) is
specified by, for t < ζ,
Xt = f (ξt ) =
∞
X
n=0
f (ξ˜n )1{t ∈ [Jn , Jn+1 )} = X0 +
∞
X
n=0
∆fn+1 1{Jn ∈ (0, t]}.
If there is no explosion, the process Xt is an Ft -semimartingale admitting the
decomposition Xt = X0 + Mt + At (see e.g. [174]), where Mt is a martingale
338
Chapter 7. Markov chains in continuous time
vanishing at 0 and At is the predictable compensator given by
Z
Z
At =
Γf (ξs− )ds =
Γf (ξs )ds.
(0,t]
[0,t)
Note that, although not explicitly marked, Xt , Mt , and At all depend on f .
We use in the sequel also the infinitesimal form of the above decomposition.
For any admissible f we have dXt = dMt + dAt = dMt + Γf (ξt− )dt; in
particular, since Mt is an Ft -martingale,
E[dXt | Ft− ] = E[dAt | Ft− ] = Γf (ξt− )dt
(7.5)
represents the conditional increment of Xt as an ordinary differential multiplied by a previsible random factor.
We also recall Itô’s formula, which for a real-valued semimartingale
(St , t ∈ R+ ) and any twice continuously differentiable g : R → R, says
dg(St ) = g 0 (St− )dStc + g(St ) − g(St− ),
(7.6)
where Stc denotes the continuous part of St .
The next result is a continuous-time analogue of Theorem 2.6.2.
Lemma 7.3.6. Let (Yt , t ∈ R+ ) be an R+ -valued process adapted to a filtration (Gt , t ∈ R+ ), and let T be a stopping time. Suppose that there exists
ε > 0 such that
E[dYt | Gt− ] ≤ −εdt, on {t ≤ T }.
Then E[T | G0 ] ≤ ε−1 Y0 .
Proof. Since {s ≤ T } ∈ Gs− , the hypothesis of the lemma can be written as
E[dYs∧T | Gs− ] ≤ −ε1{T ≥ s}ds.
Taking expectations conditional on G0 and integrating over s ∈ [0, t] yields
0 ≤ E[Yt∧T | G0 ] ≤ Y0 − ε
Z
0
t
P[T ≥ s | G0 ]ds, for all t ∈ R+ .
Taking t → ∞ it follows from monotone convergence that
Z ∞
0 ≤ Y0 − ε
P[T ≥ s | G0 ]ds = Y0 − ε E[T | G0 ],
0
which gives the result.
7.3. Existence and non-existence of moments of passage times
339
We can now present the proof of Theorem 7.3.4.
Proof of Theorem 7.3.4. We assume that P[ζ = ∞] = 1, i.e., there is no
explosion. First, we prove that (b) implies (a). Without loss of generality,
using Remark 7.3.5, possibly modifying f , we can always assume that a0 :=
inf x∈F
/ f (x) > 0 and f (x) = 0 for all x ∈ F . Choose then an arbitrary
c ∈ (0, a0 ). Let Xt = f (ξt ) and recall that τF = inf{t ≥ 0 : ξt ∈ F }. Then
using (7.5) condition (b) can be expressed as
E[dXt | Ft− ] = Γf (ξt− )dt ≤ −εdt, on {t ≤ τF }.
Hence we may apply Lemma 7.3.6 with Yt = Xt , Gt = Ft and T = τF to get
Ex τF ≤ ε−1 f (x) < ∞ for every x ∈
/ F ; clearly Ex τF is also finite for x ∈ F .
It follows that ξt is positive-recurrent by the continuous-time analogue of
Lemma 2.6.1 (we omit the details).
Now, we prove that (a) implies (b). Let F = {z} for some fixed z ∈ Σ;
positive recurrence of the chain implies that Ex τF < ∞ for all x ∈ Σ. Define
(
Ex τF if x ∈
/ F,
f (x) =
0
if x ∈ F.
Then, for all x ∈
/ F,
mf (x) =
X
y6=z
Pxy Ey τF − Ex τF = Ex [τF − σ1 ] − Ex τF = − Ex σ1 = −
1
.
γx
It follows that Γf (x) = γx mf (x) = −1 for x ∈
/ F . By adding the constant 1
to the function f determined above (see Remark 7.3.5), we see that f meets
all the requirements.
Next we work towards the proof of Theorem 7.3.1. We proceed via
two technical results that deal separately with the cases p ≥ 2 and p < 2,
analogously to the proof of Theorem 2.7.1.
Lemma 7.3.7. Let f ∈ Dom+ (Γ) with f → ∞, p ≥ 2, and a > 0. Denote
Xt = f (ξt ) and assume further that f p ∈ Dom+ (Γ). Suppose that there
exists c > 0 such that
Γf p (x) ≤ −cf p−2 (x), for all x ∈
/ Sa (f ).
Then there exists C ∈ R+ such that for all q ∈ [0, p/2],
Ex [τSqa (f ) ] ≤ Cf (x)2q , for all x ∈ Σ.
340
Chapter 7. Markov chains in continuous time
Proof. Use the abbreviation A := Sa (f ). We first show that under the
hypotheses of the lemma, the process (Zt , t ∈ R+ ) defined by
p/2
c
2
Zt = Xt∧τA +
(t ∧ τA )
p/2
is a non-negative supermartingale. Introducing the predictable decomposition 1 = 1{t > τA } + 1{t ≤ τA }, we get
i
h c p/2 t
E[dZt | Ft− ] = E d Xt2 +
Ft− 1{t ≤ τA }.
p/2
Itô’s formula (7.6) applied to St = Xt2 +
c
p/2 t
with g(x) = xp/2 yields
c p/2
c (p/2)−1
2
t
t
d Xt2 +
= c Xt−
+
dt
p/2
p/2
c p/2 2
c p/2
+ Xt2 +
t
t
− Xt− +
.
p/2
p/2
(7.7)
Writing the semimartingale decomposition for the process Xtp , similarly
to (7.5), we note that the hypothesis of the lemma implies that
p−2
E[dXtp | Ft− ] = Γf (ξt− )p dt ≤ −cXt−
dt, on {t ≤ τA }.
(7.8)
Since p ≥ 2, we may apply the conditional Minkowski inequality to obtain
h
i 2/p
c p/2 c p/2
p
2
E Xt +
t
t
+
Ft− ≤ E[Xt | Ft− ]
p/2
p/2
2/p
c p/2
p
= Xt−
+ E[dXtp | Ft− ]
+
t
p/2
c p/2
p
p−2
≤ (Xt−
− cXt−
dt)2/p +
t
,
p/2
on {t ≤ τA }, by (7.8). Hence, on {t ≤ τA },
i h
2/p
c p/2 c
c p/2
2
E Xt2 +
t
F
≤
X
1
−
dt
+
t
t−
t−
2
p/2
p/2
Xt−
c 1
c p/2
2
≤ Xt−
1−
dt
+
t
,
2
p/2 Xt−
p/2
using the inequality (1 − y)r ≤ 1 − ry for r ∈ [0, 1] and all y ≥ 0. Hence
h
i c p/2 c
c p/2
2
2
E Xt +
t
−
dt +
t
Ft− ≤ Xt−
p/2
p/2
p/2
7.3. Existence and non-existence of moments of passage times
341
c (p/2)−1
c p/2
2
2
− c Xt−
+
dt,
= Xt−
+
t
t
p/2
p/2
by Taylor’s formula. Thus taking expectations in (7.7), we obtain E[dZt |
Ft− ] ≤ 0. Hence Zt is a supermartingale as claimed.
To prove the statement in the lemma, it suffices to suppose that x ∈
/ A.
2q/p
Since the function x 7→ x
is concave for q ∈ [0, p/2], we have that the
2q/p
supermartingale property holds for Zt
as well as for Zt . Since Xt is
non-negative,
c
2q/p
2q/p
Ex [(t ∧ τA )q ] ≤ Ex [Zt ] ≤ Ex [Z0 ] = f (x)2q .
p/2
Fatou’s lemma now completes the proof.
Lemma 7.3.8. Let f ∈ Dom+ (Γ) with f → ∞, 0 < p ≤ 2, and a > 0.
Denote Xt = f (ξt ) and assume further that f p ∈ Dom+ (Γ). Suppose that
there exists c > 0 such that
Γf p (x) ≤ −cf p−2 (x), for all x ∈
/ Sa (f ).
Then for any q ∈ [0, p/2) there is a constant C ∈ R+ such that Ex [τSqa (f ) ] ≤
Cf (x)p for all x ∈ Σ.
Proof. Let q ∈ [0, p/2). Again use the abbreviation A := Sa (f ). We first
show that under the hypotheses of the lemma, the process (Zt , t ∈ R+ )
defined by
c
p
Zt = Xt∧τ
+ (1 + (t ∧ τA ))q
A
q
satisfies for some C ∈ R+ ,
Ex Zt ≤ Cf (x)p , for all x ∈ Σ.
(7.9)
Once more, the hypothesis of the lemma implies that (7.8) holds. Then since
d(Xtp + qc (1 + t)q ) = dXtp + c(1 + t)q−1 dt, we have
p−2
E[dZt | Ft− ] ≤ c(−Xt−
+ (1 + t)q−1 )dt, on {t ≤ τA }.
Since t ≤ τA implies Xt ≥ a, we have that, on {t ≤ τA },
p−2
E[dZt | Ft− ] ≤ c(−Xt−
+ (1 + t)q−1 )1{Xt− ∈ [a, (1 + t)1/2 )}dt
p−2
+ c(−Xt−
+ (1 + t)q−1 )1{Xt− ∈ [(1 + t)1/2 , ∞)}dt.
342
Chapter 7. Markov chains in continuous time
Here we note that if Xt− ≤ (1 + t)1/2 ,
p−2
−Xt−
+ (1 + t)q−1 ≤ −(1 + t)(p/2)−1 + (1 + t)q−1 ≤ 0,
for all t ≥ 0, since 0 < q < p/2 ≤ 1. Hence
Ex [dZt ] ≤ c(1 + t)q−1 Px [XτA ∧t− ≥ (1 + t)1/2 ]dt
≤ c(1 + t)q−1−(p/2) Ex [XτpA ∧t− ],
by Markov’s inequality. A simple consequence of the hypothesis of the
lemma is that E[dXτpA ∧t ] ≤ 0, so that Ex [XτpA ∧t− ] ≤ Ex [X0p ] = f (x)p . Hence
Ex [dZt ] ≤ cf (x)p (1 + t)q−1−(p/2) .
Integrating this differential inequality yields
Z ∞
(1 + t)q−1−(p/2) dt;
Ex Zt ≤ cf (x)p
0
the condition q < p/2 ensures the finiteness of the last integral, which establishes the claim (7.9). Since Xt is non-negative, we obtain from (7.9)
that
c
Ex [(t ∧ τA )q ] ≤ Ex Zt ≤ Cf (x)p ;
q
Fatou’s lemma completes the proof.
Theorem 7.3.1 is now immediate from the preceding two lemmas.
Proof of Theorem 7.3.1. The statement is a combination of Lemmas 7.3.7
and 7.3.8.
Now we turn to the proof of Theorem 7.3.2 on conditions for some
passage-time moments to be infinite. We first need the following crucial estimate, which is a continuous-time analogue of Lemma 2.7.5. Lemma 7.3.9
serves twice in this chapter: once on the road to Theorem 7.3.2 and once
more in a different context, namely for finding conditions for non-implosion
(Theorem 7.4.13).
Lemma 7.3.9. Let (Yt , t ∈ R+ ) be an R+ -valued process adapted to a filtration (Gt , t ∈ R+ ). Let a > 0 and Ta := inf{t ≥ 0 : Yt ≤ a}. Suppose that
there exist constants c1 > 0, c2 > 0, and r > 1 such that
(a) E[dYt | Gt− ] ≥ −c1 dt on {t ≤ Ta }; and
343
7.3. Existence and non-existence of moments of passage times
r−1
(b) E[dYtr | Gt− ] ≤ c2 Yt−
dt on {t ≤ Ta }.
Then, for all α ∈ (0, 1), there exist ε > 0 and δ > 0 such that, for all t ≥ 0,
P[Ta > t + εYt∧Ta | Gt ] ≥ 1 − α, on {Ta > t, Yt > a(1 + δ)}.
Proof. Fix t ∈ R+ and let σ = (Ta − t)+ ; then for all s > 0 we have
{σ > s} = {Ta > t + s} ∈ Gt+s . To prove the lemma, it is enough to
establish P[σ > εYt | Gt ] ≥ 1 − α on {σ > 0, Yt > a(1 + δ)}. Observe that
P[σ > εYt | Gt ] = P[Yt+(εYt )∧σ > a | Gt ], on {σ > 0, Yt > a(1 + δ)}.
Write Ut := Yt+(εYt )∧σ . Let r > 1 be as given in condition (b) of the lemma.
Then, by Hölder’s inequality,
E[Ut | Gt ] = E[Ut 1{Ut ≤ a} | Gt ] + E[Ut 1{Ut > a} | Gt ]
1/r
1−(1/r)
≤ a + E[Utr | G]
P[Ut > a | Gt ]
;
therefore
P[Ut > a | Gt ] ≥
(E[U | G ] − a)+ r/(r−1)
t
t
.
(E[Utr | G])1/r
(7.10)
For a lower bound on the numerator on the right-hand side of (7.10),
E[Ut | Gt ] = E[Yt+(εYt )∧σ − Yt | Gt ] + Yt
Z t+(εYt )∧σ
=
E[dYs | Gt ] + Yt ≥ −c1 εYt + Yt ,
t
by hypothesis (a). Obtaining an upper bound on the denominator on the
right-hand side of (7.10) requires more work. We obtain an upper bound
r
for E[Yt+s∧σ
| Gt ] for arbitrary s > 0. Let τ ∈ (0, ∞) be a Gt -measurable
random variable. For c3 = c2 /r and any s ∈ (0, τ ], define
r Fτ (s) = E Yt+s∧σ + c3 τ − c3 (s ∧ σ) Gt .
We shall show that Fτ (s) ≤ Fτ (s−) for all s ∈ (0, τ ]. It is enough to show this
inequality on {s ≤ σ} since otherwise Fτ (s) = Fτ (s−) and there is nothing
to prove. To show that Fτ is decreasing in (0, τ ], it is enough to show that
E[dZs | Gt+s− ] ≤ 0 for all s ∈ (0, τ ], where Zs = (Yt+s∧σ + c3 τ − c3 (s ∧ σ))r .
Now, on {s ≤ σ}, Itô’s formula (7.6) yields
dZs = −rc3 (Yt+s− +c3 τ −c3 s)r−1 ds+(Yt+s +c3 τ −c3 s)r −(Yt+s− +c3 τ −c3 s)r .
344
Chapter 7. Markov chains in continuous time
Moreover, using Minkowski’s inequality, we get
r
r
E[(Yt+s + c3 τ − c3 s)r | Gt+s− ] ≤ E[Yt+s
| Gt+s− ]1/r + c3 τ − c3 s ,
using the fact that τ is Gt -measurable. Also, by hypothesis (b),
r−1
r
r
E[Yt+s
| Gt+s− ] ≤ Yt+s−
+ c2 Yt+s−
1{s ≤ σ}ds
c2
r
1{s ≤ σ}ds
= Yt+s−
1+
Yt+s−
r
≤ Yt+s− + c3 1{s ≤ σ}ds ,
using the elementary inequality 1 + y ≤ (1 + r−1 y)r for all r > 1 and all
y > 0. Therefore,
E[(Yt+s + c3 τ − c3 s)r | Gt+s− ] ≤ (Yt+s− + c3 ds + c3 τ − c3 s)r
= (Yt+s− + c3 τ − c3 s)r + rc3 (Yt+s− + c3 τ − c3 s)r−1 ds.
It follows that, for all s ∈ (0, τ ], E[dZs | Gt+s− ] ≤ 0. So, for all τ > 0,
r
E[Yt+τ
| Gt ] = Fτ (τ ) ≤ lim Fτ (s) = (Yt + c3 τ )r , on {τ ≤ σ}.
s→0+
Choosing τ = εYt , we get, on {εYt ≤ σ},
1/r
r
E[Yt+(εY
| Gt ]
≤ Yt + c3 εYt .
t )∧σ
On the other hand, on {εYt > σ} ∩ {Yt > a(1 + δ)},
1/r
r
| Gt ]
≤ a ≤ Yt + c3 εYt .
E[Yt+(εY
t )∧σ
Thus it follows from (7.10) that, on {Yt > a(1 + δ)},
(1 − c ε − Y −1 a)+ r/(r−1)
1
t
1 − c3 ε
(1 − c ε − (1 + δ)−1 )+ r/(r−1)
1
≥
,
1 − c3 ε
P[Ut > a | Gt ] ≥
which exceeds any 1 − α, α ∈ (0, 1), for suitable ε > 0 and δ > 0.
Lemma 7.3.10. Let (Yt , t ∈ R+ ) be an R+ -valued process adapted to a
filtration (Gt , t ∈ R+ ). Let a > 0 and Ta := inf{t ≥ 0 : Yt ≤ a}. Suppose
that there exist positive constants a, c1 , c2 , r, p such that
345
7.4. Explosion and implosion
(a) Y0 = y > a;
(b) E[dYt | Gt− ] ≥ −c1 dt on {t ≤ Ta };
r−1
(c) E[dYtr | Gt− ] ≤ c2 Yt−
dt on {t ≤ Ta };
p
(d) (Yt∧T
) is a submartingale.
a
Then E[Taq ] = ∞ for all q > p.
Proof. It suffices to suppose that P[Ta < ∞] = 1. Let q > p. Suppose,
for the purposes of deriving a contradiction, that E[Taq ] < ∞. Under the
conditions of the lemma, Lemma 7.3.9 applies, and taking α = 1/2 there
shows that there exist positive constants ε and δ such that
P[Ta > t + εYt∧Ta | Gt ] ≥ 1/2, on {Yt∧Ta > a(1 + δ)},
for all t ∈ R+ . Hence
E Taq ≥ E E[Taq 1{Ta > t + εYt∧Ta } | Gt ]1{Yt∧Ta > a(1 + δ)}
1 ≥ E (εYt∧Ta )q 1{Yt∧Ta > a(1 + δ)}
2
εq q
εq
q
]
−
≥
E[Yt∧T
a (1 + δ)q .
a
2
2
Thus under the assumption E[Taq ] < ∞, re-arranging the preceding inequalq
ity shows that there exists a finite constant K1 such that E[Yt∧T
] ≤ K1 for
a
p
all t ∈ R+ . Hence Yt∧T
is
uniformly
integrable;
by
hypothesis
(d) it is a
a
p
p
q
submartingale. Hence limt→∞ E[Yt∧Ta ] = E[YTa ] ≤ a . On the other hand,
q
] ≥ E Y0q = y q . Choosing
the submartingale property shows that E[Yt∧T
a
Y0 = y > a leads to the desired contradiction.
Proof of Theorem 7.3.2. On identifying Yt in Lemma 7.3.10 with f (ξt ), we
see that the conditions of the theorem imply the hypotheses of the lemma.
The non-existence of moments immediately follows.
7.4
Explosion and implosion
The classical criterion for explosion due to Chung [45] is in terms of the
jump chain and states
P[ζ < ∞] = 1 if and only if
∞
X
n=0
γξ̃−1 < ∞.
n
(7.11)
346
Chapter 7. Markov chains in continuous time
This condition is usually difficult to check since it is global, i.e., requires the
knowledge of the entire trajectory of the embedded Markov chain. The purpose of this section is to give some conditions whose validity can be verified
by local estimates. The first result gives a Lyapunov function criterion for
the explosion time to be integrable; it is worth noting that although explosion can only occur in the transient case, the result is strongly reminiscent
of Foster’s criterion in discrete time (Theorem 2.6.4).
Theorem 7.4.1. Suppose that (CT) holds. The following are equivalent.
(a) The explosion time ζ satisfies Ex ζ < ∞ for all x ∈ Σ.
(b) There exist f ∈ Dom+ (Γ) strictly positive, a finite set A ⊆ Σ, and
ε > 0 such that Γf (x) ≤ −ε for all x ∈
/ A, and either (i) A = ∅; or
(ii) the chain is transient.
Remarks 7.4.2. (a) If A = ∅, then the function f in condition (b) cannot
have f → ∞, or else Theorem 7.4.9 below would imply non-explosion; note
however, that f need not be uniformly bounded.
(b) By Theorem 7.2.2, we may replace the transience condition in (b)(ii) by
the condition f (y) < inf x∈A f (x) for some y ∈
/ A.
The following result will be shown to follow from Theorem 7.4.1.
Proposition 7.4.3. Suppose that (CT) holds. Let f ∈ Dom+ (Γ) be bounded
and strictly positive, and denote b = supx∈Σ f (x). Suppose that there exist
a finite set A ⊆ Σ and a non-decreasing function g : R+ → (0, ∞) such that
R b dy
/ A. Then Ex ζ < ∞ for all
0 g(y) < ∞, and Γf (x) ≤ −g(f (x)) for all x ∈
x ∈ Σ.
Remark 7.4.4. The interest of Proposition 7.4.3 is in the case inf x∈Σ g(f (x)) =
0, because then the hypothesis is weaker than the requirement of Theorem 7.4.1(b) that Γf (x) ≤ −ε.
The following result permits an infinite exceptional set, and, roughly
speaking, gives a condition for explosion to occur in a direction in which the
Lyapunov function f tends to zero.
Theorem 7.4.5. Suppose that (CT) holds. Suppose that there exist an
increasing sequence E1 ⊆ E2 ⊆ · · · of (finite or infinite) sets in E with
∪n En = Σ, and a function f ∈ Dom+ (Γ), such that
(a) for each n ∈ N, σn := inf{t ≥ 0 : ξt ∈
/ En } is finite a.s.;
(b) supx∈Σ f (x) < ∞, inf x∈En f (x) > 0, and limn→∞ supx∈E
/ n f (x) = 0;
7.4. Explosion and implosion
347
(c) for some ε > 0, Γf (x) ≤ −ε for all x ∈
/ E1 .
Then Px [ζ < ∞] = 1 for all x ∈ Σ.
If Px [ζ < ∞] > 0 for some x ∈ Σ, then irreducibility of the chain implies
that Px [ζ < ∞] > 0 for all x, i.e., explosion can occur from any starting
state. However, we may have (see Example 7.4.8) 0 < Px [ζ < ∞] < 1.
Additionally, the results in this section establish conditions that guarantee
Ex ζ < ∞, implying explosion, but we may have (see Example 7.4.10) Px [ζ <
∞] = 1 but Ex ζ = ∞. The following conditional explosion result is useful
in such circumstances.
Theorem 7.4.6. Suppose that (CT) holds. Suppose that there exists a triple
(ε, A, f ) with ε > 0, A a proper (finite or infinite) subset of Σ such that Σ\A
is infinite, and f ∈ Dom+ (Γ), such that
(a) there exists x0 ∈
/ A with f (x0 ) < inf x∈A f (x);
(b) Γf (x) ≤ −ε for all x ∈
/ A.
Then Px0 [τA = ∞] > 0 and Ex0 [ζ | τA = ∞] < ∞; in particular, Px0 [ζ <
∞] > 0, so explosion occurs with positive probability.
Remark 7.4.7. Note that if the conditions of Theorem 7.4.6 hold, then the
chain is necessarily transient (cf. Theorem 7.2.2). If the set A is finite, then
the conditions of Theorem 7.4.6 imply that E ζ < ∞, by Theorem 7.4.1.
Example 7.4.8. Consider a nearest-neighbour random walk on Z with jump
rates given for x ≤ 0 by Γx x−1 = 2 and Γx x+1 = 1, and for x > 0 by
Γx x−1 = x2 and Γx x+1 = 2x2 .
Let A = {0, −1, −2, . . .} and define f : Z → R+ by f (x) = 1, x ≤ 0 and
f (x) = x−α , x ≥ 1, where α > 0. Then for x ≥ 1,
Γf (x) = 2x2 (x + 1)−α − x−α + x2 (x − 1)−α − x−α
= 2x2 x−α (−αx−1 + O(x−2 )) + x2 x−α (αx−1 + O(x−2 ))
= −αx1−α + O(x−α ).
Thus for x ∈
/ A we have Γf (x) ≤ −ε provided α ∈ (0, 1]. Thus an application
of Theorem 7.4.6 shows that, for any x ≥ 1, Ex [ζ | τA = ∞] < ∞ and
Px [ζ < ∞] > 0, so explosion occurs with positive probability.
On the other hand, it may be shown that Px [ζ < ∞] < 1; we sketch the
argument. Indeed, we have Px [ξ˜n → −∞] > 0 for any x, and if the process
is transient to −∞ it will not explode, as shown by Chung’s criterion (7.11)
and the fact that the rates in that direction are uniformly bounded.
4
348
Chapter 7. Markov chains in continuous time
A sufficient condition for non-explosion is the next result.
Theorem 7.4.9. Suppose that (CT) holds. Let f ∈ Dom+ (Γ) with f → ∞.
Suppose also that
(a) there
R z dyexists a non-decreasing function g : R+ → R+ for which G(z) :=
0 g(y) < ∞ for all z ∈ R+ but limz→∞ G(z) = ∞; and
(b) Γf (x) ≤ g(f (x)) for all x ∈ Σ.
Then Px [ζ = ∞] = 1 for all x ∈ Σ.
Example 7.4.10. Consider a nearest-neighbour random walk on Z with
jump rates given for x ≤ 0 by Γx x−1 = Γx x+1 = 1, and for x > 0 by
Γx x−1 = xβ and Γx x+1 = 2xβ , where β ∈ R is a fixed parameter.
First we show that explosion does not occur if β ≤ 1. Define f : Z → R+
by f (x) = 1 + |x|. Then, Γf (0) = 1 + 1 = 2 ≤ 2f (0), while, for x ≤ −1,
Γf (x) = (|x| + 1 − |x|) + (|x| − 1 − |x|) = 0 ≤ 2f (x),
and, for x ≥ 1,
Γf (x) = 2xβ (|x| + 1 − |x|) + xβ (|x| − 1 − |x|) = |x|β ≤ 2f (x),
provided β ≤ 1. Thus an application of Theorem 7.4.9 shows that, if β ≤ 1,
Px [ζ = ∞] = 1 for all x ∈ Z.
Next, we show that explosion is certain if β > 1. Let En = (−∞, n] ∩ Z.
Then σn = inf{t ≥ 0 : ξt ∈
/ En } < ∞ for each n. Consider the Lyapunov
function f : Z → (0, 1] given by f (x) = 1, x ≤ 0 and f (x) = x−α , x ≥ 1,
where α > 0. Then for x ≥ 1,
Γf (x) = 2xβ (x + 1)−α − x−α + xβ (x − 1)−α − x−α
= 2xβ x−α (−αx−1 + O(x−2 )) + xβ x−α (αx−1 + O(x−2 ))
= −αxβ−α−1 + O(xβ−α−2 ).
Thus for x ∈
/ A we have Γf (x) ≤ −ε provided α ∈ (0, β − 1], which requires
β > 1. Thus an application of Theorem 7.4.5 shows that, if β > 1, Px [ζ <
∞] = 1 for all x ∈ Z. Moreover, an application of Theorem 7.4.6 shows that
for A = {0, −1, −2, . . .} and any x ≥ 1, Ex [ζ | τA = ∞] < ∞.
It may be shown that in this example Ex ζ = ∞. We do not have any
suitable Lyapunov function result here, so we just sketch the argument. (In
principle Theorem 7.4.1 gives a condition for Ex ζ = ∞, but in practice one
7.4. Explosion and implosion
349
cannot show that no appropriate function exists.) The idea is that there is
positive probability that before the process explodes it takes a sojourn on
the negative integers, and in so doing the expected time to return to the
positive integers is infinite.
4
Finally, we state results about implosion. The first result shows that
implosion implies the existence of all moments and even of exponential moments.
Proposition 7.4.11. Suppose that (CT) holds. Suppose that there exists
a finite set A ∈ E and a constant CA ∈ R+ such that supx∈Σ Ex τA ≤ CA
holds. Then the chain implodes towards any state z ∈ Σ, and, moreover, for
each z ∈ Σ there exists α > 0 such that supx∈Σ Ex eατz < ∞.
Next we give a criterion for implosion; as before, we remark that to check
that the chain is non-explosive it is sufficient to show that it is recurrent.
Theorem 7.4.12. Suppose that (CT) holds. Suppose that the chain is nonexplosive, i.e., P[ζ < ∞] = 0. The following are equivalent.
(a) There exists a triple (ε, F, f ) with ε > 0, F a finite set and f ∈
/ F.
Dom+ (Γ) such that supx∈Σ f (x) < ∞ and Γf (x) ≤ −ε for all x ∈
(b) For every finite A ∈ E, there exists a constant CA ∈ R+ such that
Ex τA ≤ CA for all x ∈ Σ, i.e., there is implosion towards A.
Next we give a sufficient condition for non-implosion.
Theorem 7.4.13. Suppose that (CT) holds. Let f ∈ Dom+ (Γ) be such that
f → ∞ and suppose that there exist constants a > 0, c > 0, ε > 0, and
r > 1 such that f r ∈ Dom+ (Γ). Suppose also that
(a) Γf (x) ≥ −ε, for all x ∈
/ Sa (f ); and
(b) Γf r (x) ≤ cf r−1 (x), for all x ∈
/ Sa (f ).
Then the chain does not implode towards Sa (f ).
In some applications, it is quite difficult to guess the form of the function f satisfying the uniform condition Γf (x) ≤ −ε required to apply
Theorem 7.4.12. It is sometimes more convenient to check merely that
Γf (x) ≤ −g(f (x)) for some function g vanishing at 0 in some controlled
way. Thus the following result provides us with a convenient alternative
condition to be checked.
350
Chapter 7. Markov chains in continuous time
Proposition 7.4.14. Suppose that (CT) holds. Suppose that the chain is
non-explosive, i.e., P[ζ < ∞] = 0. Let f ∈ Dom+ (Γ) be strictly positive and
such that supx∈Σ f (x) = b < ∞; assume further that for any a ∈ (0, b) the
sublevel set Sa (f ) is finite. Let g : [0, b] → R+ be an increasing function
R b dy
such that B := 0 g(y)
< ∞. If Γf (x) ≤ −g(f (x)) for all x ∈
/ Sa (f ) then
Ex τSa (f ) ≤ B for all x ∈
/ Sa (f ), i.e., the chain implodes towards Sa (f ).
The remainder of this section is devoted to the proofs of the results on
explosion and implosion stated above.
Since we wish to treat explosion, we work with the Markov chain ξt
evolving on the augmented state space Σ̂ = Σ ∪ {∂} with the generator Γ
extended in the natural way to a generator Γ̂; we work with augmented
functions fˆ ∈ Dom(Γ̂) obtained from f ∈ Dom(Γ) by setting fˆ(∂) = 0 as
explained earlier.
First we treat our results on conditional explosion. We will need the
following continuous-time analogue of Lemma 2.5.10.
Lemma 7.4.15. Suppose that there exist f ∈ Dom+ (Γ) and A ⊂ Σ such
that Γf (x) ≤ 0 for all x ∈
/ A. Then
Px [τA = ∞] ≥ 1 −
f (x)
.
inf y∈A f (y)
Proof. Let Yt = f (ξt∧τA ). Then since Γf (x) ≤ 0 for x ∈
/ A, we have that
Yt is a non-negative supermartingale, and hence converges a.s. to some Y∞ .
For every x ∈
/ A, Fatou’s lemma implies that Ex Y∞ ≤ limt→∞ Ex Yt ≤ f (x).
Hence
f (x) ≥ Ex Y∞ ≥ Ex [Y∞ 1{τA < ∞}] = Ex [f (ξτA )1{τA < ∞}]
≥ inf f (y) Px [τA < ∞].
y∈A
This gives the result.
Proof of Theorem 7.4.6. First of all, Lemma 7.4.15 shows that
Px0 [τA = ∞] ≥ 1 −
f (x0 )
> 0.
inf y∈A f (y)
An application of Lemma 7.3.6 with Yt = f (ξt∧τA ∧ζ ) and T = τA ∧ ζ shows
that Ex0 [τA ∧ ζ] ≤ ε−1 f (x0 ). Hence
f (x0 )
≥ Ex0 [τA ∧ ζ] ≥ Ex0 [ζ | τA = ∞] Px0 [τA = ∞].
ε
Re-arranging and using the fact that Px0 [τA = ∞] > 0 gives the result.
351
7.4. Explosion and implosion
Proof of Theorem 7.4.5. First we show that for any δ > 0 we can choose
n ∈ N so that
/ En .
(7.12)
Px [τE1 = ∞] ≥ 1 − δ, for all x ∈
Indeed, we have from Lemma 7.4.15 that
inf Px [τE1 = ∞] ≥ 1 −
x∈E
/ n
supx∈E
/ n f (x)
;
inf y∈E1 f (y)
then since supx∈E
/ n f (x) → 0 and inf x∈E1 f (x) > 0 we can choose n so
that (7.12) holds. Then we can apply Theorem 7.4.6 with A = E1 to show
that,
Px [ζ < ∞] ≥ Px [ζ < ∞ | τA = ∞] Px [τA = ∞] = Px [τA = ∞] ≥ 1 − δ.
Let σn = inf{t ≥ 0 : ξt ∈
/ En }; by hypothesis, σn < ∞ a.s., and the strong
Markov property shows that for any x ∈ Σ,
Px [ζ < ∞] ≥ Ex P[ζ < ∞ | Fσn ] ≥ 1 − δ.
Since δ > 0 was arbitrary, the result follows.
Now we turn to the criterion for integrability of the explosion time,
Theorem 7.4.1.
Proof of Theorem 7.4.1. Suppose that (b) holds. First suppose that A = ∅.
Let Yt = fˆ(ξt ) for t ≥ 0. Then (Yt , t ∈ R+ ) is an R+ -valued process adapted
to (Ft , t ∈ R+ ). We write τ∂ := inf{t ≥ 0 : ξt = ∂} for the hitting time of ∂;
note
τ∂ = ζ = inf{t ≥ 0 : Yt = 0}.
The condition Γf (x) ≤ −ε for all x ∈ Σ shows that E[dYt∧τ∂ | Ft− ] ≤
−ε1{t ≤ τ∂ }dt. Then (a) follows from an application of Lemma 7.3.6.
Next suppose that A 6= ∅ is finite. First let us show that we can modify
the function f so that (b) holds with A a singleton. So suppse A is a
set with at least two elements, and choose x, y distinct elements of A. By
irreducibility, there exists a path of distinct elements z0 , z1 , . . . , zm ∈ Σ with
z0 = x and zm = y, such that Γzk zk+1 ≥ δ > 0 for k = 0, 1, . . . , m − 1. Define
a new function f1 : Σ → R by taking f1 (z) = f (z) for z 6= z1 and f1 (z1 ) =
f (z1 ) − M1 for some constant M1 ∈ R+ chosen so that Γf1 (z0 ) ≤ −ε; then
Γf1 (z) ≤ −ε for all z ∈ (Σ ∪ {z0 }) \ (A ∪ {z1 }). Iteratively, define fk (k =
2, . . . , m) by setting fk (z) = fk−1 (z) for z 6= zk and fk (zk ) = fk−1 (zk ) − Mk
for Mk ∈ R+ chosen so that Γfk (zk−1 ) ≤ −ε; then Γfk (z) ≤ −ε for all
352
Chapter 7. Markov chains in continuous time
z ∈ (Σ ∪ {z0 , z1 , . . . , zk−1 }) \ (A ∪ {zk }). At the
P last step of this procedure,
we obtain fm : Σ → R with inf x∈Σ fm (x) ≥ − m
k=1
P Mk and fm (z) ≤ −ε for
all z ∈ Σ \ (A \ {x}). So taking f 0 (z) = fm (z) + m
k=1 Mk we have replaced
0
f by a function f that satisfies the conditions in (b) but with the size of the
exceptional set A reduced by 1. Iterating this procedure we can ultimately
reduce the exceptional set to a singleton. Thus it suffices to suppose that
A = {x}.
Define N := #{t ∈ (0, ζ) : ξt− 6= y, ξt = y}, the total number of returns
to y. Then set T0 := 0 and for 1 ≤ k < N +2 define recursively the quantities
Hk := inf{t ≥ 0 : ξTk−1 +t 6= y};
τk := (ζ − Tk−1 ) ∧ inf{t > Hk : ξTk−1 +t = y};
Tk := τ1 + · · · + τk .
In words, τk are the successive times between returns to y, Hk are the
associated holding times, and Tk is the total time elapsed on the kth return
to y. For k ≥ N + 2, set Tk = ζ and τk = 0. Let Ak := {Tk−1 + τk < ζ} =
{k ≤ N }, the event that the process returns to y for the kth time. Note
that Tk−1 + τk = ζ for k ≥ N + 1.
With this notation in place, we may write
ζ = τ1 + τ2 + · · · + τN +1 =
∞
X
k=1
τk
k−1
Y
1(Aj ).
(7.13)
j=1
By the strong Markov property, we have that on Ak−1 the distribution of τk
is the same as the distribution of τ1 , so taking expectations in (7.13) gives
Ex ζ =
∞
X
k=1
Ex [τ1 ] Px [A1 ∩ · · · ∩ Ak−1 ].
Using the strong Markov property and the fact that, by (b)(ii), the Markov
chain is transient, we have that there exists δ > 0 such that Px [A1 ∩ · · · ∩
Ak−1 ] ≤ (1 − δ)k−1 . Thus we get Ex ζ ≤ δ −1 Ex τ1 . Now let Yt = f (ξt ); then
if ξ0 = y 6= x, on {t ≤ τ1 } we have E[dYt | Ft− ] ≤ −εdt, and we may apply
Lemma 7.3.6 to deduce that Ey τ1 ≤ ε−1 f (y). Then
Ex τ1 ≤ γx−1 + ε−1
X
y
Pxy f (y) < ∞
since f ∈ Dom+ (Γ). So we conclude that Ex ζ < ∞, which establishes (a).
353
7.4. Explosion and implosion
On the other hand, suppose that (a) holds. Define fˆ(x) = ε Ex τ∂ for
x ∈ Σ̂. Obviously fˆ(∂) = 0 while 0 < fˆ(x) < ∞ for all x ∈ Σ. Conditioning
on the first move of the embedded Markov chain ξ˜ we get
mf (x) = E[f (ξ˜1 ) − f (x) | ξ˜0 = x] = E[f (ξJ1 ) − f (ξ0 ) | ξ0 = x]
X
Pxy Ey τ∂ − ε Ex τ∂ = ε(Ex τ∂ − Ex σ1 ) − ε Ex τ∂
=ε
y∈Σ\{x}
= −ε Ex σ1 = −
ε
.
γx
Hence Γf (x) ≤ −ε for all x ∈ Σ, and we have the A = ∅ case of (b).
R z dy
Proof of Proposition 7.4.3. Let G(z) = 0 g(y)
. Then G is differentiable,
1
0
with G (z) = g(z) > 0 hence an increasing function of z ∈ [0, b]. Since g is
increasing, G0 is decreasing and hence G is concave satisfying limz→0 G(z) =
0 and limz→∞ G(z) < ∞. Additionally, boundedness of G implies that
G ◦ f ∈ `1+ (Γ). Due to differentiability and concavity of G, we have:
Γ(G ◦ f )(x) = γx E G(f (ξ˜n ) + ∆fn+1 ) − G(f (ξ˜n )) ξ˜n = x
≤ γx G0 (f (x)) E ∆f ξ˜n = x
n+1
1
Γf (x)
= γx
mf (x) =
≤ −1,
g(f (x))
g(f (x))
for all x ∈
/ A; we conclude by Theorem 7.4.1 because G ◦ f is strictly positive
and bounded.
Next we give the proof of the condition for non-explosion, Theorem 7.4.9.
R z dy
Proof of Theorem 7.4.9. Let G(z) = 0 g(y)
. Then G is differentiable, with
1
0
G (z) = g(z) > 0, hence increasing. Since g is increasing, G0 is decreasing,
and hence G is concave. Concavity and differentiability of G imply that
G(y + d) − G(y) ≤ dG0 (y) for all y and d with y ≥ 0 and y + d ≥ 0. Thus
0 ≤ E[G(f (ξ˜n ) + ∆fn+1 ) | ξ˜n = x] ≤ G(f (x)) +
mf (x)
< ∞,
g(f (x))
so that, setting Un := G(f (ξ˜n )), we obtain
mf (x)
E[Un+1 − Un | ξ˜n = x] ≤
≤ γx−1 ,
g(f (x))
(7.14)
354
Chapter 7. Markov chains in continuous time
by hypothesis (b). Now let Vn := Un −
Pn−1
k=0
γξ̃−1 . Then Vn is a superk
martingale adapted to F̃n . Let y ∈ R+ and define
n
n
o
X
σ := min n ≥ 0 :
γξ̃−1 ≥ y .
k=0
k
Then Vn∧σ is a supermartingale bounded below by −y, hence VP
n∧σ converges
−1
to some V∞ with V∞ < ∞ a.s. On the event {σ = ∞} we have ∞
k=0 γξ̃k < y
and so γ → ∞. This can only happen if ξ˜n visits every state only finitely
ξ̃n
often, and so f (ξ˜n ) and G(f (ξ˜n )) both tend to ∞. But on {σ = ∞},
Vn∧σ ≥ G(f (ξ˜n )) − y → ∞.
But since V∞ < ∞ a.s., we must have P[σ = ∞] = 0. In other words, we
have shown that
∞
X
γξ̃−1 ≥ y, a.s.
k=0
k
Since y was arbitrary, the result follows by Chung’s criterion (7.11).
Now we turn to the results on implosion.
Proof of Proposition 7.4.11. Note that, for x ∈
/ A, σ0 ≤ τA , where σ0 is the
holding time at the initial state, so that γx−1 = Ex σ0 ≤ Ex τA ≤ C for all
x∈
/ A. Hence, since A is finite, γ := inf x∈Σ γx > 0.
Fix z ∈ Σ. Let λ = min{n ∈ Z+ : ξ˜n = z}. Irreducibility and finiteness
of A means that there exist k ∈ N and δ > 0 such that
min P[λ ≤ k | ξ˜0 = a] ≥ δ.
a∈A
By Markov’s inequality,
P[σn ≥ t] ≤ t−1 E σn ≤ t−1 γ −1 ≤
δ
,
2k
2k
provided we choose t = t0 := γδ
. Then
h
i
Pa λ ≤ k, max σi ≤ t0 ≥ δ/2,
0≤i≤k−1
which means that Pa [τz ≤ kt0 ] ≥ δ/2 for all a ∈ A. Moreover, Markov’s
inequality also shows that Px [τA ≤ 2C] ≥ 1/2, so, by the Markov property,
with K = 2C + kt0 and ε = δ/4, we have
Px [τz ≤ K] ≥ ε, for all x ∈ Σ.
7.5. Applications
355
It follows that P[τz ≥ (m + 1)K | τz ≥ mK] ≤ (1 − ε) for all m ∈ Z+ , which
yields an exponential tail bound. The result now follows.
Proof of Theorem 7.4.12. Let us prove that (a) implies (b). The condition
Γf (x) ≤ −ε for x ∈
/ F shows, by an application of Lemma 7.3.6, that
Ex τF ≤ ε−1 f (x) so that supx∈Σ Ex τF ≤ ε−1 supx∈Σ f (x) < ∞.
Now, we show that (b) implies (a). Suppose that for a finite A ∈ E,
there exists a constant C such that Ex τA ≤ C for all x ∈ Σ. We use the
same observation as at the start of the proof of Proposition 7.4.11, that
γx−1 ≤ Ex τA ≤ C, and so γ = inf x∈Σ γx > 0. Define
(
0
if x ∈ A,
f (x) =
Ex τA if x ∈
/ A.
Then it is immediate to show that for x 6∈ A, we have Γf (x) ≤ −1.
Proof of Theorem 7.4.13. Under the conditions of the theorem, we may apply Lemma 7.3.9 with Yt = f (ξt ) and Gt = Ft , so that Ta := inf{t ≥ 0 :
Yt ≤ a} = τSa (f ) . We conclude that there exists ε > 0 such that, for all x0
sufficiently large,
1
Px0 [τSa (f ) > t + εf (x0 )] ≥ .
2
Therefore Ex0 τSa (f ) ≥ 12 εf (x0 ); since f → ∞, this expectation cannot be
bounded uniformly in x0 , so implosion is excluded.
R z dy
1
Proof of Proposition 7.4.14. Let G(z) = 0 g(y)
. Since G0 (z) = g(z)
> 0 the
function G is increasing with G(0) = 0 and G(b) = B. Since g is increasing
G0 = g1 is decreasing, hence the function G is concave. Then concavity gives
the bound
mf (x)
mG◦f (x) ≤ G0 (f (x))mf (x) =
.
g(f (x))
The condition imposed in the statement of the proposition implies that
ΓG ◦ f (x) ≤ −1. We conclude from Lemma 7.3.6.
7.5
7.5.1
Applications
Lamperti processes in continuous time
Let (ξt , t ∈ R+ ) be an irreducible time-homogeneous continuous-time Markov
chain on Σ ⊂ R+ assumed to be locally finite with inf Σ = 0 and sup Σ =
+∞; then transience can occur only to +∞. We suppose that the associated
356
Chapter 7. Markov chains in continuous time
jump chain ξ˜n is of Lamperti-type in the manner of Section 3.2; specifically,
writing ∆n := ξ˜n+1 − ξ˜n for the increments of the jump chain, we suppose
that (M1) holds, and denoting the increment moment functions by
µk (x) := E[(ξ˜n+1 − ξ˜n )k | ξ˜n = x], k ∈ N, x ∈ Σ,
we also suppose that (M2) holds. For the jump rates of the continuous-time
chain, we suppose that γx = x2−κ for some κ ∈ R.
First we give a result on passage-time moments in the recurrent case
(compare Theorem 3.2.6).
Proposition 7.5.1. Suppose that (M1) and (M2) hold, and that γx = x2−κ
for some κ ∈ R+ . Suppose that 2a < b. Define s0 := b−2a
bκ .
(i) If s < s0 and s < κp , then Ex [τAs ] < ∞.
(ii) If s0 < s < κp , then Ex [τAs ] = ∞.
Proof. We use the Lyapunov function g(x) = (1 + x)α , where α ∈ (0, p) will
be specified later. Similarly to Lemma 3.4.1, a Taylor’s formula calculation
shows that
mg (x) = E[(1 + x + ∆n )α − (1 + x)α | ξ˜n = x]
α
= (2xµ1 (x) + (α − 1)µ2 (x)) xα−2 + o(xα−2 ),
2
so assuming (M2) we have from (7.4) that
Γg(x) =
α
(2a + (α − 1)b + o(1)) xα−κ .
2
Take f (x) = (1 + x)κ/2 and set α = sκ/2; then provided α <
from (7.15) that for some c > 0 and all x sufficiently large
(7.15)
b−2a
b ,
we have
Γf s (x) = Γg(x) ≤ −c(1 + x)α−κ = −cf s−2 (x).
2(b−2a)
Hence Theorem 7.3.1 applies with p = s for any s < 2p
κ ∧
bκ , proving (i).
α
For part (ii), we again use the function g(x) = (1 + x) , α ∈ (0, p). Take
f (x) = (1 + x)κ and α = sκ. Then from (7.15) we see that if α > b−2a
b , then
s
Γf (x) = Γg(x) ≥ 0 for all x sufficiently large. Equation (7.15) (applied
with α = κ and again with α = κr) also shows that Γf (x) ≥ −c1 and, for
any r ∈ (1, p/α), that Γf r (x) ≤ c2 f r−1 (x) for all x sufficiently large. Thus
p
Theorem 7.3.2 applies whenever b−2a
bκ < s < κ .
357
7.5. Applications
Next we turn to explosion in the transient case.
Proposition 7.5.2. Suppose that (M1) and (M2) hold, and that γx = x2−κ
for some κ ∈ R. Suppose that 2a > b.
(i) If κ < 0, then Ex ζ < ∞ for all x ∈ Σ.
(ii) If κ ≥ 0, then Px [ζ = ∞] = 1 for all x ∈ Σ.
Proof. We use the Lyapunov function f (x) = (1 + x)−α for α ∈ (0, p − 2).
Similarly to Lemma 3.4.1, a Taylor’s formula calculation shows that
mf (x) = E[(1 + x + ∆n )−α − (1 + x)−α | ξ˜n = x]
α
= − (2xµ1 (x) − (1 + α)µ2 (x)) x−α−2 + o(x−α−2 ),
2
so assuming (M2) we have from (7.4) that
Γf (x) = −
α
(2a − (1 + α)b + o(1)) x−α−κ .
2
Then for any κ < 0 we may choose α ∈ (0, 2a−b
b ) with α + κ ≤ 0, and then
we may apply Theorem 7.4.1 to establish Ex ζ < ∞. This proves part (i).
For part (ii) we use the Lyapunov function f (x) = log(1 + x). Then
∆n
˜
mf (x) = E log 1 +
− 1 ξn = x
1+x
1
= (2xµ1 (x) − µ2 (x)) x−2 + o(x−2 ),
2
so assuming (M2) we have from (7.4) that
Γf (x) =
1
(2a − b + o(1)) x−κ .
2
(7.16)
Thus for κ ≥ 0 we may apply Theorem 7.4.9 with g(x) = x to establish
non-explosion.
Finally, we give a result on implosion.
Proposition 7.5.3. Suppose that (M1) and (M2) hold, and that γx = x2−κ
for some κ ∈ R+ . Suppose that 2a < b.
(i) If κ < 0, then the chain implodes.
(ii) If κ ≥ 0, then the chain does not implode.
358
Chapter 7. Markov chains in continuous time
Proof. For part (i), we use the Lyapunov function f (x) = 1 − (1 + x)−α ,
where α ∈ (0, p − 2). Then a Taylor’s formula calculation shows that
mf (x) = E[(1 + x)−α − (1 + x + ∆n )−α | ξ˜n = x]
α
= (2xµ1 (x) − (1 + α)µ2 (x)) x−α−2 + o(x−α−2 ),
2
so assuming (M2) we have from (7.4) that
Γf (x) =
α
(2a − (1 + α)b + o(1)) x−α−κ .
2
Then for any κ < 0 we may choose α > 0 with α + κ ≤ 0, and then we may
apply Theorem 7.4.12 to deduce implosion.
For part (ii) we use the Lyapunov function f (x) = logα (1+x), α ∈ (0, 1).
Then a Taylor’s formula calculation shows that
mf (x) =
α −2
x logα−1 x (2a − b + o(1)) ,
2
Γf (x) =
α −κ
x logα−1 x (2a − b + o(1)) .
2
so that
For κ ≥ 0, it follows that Γf (x) ≥ −ε for all x sufficiently large, and for
r > 1 such that αr < 1 we have Γf r (x) ≤ cf r−1 (x) for all x sufficiently
large. We conclude by Theorem 7.4.13 that implosion does not occur.
7.5.2
Simple symmetric random walk
Take as state space Σ = Zd , d ∈ N, and suppose that the jump chain is
simple symmetric random walk on Σ.
First we study explosion for the transient case, d ≥ 3.
Proposition 7.5.4. Suppose that d ≥ 3 and γx = kxk2−κ for κ ∈ R. Then
the random walk ξ˜t is explosive if κ < 0 and non-explosive if κ ≥ 0.
Proof. To prove explosion, we use the Lyapunov function f (x) = (1 +
kxk)−α . Here a Taylor’s formula calculation similar to that in Example 2.5.9
shows that
α
mf (x) = kxk−2−α (α + 2 − d + o(1)),
d
so that
α
Γf (x) = kxk−α−κ (α + 2 − d + o(1)).
d
359
Bibliographical notes
Thus for κ < 0 and d ≥ 3 we may choose α > 0 such that α + κ ≤ 0 and
Γf (x) ≤ −ε for all x outside a finite set. Then we may apply Theorem 7.4.5
with En = B(0; n) ∩ Σ to establish explosion.
To prove non-explosion, we use the Lyapunov function f (x) = log(1 +
kxk). Then a Taylor’s formula calculation gives
1
2
mf (x) =
1 − + o(1) kxk−2 ,
2
d
so that
Γf (x) =
1
2
2
1 − + o(1) kxk−κ .
d
Thus for κ ≥ 0 we have Γf (x) ≤ cf (x) and we may apply Theorem 7.4.9
with g(x) = x to establish non-explosion.
Finally we study implosion in the recurrent case d ∈ {1, 2}.
Proposition 7.5.5. Suppose that d ∈ {1, 2} and γx = kxk2−κ for κ ∈ R.
Then the random walk ξ˜t implodes if κ < 0 but does not implode if κ ≥ 0.
Proof. The proof is similar to our previous calculations: for implosion we
use the function f (x) = 1 − (1 + kxk)−α with Theorem 7.4.12, while for
non-implosion we use f (x) = logα (1 + kxk) with Theorem 7.4.13.
Bibliographical notes
Section 7.1
The results of this chapter are largely based on [220] For definitions of
continuous-time Markov chains, and the exponential holding-time and jump
chain constructions, we refer to standard texts such as [45, 4, 243].
Section 7.2
As stated in the text, Theorems 7.2.1 and 7.2.2 are direct consequence of the
corresponding discrete-time results, and so can be traced back to Foster [108]
(see the bibliographical notes to Chapter 2). The sufficient condition for recurrence in Theorem 7.2.1 can be found for example for the case of F a
singleton in an unpublished technical report by Miller [240], and for finite
F in [303]; the if and only if statement is given in Theorem 4 of [305].
Miller [240] also gives a more restrictive version of the condition for transience in Theorem 7.2.2.
360
Chapter 7. Markov chains in continuous time
Section 7.3
Theorems 7.3.1 and 7.3.2 on existence and non-existence of passage-time
moments are contained in Theorem 1.5 of [220]. The proofs mirror their
discrete-time analogues from [10] (see also Section 2.7) and also parallel
those for diffusion processes from [224].
If γx is bounded away from 0 and ∞, then since the Markov chain can
be stochastically controlled between two Markov chains with constant jump
rates, the asymptotics for the continuous-time process are identical to those
of the jump chain and the discrete-time results of [10]; thus Theorems 7.3.1
and 7.3.2 are of most interest when supx∈Σ γx = ∞ or inf x∈Σ γx = 0. As
in discrete time, existence and non-existence of moments of passage times
leads to estimates on rates of convergence to stationarity: see e.g. [285].
The continuous-time analogue of Foster’s criterion for positive recurrence, Theorem 7.3.4, is essentially Theorem 3(i) of Tweedie [305]; compare
also Theorem 1.7 in [220]. The fact that the drift condition (b) implies positive recurrence (a) was shown by Reuter [268, p. 426] in the case where F
is a singleton, and, in the case of finite F , in Lemma 1 of Kingman [171]; see
also Theorem 2.3(i) of [303]. Both directions of the statement were given for
the case of F a singleton in Theorem 2 of Miller [240]. The key integrability
condition for stopping times, Lemma 7.3.6, is an improvement of Lemma 2.3
of [220].
Criteria for a continuous-time Markov chain to be not positive-recurrent
analogous to the discrete-time Corollary 2.6.11 are given in [38], and extensions based on a continuous-time version of Kaplan’s condition are given
in [169]; these results are typically not as sharp as Theorem 7.3.2.
Section 7.4
In the case where A = ∅, Theorem 7.4.1 is Theorem 1.9 of [220], while the
fact that the uniform global drift condition (b) implies Px [ζ < ∞] = 1 in
this case is contained in Theorem 4.3.6 of [296]; similar sufficient conditions
of explosion are established in [313] for Markov chains on locally compact
separable metric spaces. The case where A 6= e∅ of Theorem 7.4.1 is new.
Permitting a non-empty exceptional set is very useful in applications, where
often Taylor’s theorem is used.
Theorem 7.4.5 is new. The conditional explosion result, Theorem 7.4.6,
is Theorem 1.12 in [220]. Results for establishing P[ζ < ∞ | ξt → ∞] are
given in [161, 162].
Theorem 7.4.9 on non-explosion is Theorem 1.14 of [220]; earlier versions
Bibliographical notes
361
of this result were established by Khas’minskii [168], in Theorem (1.11)
of Chen [37] (both with g(x) = x), and in Theorem 1 of Kersting and
Klebaner [161] (in the case of processes on R+ , with f the identity); see
also Theorem 1 of [162] and Theorem 1 of [125], which has g(x) = x. The
argument of [161] is the basis for the one given here. A similar idea was
used in [85] to prove non-explosion for Markov chains on separable metric
spaces, while Lyapunov functions are used in [39] for the study of explosion
for Markov chains on Z and time-dependent holding times.
The notion of implosion was introduced in [220]; it is reminiscent of
Doeblin’s condition for general Markov chains [76]. Theorems 7.4.12 and 7.4.13
improve slightly on results contained in Theorem 1.15 of [220].
362
Chapter 7. Markov chains in continuous time
Glossary of named
assumptions
Section 2.8: Growth bounds on trajectories
(A0) Suppose that for some na ∈ Z+ the function a : [na , ∞) → (0, ∞) is
such thatP
(i) x 7→ a(x) is increasing on x ≥ na ; (ii) limx→∞ a(x) = ∞;
1
and (iii) n≥na a(n)
< ∞.
(A1) Suppose that for some na ∈ Z+ the function a : [na , ∞) → (0, ∞) is
such thatP
(i) x 7→ a(x) is increasing on x ≥ na ; (ii) limx→∞ a(x) = ∞;
1
and (iii) n≥na na(n)
< ∞.
Section 3.2: Makovian Lamperti problem
(M0) Let Xn be an irreducible, time-homogeneous Markov chain on Σ, a
locally finite, unbounded subset of R+ , with 0 ∈ Σ.
(M1) Suppose that for some p > 2,
sup E[|∆n |p | Xn = x] < ∞.
x∈Σ
(M2) Suppose that there exist a ∈ R and b ∈ (0, ∞) such that
lim µ2 (x) = b, and lim xµ1 (x) = a.
x→∞
x→∞
Section 3.3: General Lamperti problem
(L0) Let Xn be a stochastic process adapted to a filtration Fn and taking
values in S ⊆ R+ with inf S = 0 and sup S = +∞.
(L1) Suppose that for some p > 2, δ ∈ (0, p − 2], and C ∈ R+ ,
E[|∆n |p | Fn ] ≤ C(1 + Xn )p−2−δ , a.s., for all n ≥ 0.
363
364
Glossary of named assumptions
(L2) Suppose that lim supn→∞ Xn = +∞, a.s.
(L3) Suppose that for each x ∈ R+ there exist rx ∈ N and δx > 0 such
that, for all n ≥ 0,
i
h
P
max Xm ≥ x Fn ≥ δx , on {Xn ≤ x}.
n≤m≤n+rx
Section 3.6: Irreducibility and regeneration
(I) Suppose that for each x, y ∈ S there exist constants m(x, y) ∈ N
and ϕ(x, y) > 0, and a collection of Fn -measurable random variables
Mn (y) ∈ Z+ , such that, for all n ≥ 0, Mn (y) ≤ m(Xn , y), a.s., and
P[Xn+Mn (y) = y | Fn ] ≥ ϕ(Xn , y), a.s.
(I0 ) Suppose that for each x, y ∈ S there exist constants m(x, y) ∈ N and
ϕ(x, y) > 0 such that, for all n ≥ 0,
i
h
m(X ,y)
P ∪k=0 n {Xn+k = y} Fn ≥ ϕ(Xn , y), a.s.
(Ra) Suppose that, for all k ∈ N, the distribution of Ek on {νk−1 < ∞} is
the same as the distribution of E1 .
(Rb) Suppose that, on {N = ∞}, (Ek )k∈N is an i.i.d. sequence.
(R) Suppose that X0 = 0 and both (Ra) and (Rb) hold.
Section 3.12: Supercritical Lamperti problem
(L5) Suppose that for some C ∈ R+ , E[|∆n |p | Fn ] ≤ C, a.s.
Section 4.1: Many-dimensional random walks
(MD0) Let d ≥ 1. Suppose that (ξn , n ≥ 0) is a discrete-time, timehomogeneous Markov process on an unbounded subset Σ of Rd , with
0 ∈ Σ.
(MD1) Suppose that µ(x) = 0 for all x ∈ Σ.
(MD2) There exists v > 0 such that tr M (x) ≥ v for all x ∈ Σ.
(MD3) Suppose that lim supn→∞ kξn k = +∞, a.s.
(MD4) Suppose that there exists v0 > 0 such that,
inf e> M (x)e ≥ v0 , for all x ∈ Σ.
e∈Sd
365
Glossary of named assumptions
Section 4.2: Elliptic random walk
(E1) Suppose that there exists a positive-definite matrix function σ 2 with
domain Sd−1 such that, as r → ∞,
ε(r) :=
sup
x∈Σ:kxk≥r
kM (x) − σ 2 (x̂)kop → 0.
(E2) Suppose that there exist constants U and V with 0 < U ≤ V < ∞
such that, for all u ∈ Sd−1 ,
hu, uiu = U, and tr σ 2 (u) = V.
Section 4.4: Centrally biased random walk
(CB1) Suppose that for some β ∈ [0, 1), ρ ∈ R+ , and p > 1+β, supx∈Σ Ex [kθ0 kp ] <
∞, and
µ(x) = ρkxk−β x̂ + O(kxk−β log−2 kxk),
(CB2) Suppose that, for some p > 2, supx∈Σ Ex [kθ0 kp ] < ∞, and that
there exists C < ∞ such that
kµ(x)k ≤ C(1 + kxk)−1 , for all x ∈ Σ.
(CB3) Suppose that there exist ρ ∈ R and σ 2 ∈ (0, ∞) for which, as
kxk → ∞,
µ(x) = ρx̂kxk−1 + o(kxk−1 log−1 kxk); and
kM (x) − σ 2 Ikop = o(log−1 kxk).
Section 5.2: Directional transience
(H0) Let Xn be a time-homogeneous Markov chain on Σ ⊆ R with 0 ∈ Σ,
inf Σ = −∞ and sup Σ = +∞. Suppose that X0 = 0.
(H1) There exist α > 0, c > 0, and y0 < ∞ for which, for all x ∈ Σ and all
−α .
y ≥ y0 , P[∆+
n > y | Xn = x] ≥ cy
(H2) There exist α ∈ (0, 1), c > 0, and y0 < ∞ for which, for all x ∈ Σ and
+
1−α .
all y ≥ y0 , E[∆+
n 1{∆n ≤ y} | Xn = x] ≥ cy
β
(H3) There exist β > 0 and C < ∞ for which E[(∆−
n ) | Xn = x] ≤ C for
all x ∈ Σ.
366
Glossary of named assumptions
Section 5.3: Oscillating random walk
(Os1) Let vα ∈ Dα and vβ ∈ Dβ , for some α, β > 0. For x, y ∈ R, let
(
vα (−y) if x ≥ 0,
wx (y) :=
vβ (y)
if x < 0.
(Os2) Let P = (p(i, j), i, j ∈ S) be an irreducible stochastic matrix, and
let (µk , k ∈ S) denote the corresponding invariant distribution.
(Os3) Suppose that, for each k ∈ S, we have an exponent αk ∈ (0, ∞) and
a density function vk ∈ Dαk . Then suppose that, for all y ∈ R, wk is
given by wk (y) = vk (−y) .
Section 6.2: Random strings in random environment
(N) E log+ kAkop < ∞, where log+ x = max{log x, 0}.
(D) E log(1/ri` ) < ∞, for all i ∈ A.
Section 6.3: Stochastic billiards
(SB) Suppose that for some α0 ∈ (0, π/2),
P[0 < |α| < α0 ] = 1 and E tan α = 0,
and that for some γ < 1 and A ≥ A0 as in Lemma 6.3.1,
D := D(γ, A) := {(x, y) ∈ R2 : x ≥ A, |y| ≤ xγ }.
Section 7.1: Markov chains in continuous time
(CT) Suppose that 0 < γx < ∞ for all x ∈ Σ, and that the jump chain is
irreducible.
Bibliography
[1] M. Abramowitz and I. A. Stegun, Handbook of mathematical functions with formulas, graphs, and mathematical tables, National Bureau of Standards Applied Mathematics Series, vol. 55, For sale by
the Superintendent of Documents, U.S. Government Printing Office,
Washington, D.C., 1964.
[2] K. S. Alexander, Excursions and local limit theorems for Bessel-like
random walks, Electron. J. Probab. 16 (2011), no. 1, 1–44.
[3] O. S. M. Alves, F. P. Machado, and S. Yu. Popov, The shape theorem
for the frog model, Ann. Appl. Probab. 12 (2002), no. 2, 533–546.
[4] W. J. Anderson, Continuous-time Markov chains, Springer Series in
Statistics: Probability and its Applications, Springer-Verlag, New
York, 1991.
[5] E. D. Andjel, M. V. Menshikov, and V. V. Sisko, Positive recurrence
of processes associated to crystal growth models, Ann. Appl. Probab.
16 (2006), no. 3, 1059–1085.
[6] S. Asmussen, Applied probability and queues, second ed., Applications
of Mathematics (New York), vol. 51, Springer-Verlag, New York, 2003,
Stochastic Modelling and Applied Probability.
[7] S. Aspandiiarov and R. Iasnogorodski, Tails of passage-times and an
application to stochastic processes with boundary reflection in wedges,
Stochastic Process. Appl. 66 (1997), no. 1, 115–145.
[8]
, Asymptotic behaviour of stationary distributions for countable
Markov chains, with some applications, Bernoulli 5 (1999), no. 3, 535–
569.
367
368
[9]
Bibliography
, General criteria of integrability of functions of passage-times
for nonnegative stochastic processes and their applications, Theory
Probab. Appl. 43 (1999), 343–369, translated from Teor. Veroyatnost.
i Primenen. 43 (1998) 509–539 (in Russian).
[10] S. Aspandiiarov, R. Iasnogorodski, and M. Menshikov, Passage-time
moments for nonnegative stochastic processes and an application to
reflected random walks in a quadrant, Ann. Probab. 24 (1996), no. 2,
932–960.
[11] K. B. Athreya and P. E. Ney, Branching processes, Dover Publications,
Inc., Mineola, NY, 2004, reprint of the 1972 original.
[12] O. Aydogmus, A. P. Ghosh, S. Ghosh, and A. Roitershtein, Colored
maximal branching processes, Theory Probab. Appl. 59 (2015), no. 4,
663–672, translated from Teor. Veroyatnost. i Primenen. 59 (2014)
790–800 (in Russian).
[13] K. Azuma, Weighted sums of certain dependent random variables,
Tôhoku Math. J. (2) 19 (1967), 357–367.
[14] L. Bachelier, Théorie de la spéculation, Ann. Sci. École Norm. Sup.
(3) 17 (1900), 21–86.
[15] M. N. Barber and B. W. Ninham, Random and restricted walks: theory
and applications, Gordon and Breach, New York, 1970.
[16] L. E. Baum, On convergence to +∞ in the law of large numbers, Ann.
Math. Statist. 34 (1963), 219–222.
[17] V. Belitsky, P. A. Ferrari, M. V. Menshikov, and S. Yu. Popov, A mixture of the exclusion process and the voter model, Bernoulli 7 (2001),
no. 1, 119–144.
[18] M. Benaı̈m, Vertex-reinforced random walks and a conjecture of Pemantle, Ann. Probab. 25 (1997), no. 1, 361–392.
[19] I. Benjamini, R. Izkovsky, and H. Kesten, On the range of the simple
random walk bridge on groups, Electron. J. Probab. 12 (2007), no. 20,
591–612.
[20] I. Benjamini, G. Kozma, and B. Schapira, A balanced excited random
walk, C. R. Math. Acad. Sci. Paris 349 (2011), no. 7-8, 459–462.
Bibliography
369
[21] I. Benjamini and D. B. Wilson, Excited random walk, Electron. Comm.
Probab. 8 (2003), 86–92 (electronic).
[22] J. Bérard and A. Ramı́rez, Central limit theorem for the excited random walk in dimension D ≥ 2, Electron. Comm. Probab. 12 (2007),
303–314 (electronic).
[23] H. C. Berg, Random walks in biology, expanded edition ed., Princeton
University Press, New Jersey, 1993.
[24] J. Bertoin and I. Kortchemski, Self-similar scaling limits of markov
chains on the positive integers, Preprint. (2016).
[25] P. Billingsley, Probability and measure, third ed., Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons Inc., New
York, 1995.
[26] N. H. Bingham, Random walk on spheres, Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 22 (1972), 169–192.
[27] D. Blackwell, On transient Markov processes with a countable number
of states and stationary transition probabilities, Ann. Math. Statist.
26 (1955), 654–658.
[28] D. Blackwell and D. Freedman, A remark on the coin tossing game,
Ann. Math. Statist 35 (1964), 1345–1347.
[29] J. Bojarski, R. Smolenski, A. Kempski, and P. Lezynski, Pearson’s
random walk approach to evaluating interference generated by a group
of converters, Appl. Math. Comput. 219 (2013), no. 12, 6437–6444.
[30] P. Bougerol, Oscillation des produits de matrices aléatoires dont
l’exposant de Lyapounov est nul, Lyapunov exponents (Bremen, 1984),
Lecture Notes in Math., vol. 1186, Springer, Berlin, 1986, pp. 27–36.
[31] P. Bougerol and J. Lacroix, Products of random matrices with applications to Schrödinger operators, Progress in Probability and Statistics,
vol. 8, Birkhäuser Boston Inc., Boston, MA, 1985.
[32] P. Bovet and S. Benhamou, Spatial analysis of animals’ movements
using a correlated random walk model, J. Theoret. Biol. 131 (1988),
419–433.
[33] L. Breiman, Transient atomic Markov chains with a denumerable number of states, Ann. Math. Statist. 29 (1958), 212–218.
370
Bibliography
[34] M. Campanino and D. Petritis, Random walks on randomly oriented
lattices, Markov Process. Related Fields 9 (2003), no. 3, 391–412.
[35] B. Carazza, The history of the random-walk problem: considerations
on the interdisciplinarity in modern physics, Riv. Nuovo Cimento (2)
7 (1977), no. 3, 419–427.
[36] J. Černý, Moments and distribution of the local time of a twodimensional random walk, Stochastic Process. Appl. 117 (2007), no. 2,
262–270.
[37] M.-F. Chen, On three classical problems for Markov chains with continuous time parameters, J. Appl. Probab. 28 (1991), no. 2, 305–320.
[38] B. D. Choi and B. Kim, Non-ergodicity criteria for denumerable continuous time Markov processes, Oper. Res. Lett. 32 (2004), no. 6,
574–580.
[39] P.-L. Chou and R. Z. Khas0 minskiı̆, The method of Lyapunov function
for the analysis of the absorption and explosion of Markov chains,
Probl. Inf. Transm. 47 (2011), no. 3, 232–250, translated from Problemy Peredachi Informatsii 47 (2011) 19–38 (in Russian).
[40] Y. S. Chow, H. Robbins, and H. Teicher, Moments of randomly stopped
sums, Ann. Math. Statist. 36 (1965), 789–799.
[41] Y. S. Chow and H. Teicher, Probability theory: Independence, interchangeability, martingales, third ed., Springer Texts in Statistics,
Springer-Verlag, New York, 1997.
[42] Y. S. Chow and C.-H. Zhang, A note on Feller’s strong law of large
numbers, Ann. Probab. 14 (1986), no. 3, 1088–1094.
[43] F. Chung and L. Lu, Complex graphs and networks, CBMS Regional
Conference Series in Mathematics, vol. 107, Published for the Conference Board of the Mathematical Sciences, Washington, DC; by the
American Mathematical Society, Providence, RI, 2006.
[44] K. L. Chung, On the maximum partial sums of sequences of independent random variables, Trans. Amer. Math. Soc. 64 (1948), 205–233.
[45]
, Markov chains with stationary transition probabilities, Second
edition. Die Grundlehren der mathematischen Wissenschaften, Band
104, Springer-Verlag New York, Inc., New York, 1967.
Bibliography
[46]
371
, A course in probability theory, third ed., Academic Press Inc.,
San Diego, CA, 2001.
[47] K. L. Chung and W. H. J. Fuchs, On the distribution of values of sums
of random variables, Mem. Amer. Math. Soc. 1951 (1951), no. 6, 12.
[48] Peter Clifford and Aidan Sudbury, A model for spatial conflict, Biometrika 60 (1973), 581–588.
[49] E. A. Codling, M. J. Plank, and S. Benhamou, Random walk models
in biology, J. R. Soc. Interface 5 (2008), no. 25, 813–834.
[50] J. W. Cohen, Analysis of random walks, Studies in Probability, Optimization and Statistics, vol. 2, IOS Press, Amsterdam, 1992.
[51]
, On the random walk with zero drifts in the first quadrant of
R2 , Comm. Statist. Stochastic Models 8 (1992), no. 3, 359–374.
[52] F. Comets, M. V. Menshikov, and S. Yu. Popov, One-dimensional
branching random walk in a random environment: a classification,
Markov Process. Related Fields 4 (1998), no. 4, 465–477, I Brazilian
School in Probability (Rio de Janeiro, 1997).
[53] F. Comets, M. V. Menshikov, S. Volkov, and A. R. Wade, Random
walk with barycentric self-interaction, J. Stat. Phys. 143 (2011), no. 5,
855–888.
[54] F. Comets and S. Popov, On multidimensional branching random
walks in random environment, Ann. Probab. 35 (2007), no. 1, 68–
114.
[55]
, Shape and local growth for multidimensional branching random walks in random environment, ALEA Lat. Am. J. Probab. Math.
Stat. 3 (2007), 273–299.
[56] F. Comets, S. Popov, G. M. Schütz, and M. Vachkovskaia, Billiards in
a general domain with random reflections, Arch. Ration. Mech. Anal.
191 (2009), no. 3, 497–537.
[57]
, Knudsen gas in a finite random tube: transport diffusion and
first passage properties, J. Stat. Phys. 140 (2010), no. 5, 948–984.
[58] Francis Comets, Mikhail Menshikov, and Serguei Popov, Lyapunov
functions for random walks and strings in random environment, Ann.
Probab. 26 (1998), no. 4, 1433–1445.
372
Bibliography
[59] P. Coolen-Schrijner and E. A. van Doorn, Analysis of random walks using orthogonal polynomials, J. Comput. Appl. Math. 99 (1998), no. 12, 387–399.
[60] E. Crane, N. Georgiou, S. Volkov, A. R. Wade, and R. J. Waters, The
simple harmonic urn, Ann. Probab. 39 (2011), no. 6, 2119–2177.
[61] E. Csáki, On the lower limits of maxima and minima of Wiener process
and partial sums, Z. Wahrsch. Verw. Gebiete 43 (1978), no. 3, 205–
221.
[62] E. Csáki, M. Csörgő, A. Földes, and P. Révész, On the local time of
random walk on the 2-dimensional comb, Stochastic Process. Appl.
121 (2011), no. 6, 1290–1314.
[63] E. Csáki, A. Földes, and P. Révész, Joint asymptotic behavior of local
and occupation times of random walk in higher dimension, Studia Sci.
Math. Hungar. 44 (2007), no. 4, 535–563.
[64]
, Transient nearest neighbor random walk and Bessel process,
J. Theoret. Probab. 22 (2009), no. 4, 992–1009.
[65]
, Transient nearest neighbor random walk on the line, J. Theoret. Probab. 22 (2009), no. 1, 100–122.
[66] E. Csáki, P. Révész, and J. Rosen, Functional laws of the iterated
logarithm for local times of recurrent random walks on Z2 , Ann. Inst.
H. Poincaré Probab. Statist. 34 (1998), no. 4, 545–563.
[67] J. De Coninck, F. Dunlop, and T. Huillet, Random walk weakly attracted to a wall, J. Stat. Phys. 133 (2008), no. 2, 271–280.
[68] D. DeBlassie and R. Smits, The influence of a power law drift on the
exit time of Brownian motion from a half-line, Stochastic Process.
Appl. 117 (2007), no. 5, 629–654.
[69] P. Deheuvels and P. Révész, Simple random walk on the line in random
environment, Probab. Theory Relat. Fields 72 (1986), no. 2, 215–230.
[70] F. den Hollander, Random polymers, Lecture Notes in Mathematics,
vol. 1974, Springer-Verlag, Berlin, 2009, Lectures from the 37th Probability Summer School held in Saint-Flour, 2007.
Bibliography
373
[71] F. den Hollander, M. V. Menshikov, and S. Yu. Popov, A note on
transience versus recurrence for a branching random walk in random
environment, J. Statist. Phys. 95 (1999), no. 3-4, 587–614.
[72] D. Denisov, D. Korshunov, and V. Wachtel, Potential analysis for
positive recurrent Markov chains with asymptotically zero drift: powertype asymptotics, Stochastic Process. Appl. 123 (2013), no. 8, 3027–
3051.
[73] D. È. Denisov and S. G. Foss, On transience conditions for Markov
chains and random walks, Siberian Math. J. 44 (2003), no. 1, 44–57,
translated from Sibirsk. Mat. Zh. 44 (2003) 53–68, in Russian.
[74] C. Derman and H. Robbins, The strong law of large numbers when the
first moment does not exist, Proc. Nat. Acad. Sci. U.S.A. 41 (1955),
586–587.
[75] B. Derrida, S. Goldstein, J. L. Lebowitz, and E. R. Speer, Shift equivalence of measures and the intrinsic structure of shocks in the asymmetric simple exclusion process, J. Statist. Phys. 93 (1998), no. 3-4,
547–571.
[76] W. Doeblin, Éléments d’une théorie générale des chaı̂nes simples constantes de Markoff, Ann. Sci. École Norm. Sup. (3) 57 (1940), 61–111.
[77] M. D. Donsker and S. R. S. Varadhan, On the number of distinct sites
visited by a random walk, Comm. Pure Appl. Math. 32 (1979), no. 6,
721–747.
[78] J. L. Doob, Stochastic processes, John Wiley & Sons Inc., New York,
1953.
[79] R. Douc, G. Fort, E. Moulines, and P. Soulier, Practical drift conditions for subgeometric rates of convergence, Ann. Appl. Probab. 14
(2004), no. 3, 1353–1377.
[80] D. Y. Downham and S. B. Fotopoulos, The transient behaviour of the
simple random walk in the plane, J. Appl. Probab. 25 (1988), no. 1,
58–69.
[81] P. G. Doyle and J. L. Snell, Random walks and electric networks,
Carus Mathematical Monographs, vol. 22, Mathematical Association
of America, Washington, DC, 1984.
374
Bibliography
[82] R. Durrett, Ten lectures on particle systems, Lectures on probability theory (Saint-Flour, 1993), Lecture Notes in Math., vol. 1608,
Springer, Berlin, 1995, pp. 97–201.
[83]
, Probability: theory and examples, fourth ed., Cambridge
Series in Statistical and Probabilistic Mathematics, Cambridge University Press, Cambridge, 2010.
[84] A. Dvoretzky and P. Erdős, Some problems on random walk in space,
Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, 1950 (Berkeley and Los Angeles), University of
California Press, 1951, pp. 353–367.
[85] A. Eibeck and W. Wagner, Stochastic interacting particle systems and
nonlinear kinetic equations, Ann. Appl. Probab. 13 (2003), no. 3, 845–
889.
[86] A. Einstein, Investigations on the theory of the Brownian movement,
Dover Publications Inc., New York, 1956, Edited with notes by R.
Fürth, translated by A. D. Cowper.
[87] R. Eldan, F. Nazarov, and Y. Peres, How many matrices can be spectrally balanced simultaneously?, Preprint. (2016).
[88] P. Erdős and S. J. Taylor, Some problems concerning the structure of
random walk paths, Acta Math. Acad. Sci. Hungar. 11 (1960), 137–
162.
[89] K. B. Erickson, The strong law of large numbers when the mean is
undefined, Trans. Amer. Math. Soc. 185 (1973), 371–381 (1974).
[90] S. N. Evans, Stochastic billiards on general tables, Ann. Appl. Probab.
11 (2001), no. 2, 419–437.
[91] A. M. Fal0 , Recurrence times for certain Markov random walks,
Ukrainian Math. J. 23 (1971), 676–681, translated from Ukrain. Mat.
Zh. 23 (1971) 824–830 (in Russian).
[92]
, Generalization of the results obtained by R. L. Dobrushin for
additive functionals of a Markov random walk, Theory Probab. Appl.
22 (1978), 569–571, translated from Teor. Veroyatnost. i Primenen.
22 (1977) 582–584 (in Russian).
Bibliography
[93]
375
, Certain limit theorems for an elementary Markov random
walk, Ukrainian Math. J. 33 (1981), 433–435, translated from Ukrain.
Mat. Zh. 33 (1981) 564–566 (in Russian).
[94] E. F. Fama, The behavior of stock-market prices, J. Business 38 (1965),
34–105.
[95] G. Fayolle, V. A. Malyshev, and M. V. Men0 shikov, Random walks
in a quarter plane with zero drifts. I. Ergodicity and null recurrence,
Ann. Inst. H. Poincaré Probab. Statist. 28 (1992), no. 2, 179–194.
[96] G. Fayolle, V. A. Malyshev, and M. V. Menshikov, Topics in the constructive theory of countable Markov chains, Cambridge University
Press, Cambridge, 1995.
[97] W. Feller, A limit theorem for random variables with infinite moments,
Amer. J. Math. 68 (1946), 257–262.
[98]
, An introduction to probability theory and its applications. Vol.
I, Third edition, John Wiley & Sons Inc., New York, 1968.
[99]
, An introduction to probability theory and its applications. Vol.
II., Second edition, John Wiley & Sons Inc., New York, 1971.
[100] P. A. Ferrari, Shock fluctuations in asymmetric simple exclusion,
Probab. Theory Related Fields 91 (1992), no. 1, 81–101.
[101]
, Shocks in one-dimensional processes with drift, Probability
and phase transition (Cambridge, 1993), NATO Adv. Sci. Inst. Ser.
C Math. Phys. Sci., vol. 420, Kluwer Acad. Publ., Dordrecht, 1994,
pp. 35–48.
[102] P. A. Ferrari, C. Kipnis, and E. Saada, Microscopic structure of travelling waves in the asymmetric simple exclusion process, Ann. Probab.
19 (1991), no. 1, 226–244.
[103] Yu. P. Filonov, Criterion for ergodicity of homogeneous discrete
Markov chains, Ukrain. Math. J. 41 (1989), 1223–1225, translated
from Ukrain. Mat. Zh. 41 (1989) 1421–1422.
[104] L. Flatto, A problem on random walk, Quart. J. Math. Oxford Ser. (2)
9 (1958), 299–300.
376
Bibliography
[105] S. Foss, D. Korshunov, and S. Zachary, An introduction to heavytailed and subexponential distributions, second ed., Springer Series in
Operations Research and Financial Engineering, Springer, New York,
2013.
[106] S. G. Foss and D. È. Denisov, On transience conditions for Markov
chains, Siberian Math. J. 42 (2001), no. 2, 364–371, translated from
Sibirsk. Mat. Zh. 42 (2001) 425–433, in Russian.
[107] F. G. Foster, Markoff chains with an enumerable number of states and
a class of cascade processes, Math. Proc. Cambridge Philos. Soc. 47
(1951), 77–85.
[108]
, On the stochastic matrices associated with certain queuing
processes, Ann. Math. Statistics 24 (1953), 355–360.
[109] J.-D. Fouks, E. Lesigne, and M. Peigné, Étude asymptotique d’une
marche aléatoire centrifuge, Ann. Inst. H. Poincaré Probab. Statist.
42 (2006), no. 2, 147–170.
[110] A. S. Gaı̆rat, V. A. Malyshev, M. V. Men0 shikov, and K. D. Pelikh,
Classification of Markov chains that describe the evolution of random
strings, Uspekhi Mat. Nauk 50 (1995), no. 2(302), 5–24.
[111] A. S. Gajrat, V. A. Malyshev, and A. A. Zamyatin, Two-sided evolution of a random chain, Markov Process. Related Fields 1 (1995),
no. 2, 281–316.
[112] L. Gallardo, Comportement asymptotique des marches aléatoires associées aux polynômes de Gegenbauer et applications, Adv. in Appl.
Probab. 16 (1984), no. 2, 293–323.
[113] C. Gallesco, S. Popov, and G. M. Schütz, Localization for a random
walk in slowly decreasing random potential, J. Stat. Phys. 150 (2013),
no. 2, 285–298.
[114] N. Georgiou, M. V. Menshikov, A. Mijatović, and A. R. Wade, Anomalous recurrence properties of many-dimensional zero-drift random
walks, Adv. in Appl. Probab. (2016), To appear.
[115] N. Georgiou and A. R. Wade, Non-homogeneous random walks on a
semi-infinite strip, Stochastic Process. Appl. 124 (2014), no. 10, 3179–
3205.
Bibliography
377
[116] G. Giacomin, Random polymer models, Imperial College Press, London, 2007.
[117] J. Gillis, Centrally biased discrete random walk, Quart. J. Math. Oxford Ser. (2) 7 (1956), 144–152.
[118] M. L. Glasser and I. J. Zucker, Extended Watson integrals for the cubic
lattices, Proc. Nat. Acad. Sci. U.S.A. 74 (1977), no. 5, 1800–1801.
[119] A. O. Golosov, Limit distributions for random walks in a random environment, Dokl. Akad. Nauk SSSR 271 (1983), no. 1, 25–29.
[120] M. González, M. Molina, and I. del Puerto, Asymptotic behaviour of
critical controlled branching processes with random control functions,
J. Appl. Probab. 42 (2005), no. 2, 463–477.
[121] P. S. Griffin, An integral test for the rate of escape of d-dimensional
random walk, Ann. Probab. 11 (1983), no. 4, 953–961.
[122] R. F. Gundy and D. Siegmund, On a stopping rule and the central
limit theorem, Ann. Math. Statist. 38 (1967), 1915–1917.
[123] A. Gut, Probability: A graduate course, Springer, 2005.
[124] Y. Hamana, An almost sure invariance principle for the range of random walks, Stochastic Process. Appl. 78 (1998), no. 2, 131–143.
[125] K. Hamza and F. C. Klebaner, Conditions for integrability of Markov
chains, J. Appl. Probab. 32 (1995), no. 2, 541–547.
[126] C. M. Harris and P. G. Marlin, A note on feedback queues with bulk
service, J. Assoc. Comput. Mach. 19 (1972), 727–733.
[127] T. E. Harris, First passage and recurrence distributions, Trans. Amer.
Math. Soc. 73 (1952), 471–486.
[128] W. M. Hirsch, A strong law for the maximum cumulative sum of independent random variables, Comm. Pure Appl. Math. 18 (1965), 109–
127.
[129] J. L. Hodges, Jr. and M. Rosenblatt, Recurrence-time moments in
random walks, Pacific J. Math. 3 (1953), 127–136.
[130] W. Hoeffding, Probability inequalities for sums of bounded random
variables, J. Amer. Statist. Assoc. 58 (1963), 13–30.
378
Bibliography
[131] P. Holgate, Random walk models for animal behavior, Statistical Ecology: vol. 2 (G. Patil, E. Pielou, and W. Walters, eds.), Penn State
University Press, University Park, PA, 1971, pp. 1–12.
[132] R. A. Holley and T. M. Liggett, Ergodic theorems for weakly interacting infinite systems and the voter model, Ann. Probability 3 (1975),
no. 4, 643–663.
[133] O. Hryniv, I. M. MacPhee, M. V. Menshikov, and A. R. Wade, Nonhomogeneous random walks with non-integrable increments and heavytailed random walks on strips, Electron. J. Probab. 17 (2012), article
59.
[134] O. Hryniv and M. Menshikov, Long-time behaviour in a model of microtubule growth, Adv. in Appl. Probab. 42 (2010), no. 1, 268–291.
[135] O. Hryniv, M. V. Menshikov, and A. R. Wade, Excursions and path
functionals for stochastic processes with asymptotically zero drifts,
Stochastic Process. Appl. 123 (2013), no. 6, 1891–1921.
[136]
, Random walk in mixed random environment without uniform
ellipticity, Proc. Stekov. Inst. Math. 282 (2013), 106–123.
[137] Y. Hu and H. Nyrhinen, Large deviations view points for heavy-tailed
random walks, J. Theoret. Probab. 17 (2004), no. 3, 761–768.
[138] Y. Hu and Z. Shi, The limits of Sinai’s simple random walk in random
environment, Ann. Probab. 26 (1998), no. 4, 1477–1521.
[139] B. D. Hughes, Random walks and random environments. Vol. 1, Oxford Science Publications, The Clarendon Press Oxford University
Press, New York, 1995, Random walks.
[140] T. Huillet, Random walk with long-range interaction with a barrier and
its dual: exact results, J. Comput. Appl. Math. 233 (2010), no. 10,
2449–2467.
[141] N. C. Jain and W. E. Pruitt, Maxima of partial sums of independent
random variables, Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 27
(1973), 141–151.
[142] S. F. Jarner and G. O. Roberts, Polynomial convergence rates of
Markov chains, Ann. Appl. Probab. 12 (2002), no. 1, 224–247.
379
Bibliography
[143] R. C. Jones, On the theory of fluctuations in the decay of sound, J.
Acoust. Soc. Amer. 11 (1940), 324–332.
[144] Paul Jung, The noisy voter-exclusion process, Stochastic Process.
Appl. 115 (2005), no. 12, 1979–2005.
[145] V. V. Kalashnikov, Practical stability of difference equations, Automation and Remote Control 9 (1967), 1404–1407, translated from Avtomatika i Telemekhanika 9 (1967) 172–175 (in Russian).
[146]
, The use of Lyapunov’s method in the solution of queueing
theory problems, Engrg. Cybernetics (1968), no. 5, 77–84, translated
from Izv. Akad. Nauk SSSR Tehn. Kibernet. 1968 89–95 (in Russian).
[147]
, Reliability analysis by Lyapunov’s method, Engrg. Cybernetics
(1970), no. 2, 257–269, translated from Izv. Akad. Nauk SSSR Tehn.
Kibernet. 1970 65–76 (in Russian).
[148]
, Analysis of ergodicity of queueing systems by means of the direct method of Lyapunov, Automation and Remote Control 32 (1971),
559–566, translated from Avtomatika i Telemekhanika 32 (1971) 46–
54 (in Russian).
[149]
, The property of γ-reflexivity for Markov sequences, Soviet
Math. Dokl. 14 (1973), 1869–1873, translated from Dokl. Akad. Nauk
SSSR 213 (1973) 1243–1246 (in Russian).
[150] O. Kallenberg, Foundations of modern probability, second ed., Probability and its Applications (New York), Springer-Verlag, New York,
2002.
[151] M. Kaplan, A sufficient condition for nonergodicity of a Markov chain,
IEEE Trans. Inform. Theory 25 (1979), no. 4, 470–471.
[152] S. Karlin and J. McGregor, Representation of a class of stochastic
processes, Proc. Nat. Acad. Sci. U. S. A. 41 (1955), 387–391.
[153]
, Random walks, Illinois J. Math. 3 (1959), 66–81.
[154] K. Kawazu, Y. Tamura, and H. Tanaka, Limit theorems for onedimensional diffusions and random walks in random environments,
Probab. Theory Related Fields 80 (1989), no. 4, 501–541.
380
Bibliography
[155] G. Keller, G. Kersting, and U. Rösler, On the asymptotic behaviour
of discrete time stochastic growth processes, Ann. Probab. 15 (1987),
no. 1, 305–343.
[156] F. P. Kelly, Markovian functions of a Markov chain, Sankhyā Ser. A
44 (1982), no. 3, 372–379.
[157] J. H. B. Kemperman, The oscillating random walk, Stochastic Processes Appl. 2 (1974), 1–29.
[158] D. G. Kendall, Some problems in the theory of queues, J. Roy. Statist.
Soc. Ser. B. 13 (1951), 151–173; discussion: 173–185.
[159] G. Kersting, On recurrence and transience of growth models, J. Appl.
Probab. 23 (1986), no. 3, 614–625.
[160]
, Asymptotic Γ-distribution for stochastic difference equations,
Stochastic Process. Appl. 40 (1992), no. 1, 15–28.
[161] G. Kersting and F. C. Klebaner, Sharp conditions for nonexplosions
and explosions in Markov jump processes, Ann. Probab. 23 (1995),
no. 1, 268–272.
[162]
, Explosions in Markov processes and submartingale convergence, Athens Conference on Applied Probability and Time Series
Analysis, Vol. I (1995), Lecture Notes in Statist., vol. 114, Springer,
New York, 1996, pp. 127–136.
[163] H. Kesten, The limit points of a normalized random walk, Ann. Math.
Statist. 41 (1970), 1173–1205.
[164]
, The limit distribution of Sinai’s random walk in random environment, Phys. A 138 (1986), no. 1-2, 299–309.
[165] H. Kesten and R. A. Maller, Two renewal theorems for general random
walks tending to infinity, Probab. Theory Related Fields 106 (1996),
no. 1, 1–38.
[166]
, Random walks crossing power law boundaries, Studia Sci.
Math. Hungar. 34 (1998), no. 1-3, 219–252.
[167] R. Z. Khas’minskii, On the stability of the trajectory of Markov processes, J. Appl. Math. Mech. 26 (1962), 1554–1565.
Bibliography
[168]
381
, Stochastic stability of differential equations, Monographs and
Textbooks on Mechanics of Solids and Fluids: Mechanics and Analysis,
vol. 7, Sijthoff & Noordhoff, Alphen aan den Rijn—Germantown, Md.,
1980, translated and updated version of the original Russian edition,
Nauka, Moscow, 1969.
[169] B. Kim and I. Lee, Tests for nonergodicity of denumerable continuous
time Markov processes, Comput. Math. Appl. 55 (2008), no. 6, 1310–
1321.
[170] J. F. C. Kingman, The ergodic behaviour of random walks, Biometrika
48 (1961), 391–396.
[171]
, Two similar queues in parallel, Ann. Math. Statist. 32 (1961),
1314–1323.
[172]
, Random walks with spherical symmetry, Acta Math. 109
(1963), 11–53.
[173] F. C. Klebaner, Stochastic difference equations and generalized gamma
distributions, Ann. Probab. 17 (1989), no. 1, 178–188.
[174]
, Introduction to stochastic calculus with applications, second
ed., Imperial College Press, London, 2005.
[175] L. A. Klein Haneveld, Random walk on the quadrant, Stochastic Process. Appl. 26 (1987), 228.
[176]
, Random walk in the quadrant, Ph.D. thesis, University of
Amsterdam, 1996.
[177] L. A. Klein Haneveld and A. O. Pittenger, Escape time for a random
walk from an orthant, Stochastic Process. Appl. 35 (1990), no. 1, 1–9.
[178] H. Kleinert, Path integrals in quantum mechanics, statistics, polymer
physics, and financial markets, fourth ed., World Scientific Publishing
Co. Pte. Ltd., Hackensack, NJ, 2006.
[179] M. Knudsen, Kinetic theory of gases: some modern aspects, Methuen’s
Monographs on Physical Subjects, Methuen, London, 1952.
[180] D. A. Korshunov, Moments of stationary Markov chains with asymptotically zero drift, Sibirsk. Mat. Zh. 52 (2011), no. 4, 829–840.
382
Bibliography
[181] E. Kosygina and M. P. W. Zerner, Excited random walks: results,
methods, open problems, Bull. Inst. Math. Acad. Sin. (N.S.) 8 (2013),
no. 1, 105–157.
[182] Y. Kovchegov and N. Michalowski, A class of Markov chains with no
spectral gap, Proc. Amer. Math. Soc. 141 (2013), no. 12, 4317–4326.
[183] M. V. Kozlov, Random walk in a one-dimensional random medium,
Teor. Verojatnost. i Primenen. 18 (1973), 406–408.
[184] G. Kozma, T. Orenshtein, and I. Shinkar, Excited random walk with
periodic cookies, Ann. Inst. H. Poincaré Probab. Statist. 52 (2016).
[185] V. M. Kruglov, A strong law of large numbers for pairwise independent
identically distributed random variables with infinite means, Statist.
Probab. Lett. 78 (2008), no. 7, 890–895.
[186] I. Kurkova and K. Raschel, Passage time from four to two blocks of
opinions in the voter model and walks in the quarter plane, Queueing
Syst. 74 (2013), no. 2-3, 219–234.
[187] H. J. Kushner, On the stability of stochastic dynamical systems, Proc.
Nat. Acad. Sci. U.S.A. 53 (1965), 8–12.
[188]
, Finite time stochastic stability and the analysis of tracking
systems, IEEE Trans. Automatic Control AC-11 (1966), 219–227.
[189] P. Küster, Asymptotic growth of controlled Galton-Watson processes,
Ann. Probab. 13 (1985), no. 4, 1157–1178.
[190] J. Lamperti, Criteria for the recurrence or transience of stochastic
process. I., J. Math. Anal. Appl. 1 (1960), 314–330.
[191]
, A new class of probability limit theorems, J. Math. Mech. 11
(1962), 749–772.
[192]
, Criteria for stochastic processes. II. Passage-time moments,
J. Math. Anal. Appl. 7 (1963), 127–145.
[193]
, Maximal branching processes and ‘long-range percolation’, J.
Appl. Probability 7 (1970), 89–98.
[194]
, Remarks on maximal branching processes, Teor. Verojatnost.
i Primenen. 17 (1972), 46–54.
Bibliography
383
[195] G. F. Lawler and V. Limic, Random walk: a modern introduction,
Cambridge Studies in Advanced Mathematics, vol. 123, Cambridge
University Press, Cambridge, 2010.
[196] F. W. Leysieffer, Functions of finite Markov chains, Ann. Math. Statist. 38 (1967), 206–212.
[197] T. M. Liggett, Coupling the simple exclusion process, Ann. Probability
4 (1976), no. 3, 339–356.
[198]
, Interacting particle systems, Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 276, Springer-Verlag, New York, 1985.
[199]
, Stochastic interacting systems: contact, voter and exclusion
processes, Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 324, Springer-Verlag,
Berlin, 1999.
[200] M. Loève, Probability theory. I, fourth ed., Springer-Verlag, New York,
1977, Graduate Texts in Mathematics, Vol. 45.
[201]
, Probability theory. I, fourth ed., Springer-Verlag, New York,
1977, Graduate Texts in Mathematics, Vol. 45.
[202] G. G. Lowry (ed.), Markov chains and monte carlo calculations in
polymer science, Marcel Dekker, New York, 1970.
[203] F. P. Machado, M. V. Menshikov, and S. Yu. Popov, Recurrence and
transience of multitype branching random walks, Stochastic Process.
Appl. 91 (2001), no. 1, 21–37.
[204] F. P. Machado and S. Yu. Popov, One-dimensional branching random walks in a Markovian random environment, J. Appl. Probab. 37
(2000), no. 4, 1157–1163.
[205]
, Branching random walk in random environment on trees,
Stochastic Process. Appl. 106 (2003), no. 1, 95–106.
[206] I. MacPhee, M. Menshikov, D. Petritis, and S. Popov, A Markov chain
model of a polling system with parameter regeneration, Ann. Appl.
Probab. 17 (2007), no. 5-6, 1447–1473.
384
Bibliography
[207] I. MacPhee, M. V. Menshikov, and M. Vachkovskaia, Dynamics of the
non-homogeneous supermarket model, Stoch. Models 28 (2012), no. 4,
533–556.
[208] I. M. MacPhee and M. V. Menshikov, Critical random walks on
two-dimensional complexes with applications to polling systems, Ann.
Appl. Probab. 13 (2003), no. 4, 1399–1422.
[209] I. M. MacPhee, M. V. Menshikov, S. Popov, and S. Volkov, Periodicity in the transient regime of exhaustive polling systems, Ann. Appl.
Probab. 16 (2006), no. 4, 1816–1850.
[210] I. M. MacPhee, M. V. Menshikov, S. Volkov, and A. R. Wade,
Passage-time moments and hybrid zones for the exclusion-voter model,
Bernoulli 16 (2010), no. 4, 1312–1342.
[211] I. M. MacPhee, M. V. Menshikov, and A. R. Wade, Angular asymptotics for multi-dimensional non-homogeneous random walks with asymptotically zero drift, Markov Process. Related Fields 16 (2010), no. 2,
351–388.
[212]
, Moments of exit times from wedges for non-homogeneous random walks with asymptotically zero drifts, J. Theoret. Probab. 26
(2013), 1–30.
[213] V. A. Malyšev and M. V. Men0 šikov, Ergodicity, continuity, and analyticity of countable Markov chains, Trans. Moscow Math. Soc. 39
(1979), 1–48, translated from Trudy Moskov. Mat. Obshch. 39 (1979)
3–48 (in Russian).
[214] V. A. Malyshev, Classification of two-dimensional positive random
walks and almost linear semimartingales, Soviet Math. Dokl. 13
(1972), 136–139, translated from Dokl. Akad. Nauk SSSR 202 (1972)
526–528 (in Russian).
[215]
, Random grammars, Uspekhi Mat. Nauk 53 (1998), no. 2(320),
107–134.
[216] X. Mao, Stochastic differential equations and applications, second ed.,
Horwood Publishing Limited, Chichester, 2008.
[217] M. B. Marcus and J. Rosen, Logarithmic averages for the local times of
recurrent random walks and Lévy processes, Stochastic Process. Appl.
59 (1995), no. 2, 175–184.
Bibliography
385
[218] J. G. Mauldon, On non-dissipative Markov chains, Proc. Cambridge
Philos. Soc. 53 (1957), 825–835.
[219] W. H. McCrea and F. J. W. Whipple, Random paths in two and three
dimensions, Proc. Roy. Soc. Edinburgh 60 (1940), 281–298.
[220] M. Menshikov and D. Petritis, Explosion, implosion, and moments of
passage times for continuous-time Markov chains: A semimartingale
approach, Stochastic Process. Appl. 124 (2014), no. 7, 2388–2414.
[221] M. Menshikov and S. Popov, On range and local time of manydimensional submartingales, J. Theoret. Probab. 27 (2014), no. 2,
601–617.
[222] M. Menshikov, S. Popov, A. Ramı́rez, and M. Vachkovskaia, On a general many-dimensional excited random walk, Ann. Probab. 40 (2012),
no. 5, 2106–2130.
[223] M. Menshikov and S. Volkov, Urn-related random walk with drift
ρxα /tβ , Electron. J. Probab. 13 (2008), no. 31, 944–960.
[224] M. Menshikov and R. J. Williams, Passage-time moments for continuous non-negative stochastic processes and applications, Adv. in Appl.
Probab. 28 (1996), no. 3, 747–762.
[225] M. Menshikov and S. Zuyev, Polling systems in the critical regime,
Stochastic Process. Appl. 92 (2001), no. 2, 201–218.
[226] M. V. Menshikov, Ergodicity and transience conditions for random
walks in the positive octant of space, Soviet Math. Dokl. 15 (1974),
1118–1121, translated from Dokl. Akad. Nauk SSSR 217 (1974) 755–
758 (in Russian).
[227]
, Martingale approach for Markov processes in random environment and branching Markov chains, Resenhas 3 (1997), no. 2, 159–
171.
[228] M. V. Menshikov, I. M. Asymont, and R. Iasnogorodskii, Markov processes with asymptotically zero drifts, Probl. Inf. Transm. 31 (1995),
248–261, translated from Problemy Peredachi Informatsii 31 (1995)
60–75 (in Russian).
[229] M. V. Menshikov, D. Petritis, and A. R. Wade, Heavy-tailed random
walks on complexes of half-lines, Preprint. (2016).
386
Bibliography
[230] M. V. Menshikov and S. Yu. Popov, Exact power estimates for countable Markov chains, Markov Process. Related Fields 1 (1995), no. 1,
57–78.
[231] M. V. Menshikov, S. Yu. Popov, V. Sisko, and M. Vachkovskaia, On
a many-dimensional random walk in a rarefied random environment,
Markov Process. Related Fields 10 (2004), no. 1, 137–160.
[232] M. V. Menshikov, M. Vachkovskaia, and A. R. Wade, Asymptotic behaviour of randomly reflecting billiards in unbounded tubular domains,
J. Stat. Phys. 132 (2008), no. 6, 1097–1133.
[233] M. V. Menshikov and S. E. Volkov, Branching Markov chains: qualitative characteristics, Markov Process. Related Fields 3 (1997), no. 2,
225–241.
[234] M. V. Menshikov and A. R. Wade, Random walk in random environment with asymptotically zero perturbation, J. Eur. Math. Soc. (JEMS)
8 (2006), no. 3, 491–513.
[235]
, Logarithmic speeds for one-dimensional perturbed random
walks in random environments, Stochastic Process. Appl. 118 (2008),
no. 3, 389–416.
[236]
, Rate of escape and central limit theorem for the supercritical
Lamperti problem, Stochastic Process. Appl. 120 (2010), no. 10, 2078–
2099.
[237] F. Merkl and S. W. W. Rolles, Recurrence of edge-reinforced random
walk on a two-dimensional graph, Ann. Probab. 37 (2009), no. 5, 1679–
1714.
[238] J.-F. Mertens, E. Samuel-Cahn, and S. Zamir, Necessary and sufficient
conditions for recurrence and transience of Markov chains, in terms
of inequalities, J. Appl. Probab. 15 (1978), no. 4, 848–851.
[239] S. Meyn and R. L. Tweedie, Markov chains and stochastic stability,
second ed., Cambridge University Press, Cambridge, 2009, With a
prologue by Peter W. Glynn.
[240] R. G. Miller, Jr., Foster’s Markov chain theorems in continuous time,
Tech. Report 88, Applied mathematics and statistics laboratory, Stanford University, 1963.
Bibliography
387
[241] P. A. P. Moran, An introduction to probability theory, Oxford Science Publications, The Clarendon Press, Oxford University Press, New
York, 1984, Corrected reprint of the 1968 original.
[242] M. D. Moustafa, Input-output Markov processes, Proc. Konin. Neder.
Akad. Wetens. A. Math. Sci. 19 (1957), 112–118.
[243] J. R. Norris, Markov chains, Cambridge Series in Statistical and Probabilistic Mathematics, vol. 2, Cambridge University Press, Cambridge,
1998.
[244] R. Nossal, Stochastic aspects of biological locomotion, J. Stat. Phys.
30 (1983), 391–400.
[245] R. J. Nossal and G. H. Weiss, A generalized Pearson random walk
allowing for bias, J. Stat. Phys. 10 (1974), 245–253.
[246] E. Nummelin, General irreducible Markov chains and nonnegative operators, Cambridge Tracts in Mathematics, vol. 83, Cambridge University Press, Cambridge, 1984.
[247] S. Orey, Lecture notes on limit theorems for Markov chain transition
probabilities, Van Nostrand Reinhold Co., London-New York-Toronto,
Ont., 1971, Van Nostrand Reinhold Mathematical Studies, No. 34.
[248] R. A. Orwoll and W. H. Stockmayer, Stochastic models for chain dynamics, Adv. Chem. Phys. 15 (1969), 305–324.
[249] V. I. Oseledec, A multiplicative ergodic theorem. Characteristic Ljapunov, exponents of dynamical systems, Trudy Moskov. Mat. Obšč.
19 (1968), 179–210.
[250] A. G. Pakes, Some conditions for ergodicity and recurrence of Markov
chains, Operations Res. 17 (1969), 1058–1061.
[251]
, Some remarks on a one-dimensional skip-free process with
repulsion, J. Austral. Math. Soc. Ser. A 30 (1980/81), no. 1, 107–
128.
[252] E. Parzen, Stochastic processes, Classics in Applied Mathematics,
vol. 24, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1999, Reprint of the 1962 original.
[253] K. Pearson, The problem of the random walk, Nature 72 (1905), 342.
388
Bibliography
[254] K. Pearson and J. Blakeman, A mathematical theory of random migration, Drapers’ Company Research Memoirs Biometric Series, Dulau
and co., London, 1906.
[255] R. Pemantle, A survey of random processes with reinforcement,
Probab. Surv. 4 (2007), 1–79.
[256] R. Pemantle and S. Volkov, Vertex-reinforced random walk on Z has
finite range, Ann. Probab. 27 (1999), no. 3, 1368–1388.
[257] Y. Peres, S. Popov, and P. Sousi, On recurrence and transience of
self-interacting random walks, Bull. Braz. Math. Soc. 44 (2013), no. 4,
1–27.
[258] R. G. Pinsky, Positive harmonic functions and diffusion, Cambridge
Studies in Advanced Mathematics, vol. 45, Cambridge University
Press, Cambridge, 1995.
[259] G. Pólya, Über eine Aufgabe der Wahrscheinlichkeitsrechnung betreffend die Irrfahrt im Straßennetz, Math. Ann. 84 (1921), no. 1-2, 149–
160.
[260] N. N. Popov, The rate of convergence for countable Markov chains,
Theor. Probability Appl. 24 (1979), no. 2, 401–405, translated from
Teor. Verojatnost. i Primenen. 24 (1979) 395–399 (in Russian).
[261] W. E. Pruitt, The rate of escape of random walk, Ann. Probab. 18
(1990), no. 4, 1417–1461.
[262] O. Raimond and B. Schapira, On some generalized reinforced random
walk on integers, Electron. J. Probab. 14 (2009), no. 60, 1770–1789.
[263] C. Rau, Sur le nombre de points visités par une marche aléatoire sur
un amas infini de percolation, Bull. Soc. Math. France 135 (2007),
no. 1, 135–169.
[264] Lord Rayleigh, On the resultant of a large number of vibrations of the
same pitch and of arbitrary phase, Phil. Mag. Ser. 5 10 (1880), 73–78.
[265] J. A. F. Regnault, Calcul des chances et philosophie de la bourse,
Mallet-Bachelier/Castel, Paris, 1863.
[266] P. H. F. Reimberg and L. R. Abramo, Random flights through spaces
of different dimensions, J. Math. Phys. 56 (2015), no. 1, 013512.
Bibliography
389
[267] S. I. Resnick, A probability path, Birkhäuser Boston Inc., Boston, MA,
1999.
[268] G. E. H. Reuter, Competition processes, Proc. 4th Berkeley Sympos.
Math. Statist. and Prob., Vol. II, Univ. California Press, Berkeley,
Calif., 1961, pp. 421–430.
[269] Pál Révész, Random walk in random and non-random environments,
third ed., World Scientific Publishing Co. Pte. Ltd., Hackensack, NJ,
2013.
[270] D. Revuz, Markov chains, second ed., North-Holland Mathematical
Library, vol. 11, North-Holland Publishing Co., Amsterdam, 1984.
[271] P. Robert, Stochastic networks and queues, Applications of Mathematics, vol. 52, Springer-Verlag, Berlin, 2003.
[272] L. C. G. Rogers and J. W. Pitman, Markov functions, Ann. Probab.
9 (1981), no. 4, 573–582.
[273] B. A. Rogozin and S. G. Foss, The recurrence of an oscillating random
walk, Theor. Probability Appl. 23 (1978), no. 1, 155–162, translated
from Teor. Verojatnost. i Primenen. 23 (1978) 161–169 (in Russian).
[274] M. Rosenblatt, Functions of a Markov process that are Markovian, J.
Math. Mech. 8 (1959), 585–596.
[275] W. A. Rosenkrantz, A local limit theorem for a certain class of random
walks, Ann. Math. Statist. 37 (1966), 855–859.
[276] N. Sandrić, Recurrence and transience property for a class of Markov
chains, Bernoulli 19 (2013), no. 5B, 2167–2199.
[277]
, Recurrence and transience criteria for two cases of stable-like
markov chains, J. Theoret. Probab. 27 (2014), 754–788.
[278] K.-I. Sato, Lévy processes and infinitely divisible distributions, Cambridge Studies in Advanced Mathematics, vol. 68, Cambridge University Press, Cambridge, 1999, translated from the 1990 Japanese
original, Revised by the author.
[279] S. Schumacher, Diffusions with random coefficients, Ph.D. thesis, University of California, Los Angeles, 1984.
390
[280]
Bibliography
, Diffusions with random coefficients, Particle systems, random media and large deviations (Brunswick, Maine, 1984), Contemp.
Math., vol. 41, Amer. Math. Soc., Providence, RI, 1985, pp. 351–356.
[281] L. I. Sennott, P. A. Humblet, and R. L. Tweedie, Mean drifts and the
nonergodicity of Markov chains, Oper. Res. 31 (1983), no. 4, 783–789.
[282] V. Shcherbakov and S. Volkov, Stability of a growth process generated by monomer filling with nearest-neighbour cooperative effects,
Stochastic Process. Appl. 120 (2010), no. 6, 926–948.
[283] L. A. Shepp, Symmetric random walk, Trans. Amer. Math. Soc. 104
(1962), 144–153.
[284]
, Recurrent random walks with arbitrarily large steps, Bull.
Amer. Math. Soc. 70 (1964), 540–542.
[285] T. Shiga, A. Shimizu, and T. Soshi, Passage-time moments for positively recurrent Markov chains, Nagoya Math. J. 162 (2001), 169–185.
[286] A. N. Shiryaev, Probability, second ed., Graduate Texts in Mathematics, vol. 95, Springer-Verlag, New York, 1996, translated from the first
(1980) Russian edition by R. P. Boas.
[287] M. F. Shlesinger and B. J. West (eds.), Random walks and their applications on the physical and biological sciences, American Institute
of Physics, New York, 1984.
[288] Ya. G. Sinaı̆, The limiting behavior of a one-dimensional random walk
in a random medium, Theor. Probability Appl. 27 (1983), 256–268,
translated from Teor. Veroyatnost. i Primenen. 27 (1982) 247–258 (in
Russian).
[289] A. Singh, A slow transient diffusion in a drifted stable potential, J.
Theoret. Probab. 20 (2007), no. 2, 153–166.
[290] P. E. Smouse, S. Focardi, P. R. Moorcroft, J. G. Kie, J. D. Forester,
and J. M. Morales, Stochastic modelling of animal movement, Phil.
Trans. Roy. Soc. Ser. B Biol. Sci. 365 (2010), 2201–2211.
[291] F. Solomon, Random walks in a random environment, Ann. Probability 3 (1975), 1–31.
Bibliography
391
[292] F. Spitzer, Renewal theorems for Markov chains, Proc. 5th Berkeley
Sympos. Math. Statist. and Prob., Vol. II, Univ. California Press,
Berkeley, Calif., 1967, pp. 311–320.
[293]
, Principles of random walk, second ed., Springer-Verlag, New
York, 1976, Graduate Texts in Mathematics, Vol. 34.
[294] A. Stannard and P. Coles, Random-walk statistics and the spherical
harmonic representation of cosmic microwave background maps, Mon.
Not. Roy. Astron. Soc. 364 (2005), no. 3, 929–933.
[295] W. F. Stout, Almost sure convergence, Academic Press, 1974, Probability and Mathematical Statistics, Vol. 24.
[296] D. W. Stroock, An introduction to Markov processes, Graduate Texts
in Mathematics, vol. 230, Springer-Verlag, Berlin, 2005.
[297] A. Sturm and J. M. Swart, Tightness of voter model interfaces, Electron. Commun. Probab. 13 (2008), 165–174.
[298] R. Syski, Exit time from a positive quadrant for a two-dimensional
Markov chain, Comm. Statist. Stochastic Models 8 (1992), no. 3, 375–
395.
[299] G. J. Székely, On the asymptotic properties of diffusion processes, Ann.
Univ. Sci. Budapest. Eötvös Sect. Math. 17 (1974), 69–71 (1975).
[300] A.-S. Sznitman, Topics in random walks in random environment,
School and Conference on Probability Theory, ICTP Lect. Notes,
XVII, Abdus Salam Int. Cent. Theoret. Phys., Trieste, 2004, pp. 203–
266 (electronic).
[301] P. Tarrès, Vertex-reinforced random walk on Z eventually gets stuck
on five points, Ann. Probab. 32 (2004), no. 3B, 2650–2701.
[302] R. L. Tweedie, Sufficient conditions for ergodicity and recurrence of
Markov chains on a general state space, Stochastic Processes Appl. 3
(1975), no. 4, 385–403.
[303]
, Sufficient conditions for regularity, recurrence and ergodicity
of Markov processes, Math. Proc. Cambridge Philos. Soc. 78 (1975),
125–136.
[304]
, Criteria for classifying general Markov chains, Advances in
Appl. Probability 8 (1976), no. 4, 737–771.
392
[305]
Bibliography
, Criteria for ergodicity, exponential ergodicity and strong ergodicity of Markov processes, J. Appl. Probab. 18 (1981), no. 1, 122–
130.
[306] R. van der Hofstad and M. Holmes, Monotonicity for excited random
walk in high dimensions, Probab. Theory Related Fields 147 (2010),
no. 1-2, 333–348.
[307] Y. Velenik, Localization and delocalization of random interfaces,
Probab. Surv. 3 (2006), 112–169.
[308] A. Yu. Veretennikov, On the rate of convergence for infinite server
Erlang–Sevastyanov’s problem, Queueing Syst. 76 (2014), no. 2, 181–
203.
[309] A. Yu. Veretennikov and G. A. Zverkina, Simple proof of Dynkin’s
formula for single-server systems and polynomial convergence rates,
Markov Process. Related Fields 20 (2014), no. 3, 479–504.
[310] M. Voit, Central limit theorems for random walks on N0 that are associated with orthogonal polynomials, J. Multivariate Anal. 34 (1990),
no. 2, 290–322.
[311]
, Strong laws of large numbers for random walks associated with
a class of one-dimensional convolution structures, Monatsh. Math.
113 (1992), no. 1, 59–74.
[312] S. Volkov, Vertex-reinforced random walk on arbitrary graphs, Ann.
Probab. 29 (2001), no. 1, 66–91.
[313] W. Wagner, Explosion phenomena in stochastic coagulationfragmentation models, Ann. Appl. Probab. 15 (2005), no. 3, 2081–
2112.
[314] G. H. Weiss, Random walks and their applications, Amer. Sci. 71
(1983), 65–71.
[315] G. H. Weiss and R. J. Rubin, Random walks: Theory and selected
applications, Adv. Chem. Phys. 52 (1983), 363–505.
[316] W. M. Wonham, Liapunov criteria for weak stochastic stability, J.
Differential Equations 2 (1966), 195–207.
[317]
, A Liapunov method for the estimation of statistical averages,
J. Differential Equations 2 (1966), 365–377.
Bibliography
393
[318] S. Zachary, On two-dimensional Markov chains in the positive quadrant with partial spatial homogeneity, Markov Process. Related Fields
1 (1995), no. 2, 267–280.
[319] O. Zeitouni, Random walks in random environment, Lectures on
probability theory and statistics, Lecture Notes in Math., vol. 1837,
Springer, Berlin, 2004, pp. 189–312.
[320]
, Random walks in random environments, J. Phys. A 39 (2006),
no. 40, R433–R464.
[321] M. P. W. Zerner, Recurrence and transience of excited random walks
on Zd and strips, Electron. Comm. Probab. 11 (2006), 118–128 (electronic).
Index
adapted, 24
almost every, 276
almost-sure bounds, 82, 85, 102, 227,
244, 277, 279, 283
angular asymptotics, xiii, 13, 15, 217,
222
annealed, 324
anomalous recurrence, xiii, 11, 19, 185
asymptotically zero drift, 14, 84, 98,
185, 188, 215, 227, 283
Azuma–Hoeffding inequalities, 50, 91,
286, 289
Bessel process, 181, 182
birth-and-death chain, 174
Borel sets, 23
Borel–Cantelli lemma, 40
branching process, 117, 121, 176, 182,
329
branching random walk, 329
Burkholder’s inequality, 321
Doob’s inequality, 38
drift condition, x
Dvoretzky–Erdős theorem, 149
Dynkin’s formula, 38, 46, 67, 91
ergodic theorem, 89, 129, 177, 295
exclusion process, 306, 316, 326
exclusion-voter process, 306, 321, 327
excursion, 127, 131
excursion maximum, 42, 144
explosion, 331, 332, 346
filtration, 24
Foster’s criterion, 67
gambler’s ruin, 1, 42
Gamma distribution, 157
generator, 332
growth model, 182, 329
half strip, 175, 180
Heaviside configuration, 307
central limit theorem, 164, 165, 181 heavy tails, xiii, 4, 15, 239
hitting time, 25, 26, 333
Chapman–Kolmogorov relation, 26
defective, 57, 350
Chung–Fuchs theorem, 10, 19, 176,
existence of moments, 74, 102,
185, 190
134, 227, 336
convexity, 35
integrable, 66, 338
diffusion, 96, 181
non-existence of moments, 4, 77,
102, 134, 336
diffusive, 145, 149, 179
Doob decomposition, 34
non-integrable, 72, 134
394
Index
implosion, 331, 333, 349, 361
increment covariance matrix, 11, 69,
116, 135, 189, 190, 199
increment moment function, 6, 8, 97,
103, 104, 194, 300, 356
inequality
Azuma–Hoeffding inequalities, 50,
91, 286, 289
Burkholder’s inequality, 321
maximal inequality, 44, 45, 47,
48, 90
for submartingales, 38
for supermartingales, 44
Kolmogorov’s inequalities, 39,
48, 91, 191, 234
Kushner–Kalashnikov inequality, 47
integrable, 35
invariance principle, 181
irreducibility, 26, 124
irreducible, 333
Itô’s formula, 338
395
local time, see occupation time
locally finite, 29, 125, 128
Lyapunov exponent, 292
Lyapunov function, x, 4
Lyapunov function criterion
for positive recurrence, 67, 337
for recurrence, 53, 334
for transience, 56, 335
Marcinkiewicz–Zygmund strong law,
133
Markov chain
aperiodic, 26
conservative, 332
continuous-time, 333
countable, 26
criterion for positive recurrence,
67
criterion for recurrence, 53
criterion for transience, 56
discrete-time, 25
generator, 332
irreducible, 26
jump chain, 332
life time, 332
null recurrence, 27, 73, 101, 333
Kaplan’s condition, 93, 360
period, 26
Kolmogorov’s inequalities, 39, 48, 91,
positive recurrence, 27, 67, 72,
191, 234
101, 333, 337
recurrence, 27, 52, 53, 333, 334
Lamperti problem, 7, 97, 194, 300
regular, 332
supercritical, 163, 218
stationary measure, 27, 72
lamperti problem, 356
time-homogeneous, 26
Lamperti’s problem, xiii
transience, 27, 52, 56, 64, 100,
last exit time, 150, 180, 239, 251
333, 335
law of large numbers, 10, 16, 20, 131–
transition function, 26
133, 164, 181, 217, 218
transition probabilities, 26
law of the iterated logarithm, 39, 281,
Markov property, 18, 25
285
strong, 30
limiting direction, 13, 16, 217
martingale, 24
linear transformation, 11
local escape, 106, 125, 149, 154
convergence theorem
396
Index
for submartingales, 35
quenched, 275
for supermartingales, 35
queue, 329
difference, 104, 108
optional stopping theorem, see op- random environment, 275, 291
tional stopping theorem
random matrix, 292
orthogonality of increments, 35 random polymer, 182
semimartingale, 25
random walk, ix, 1, 8
submartingale, 24
branching, 329
supermartingale, 24
centrally biased, 14, 185, 188, 215,
maximal inequality, see inequality
216, 227, 236
mean drift, 7, 8, 189, 190
centrally biased, discrete, 200, 235
method of moments, 159, 162
controlled, 19, 188
moments Markov property, 108, 181
elliptic, 19, 187, 203
excited, 329
natural filtration, 24
in random environment, 275, 292,
non-confinement, 105, 125, 126, 175,
324
190, 198, 234, 240, 267, 272,
linear transformation of, 11
300
multidimensional, 56, 84, 98, 106,
non-degeneracy, 48, 54, 190, 191, 199
115, 135, 149, 185, 189, 256,
non-dissipative, 92
358
normal distribution, 157, 164
one dimensional, 28, 31, 87, 101,
nullity, 4, 73, 129, 156
102, 150
occupation time, 71, 129, 156, 230,
oscillating, 92, 239, 255, 274
238
Pearson–Rayleigh, 9, 19, 186
open problem, 238, 326, 328
reinforced, 329
optional stopping theorem
simple, 2, 9, 17, 28, 31, 51, 53,
for submartingales, 36
56, 87, 98, 104, 115, 149, 256,
for supermartingales, 36
358
Oseledets multiplicative ergodic the- range, 188, 230, 237, 238
orem, 295
rate of escape, 10, 149, 163, 217, 228,
239, 244, 283
Pólya’s theorem, 3, 53, 57, 115, 190
reaction-diffusion process, 328
partial sums, 3, 9
recurrence, 3, 7, 9, 27, 53, 114, 127,
passage time, see hitting time
191, 199, 204, 209, 216, 227,
polling system, 329
269, 276, 283, 294, 300, 316,
polynomial ergodicity, 178, 179
318, 321, 322
power set, 23, 26
regeneration, 127
previsible, 34
renewal function, 153, 180
quadratic variation, 40
reversible, 325
Index
semimartingale, x
Sinai’s regime, 277, 282, 324
Solomon’s theorem, 276
spatially homogeneous, 9, 13, 185
square-difference identity, 48
square-integrable, 35
square-root boundary, 70, 93
staircase, 309
state space, 23
stochastic billiards, 299, 326
stochastic difference equation, 104, 108,
175
stochastic process, 23
stopping time, 24, 337
string, 291
strong law of large numbers, see law
of large numbers
strong Markov property, see Markov
property
sub-diffusive, 102, 145, 179
super-diffusive, 163, 217
supermarket model, 329
transience, 3, 7, 9, 27, 56, 58, 59, 114,
127, 191, 199, 200, 204, 208,
216, 227, 241, 270, 276, 283,
294, 300, 317
truncation, 13
uniform ellipticity, 14, 106, 175, 190,
199, 231, 282
urn model, 330
voter model, 306, 318, 326
weak renewal theorem, 49, 91
zero drift, 10, 54, 91, 98, 116, 135,
174, 185, 186, 190, 200, 234
zero-one law, 20
397