Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Stochastic Convergence in Metric Spaces and
Empirical Processes
Marcel Richter
Seminar Mathematical Statistics
WS14/15
Supervision: Joseph Tadjuidje
February 13, 2015
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Outline
1
Stochastic Convergence in Metric Spaces
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
2
Empirical Processes as Stochastic Processes
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and
Donsker II
3
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
Outline
1
Stochastic Convergence in Metric Spaces
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
2
Empirical Processes as Stochastic Processes
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and
Donsker II
3
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
Basic Definitions
Definition
A metric space is a set D equipped with a metric. A metric or
distance function is a map d : D × D 7→ [0, ∞) with the
properties:
d(x, y ) = d(y , x)
d(x, z) ≤ d(x, y ) + d(y , z)
d(x, y ) = 0 ⇔ x = y
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
Basic Definitions
Definition
The Borel σ − Field (or Borel σ − Algebra) on a metric space D is
the smallest σ − Algebra (a subset of P(D) that is not empty,
closed under complementation as well as under countable unions),
that contains all the open subsets of D.
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
Basic Definitions
Definition
Assume D1 , D2 to be two metric spaces and Σ1 , Σ2 the
corresponding Borel σ − Fields. A function
f : (D1 , Σ1 ) 7→ (D2 , Σ2 ) is called borel − measurable if
f −1 (A) ∈ Σ1 for any A ∈ Σ2 .
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
Basic Definitions: Random Elements
Definition
A borel-measurable function X : (D1 , Σ1 ) 7→ (D2 , Σ2 ) on a
probability space (D1 , Σ1 , P) is called random element with values
in D2 .
Random Elements generalizeRandom Variables
Usual types of Random Elements: Random Vectors, Random
Matrices, Random Functions, Stochastic Processes
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
Examples for metric spaces
Example
The Euclidean space Rn is a metric space with respect to the
metric induced the euclidean norm: d(x, y ) =k x − y k2 . Of
course, any p-norm induces a metric. This implies Random
Variables, Random Vectors and as well Random Matrices to be
Random Elements.
Example
The extended real line R = [−∞, ∞] is a metric space with respect
to d(x, y ) = |Φ(x) − Φ(y )| where Φ is any bounded, strictly
increasing continous function as is the cumulative distribution
function of the normal distribution.
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
Examples for metric spaces
Example
The Euclidean space Rn is a metric space with respect to the
metric induced the euclidean norm: d(x, y ) =k x − y k2 . Of
course, any p-norm induces a metric. This implies Random
Variables, Random Vectors and as well Random Matrices to be
Random Elements.
Example
The extended real line R = [−∞, ∞] is a metric space with respect
to d(x, y ) = |Φ(x) − Φ(y )| where Φ is any bounded, strictly
increasing continous function as is the cumulative distribution
function of the normal distribution.
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
Examples for metric spaces
Definition
Given an arbitrary set T, let l ∞ (T ) be the notation to denote the
set of all bounded functions z : T 7→ R. Let sums and scalar
multiplication be defined pointwise. The uniform norm is thus
defined by:
k z kT = sup |z(t)|
t∈T
Equipped with the uniform norm, l ∞ (T ) is a normed (and thus
metric) space.
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
Outline
1
Stochastic Convergence in Metric Spaces
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
2
Empirical Processes as Stochastic Processes
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and
Donsker II
3
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
Stochastic Convergence of Random Elements
Usually: convergence in distribution of random vectors defined
by reference to their distribution functions
But: Distribution functions do not extend in a natural way to
random elements with values in metric spaces.
Remedy: Alternative Characterizations as given by
Portmanteau Lemma: P(Xn ≤ x) → P(X ≤ x) for all
continuity points of x → P(X ≤ x) ⇔Ef (Xn ) → Ef (X ) for all
bounded, continuous functions f
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
Stochastic Convergence of Random Elements
First Definition-Attempt: A sequence of random elements Xn
with values in a metric space D is said to converge in
distribution to a random element X if Ef (Xn ) → Ef (X ) for
every bounded, continuous function f : D 7→ R
in some applications the ”random elements” of interest tum
out not to be Borel-measurable
extend the preceding definition to a sequence of arbitrary maps
Xn : Ωn 7→ D, defined on probability spaces (Ωn , Un , Pn )
Problem: Ef (Xn ) need no longer make sense
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
Stochastic Convergence of Random Elements
Remedy: Replace E by the outer expectation E ∗ defined by
E ∗ f (X ) = inf {EU : U : Ω 7→ R, measurable, U ≥
f (X ), EU exists}
Definition
A sequence of arbitrary maps Xn : Ω 7→ D converges in
distribution to a random element X if E ∗ f (Xn ) → Ef (X ) for every
bounded, continuous function f : D → R. Here we insist that the
limit X be Borel-measurable.
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
Stochastic Convergence of Random Elements
Central Results on Convergence of Random Vectors (see
Chapter2):
Portmanteau-Lemma: Several characterizations of weak
convergence (such as P(Xn ≤ x) → (X ≤ x) for all continuity
points of x 7→ P(X ≤ x) and Ef (Xn ) 7→ Ef (X ) for all
bounded, continuous functions f )
Results regarding the realations between weak convergence,
convergence in probability and almost sure convergence
The Continous Mapping Theorem that specifies under what
conditions Xn → X implies f (Xn ) → f (X )
What about the Laws of Large Numbers and the CLT?
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
Outline
1
Stochastic Convergence in Metric Spaces
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
2
Empirical Processes as Stochastic Processes
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and
Donsker II
3
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
What is a Stochastic Process?
Definition
A stochastic process X = {Xt : t ∈ T } is a collection of random
variables Xt : Ω 7→ R, indexed by an arbitrary set T and defined
on the same probability space (Ω, U, P).
For a fixed ω, the map t 7→ Xt (ω) is called a sample path
for our purpose: think of X as a random function, whose
realizations are the sample paths
Examples: Random Walk, Poisson-Processes and
Markov-Chains.
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
What is a Stochastic Process?
Figure: Realization of a random function
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
What is a Stochastic Process?
Figure: Another R. of the same RF
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
What is a Stochastic Process?
Figure:
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
Example: The Brownian Bridge
Definition
A Gaussian process is a stochastic process X = {Xt : t ∈ T }, for
which (Xt1 , Xt2 , . . . , Xn ) is multi-variate normally distributed for any finite
set t1 , t2 , . . . , tn ∈ T .
Definition
A continous time gaussian random process Gλ on [0, 1] (λ denoting
the uniform distribution on [0,1]) with Gλ (0) = 0 and Gλ (1) = 0 with
0 mean and covariance function E Gλ (t1 )Gλ (t2 ) = min(t1 , t2 ) − t1 · t2 is
called Standard Brownian Bridge. GF = Gλ ◦ F is called
F-Brownian Bridge and has 0 mean and covariance function
E GF (t1 )GF (t2 ) = F (min(t1 , t2 )) − F (t1 ) · F (t2 ).
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
Example: The Brownian Bridge
Figure: Realization of a Brownian Bridge
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
Example: The Brownian Bridge
Figure:
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
Example: The Brownian Bridge
Figure:
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Outline
1
Stochastic Convergence in Metric Spaces
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
2
Empirical Processes as Stochastic Processes
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and
Donsker II
3
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Basic Definitions
Definition
Let X1 , ..., Xn be a random sample from a distribution function F
on the real line. The empirical distribution function is defined as:
n
1X
Fn (t) =
1{Xi ≤ t}
n
i=1
The stochastical process defined by Gn =
empirical process
Marcel Richter
√
n(Fn − F ) is called
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Basic Definitions
Figure: Example of an edf and corresponding empirical process (N=10)
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Glivenko-Cantelli-Theorem I
Figure: The sequence of Fn approaches F (displayed: N=3, N=10,
N=50)
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Glivenko-Cantelli-Theorem I
The Glivenko-Cantelli theorem extends the law of large
numbers and gives uniform convergence. It uses the uniform
distancek Fn − F k∞ = sup |Fn (t) − F (t)| that is known as the
t
Kolmogorov-Smimov statistic.
Theorem
(Glivenko-Cantelli). If X1 , X2 , . . . are i.i.d. random variables with
as
distribution function F , then k Fn − F k∞ → 0.
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Donsker-Theorem I
Figure: Example of an edf and corresponding empirical process (N=10)
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Donsker-Theorem I
Figure: Simulation of a Uniform Empirical Process for N=50
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Donsker-Theorem I
Figure: N=200 and N=1000
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Donsker-Theorem I
We observe: The sequence of empirical processes
random functions, converges in distribution.
√
n(Fn − F ), viewed as
Theorem
(Donsker). If X1 , X2 , ... are i.i.d. random variables with
√ distribution
function F , then the sequence of empirical processes n(Fn − F )
converges in distribution in the space D[−∞, ∞] to a tight random
element GF , whose marginal distributions are zero-mean normal with
covariance function E GF (ti )GF (tj ) = F (min(t1 , t2 )) − F (ti ) · F (tj ).
The limit process GF is a F Brownian bridge process (or standard
Brownian bridge if F is the uniform distribution on [0, 1] as in the
example).
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Donsker-Theorem I
Figure: Convergence of an [0,1]-uniform empirical process
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Outline
1
Stochastic Convergence in Metric Spaces
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
2
Empirical Processes as Stochastic Processes
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and
Donsker II
3
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Basic Definitions
Definition
Let X1 , ..., Xn be a random sample from a probability distribution P on a
measurableP
space (X , A). The
Pnempirical measure Pn is defined by
n
Pn (A) = n1 i=1 IA (Xi ) = n1 i=1 δXi (A) for A ∈ A where IA is the
indicator function and δX is the Dirac measure.
Pn (A) describes the relative frequency of an event A ∈ A within a
sample
to generalize observe: by the following definition, the empirical
measure Pn maps measurable functions f : X → R to their
empirical mean
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Basic Definitions
Definition
Pn
We also call Pn f := n1 i=1 f (Xi ) the expectation of f under the
empirical measure Pn .
In particular, the empirical measure of A is simply the empirical mean of
the indicator function, Pn (A) = Pn IA .
Definition
We call Pf :=
R
X
fdP the expectation of f under P.
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Glivenko-Cantelli-Theorem II
By the law of large numbers, the sequence Pn f converges
almost surely to Pf (for every f such that Pf is defined)
The abstract Glivenko-Cantelli theorems make this result
uniform in f ranging over a class of functions
Definition
A class F of measurable functions f : X 7→ R is called P
as∗
-Glivenko-Cantelli if k Pn f − Pf kF = sup |Pn f − Pf | → 0
f ∈F
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Glivenko-Cantelli-Theorem II
Definitions
We shall mainly consider the bracketing entropy relative to the
Lr (P)−norm k f kP,r = (P|f |r )1/r .Given two functions l and u, the
bracket [l, u] is the set of all functions f with l ≤ f ≤ u. An ε-bracket in
Lr (P) is a bracket [l, u] with P(u − l)r < εr . The bracketing number
N[ ] (ε, F, Lr (P)) is the minimum number of ε-brackets needed to cover F
. The entropy with bracketing is the logarithm of the bracketing
number.
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Glivenko-Cantelli-Theorem II
Theorem
(“Abstract” Glivenko-Cantelli). Every class F of measurable
functions such that N[ ] (ε, F, L1 (P)) < ∞ is P-Glivenko-Cantelli.
“abstract” Glivenko-Cantelli generalizes the result of the
preceeding version
to see how the older version is a special case of the just
presented one: set F the collection of all indicator functions
of the form ft = 1(−∞,t] with t ranging overR
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Donsker-Theorem II
Definition
The empirical process evaluated at f is defined as
√
Gn f = n(Pn f − Pf ).
By the multivariate central limit theorem , given any finite set of
measurable functions fi with Pfi 2 < ∞:
(Gn f1 , ..., Gn fk )
(GP f1 , ..., GP fk )
where the vector on the right possesses a multivariate-normal
distribution with mean zero and covariances
E GP f GP g = Pfg − PfPg
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Donsker-Theorem II
abstract Donsker theorems make this result ”uniform” in
classes of functions
Definition
A class F of measurable functions f : X 7→ R is called
P-Donsker if the sequence of processes {Gn f : f ∈ F} converges
in distribution to a tight limit process in the space l ∞ (F)
Then the limit process is a Gaussian process GP with zero
mean and covariance function E GP f GP g = Pfg − PfPg
P-Brownian bridge
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Donsker-Theorem II
a sufficient condition for a class to be Donsker is that they do not
grow too fast (usullay: N[ ] → ∞ as ε → 0)
to measure this “speed”, use the bracketing integral:
Rδp
J[ ] (δ, F, L2 (P)) = 0 log N[ ] (ε, F, L2 (P))dε.
Theorem
(Donsker). Every class F of measurable functions with
J[ ] (1, F, L2 (P)) < ∞ is P-Donsker.
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Examples of Glivenko-Cantelli and Donsker Classes of
Functions
Example
(Distribution function I ). Assume F to be the collection of all
indicator functions of the form ft = 1(−∞,t] with t ranging over R. Since
F is Donsker:
k Pn f − Pf kF =sup |Pn f − Pf | = sup |Pn 1(−∞,t] − P1(−∞,t] |
t
f ∈F
Z
n
1X
1(−∞,t] (Xi ) − 1(−∞,t] d P|
=sup |
t n
i=1
as ∗
=sup |Fn (t) − F (t)| → 0
t
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Examples of Glivenko-Cantelli and Donsker Classes of
Functions
Example
(Distribution
function II ). Equally can be shown that
F = ft = 1(−∞,t] |t ∈ R suffices the bracket-number-growth-rate-criteria of
the Donsker-Theorem. We get
√
n Pn 1(−∞,t] − P1(−∞,t] , t ∈ R
{Gn ft , t ∈ R} =
√
n→∞
=
n (Fn (t) − F (t)) , t ∈ R −→ GF
with GF = {Xt , t ∈ R} being a Gaussian Process with 0 mean and Covariance
Function E GP f Gp g = Pfg − PfPg . Hence:
Z
cov (t1 , t2 ) = Pft1 ft2 − Pft1 Pft2 = 1(−∞,min(t1 ,t2 )] dP − . . .
= F (min(t1, t2 )) − F (t1 ) · F (t2 )
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and Donsker II
Examples of Glivenko-Cantelli and Donsker Classes of
Functions
Example
(Parametric class). Let F = {fθ , θ ∈ Θ} be a collection of measurable
functions indexed by a bounded subset Θ ⊂ Rd . Suppose that there exists a
measurable function m such that |fθ1 (x) − fθ2 (x)|≤ m(x) k θ1 − θ2 k, every
θ1 , θ2 . If P|m|r < ∞, then there exists a Konstant K, depending on Θ and d
only, such that the bracketing numbers satisfy
d
diamΘ
N[ ] (ε k m kP,r , F, Lr (P)) ≤ K
, every 0 < ε < diamΘ.
ε
Hence, the entropy is of smaller order than log (1/ε) and it follows, that the
bracketing number integral certainly converges so that F is P-Donsker.
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Outline
1
Stochastic Convergence in Metric Spaces
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
2
Empirical Processes as Stochastic Processes
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and
Donsker II
3
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Goodness-of-Fit-Statistics
Practical Problem: Given a sample of idd. Random Elements (here:
Random Variables) with unknown distribution, how to design a statistical
test to determine whether an assumed Distribution is the real distribution
Idea: Since Pn always is a reasonable estimator for P, use a measure for
the discrepancy between Pn and P
One such commonly used is the already mentioned
Kolmogorov-Smirnov-Statistic:
√
n k Fn − F k∞
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Goodness-of-Fit-Statistics: Kolmogorov-Smirnov I
√
Observe: the Kolmogorov-Smirnov-Statistic n k Fn − F k∞ (as
well as many others!) is a continuous function of the empirical
process
hence, the continuous-mapping theorem and (yet) the (simple)
Donsker-Theorem imply:
Corollary
If X1 , X2 , ... are i.i.d. random variables with distribution function F , then
the sequences of Kolmogorov-Smirnov statistics converge in distribution
to k GF k∞ (with GF being the F -brownian bridge). The distribution of
this limit is the same for every continuous distribution function F .
the test offers the great advantage of not requiring the underlying
Random Variable to be normally distributed
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Goodness-of-Fit-Statistics: Kolmogorov-Smirnov I
Constructing a Test:
According to the corollary, under the null hypothesis (that F
√
is the real Distribution of a givin sample) n k Fn − F k∞
converges in distribution to the supremum of a F -Brownian
Bridge: k GF k∞
Luckily, the distribution function
of this is analystical:
P
k−1 e −2k 2 x 2
P(k GF k∞ ≤ x) = 1 − 2 ∞
(−1)
k=1
√
the null hypothesis is rejected at level α if n k Fn − F k∞ is
greater than the (1 − α2 )-quantile of the distribution of
k GF k∞
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Goodness-of-Fit-Statistics: Kolmogorov-Smirnov I
Figure: Visualization of the Kolmogorov-Smirnov Test
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Goodness-of-Fit-Statistics: Kolmogorov-Smirnov II
With the generalized notions presented in the last chapte: modify the the
Kolmogorov-Smirnov Test:
to test the null hypothesis that P belongs to a certain family
{Pθ : θ ∈ Θ} of distributions
measure of the discrepancy between Pn and Pθ̂ , for a reasonable
estimator θ̂ of θ: modified
Kolmogorov-Smirnov statistic for
√
testing normality: sup n|F(t) − Φ( t−SX̄ )|
t
for many goodness-of-fit statistics of this type, the limit distribution
√
follows from the limit distribution of n(Pn − Pθ )
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Goodness-of-Fit-Statistics: Kolmogorov-Smirnov II
Theorem
Let X1 , ..., Xn be a random sample from a distribution Pθ indexed
by θ ∈ Rk . Let F be a Pθ -Donsker class of measurable functions
and let θ̂n be estimators that are asymptotically linear with
influence function ψθ . Assume that the map θ 7→ Pθ from Rk to
l ∞ (F) is Frichet differentiable at θ. Then the sequence
√
n(Pn − Pθ̂ ) converges under θ in distribution in l ∞ (F) to the
process f 7→ GPθ f − GPθ ψθT Ṗθ f .
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Goodness-of-Fit-Statistics: Kolmogorov-Smirnov II
The preceding theorem implies (for instance), that
√
n k Fn − Fθ k∞ converges in distribution to the supremum of a
certain Gaussian process:
√
√
n k Fn (t) − Fθ (t) kt = n k (Pn − Pθ̂ )(ft ) kt
n→∞
→ k GPθ (ft ) − GPθ (ψθT Ṗθ ft ) kt
Note: The distribution of the limit may depend on the model
θ 7→ Fθ ,the estimators θ̂n , and even on the parameter value θ
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Example
We want to test, whether the distribution of a sample belongs to a
certain family (for matters of simplicity, let’s assume we know the
reall scale parameter σ 2 to be 2)
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Example
Figure: Sample of idd Random Variables with unknown distribution (but
known Variance)
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Example
We guess, that the data is normally distributed θ = µ and
estimate θ̂ = X̄n
We thus have, that under the 0-Hypothesis that the reall
Distribution of the Data belongs to the normal family:
√
n k Fn − Fθ̂ k∞ converges to the supremum of the difference
of a Φ-Brownian Bridge and a gaussian process, which Xt ’s
are distributed like (θ̂ − θ0 ) · Φθ (θ0 , t)
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Example
(θ̂ − θ0 ) can be written as
N (0, σ 2 )-distributed
√1
n
Pn
i=1 (Xi
− θ0 ) which is
by CLT
dΦ
dθ (θ0 , t)
is a scalar that is given by the negative of the
density function of a (θ0 , σ 2 )-normal distributed RV
It follows, that the part on the right is a √
guassian process with
0 mean and cov (t1 , t2 ) = ϕ(t1 ) · ϕ(t2 ) · σ 2 where ϕ is the
above mentioned density function
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Example
With the given variance (here:2) and empirical mean (here:2.48),
we can compute the covariance function of the limit process of
(Fn − Fθ̂ )
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Example
Figure: Realization of the Limit Process
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Example
Figure: Results of the Simulation (600 iterations)
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Example
From the Simultaion, we derive the 1 − α-quantile of the
simulated distribution of k Fn − Fθ̂ k∞
For the Example, let we get the 0.95-quantile to be
approximately 1.4
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Example
Figure: Fn of the Sample and Fθ̂
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Example
From the Simultaion, we derive the 1 − α-quantile of the
simulated distribution of k Fn − Fθ̂ k∞ under the 0-Hypothesis
For the Example, let we get the 0.95-quantile to be
approximately 1.4
√
Finally, we compute n k Fn − Fθ̂ k∞ from the Data. We get:
1.212
We cannot refuse the Hypothesis, that the Data is normally
distributed, although it’s reall distribution is a
(2.7,1.75)-weibull-distribution
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Example
Figure: Densities of (λ, k)-Weibull-Distributions
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Outline
1
Stochastic Convergence in Metric Spaces
Basic Definitions: Metric Spaces and Random Elements
Stochastic Convergence of Random Elements
Stochastic Processes as Random Elements
2
Empirical Processes as Stochastic Processes
The Theorems of Glivenko-Cantelli and Donsker I
Generalization: The Theorems of Glivenko-Cantelli and
Donsker II
3
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Random Functions
If a class F of functions is P -Glivenko-Cantelli, then:
as
|Pn f − Pf | → 0 uniformly in f varying over F
as
It is immediate that: |Pn fˆn − P fˆn | → 0 for every sequence of
random functions fn ∈ F
as
as
If fn → f0 and fn is dominated, so that P fˆn → Pf0 , then
as
Pn fˆn → Pf0
Can we similarly apply the Donsker-Theorem?
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Random Functions
Lemma. Suppose that F is a P -Donsker class of measurable
functions and fˆn is a sequence of random functions that take their
2
R
values in F such that
fˆn (x) − f0 (x) dP(x) converges in
as
probability to 0 for some fo f0 ∈ l2 (P). Then Gn (fˆn − f0 ) → 0 and
hence Gn fˆn
Gp f0 .
Note:
Pn This can help to handle random sequences of the form
i=1 fn,θ̂n (Xi ) for functions fn,θ that change with n and depend on
an estimated parameter
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Random Functions: Mean Absolute Deviation
Definition
The mean absolute deviation of a random sample X1 , ..., Xn is the scale
estimator
n
1X
Mn =
|Xi − X̄n |
n i=1
Why study mean absolute deviation?
We’ll show, that Mn is asymptotically normal with mean 0
and variance equal to the variance of 2(F (0) − 1)X + |X |
Note: Variance is never hurt by not knowing the true mean
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
if Fx 2 < ∞ , then the set of functions x 7→ |x − θ| with θ
ranging over a compact, such as [−1, 1], is F- Donsker (by
geometric class example)
because, by the triangle inequality,
2
P
F |x − X̄n | − |x| ≤ |X̄n |2 → 0,the preceding lemma shows
P
that Gn |x − X̄n | − Gn |x| → 0
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
Random Functions: Mean Absolute Deviation
P
From Gn |x − X̄n | − Gn |x| → 0 we can derive, that
√
√
n (Mn − F |x|) = n F |x − X̄n | − F |x| + Gn |x| + oP (1)
If the map θ 7→ F |x − θ| is differentiable at 0, then, with the
derivative written as 2F(0)-1, the first term on the right is
asymptotically equivalent to (2F (0) − 1) Gn x, by the delta
method
It follows: the mean absolute deviation is asymptotically
normal with mean zero and asymptotic variance equal to the
variance of (2F (0) − 1) X1 + |X1 |
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
What else?
Glivenko-Cantelli classes, Donsker classes, and entropy
appears are convenient to state ”regularity conditions” in
many fields (see M- and Z-Estimators)
Worth studying: Glivenko-Cantelli and Donsker-Theorem
analogous theorems for the case that the class of functions is
not constant
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes
Stochastic Convergence in Metric Spaces
Empirical Processes as Stochastic Processes
Applications of Glivenko-Cantelli and Donsker in Statistics
Goodness-of-Fit Statistics
Random Functions
The End
Thank you for your attention
Marcel Richter
Stochastic Convergence in Metric Spaces and Empirical Processes