Probability Theory
Order Statistics
Consider a set X₁,X₂,…,Xn of independent and identically distributed
(continuous) random variables.
Definition 1.1. Let X(k) denote the k:th smallest of X₁,X₂,…,Xn. The random
vector (X(1),X(2),…,X(n)) is called the order statistic and X(k) the k:th order
variable, k=1,2,…,n.
Chapter 4
For the order statistic we thus have that
Order Statistics
It is of natural interest to find the joint probability distributions of these
ordered random variables, and we will begin by finding the marginal
probability distributions of the ”extremes”, that is
Thommy Perlinger, Probability Theory
1
2
The probability distribution of X(n)
Example 1
Let Y₁,Y₂,…,Yn be a set of independent and identically distributed U(0,θ).
Determine the probability distribution of Y(n).
From the common distribution function F(x) it follows that
Since
independent…
it follows that
…and identically
distributed
Differentiation according to the chain rule gives us the density function of X(n).
3
Thommy Perlinger, Probability Theory
4
1
Example 2
Example 2
Let Y₁,Y₂,… be a sequence of independent and identically
distributed U(0,1). Furthermore, let X be discrete with
probability function
Find the distribution of W = min(Y₁,Y₂,…,YX).
We first observe that W | X=x is the first order statistic from a
sample of size x.
Thommy Perlinger, Probability Theory
5
The probability distribution of X(k)
Thommy Perlinger, Probability Theory
6
Example for the median
We now generalize and look at the marginal probability distribution of X(k),
i.e. the k:th order variable, k=1,2,…,n.
Let X1,X2, and X3 be a random sample from an Exp(1) distribution. Find the
distribution of the median and compute its mean.
Theorem 1.2. Consider a set X₁,X₂,…,Xn of independent and identically
distributed (continuous) random variables with density function f(x) and
distribution function F(x). For k=1,2,…,n, the density of X(k) is given by
According to Theorem 1.2 the density of the median, i.e. X(2), is given by
The mean of X(2) is thus
Proof. Both a formal and a heuristic proof can be found in the textbook.
Thommy Perlinger, Probability Theory
7
Thommy Perlinger, Probability Theory
8
2
The joint distribution of X(1) and X(n)
Exercise 2.6
Exercise 2.6. Suppose n points are chosen uniformly and independently of
each other on the unit disc. Compute the expected value of the area of the
annulus obtained by drawing circles through the extremes.
Theorem 2.1. Consider a set X₁,X₂,…,Xn of independent and identically
distributed (continuous) random variables with density function f(x) and
distribution function F(x). The joint density of the extremes is given by
What to do? Let the distance of a point to the center be Rn, n=1,2,…,n.
If R(1)=u and R(n)=v we get the following situation:
Theorem 2.2. The range is defined by Rn= X(n)- X(1). Let u=x(1). The density
of Rn is then given by
v
u
Step 1. Find the distribution of R, i.e. the distance to the
center.
Step 2. Find the distribution of W=R2, i.e. the squared distance
to the center.
Step 3. Find the distribution of the range RW = W(n) -W(1) for the squared
distance to the center.
Remark. When the domain of X has an upper bound b, then for Rn=r the
upper bound of U becomes b-r.
Thommy Perlinger, Probability Theory
The area of the annulus is thus given by π(v2-u2).
9
Step 4. Compute E(π·RW), i.e. the expected value of the area of the annulus.
Exercise 2.6
Exercise 2.6
Step 1. If (X1,Y1), (X2,Y2),…, (Xn,Yn) represents the coordinates of the
chosen points, then they are i.i.d. bivariate random variables with common
density given by
We thus want to find the marginal distribution of R. Since
it follows from the transformation theorem that
For each such point, we are interested in the distance to the center of the
circle. We therefore switch to polar coordinates, i.e. we study the bijection
and hence that
where R is the distance to the center of the circle and Θ the angle.
Thommy Perlinger, Probability Theory
11
Thommy Perlinger, Probability Theory
12
3
Exercise 2.6
The joint distribution of the order statistic
Step 2. Let W=R2. It follows from the (univariate) transformation theorem that
Theorem 3.1. Consider a set X₁,X₂,…,Xn of independent and identically
distributed (continuous) random variables with density function f(x) and
distribution function F(x). The joint density of the order statistic is given by
Step 3. The squared distances to the center can thus be seen as n i.i.d.
U(0,1). Let RW represent the range. It then follows from Theorem 2.2 that
Step 4. We conclude that RW ∈ β(n-1,2). It therefore follows that the expected
value of the area of the annulus is given by the intuitively appealing
Thommy Perlinger, Probability Theory
13
Thommy Perlinger, Probability Theory
Problem 4.4.14
Let X1, X2 and X3 be independent U(0,1)-distributed random variables.
Prove the intuitively reasonable result that X(1) and X(3) are conditionally
independent given X(2) and determine this (conditional) distribution.
14
Problem 4.4.6 a
Let X1, X2, X3 and X4 be independent U(0,1)-distributed random variables.
Compute Pr(X(3)+X(4) ≤1).
The first task is to find the joint density of X(3) and X(4). Theorem 3.1 tells us
Remark. We thus have to determine the distribution of
It follows from Theorem 3.1 and Theorem 1.2 that
which implies that
which means that
From here on it is standard technique using the transformation theorem.
and the statement is proven. The marginal distributions are U(0,x) and U(x,1).
Thommy Perlinger, Probability Theory
16
4
Problem 4.4.6 a
Problem 4.4.6 a
To find the density of X(3)+X(4) we introduce the auxiliary variable X(3), i.e.
that is
Hence, the marginal distribution of U is given by
and so
and so it follows from the transformation theorem that
and so we finally find that
A little bit of care has to be taken when
determining the marginal distribution of U.
We see that the two cases 0<u<1 and 1≤u<2
must be considered separately.
Thommy Perlinger, Probability Theory
17
Thommy Perlinger, Probability Theory
18
5