Probability theory
Mihálykóné Orbán Éva
Dept. of Math.
MOE (PE MIK)
MIMAM143V
1 / 39
The law of large numbers i)
Statement (Markov inequality)
0
ξ and E (ξ ) is …nite and 0 < λ, then
P (ξ
ξ
E (ξ )
λ 1ξ
E ( λ 1ξ
λ)
λ
=
λ if ξ λ
.
0 if ξ < λ
= λ E (1ξ
E (ξ )
λ
MOE (PE MIK)
E (ξ )
.
λ
λ)
P (ξ
MIMAM143V
λ)
= λ P (ξ
λ ),
λ ).
2 / 39
The law of large numbers ii)
Statement (Csebisev inequality)
If D (η ) is …nite, 0 < ε, then
P (jη
E (η )j
2
As 0 (η E (η )) , and E (η
apply Markov inequality:
E (η ))
E (η E (η ))2
D 2 (η )
=
ε2
ε2
P (jη
MOE (PE MIK)
D 2 (η )
.
ε2
ε)
2
= D 2 (η ) is …nite, λ = ε2 ,
P (η
E (η )j
MIMAM143V
E (η ))2
ε2 =
ε)
3 / 39
The law of large numbers iii)
Statement (Another form of Chebisev inequality)
If D (η ) is …nite, 0 < ε, then
P (jη
E (η )j < ε)
Bizonyítás Original form:: P (jη
P (jη
1
MOE (PE MIK)
E (η )j
E (η )j < ε) = P (jη
P (jη
E (η )j
D 2 (η )
.
ε2
1
ε)
MIMAM143V
ε)
D 2 (η )
.
ε2
E (η )j
1
ε) =
D 2 (η )
.
ε2
4 / 39
The law of large numbers iv)
Statement (Another form of Chebisev inequality)
If D (η ) is …nite, D (η ) 6= 0, 0 < k, then
P (jη
P (jη
E (η )j
E (η )j < k D (η ))
Bizonyítás let ε = k D (η ) Then
1
k2
k D (η ))
1
1
k2
D 2 (η )
D 2 (η )
1
=
= 2.
2
2
ε
k
(k D (η ))
Remark
We get bounds for the probabilities, not the exact values. We do not need
the distribution..
MOE (PE MIK)
MIMAM143V
5 / 39
The law of large numbers v)
Another form of the Chebysev’s inequality
P (jη
E (η )j < kD (η ))
1
1
k2
Remark
Large di¤erence-small probability, small di¤erence-large probability.
Especially:
k=1
1
k=2
1
k=3
1
MOE (PE MIK)
1
=0
k2
1
= 0.75
k2
1
= 0.89
k2
P (jη
E (η )j < D (η ))
P (jη
E (η )j < 2D (η ))
0.75
P (jη
E (η )j < 3D (η ))
0.89
MIMAM143V
0
6 / 39
The law of large numbers vi)
Statement
If ξ i i = 1, 2, . . . are independent identically distributed r.v.-s,
E (ξ i ) = m, D (ξ i ) = σ are …nite, then for any 0 < ε
Bizonyítás E
1
n
P
1 n
ξi
n i∑
=1
P
1 n
ξi
n i∑
=1
∑ni=1 ξ i = m,
m
ε
!
! 0, if n ! ∞.
!
! 1, if n ! ∞
m <ε
D2
1
n
∑ni=1 ξ i =
σ2
.
n
Apply the Chebysev’s inequality with η = n1 ∑ni=1 ξ i -re.
σ2
P n1 ∑ni=1 ξ i m
ε
, P n1 ∑ni=1 ξ i m < ε
n ε2
MOE (PE MIK)
MIMAM143V
1
σ2
.
n ε2
7 / 39
The law of large numbers vi)
P
1 n
ξi
n i∑
=1
m
P
1 n
ξi
n i∑
=1
m <ε
MOE (PE MIK)
ε
!
!
MIMAM143V
! 0, if n ! ∞
! 1, if n ! ∞
8 / 39
Bernoulli theorem i)
Statement (Bernoulli theorem)
repeat n times an experiment, let kA (n) be the frequency of the event A..
Let 0 < p = P (A) < 1.Then for any 0 < ε
P
kA (n)
n
p
ε
! 0, if n ! ∞
P
kA (n)
n
p <ε
! 1, if n ! ∞.
Bizonyítás kA (n) is binomially distributed with parameters. n, and
p = P (A)
kA (n) is the sum of n independent
p characteristically distributed r.v.-s
E (1A i ) = P (A) = p, D (1A i ) = p (1 p )
Apply the law of large numbers.
MOE (PE MIK)
MIMAM143V
9 / 39
Bernoulli theorem ii)
P
P
kA (n)
n
kA (n)
n
ε
! 0, if n ! ∞
p <ε
! 1, if n ! ∞.
p
Remark
the relative frequency is close to the probability if the number of
experiments is large.
pools, computer simulations, sampling.
MOE (PE MIK)
MIMAM143V
10 / 39
Central limit theorem-computer simulation i)
One random variable
MOE (PE MIK)
MIMAM143V
11 / 39
Central limit theorem-computer simulation
The sum of two random variables
MOE (PE MIK)
MIMAM143V
12 / 39
Central limit theorem-computer simulation
The sum of 5 random variables
MOE (PE MIK)
MIMAM143V
13 / 39
Central limit theorem-computer simulation
The sum of 100 random variables
MOE (PE MIK)
MIMAM143V
14 / 39
Central limit theorem-computer simulation
One random variable
MOE (PE MIK)
MIMAM143V
15 / 39
Central limit theorem-computer simulation
The sum of 2 random variables
MOE (PE MIK)
MIMAM143V
16 / 39
Central limit theorem-computer simulation
The sum of 5 random variables
MOE (PE MIK)
MIMAM143V
17 / 39
Central limit theorem-computer simulation
The sum of 100 random variables
MOE (PE MIK)
MIMAM143V
18 / 39
The central limit theorem i)
Preparation
Let ξ i i = 1, 2, 3, . . .be independent identically distributed r.v.-s
E (ξ i ) = m, D (ξ i ) = σ.
Take ∑ni=1 ξ i
p
We know that E (∑ni=1 ξ i ) = n m, D (∑ni=1 ξ i ) = n σ.
By the properties of the expectation
E (∑ni=1 ξ i nm )
E (∑ni=1 ξ i ) nm
∑ni=1 ξ i nm
p
p
p
E
=
=
=
n σ
n σ
n σ
nm nm
p
= 0.
n σ
By the properties of the dispersion
p
D (∑ni=1 ξ i nm )
D (∑ni=1 ξ i )
n σ
∑ni=1 ξ i nm
p
p
= p
=p
= 1.
=
D
n σ
n σ
n σ
n σ
MOE (PE MIK)
MIMAM143V
19 / 39
The central limit theorem i)
Theoreml
Let ξ i i = 1, 2, 3, . . . independent identically distributed r.v.-s,
E (ξ i ) = m, D (ξ i ) = σ.
Then lim P
n !∞
∑ni=1 ξ i nm
p
<x
n σ
= Φ(x ) for any x 2 R .
Remark
There is no requirement for the distribution of ξ i , it can be arbitrary.
Remark
On the left hand side one can see a sequence of c.d.f.-s- the limit is the
standard normal c.d.f.
MOE (PE MIK)
MIMAM143V
20 / 39
The central limit theorem iii)
Approximation of the c.d.f of a sum
lim P
n !∞
∑ni=1 ξ i nm
p
<x
n σ
= Φ (x )
Application
Let ξ i i = 1, 2, 3, . . . be independent identically distributed r.v.-s,
E (ξ i ) = m, D (ξ i ) = σ.
Then Fsum (y ) = P (∑ni=1 ξ i < y ) = P (∑ni=1 ξ i nm < y nm ) =
P
y nm
∑ni=1 ξ i nm
p
< p
n σ
n σ
MOE (PE MIK)
Φ
y nm
p
n σ
MIMAM143V
21 / 39
The central limit theorem iv)
Approximation of the c.d.f. of a sum
Fsum (y )
Φ
y nm
p
n σ
Remark
On the left hand side there is a c.d.f..
On the right hand side there is normal c.d.f..
p
The parameters of the normal distribution are: nm, and n σ.( Just the
expectation and dispersion of the sum.)
Remark
The approximation is good for 100 n in case of any distribution.
If the distribution of ξ i i = 1, 2, 3, . . .is about symmetrical, then it can be
applied in case of 30 n.
MOE (PE MIK)
MIMAM143V
22 / 39
The central limit theorem v)
Moivre-Laplace formula
lim P
n !∞
∑ni=1 ξ i nm
p
<x
n σ
= Φ (x )
Statement
If kA (n) is the frequency of the vent A if we have n independent
experiments, P (A) = p, then
P (a
kA (n) < b )
Φ
p
b
np
np (1
p)
!
Φ
p
a
np
np (1
p)
!
kA (n) is the sum of n independent characteristically distributed random
variables.
The c.d.f of the sum is approximately normal distribution
p function with
parameters m = E (kA (n)) = np, σ = D (kA (n)) = np (1 p ) .
MOE (PE MIK)
MIMAM143V
23 / 39
The central limit theorem vi)
Moivre-Laplace formula
Application
If kA (n) is the frequency of A having n independent experiments,
P (A) = p, then
Φ
P (kA (n) = i ) = P (i kA (n) < i + 1)
!
1
i + 1 np
i np
p
=
Φ p
= Φ 0 (z ) p
np (1 p )
np (1 p )
np (1 p )
z2
1
1
p
e 2 p
2π
np (1 p )
(i +0.5 np )2
1
1
p
e 2np (1 p ) p
2π
np (1
!
Remark
We can approximate the probability P (kA (n) = i ) by p.d.f.
MOE (PE
MIMAM143V
The p.d.f
is aMIK)
normal p.d.f with expectation
np and dispersion
24 / 39
Example
n = 100, p = 0.1
p 1 p exp ( x + 0.5
2π 9
p
10)2 /(2 9)
0.12
0.10
0.08
0.06
0.04
0.02
0.00
0
2
MOE (PE MIK)
4
6
8
10
MIMAM143V
12
14
16
18
20
k
25 / 39
The central limit theorem vii)
Statement
Let η be Poisson distributed r.v. with 10 λ.
Then
x λ
P (η < x ) Φ( p )
λ
Application
Let η be Poisson distributed with 30 λ parameter.
λk λ
Then P (η = k ) =
e
Φ( k +p1 λ ) Φ( kp λ )
λ
λ
k!
MOE (PE MIK)
MIMAM143V
26 / 39
The central limit theorem viii)
Approximation of the c.d.f. of the relative frequency
Statement
If kA (n) is the frequency of the event A having n independent
experiments, P (A) = p, then
0
1
1
B
C
k (n ) p
Bn A
C
lim P B p
< x C = Φ (x )
n !∞ @
A
p (1 p )
p
n
MOE (PE MIK)
MIMAM143V
27 / 39
The central limit theorem ix)
The approximation of the c.d.f
Frelfreq (y )
Application
0
y
Φ @q
1
p A
.
p (1 p )
n
1
1
kA (n) p < ε = P p ε <
kA (n) < p + ε
n
n
Frelgyak (p + ε) Frelgyak (p ε)
1
0
1
0
p ε pA
p + ε pA
Φ@ q
=
Φ@ q
P
0
Φ@ q
p (1 p )
n
1
ε
p (1 p )
n
A
MOE (PE MIK)
0
Φ@
p (1 p )
n
q
ε
p (1 p )
n
1
A=2 Φ
MIMAM143V
q ε
p (1 p )
n
!
=
1
28 / 39
The central limit theorem x)
Approximation of the c.d.f. of the relative frequency
P
1
kA (n)
n
2 Φ
p <ε
q ε
p (1 p )
n
!
1
Statement
P
p (1
p)
p
Φ 2ε n
1
kA (n)
n
1 p
, p (1
4
Φ
MOE (PE MIK)
q ε
p)
p (1 p )
n
!
p
2Φ 2ε n
p <ε
1
,
2
r
p (1
p
1
p ,2ε n
2 n
p)
n
p
, 2 Φ 2ε n
MIMAM143V
1.
1
2 Φ
q ε
q
p (1 p )
n
ε
p (1 p )
! n
,
1.
29 / 39
Exercises
1
Roll 25 times a fair dice. Let ξ be the number of "6".
1
2
2
Compute the probability that ξ is in the neigborhood of its expectation
with radius dispersion!
Compute the probability that ξ is in the neigbouhood of its expectation
with radius double of the dispersion!
Roll 50 times a fair dice. Let ξ be the number of "6".
1
2
3
Compute the probability that ξ is in the neigborhood of its expectation
with radius dispersion.
Compute the probability that ξ is in the neigborhood of its expectation
with radius three times dispersion.
Compute the approximate probability applying Gauss approximation.
MOE (PE MIK)
MIMAM143V
30 / 39
Exercises
3
Post o¢ ce promises that the letters posted by "priorities" will arrive
the next time with probability 0.85.
1
2
3
4
Compute the probability that in case of 12 letters at least 10 will arrive
in time.
Compute the probability that in case of 1200 letters at least 1000 will
arrive in time.
At least how many letters will arrive with probability 0.99 in case of
1200 letters?
At most how many letters will arrive with probability 0.9 in case of
2000 letters?
MOE (PE MIK)
MIMAM143V
31 / 39
Exercises
4
The number of the errors in some fabric is Poisson distributed r.v. In
1 metre, there are 0.3 errors, on average.
1
2
3
4
5
6
Compute the probability that, in 5 metres there are less than 2 errors.
Compute the probability that, the number of errors in 5 metres is
situated in the neighborhood with dispersion of its expectation
Compute the probability that, the number of errors in 5 metres is
situated in the neighborhood with three time dispersion of its
expectation
Compute the probability that, the number of errors in 25 metres is less
than 20.
Compute the probability that, the number of errors in 5 metres is
situated in the neighborhood with dispersion of its expectation
How may meters contain less than 100 errors with probability 0.9.
MOE (PE MIK)
MIMAM143V
32 / 39
Exercises
5
The movement of a particle is a random process. During 1 step 2
units forward with probability 0.3, 1 unit forward with probability 0.2,
1 unit backward with probability 0.5 .The steps are independent.
1
2
3
4
5
6
Give the distribution of the movement belonging to 1 step.
Give the distribution of the movement belonging two steps.
After 200 steps, give the c.d.f and the p.d.f of the movement.
Compute the probability that the particle move on at least 10 units
during 200 steps.
Give the upper bound of the state belonging to 1000 steps and
probability 0.9
Construct an interval symmetrical to 60 in which the particle is
situated after 200 steps with probability 0.98.
MOE (PE MIK)
MIMAM143V
33 / 39
Exercises
6
We gamble. During one gamble we roll two fair dices If the results are
"6" 1we get 100HUF, if one of them is "6", the other not, we get 10
HUF., if neither of them is "6", we do not get anything.
1
2
3
Compute the probability that after 100 games our gain is between 500
and 600 HUF.
If the price of a game is 5 HUF, then compute the probability that we
will be in negative after 100 games.
If the price of a game is 6 HUF, then compute the probability that we
will be in negative after 1000 games.
MOE (PE MIK)
MIMAM143V
34 / 39
Exercises
7
At the desk, the total of the bill is rounded to 0 or 5.During the
payments, the ends of the bill are independent random variables.
Suppose that all ends are equally likely.
1
2
3
4
Give the distribution of the surplus in the desk in one payment
(di¤erence between the number at the end of the bill and the money to
pay after rounding.)
Compute the expectation and the dispersion of the surplus during one
payment.
Compute the probability that during 300 payments the surplus in the
desk is at most 50 HUF.
Give the bounds of the surplus during 700 payments belonging to the
probability 0.9.
MOE (PE MIK)
MIMAM143V
35 / 39
Exercises
8
People who buy ticket for a ‡ight are at the boarding with
probability 0.9 and with probability 0.1 are missing.
1
2
3
Compute the probability that in case of 300 ticket at least 280
passengers are at the boarding.
At most how many people are at the boarding with probability 0.98.
How many tickets should be sold, if one wants to have at most 300
people at the boarding with probability 0.98.
MOE (PE MIK)
MIMAM143V
36 / 39
Exercises
9
At an election, every voter takes part with probability 0.7. Voters may
make faults on the sheet and ask another one with probability 0.1
1
2
3
Compute the distribution of the number of the sheets used by a single
voter.
Compute the distribution of the number of the sheets used by two
voters.
How many sheets are enough with probability 0.999 in case of 10000
voters?
MOE (PE MIK)
MIMAM143V
37 / 39
Exercises
10
The lifetime of a bulb is exponentially distributed r.v. with
expectation 2000 hours.
1
2
3
4
Compute the probability that the lifetime of a single bulb is less than
2500 hours.
Compute the probability that the average lifetime of a 200 bulbs is less
than 2500 hours.
Construct such in interval symmetrical to 2000 hours in which the
lifetime of a single bulb is with probability 0.9.
Construct such in interval symmetrical to 2000 hours in which the
average lifetime of 300 bubbles is with probability 0.9.
MOE (PE MIK)
MIMAM143V
38 / 39
Exercises
11
Processing time of a process is a r.v. with p.d.f.
1 x2 , if 0 x 2
f(x)=
0 otherwise
1
2
3
4
5
6
7
Compute the probability that the processing time is in the neigborhood
of the expectation with radius dispersion.
Compute the probability that the processing time is less than 1.5.
Compute the probability that the processing time is between 0.5 and
0.8?
Compute the probability that the average of 100 processing time is less
than 1.5. (The processing times are supposed to be independent.)
Compute the probability that the average of 100 processing time is
between 0.5 and 0.8 ?
At most how much is average processing time in case of 100 process
with probability 0.95.?
Draw the p.d.f in case of a single process and in case of the average of
100 processes.
MOE (PE MIK)
MIMAM143V
39 / 39