23
2.2. Multivariate Probability Distributions
(I) Bivariate Probability Distribution
Joint Probability Distribution (Density) Function:

Let X 1 and X 2 be discrete random variables. The joint probability
distribution function is
f x1 , x2   P X 1  x1 , X 2  x2  ,
where x1 and x 2 are possible values of X 1 and X 2 , respectively.
f
satisfies the following conditions:
1. f x1 , x2   0 for all x1 , x2 ;
2.
 f x , x   1 .
1
x2

2
x1
Let X 1 and X 2 be continuous random variables. The joint probability
density function satisfies the following conditions:
1. f x1 , x2   0 for    x1  ,  x2   ;
 
2.
  f x , x dx d x
1
2
1
2
 1.
  
Cumulative Distribution Function:

For any random variables X 1 and X 2 , the joint (cumulative) distribution
function is given by F x1 , x2   P X 1  x1 , X 2  x2  .

As X 1 and X 2 are discrete,
Pa  X 1  b, c  X 2  d  
  f x , x  .
c x2 d a x1 b
1
2
As X 1 and X 2 are continuous,
d b
Pa  X 1  b, c  X 2  d     f x1 , x2 dx1dx2 .
c a

For any random variables X 1 and X 2 with joint cumulative distribution
function F x1 , x2  ,
1. F  ,  F  , x2   F x1 ,  0;
2. F ,   1 ;
23
24
3. if b  a, d  c , then
Pa  X1  b, c  X 2  d   F b, d   F b, c  F a, d   F a, c .
Example 1:
Form a collection of 3 white balls, 2 black balls, and 1 red ball, 2 balls is to be
randomly selected. Let X 1 denote the number of white balls and X 2 the
number of black balls.
(a) Find the joint probability distribution table of X 1 and X 2 .
(b) F 1,1 and F 2,0 .
[solution:]
(a)
 3  2  1
  
i j 2i 
P X 1  i, X 2  j   f i, j     
6
 
 2


j 
,0  i, j  2; i  j  1 or 2;
For example,
 3  2  1 
   
1 1 0
3 2 6
P X 1  1, X 2  1  f 1,1      
 .
15 15
6
 
 2
The joint probability distribution table is
x1
x2
0
1
2
0
0
3
15
3
15
1
2
15
2
1
15
Total
3
15
6
6
15
8
15
0
1
15
3
15
1
(b) F 1,1  P X 1  1, X 2  1  f 0,0  f 0,1  f 1,0  f 1,1 
F 2,0  P X 1  2, X 2  0  f 0,0  f 1,0  f 2,0 
24
15
0
15
0
9
Total
0  2  3  6 11

.
15
15
033 6

15
15
25
Example 2:
Let f x1 , x2   2x1 ,0  x1  k;0  x2  1; and f x1 , x2   0, otherwise.
(a) Find k .
(b) Find F 0.7,0.5 , F 2,0 and F 0.2,3 .
[solution:]
(a)
 


1 k
1
0 0
0
 
f x1 , x2 dx1dx2    2 x1dx1dx2   x12 0 dx2   k 2 dx2  k 2  1  k  1 .
1
k
0
 k  1.
(b)
F 0.7,0.5  P X 1  0.7, X 2  0.5 
0.5 0.7
  f x , x dx dx
1
2
1
0.50.7
2

 
 x 
0.5

2 0.7
1 0
  2 x dx dx
1
1
0 0
0.5
dx2 
0
 0.49dx
2
 0.49  0.5  0.245
0
F 2,0   P  X 1  2, X 2  0  
0 2
  f x , x dx dx
1
2
1
  
0
 
0 1
2
   2 x1dx1dx2
0 0
0
  x12 0 dx2   1dx2  1  0  0
1
0
0
F 0.2,3  P  X 1  0.2, X 2  3 
3 0.2

f  x1 , x2 dx1dx2 

 x 
1

2 0.2
1 0
1 0.2
  2 x dx dx
1
0
1
0
1
dx2 
0
 0.04dx
2
 0.04
0
Marginal Probability Distribution (Density) Function:

Let X 1 and X 2 be discrete random variables. The marginal probability
distribution function for X 1 and X 2 are
f1 x1   P X 1  x1    f x1 , x2  ,
x2
and
f 2 x2   P X 2  x2    f x1 , x2  ,
x1
respectively.

Let X 1 and X 2 be continuous random variables. The marginal
25
2
2
26
probability density function for X 1 and X 2 are
f 1  x1  

 f x , x dx
1
2
2
,

and
f 2 x2  

 f x , x dx ,
1
2
1

respectively.
Conditional Probability Distribution (Density) Function:

Let
X 1 and
X2
be discrete random variables. The conditional
probability distribution function X 1 given X 2 is
f1 x1 | x2   P X 1  x1 | X 2  x2  
P X 1  x1 , X 2  x2  f x1 , x2 
,

P X 2  x 2 
f 2 x2 
provided that f 2 x2   0 . Similarly, the conditional probability distribution
function X 2 given X 1 is
f 2 x2 | x1   P X 2  x2 | X 1  x1  
provided that f1 x1   0 .

P X 1  x1 , X 2  x2  f x1 , x2 
,

P X 1  x1 
f1 x1 
Let X 1 and X 2 be continuous random variables. The conditional
probability density function X 1 given X 2 is
 f x1 , x2 
, f 2 x2   0

f1 x1 | x2    f 2 x2 

otherwise.
0,
Similarly, the conditional probability density function X 2 given X 1 is
 f x1 , x2 
, f1 x1   0

f 2 x2 | x1    f1 x1 

otherwise.
0,
Example 1 (continue):
(c) Find the marginal probability distribution function of X 1 .
(d) Given one of chosen balls being black, what is the distribution for the number
of balls being white?
[solution:]
26
27
0  2 1 3

,
15
15
(c) f1 (0)  f 0,0  f 0,1  f 0,2 
f1 (1)  f 1,0  f 1,1  f 1,2 
3 60 9
 ,
15
15
f1 (2)  f 2,0  f 2,1  f 2,2  
(d) f 2 (1)  f 0,1  f 1,1  f 2,1 
300 3

15
15
260 8

15
15
2
f 0,1
1
f1 0 | 1  P X 1  0 | X 2  1 
 15  ,
f 2 1 8
4
15
6
f 1,1
3




f1 1 | 1  P X 1  1 | X 2  1 
 15  ,
f 2 1 8
4
15
f1 2 | 1  P X 1  2 | X 2  1 
f 2,1
0

0
f 2 1 8
15
Example 2 (continue):
(c) Find the marginal probability density function for X 1 and X 2 .
(d) Find the conditional probability density function for X 1 given X 2 and the
conditional probability density function for X 2 given X 1
[solution:]
(c)

f1 x1  


f 2 x2  
1
f x1 , x2 dx2   2 x1dx2  2 x1 ,0  x1  1; .
0

 f x , x dx   2 x dx  x 
1
1

2
1
1
1
2 1
1 0
0
(d) For 0  x1  1,0  x2  1,
f x1 , x2 
 2 x1
f 2 x2 
f1 x1 | x2  
and
f 2 x2 | x1  
f x1 , x2 
 1.
f1 x1 
27
 1,0  x2  1;
28
Independence of Two Random Variables:
Let X 1 and X 2 be random variables. If X 1 and X 2 are independent if and
only if
for all pairs of
f x1 , x2   f1 x1  f 2 x2  ,
x1 , x2  .
Note: As f 2 x2   0 and X 1 and X 2 are independent, then
Further, as X 1 and X 2

f1 x1 | x2   f1 x1  .
are independent,

P X 1  x1 , X 2  x 2  f x1 , x2   f1 x1  f 2 x2   P X 1  x1 P X 2  x2 
 P A  B   P APB , A  X 1  x1 , B  X 2  x2 .
Example 1 (continue):
(e) Are X 1 and X 2 independent?
[solutions:]
No since f 1,0 
3
5
6
9 6


   f1 1 f 2 0 .
15 25 25 15 15
Example 2 (continue):
(e) Are X 1 and X 2 independent?
[solutions:]
Yes since f x1 , x2   2 x1  f1 x1  f 2 x2 .
Example 3:
Let f x1 , x2   6 x1 x22 ,0  x1  k ;0  x2  1; and f x1 , x2   0, otherwise. Are X 1
and X 2 independent?
[solution:]
f1 x1  

 f x , x dx
1

f 2 x2  



2
1
2


 2 x1 ,0  x1  1;


 3x22 ,0  x1  1.
  6 x1 x22 dx2  2 x1 x23
1
0
0
1
f x1 , x2 dx1   6 x1 x22 dx1  3x12 x22
Thus,
Therefore, X 1 and X 2
1
0
0
f x1 , x2   6 x1 x22  f1 x1  f 2 x2  .
are independent.
28
29
(II) Multivariate Probability Distribution
Joint Probability Distribution (Density) Function:

Let X 1 ,, X n be discrete random variables. The joint probability
distribution function is
f x1 ,, xn   P X 1  x1 ,, X n  xn  ,
where x1 ,, x n are possible values of X 1 ,, X n , respectively.
f
satisfies the following conditions:
1. f x1 ,, xn   0 for all x1 ,, x n ;
2.
 f x ,, x   1 .
1
x2

n
xn
Let X 1 ,, X n be continuous random variables. The joint probability
density function satisfies the following conditions:
1. f x1 ,, xn   0 for    x1 ,, xn  ; ;
2.




   f x ,, x dx  d x
1
n
1
n
 1.
Independence of Multiple Random Variables:
Let X 1 ,, X n be random variables. If X 1 ,, X n are independent if and only
if
for all pairs of
f x1 ,, xn   f1 x1  f n xn  ,
x1 ,, xn  , where
f j is the marginal probability distribution (or
density) function of X j .
Likelihood Function:
Let x1 ,, x n be sample observations taken on corresponding random variables
X 1 ,, X n .whose joint probability distribution or density function depends on
the parameters 1 ,, k . The likelihood of sample is
l 1 ,, k   f x1 ,, xn | 1 ,, k  .
As X 1 ,, X n are a random sample from a common probability distribution (or
density) with the parameters 1 ,, k , i.e., the probability distribution or
density function equal to f x | 1 ,, k  and X 1 ,, X n being i.i.d. (identically
independent distributed), the likelihood is
29
30
n
l 1 ,  ,  k    f xi | 1 ,  ,  k   f x1 | 1 ,  ,  k  f xn | 1 ,  ,  k  .
i 1
Example 4:


Let X 1 ,, X n are a random sample from a normal distribution N  ,  2 .
Then,
l  , 
 
n
2
i 1
 xi   2
e
2
2 2
2
 2
2

n

n
  xi  2
i 1
2
e
2 2
Example 5:
Let X 1 ,, X n are a random sample from a binomial distribution Bm, p .
Then,
n
n
xi
  m  
 m  xi
m xi
nm

xi
l  p      p 1  p 
    p i 1 1  p  
i 1
i 1  xi 
 i 1  xi 
n
n
Example 6:
Let X 1 ,, X n are a random sample from a Poisson distribution Poisson  .
Then,
n
 xi
e  xi
e n  i 1
l    

n
xi !
i 1
 xi !
n
i 1
30