Sample Correlation
Mathematics 47: Lecture 5
Dan Sloughter
Furman University
March 10, 2006
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
1/8
Definition
If X and Y are random variables with means µX and µY and variances σX2
and σY2 , respectively, then we call
cov(X , Y ) = E [(X − µX )(Y − µY )]
the covariance of X and Y .
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
2/8
Theorem (Cauchy-Schwarz Inequality)
If X and Y are random variables for which E [X 2 ] and E [Y 2 ] both exist,
then
(E [XY ])2 ≤ E [X 2 ]E [Y 2 ].
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
3/8
Theorem (Cauchy-Schwarz Inequality)
If X and Y are random variables for which E [X 2 ] and E [Y 2 ] both exist,
then
(E [XY ])2 ≤ E [X 2 ]E [Y 2 ].
Proof.
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
3/8
Theorem (Cauchy-Schwarz Inequality)
If X and Y are random variables for which E [X 2 ] and E [Y 2 ] both exist,
then
(E [XY ])2 ≤ E [X 2 ]E [Y 2 ].
Proof.
I
Let f (t) = E [(X + tY )2 ] = E [X 2 ] + 2tE [XY ] + t 2 E [Y 2 ].
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
3/8
Theorem (Cauchy-Schwarz Inequality)
If X and Y are random variables for which E [X 2 ] and E [Y 2 ] both exist,
then
(E [XY ])2 ≤ E [X 2 ]E [Y 2 ].
Proof.
I
Let f (t) = E [(X + tY )2 ] = E [X 2 ] + 2tE [XY ] + t 2 E [Y 2 ].
I
Then f is a quadratic polynomial in t with f (t) ≥ 0 for all t.
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
3/8
Theorem (Cauchy-Schwarz Inequality)
If X and Y are random variables for which E [X 2 ] and E [Y 2 ] both exist,
then
(E [XY ])2 ≤ E [X 2 ]E [Y 2 ].
Proof.
I
Let f (t) = E [(X + tY )2 ] = E [X 2 ] + 2tE [XY ] + t 2 E [Y 2 ].
I
Then f is a quadratic polynomial in t with f (t) ≥ 0 for all t.
I
Hence, by the quadratic formula, 4(E [XY ])2 − 4E [X 2 ]E [Y 2 ] ≤ 0.
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
3/8
Theorem (Cauchy-Schwarz Inequality)
If X and Y are random variables for which E [X 2 ] and E [Y 2 ] both exist,
then
(E [XY ])2 ≤ E [X 2 ]E [Y 2 ].
Proof.
I
Let f (t) = E [(X + tY )2 ] = E [X 2 ] + 2tE [XY ] + t 2 E [Y 2 ].
I
Then f is a quadratic polynomial in t with f (t) ≥ 0 for all t.
I
Hence, by the quadratic formula, 4(E [XY ])2 − 4E [X 2 ]E [Y 2 ] ≤ 0.
I
Hence (E [XY ])2 ≤ E [X 2 ]E [Y 2 ].
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
3/8
Correlation coefficient
I
Applying the Cauchy-Schwarz inequality to the definition of
covariance, we have
q
q
2
|cov(X , Y )| ≤ E [(X − µX ) ] E [(Y − µY )2 ] = σX σY .
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
4/8
Correlation coefficient
I
Applying the Cauchy-Schwarz inequality to the definition of
covariance, we have
q
q
2
|cov(X , Y )| ≤ E [(X − µX ) ] E [(Y − µY )2 ] = σX σY .
I
If we let,
ρX ,Y =
cov(X , Y )
σX σY
then −1 ≤ ρX ,Y ≤ 1.
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
4/8
Correlation coefficient
I
Applying the Cauchy-Schwarz inequality to the definition of
covariance, we have
q
q
2
|cov(X , Y )| ≤ E [(X − µX ) ] E [(Y − µY )2 ] = σX σY .
I
If we let,
ρX ,Y =
cov(X , Y )
σX σY
then −1 ≤ ρX ,Y ≤ 1.
I
Moreover, |ρX ,Y | = 1 if and only if Y = aX + b for some real
numbers a and b.
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
4/8
Correlation coefficient
I
Applying the Cauchy-Schwarz inequality to the definition of
covariance, we have
q
q
2
|cov(X , Y )| ≤ E [(X − µX ) ] E [(Y − µY )2 ] = σX σY .
I
If we let,
ρX ,Y =
cov(X , Y )
σX σY
then −1 ≤ ρX ,Y ≤ 1.
I
Moreover, |ρX ,Y | = 1 if and only if Y = aX + b for some real
numbers a and b.
Definition
We call ρX ,Y the correlation coefficient of X and Y .
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
4/8
Correlation and independence
I
Note: if X and Y are independent, then cov(X , Y ) = 0 (and hence
ρX ,Y = 0).
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
5/8
Correlation and independence
I
Note: if X and Y are independent, then cov(X , Y ) = 0 (and hence
ρX ,Y = 0).
I
If cov(X , Y ) = 0, we say X and Y are uncorrelated.
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
5/8
Correlation and independence
I
Note: if X and Y are independent, then cov(X , Y ) = 0 (and hence
ρX ,Y = 0).
I
If cov(X , Y ) = 0, we say X and Y are uncorrelated.
I
However, uncorrelated does not necessarily imply independence,
although it does if (X , Y ) has a bivariate normal distribution.
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
5/8
Sample correlation
I
Now suppose (X1 , Y1 ), (X2 , Y2 ), . . . , (Xn , Yn ) are independent
identically distributed pairs of random variables (that is, a random
sample from a bivariate distribution).
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
6/8
Sample correlation
I
I
Now suppose (X1 , Y1 ), (X2 , Y2 ), . . . , (Xn , Yn ) are independent
identically distributed pairs of random variables (that is, a random
sample from a bivariate distribution).
Let
1 Pn
i=1 (Xi − X̄ )(Yi − Ȳ )
n
q P
R=q P
n
n
1
1
2
2
(X
−
X̄
)
i
i=1
i=1 (Yi − Ȳ )
n
n
Pn
i − nX̄ Ȳ
i=1 Xi Yq
= qP
Pn
n
2
2
2
2
i=1 Xi − nX̄
i=1 Yi − nȲ
P
P
n
n
Pn
i=1 Yi
i=1 Xi
i=1 Xi Yi −
n
r
.
=r
Pn
Pn
2
2
Pn
P
X
Y
( i=1 i )
( i=1 i )
n 2
2
i=1 Xi −
i Yi −
n
n
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
6/8
Sample correlation
I
I
Now suppose (X1 , Y1 ), (X2 , Y2 ), . . . , (Xn , Yn ) are independent
identically distributed pairs of random variables (that is, a random
sample from a bivariate distribution).
Let
1 Pn
i=1 (Xi − X̄ )(Yi − Ȳ )
n
q P
R=q P
n
n
1
1
2
2
(X
−
X̄
)
i
i=1
i=1 (Yi − Ȳ )
n
n
Pn
i − nX̄ Ȳ
i=1 Xi Yq
= qP
Pn
n
2
2
2
2
i=1 Xi − nX̄
i=1 Yi − nȲ
P
P
n
n
Pn
i=1 Yi
i=1 Xi
i=1 Xi Yi −
n
r
.
=r
Pn
Pn
2
2
Pn
P
X
Y
( i=1 i )
( i=1 i )
n 2
2
i=1 Xi −
i Yi −
n
n
Definition
We call R the sample correlation coefficient.
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
6/8
Example
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
7/8
Example
I
An experiment to measure the yield of wheat for seven different levels
of nitrogen gave the following observations:
Nitrogen/acre (x)
40
60
80
100 120 140 160
Yield (cwt/acre) (y) 15.9 18.8 21.6 25.2 28.7 30.4 30.7
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
7/8
Example
I
I
An experiment to measure the yield of wheat for seven different levels
of nitrogen gave the following observations:
Nitrogen/acre (x)
40
60
80
100 120 140 160
Yield (cwt/acre) (y) 15.9 18.8 21.6 25.2 28.7 30.4 30.7
If we let xi and yi , i = 1, 2, . . . , 7, represent the nitrogen levels and
wheat yields, respectively, then
7
X
xi = 700,
i=1
7
X
yi = 171.3,
i=1
7
X
i=1
Dan Sloughter (Furman University)
xi2 = 81, 200, and
7
X
xi yi = 18, 624,
i=1
7
X
yi2 = 4398.19.
i=1
Sample Correlation
March 10, 2006
7/8
Example
I
I
An experiment to measure the yield of wheat for seven different levels
of nitrogen gave the following observations:
Nitrogen/acre (x)
40
60
80
100 120 140 160
Yield (cwt/acre) (y) 15.9 18.8 21.6 25.2 28.7 30.4 30.7
If we let xi and yi , i = 1, 2, . . . , 7, represent the nitrogen levels and
wheat yields, respectively, then
7
X
xi = 700,
i=1
i=1
So
7
X
yi = 171.3,
i=1
7
X
I
7
X
xi2 = 81, 200, and
7
X
yi2 = 4398.19.
i=1
18, 624 − (700)(171.3)
q 7
r=q
(700)2
81, 200 − 7
4398.19 −
Dan Sloughter (Furman University)
xi yi = 18, 624,
i=1
Sample Correlation
(171.3)2
7
≈ 0.9830.
March 10, 2006
7/8
Example (cont’d)
I
If the nitrogen levels are in a vector x and the wheat yields are in a
vector y, then the R command > cor(x, y) returns r , in this case
0.9830173.
Dan Sloughter (Furman University)
Sample Correlation
March 10, 2006
8/8