Notes Chapter 20.notebook
May 27, 2016
Targets:
* Explain how to use significance levels in a hypothesis
test.
* Explain what a Type I error is and what a Type II error
are in context.
* Explain what is meant by the Power of a test.
1
Notes Chapter 20.notebook
May 27, 2016
* The P­value of a test is the probability of obtaining a statistic as extreme or more extreme as the one obtained
if there is really no difference/no effect (i.e. null is true). * When we perform a hypothesis test, we interpret a "rare" or "unusual" statistic as being evidence against the null hypothesis.
* The "rareness" of a statistic is quantified by the P­value: the closer to zero the P­value is, the more rare the statistic.
2
Notes Chapter 20.notebook
May 27, 2016
If the P­value is 0.000034, then our statistic is very rare,
thus we must conclude that there is a difference (reject the
null).
If the P­value is 0.19, then our statistic is "common" and
thus could be due to sampling variability. We have no evidence of a difference.
But what if the P­value is 0.036? or 0.062? or 0.018?
3
Notes Chapter 20.notebook
May 27, 2016
Significance Level: The significance level or alpha level of a test, denoted is the predetermined threshold for how low the P­value must
be in order to conclude that the statistic is significant or "rare."
Common significance (or alpha) levels are:
4
Notes Chapter 20.notebook
May 27, 2016
p. 537, #14:
Environmentalists concerned about the impact of high­
frequency radio transmissions on birds found that there was no evidence of a higher mortality rate among hatchlings in nests near cell towers. They base this conclusion on a test using a 5% significance level.
Would they have made the same decision using Maybe, if p­value less than 10%
but could be greater than 10%
How about Yes, fail to reject. If p­value greater
than 5% then definitely greater than 1%
5
Notes Chapter 20.notebook
May 27, 2016
Type I and Type II error
Whether we decide to reject the null hypothesis or we fail to
reject the null hypothesis, we never know if our decision is the
correct decision since it's based on the probability of obtaining
the particular sample that we used.
Our decision could be wrong.
6
Notes Chapter 20.notebook
May 27, 2016
TRUTH
true
My
decision
false
Reject Type I error
OK
Fail to reject OK
Type II error
7
Notes Chapter 20.notebook
May 27, 2016
p. 536, #8
For each of the following situations, state whether a Type I error, a Type II error, or neither error has been made.
a) A test of vs. fails to reject the null hypothesis. Later it is discovered that p = 0.9.
Correct decision since p is not less than 0.8.
b) A test of vs. rejects the null hypothesis. Later it is discovered that p = 0.65.
Correct decision since p does not equal 0.5.
c) A test of vs. fails to reject the null hypothesis. Later it is discovered that p = 0.6.
Type II error. p is less than 0.7 so we should have rejected the null but didn't.
8
Notes Chapter 20.notebook
May 27, 2016
p. 539, #30
Production managers on an assembly line must monitor the output to be sure that the level of defective products remains small. They periodically inspect a random sample of the items produced. If the find a significant increase in the proportion of items that must be rejected, they will halt the assembly process until the problem can be identified and repaired.
a) In this context, what is a Type I error?
We conclude there's an increase in defects, but there's not an increase
b) In this context, what is a Type II error?
We conclude there's not an increase in defects, but there actually is an increase.
c) Which type of error would the factory owner consider more
serious? Why?
d) Which type of error would the customers consider more serious? Why?
9
Notes Chapter 20.notebook
May 27, 2016
The Power of a test is the probability that it correctly rejects a false null hypothesis.
A social scientist conducts a study to see if the rate of domestic violence is higher among high school drop outs. Explain what "power" means in this context.
Note: The Power of a test is the complement of the probability of a Type II error, so it's .
10
Notes Chapter 20.notebook
May 27, 2016
How do you determine the alpha level?
The main consideration in choosing your significance level is this: Which type of error, I or II, is worse in this situation?
The significance level is the same as the probability of a Type I error.
A lower significance level (0.01 as compared to 0.05), reduces the likelihood of a Type I error, but increases the likelihood of a Type II error (because you now need more evidence to reject the null).
11
Notes Chapter 20.notebook
May 27, 2016
How can we increase the power of a test?
* Use a larger sample size.
* Use a higher significance level (making it "easier" to reject
the null.
12
Notes Chapter 20.notebook
May 27, 2016
p. 539, #36
A company is willing to renew its advertising contract with a local radio station if the station can prove that more than 20% of the residents in the city have heard the ad and recognize the company's product. The radio station proposes to conduct a phone survey of 400 randomly selected residents.
a) What are the hypotheses?
b) The station plans to conduct this test using a 10% level of significance, but the company wants the significance level lowered to 0.05. Why?
c) What is meant by the power of this test?
d) For which level of significance will the power be higher?
e) They finally agree to use alpha = 0.05, but the company
proposes that the station contact 600 residents as opposed to
400. Will that make the risk of Type II error higher or lower?
Will the power be higher or lower?
13
Notes Chapter 20.notebook
May 27, 2016
14