Lesson 9.1 - Statistical Tests: The Basics

What is the difference between a confidence interval and a significance test?
I’m a Great Free-Throw Shooter!

A basketball player claims to make 80% of the free throws that he attempts. We think he might be exaggerating. To
test this claim, we’ll ask him to shoot some free throws—virtually—using The Reasoning of a Statistical Test applet.
1. Launch the applet.
2. Set the applet to take 25 shots. Click “Shoot.” Record how many of the 25 shots the player makes: ___________
3. Click “Shoot” again for 25 more shots. Repeat until you are convinced either that the player makes less than 80%
of his shots or that the player’s claim is true.
4. Click “Show true probability.” Were you correct?
What are the only two explanations for why we get a different statistic than what is claimed about the true parameter?
Stating Hypotheses

A significance test starts with a careful statement of the claims we want to compare.

What are the hypotheses that we will compare?

In the free-throw shooter example, our hypotheses are
where _____ is the true (think “long-run”) proportion of made free throws.

In any significance test, the null hypothesis has the form

The alternative hypothesis has one of the forms
Ha : parameter ____ value
Ha : parameter ____ value
Ha : parameter ____ value

How do you determine the correct form of the alternative hypothesis?
Stating Hypotheses Cautions!

The hypotheses should express the hopes or suspicions we have ________________ we see the data. It is cheating
to look at the data first and then frame hypotheses to fit what the data show.

Hypotheses always refer to a ________________________, not to a sample. Be sure to state H0 and Ha in terms of
population ________________________.

It is _______________ correct to write a hypothesis about a sample statistic, such as 𝑝̂ = 0.64 or 𝑥̅ = 85.
Example: A better golf club?
Mike is an avid golfer who would like to improve his play. A friend suggests getting new clubs and lets Mike try out his
7-iron. Based on years of experience, Mike has established that the mean distance that balls travel when hit with his old
7-iron is 𝜇 = 175 yards with a standard deviation of 𝜎 = 15 yards. He is hoping that this new club will make his shots
with a 7-iron more consistent (less variable), so he goes to the driving range and hits 50 shots with the new 7-iron.
State appropriate hypotheses for performing a significance test. Be sure to define the parameter of interest.
The Reasoning of Significance Tests

Suppose a basketball player claimed to be an 80% free-throw shooter. To test this claim, we have him attempt 50
free-throws. He makes 32 of them. His sample proportion of made shots is 𝑝̂ = 32/50 = _________.

What can we conclude about the claim based on this sample data?

We can use software to simulate 400 sets of 50 shots assuming that the player is really an 80% shooter.
 You can say how strong the evidence
AGAINST the player’s claim is by giving
the probability that he would make as few as
32 out of 50 free throws if he really makes 80%
of his shots in the long run.
 Based on the simulation, our estimate of this
probability is ____________________________.

The observed statistic is so _____________________ if the actual parameter value is __________________ that it
gives convincing evidence that the players claim is _______ _____________.

There are two possible explanations for the fact that he made only 64% of his free throws:
1) The null hypothesis is _____________________. The player’s claim is correct (𝑝 = 0.8), and just by chance, a very
unlikely outcome occurred.
2) The alternative hypothesis is _________________. The population proportion is actually less than 0.8, so the
sample result is not an unlikely outcome.

Basic Idea: An outcome that would rarely happen if the null hypothesis were true is good evidence that the null
hypothesis is not true.
Interpreting 𝑷-Values

The null hypothesis _______ states the claim that we are seeking evidence __________________. The probability
that measures the strength of the evidence against a null hypothesis is called a _________________.
 The probability, computed assuming ______ is true, that the statistic would take a value ____ __________________
or _____________ ____________________than the one actually observed is called the 𝑷-value of the test.
 ________________ ________________ are evidence against H0 because they say that the observed result is unlikely
to occur when ______ is _____________.
 ________________ ________________ fail to give convincing evidence against H0 because they say that the
observed result is likely to occur by chance when _______ is _____________.
Example: A better golf club?
When Mike was testing a new 7-iron, the hypotheses were
𝐻0 : 𝜎 = 15
𝐻𝑎 : 𝜎 < 15
where 𝜎 = the true standard deviation of the distances Mike hits golf balls using the new 7-iron. Based on 50 shots with
the new 7-iron, the standard deviation was s x = 13.9 yards. After checking that the conditions were met, Mike
performed a significance test and obtained a 𝑃-value of 0.25.
(a) Explain what it would mean for the null hypothesis to be true in this setting.
(b) Interpret the 𝑃-value in this context.
Statistical Significance

The final step in performing a significance test is to draw a _______________________ about the competing claims
you were testing. We make one of two decistions based on the strength of the evidence (P-value)
________________ the null hypothesis (and in favor of the alternative hypothesis):
Note: A fail-to-reject H0 decision in a significance test doesn’t mean that H0 is true. For that reason, you should
_____________ “accept H0” or use language implying that you believe H0 is true.
In a nutshell, our conclusion in a significance test comes down to
𝑷-value small → reject H0 → convincing evidence for Ha
𝑷-value large → fail to reject H0 → not convincing evidence for Ha

There is no rule for how small a 𝑃-value we should require in order to reject H0. But we can compare the 𝑃-value
with a fixed value that we regard as decisive, called the significance level. We write it as ___, the Greek letter alpha.
 If the 𝑃-value is smaller than alpha, we say that the data are statistically significant at level α. In that case, we reject
the null hypothesis H0 and conclude that there is convincing evidence in favor of the alternative hypothesis Ha.
 When we use a fixed level of significance to draw a conclusion in a significance test,
P-value ____ α → ___________ H0 → convincing evidence ______ Ha
P-value ____ α → ________ _____ ___________ H0 → _______ convincing evidence _______ Ha
Example: A better golf club?
In the 7-iron example, the estimated 𝑃-value was 0.25. Because the 𝑃-value is large, we __________________________
the null hypothesis that 𝜎 = 15. We _______________ have convincing evidence that Mike is more consistent when
using the new 7-iron.
Example: Tasty chips
For his second semester project in AP Statistics, James decided to investigate whether students at his school prefer
name-brand potato chips to generic potato chips. He randomly selected 50 students and had each student try both
types of chips, in random order. Overall, 32 of the 50 (𝑝̂ = 0.64) students preferred the name-brand chips. James
performed a significance test using the hypotheses
𝐻0 : 𝑝 = 0.5
𝐻𝑎 : 𝑝 > 0.5
where 𝑝 = the true proportion of students at his school who prefer name-brand chips. The resulting 𝑃-value was
0.0239. What conclusion would you make at each of the following significance levels?
(a) 𝛼 = 0.05
(b) 𝛼 = 0.01
Type I and Type II Errors

When we draw a conclusion from a significance test, we hope our conclusion will be correct. But sometimes it will
be wrong. There are two types of mistakes we can make.
 If we reject H0 when H0 is true, we have committed a ________________________________.
 If we fail to reject H0 when Ha is true, we have committed a ________________________________.
Truth about the population
Conclusion
based on
sample
Example: Faster fast food?
The manager of a fast-food restaurant wants to reduce the proportion of drive-through customers who have to wait
longer than two minutes to receive their food after their order is placed. Based on store records, the proportion of
customers who had to wait longer than two minutes was p = 0.63. To reduce this proportion, the manager assigns an
additional employee to assist with drive-through orders. During the next month, the manager will collect a random
sample of drive-through times and test the following hypotheses:
𝐻0 : 𝑝 = 0.63
𝐻𝑎 : 𝑝 < 0.63
where p = the true proportion of drive-through customers who have to wait longer than two minutes to receive their
food. Describe a Type I and a Type II error in this setting and explain the consequences of each.

The probability of a Type I error is the probability of rejecting H0 when it is really true…This is exactly the significance
level of the test.
 Significance and Type I Error
The significance level 𝛼 of any fixed-level test is the probability of a __________________________.
That is, 𝛼 is the probability that the test will reject the null hypothesis H0 when H0 is actually true.
Consider the consequences of a Type I error before choosing a significance level.
Example: Faster fast food?
In the fast-food Example, we were testing the following hypotheses:
𝐻0 : 𝑝 = 0.63
𝐻𝑎 : 𝑝 < 0.63
where p = the true proportion of drive-through customers who have to wait longer than two minutes after their order is
placed to receive their food. Suppose that the manager decided to carry out this test using a random sample of 250
orders and a significance level of  = 0.10. What is the probability of making a Type I error?