Birthdays
R Pruim
2014-09-08
Looking at Birthdays Again
We have already looked at this plot a couple times
xyplot( births ~ date, data=Births78)
births
10000
9000
8000
7000
Jan
Apr
Jul
Oct
Jan
date
It was suggested that a possible explanation for the two parallel waves is a weekend/weekday affect. We can
check wether this is plausible by adding some features to our plot.
First let’s color by day of week. The wday function in the lubridate pacakge will convert dates to day of the
week for us.
require(lubridate)
xyplot( births ~ date, data=Births78,
groups=wday(date))
births
10000
9000
8000
7000
Jan
Apr
Jul
Oct
date
1
Jan
This looks like some good evidence to support our conjecture. There are primarily two colors in the lower
wave and 5 in the upper wave – but there are some exceptions. These are easier to spot if we “connect the
dots”.
xyplot( births ~ date, data=Births78,
groups=wday(date), type='l' )
births
10000
9000
8000
7000
Jan
Apr
Jul
Oct
Jan
date
Now it is easier to see that there are a few days that are “in the wrong group”. (These are the holidays.)
Finally, let’s add a key so we know which day is which.
xyplot( births ~ date, data=Births78,
groups=wday(date), type='l',
auto.key=TRUE)
births
1
2
3
4
5
6
7
10000
9000
8000
7000
Jan
Apr
Jul
Oct
Jan
date
Here’s a much nicer version (fancier than you need to know how to do just now).
xyplot( births ~ date, data=Births78,
groups=wday(date, label=TRUE, abbr=TRUE), type='l',
auto.key=list(columns=4, points=FALSE, lines=TRUE)
)
2
Sun
Mon
Tues
Wed
Thurs
Fri
Sat
births
10000
9000
8000
7000
Jan
Apr
Jul
Oct
Jan
date
We can also compute numbers and put them right into our text. For example, the mean number of births is
9132.1616. That’s more digits than we need. We could choose to round this to 9132.
3
Describing Distributions
R Pruim
2014-09-09
Pros and cons of Experimental vs Observational studies
• Experiments may be artificial
• Observation studies need to be wary about confounding
• Some experiments are not ethical
Distributions
When we talk about the distribution of a variable, we are talking about the answers to the following questions:
• what values?
• how often?
We can summarize distributions in several ways:
•
•
•
•
tables
graphs
numerical summaries
adjectives that describe the “shape”
HELPrct
The Health Evaluation and Linage to Primary Care data set has information collected on individuals admitted
for treatment of an adiction to alcohol, cocaine, or heroin.
tally( ~ substance, data=HELPrct)
substance
##
## alcohol cocaine
##
177
152
heroin
124
tally( ~ substance, data=HELPrct, format="prop")
##
## alcohol cocaine
## 0.3907 0.3355
heroin
0.2737
1
bargraph( ~ substance, data=HELPrct)
Frequency
150
100
50
0
alcohol
cocaine
heroin
tally( ~ sex, data=HELPrct, format="percent")
sex
##
## female
## 23.62
male
76.38
favstats( ~age, data=HELPrct )
age
##
##
min Q1 median Q3 max mean
sd
n missing
19 30
35 40 60 35.65 7.71 453
0
i1 i1 is the average number of drinks (standard units) consumed per day over the 30 days before admission
to the program.
favstats( ~ i1, data=HELPrct )
##
##
min Q1 median Q3 max mean
sd
n missing
0 3
13 26 142 17.91 20.02 453
0
bwplot( ~i1, data=HELPrct)
2
0
50
100
i1
histogram( ~ i1, data=HELPrct )
0.025
Density
0.020
0.015
0.010
0.005
0.000
0
50
100
i1
bwplot( substance ~ i1, data=HELPrct )
heroin
cocaine
alcohol
0
50
100
i1
3
150
histogram( ~ i1 | substance, data=HELPrct, width=20 )
0
alcohol
50
100 150
cocaine
heroin
Density
0.03
0.02
0.01
0.00
0
50
100 150
0
50
100 150
i1
favstats( ~ i1 | substance, data=HELPrct )
##
.group min Q1 median Q3 max
mean
sd
n missing
## 1 alcohol
0 13
25 38 142 29.192 22.43 177
0
## 2 cocaine
0 2
6 13 102 12.132 15.19 152
0
## 3 heroin
0 0
4 13 64 8.879 12.76 124
0
favstats(i1 ~ substance, data=HELPrct )
##
.group min Q1 median Q3 max
mean
sd
n missing
## 1 alcohol
0 13
25 38 142 29.192 22.43 177
0
## 2 cocaine
0 2
6 13 102 12.132 15.19 152
0
## 3 heroin
0 0
4 13 64 8.879 12.76 124
0
densityplot( ~ i1, groups=substance, data=HELPrct, width=20, auto.key=TRUE, lwd=3 )
alcohol
cocaine
heroin
0.05
Density
0.04
0.03
0.02
0.01
0.00
0
50
100
i1
4
150
Class Notes 2014-09-11
R Pruim
2014-09-11
Old Faithful
How long is a typical eruption? How much does the length of an eruption vary?
require(MASS)
# geyser data lives here
mean(~duration, data=geyser)
## [1] 3.461
median(~duration, data=geyser)
## [1] 4
favstats( ~ duration, data=geyser )
##
##
min Q1 median
Q3 max mean
sd
n missing
0.8333 2
4 4.383 5.45 3.461 1.148 299
0
We need more than numbers
Looking at some plots shows us that the mean and median might not be very good summary numbers for
this data.
densityplot( ~ duration, data=geyser )
0.5
Density
0.4
0.3
0.2
0.1
0.0
0
2
4
6
duration
1
bwplot( ~ duration, data=geyser )
1
2
3
4
5
duration
histogram( ~ duration, data = geyser, width = 0.2 )
1.0
Density
0.8
0.6
0.4
0.2
0.0
1
2
3
4
5
duration
There is also a quirky artifact in the data.
histogram( ~ duration, data = geyser, width = 0.05 )
Density
3
2
1
0
1
2
3
4
5
duration
2
Apparently during the nigth, eruptions were listed as long or short and recorded as 4 or 2 minutes. See
?geyser for more about the data.
Duration vs wait
waiting is the length of time since the previous eruption. For scatter plots, we put the explanatory variable
(if there is one) on the x-axis and the response on the y-axis. Since the eruptions come after the wait, we’ll
consider duration to be the response.
xyplot( duration ~ waiting, data=geyser )
duration
5
4
3
2
1
40
60
80
100
waiting
We can also calculate the time until the next eruption. (You don’t need to fully understand this code, but it
is shifting the waits over one row.)
geyser <- geyser %>%
mutate( time.til.next = c(tail(waiting, -1), NA) )
xyplot( time.til.next ~ duration, data=geyser )
time.til.next
100
80
60
40
1
2
3
4
5
duration
This shows a much clearer picture. After a longer eruption, we expect a longer wait.
3
Z-scores
Z-scores measure how far above or below the mean a value is in units of standard deviations. That is, they
answer the question:
How many standard deviations above or below the mean is this value?
Here’s the formula:
Z=
value − mean
standard deviation
This gives us a standardized way of comparing values across distributions.
Example: Men’s and Women’s heights Let’s compare a 6’0" woman to a 6’5" man. Which seems more
unusual (i.e., farther above average)?
We can use this information to do the calculations.
Height and Weight distributions
height (male 18-74)
height (female 18-74)
N(5'9",3") =
N(69,3)
N(5'3.5",2.5")= N(63.5,2.5)
height (male 18-24)
height (female 18-24)
N(5'10",3") =
N(70,2.8)
N(5'4.3",2.6")= N(64.3,2.6)
height (male 11)
N(146cm,8cm) =
weight (male 18-24)
weight (female 18-24)
N(162,29.1)
N(134,27)
N(57.5,3.15)
z_woman <- (72 - 64.3) / 2.6; z_woman
## [1] 2.962
z_man <- (77 - 70) / 2.8; z_man
## [1] 2.5
So it is more unusual for a woman to be 6’0" than for a man to be 6’5“.
Correlation coefficient
We would like a number that measures the strength (and direction) of linear associations between two
variables. The correleation coefficient (r) is computed by summing the products of z-scores and dividing by
n − 1. We won’t calculate this number by hand.
Here’s what you need to know about r:
• r is always between -1 and 1
• r = 1 only for a perfect linear fit with positive slope
4
• r = −1 only for a perfect linear fit with negative slope
• we get the same number if we reverse the roles of explantory and response variables
• the cor() function in R computes it for us
# get the same value either way around for cor():
cor( waiting ~ duration, data=geyser )
## [1] -0.6446
cor( duration ~ waiting, data=geyser )
## [1] -0.6446
# this doesn't work because some we have some missing data
cor( time.til.next ~ duration, data=geyser )
## [1] NA
tail( geyser, 3 )
##
waiting duration time.til.next
## 297
58
4
88
## 298
88
4
79
## 299
79
2
NA
# ignore missing data -- use only complete observations
cor( time.til.next ~ duration, data=geyser, use="complete.obs")
## [1] 0.8885
As expected, the correlation is stronger (closer to 1) for time.til.next.
Correlation Guessing Game You can get calibrated to the correlation scale by playing this game:
• Correlation Guessing Game
5
Intro to Hypothesis Testing
R Pruim
2014-09-12
Hypothesis Testing: Four Steps
We will illustrate our 4 Step Process using the Lady Tasting Tea example. Suppose we test her on 20 cups
and she correctly identifies 18 of the 20.
Step 1: State Null and Alternative Hypotheses
For the Lady Tasting Tea, our Null Hypothesis is that the lady is just guessing, so her probability of correctly
identifying a cup is 50%. The alternative is that she does better.
• H0 : p = 0.50
• H0 : p > 0.50
Step 2: Compute a test statistic
We summarize all the evidence in our data as a single number. Since the number if computed from our
data, it is a statistic. In this case we could use either the number correct (x = 18) or the proportion correct
(p̂ = 0.9)
Step 3: Determine the p-value
We want to know how unlikely it would be to get 18 or more correct just by guessing – that is, assuming H0
is true. To answer this, we can simulate a world where H0 is true. In this case, flipping coins (or getting a
computer to simulate flipping coins) can do the job. The rflip() function in R will do the flipping for us.
rflip()
# flip one coin
##
## Flipping 1 coin [ Prob(Heads) = 0.5 ] ...
##
## H
##
## Number of Heads: 1 [Proportion Heads: 1]
rflip(20)
# flip 20 coins
##
## Flipping 20 coins [ Prob(Heads) = 0.5 ] ...
##
## H H H T T T T T H H H T T T T H H H T T
##
## Number of Heads: 9 [Proportion Heads: 0.45]
1
Now we just need to do this a lot of times and keep track of the values of our test statistic. The do() function
will help us do things repeatedly. Let’s do it 10000 times and keep track of the results.
do(10000) * rflip(20) -> RandomLadies
## Loading required package: parallel
head(RandomLadies)
##
##
##
##
##
##
##
1
2
3
4
5
6
n heads tails prop
20
7
13 0.35
20
11
9 0.55
20
9
11 0.45
20
10
10 0.50
20
8
12 0.40
20
12
8 0.60
# set width = 1 to avoid artifacts in histogram
histogram( ~heads, data=RandomLadies, width=1, v=18 )
Density
0.15
0.10
0.05
0.00
5
10
15
20
heads
tally( ~ heads, data=RandomLadies )
##
##
##
##
##
2
2
17
5
3
8
18
3
4
48
19
2
5
151
6
361
7
8
9
10
11
12
758 1208 1587 1743 1606 1220
13
753
14
364
15
141
16
40
tally( ~ (heads>=18), data=RandomLadies )
##
##
##
TRUE FALSE
5 9995
In this case, only 5 of our 10000 simulations produced a test statistic as large as 18. So our estimated p-value
is 5 × 10-4.
2
Step 4: Draw a conclusion
So what do we conclude? While it is possible that someone could get 18 out of 20 correct just by guessing, it
is very unlikely. This is evidence against the null hypothesis.
So we reject the null hypothesis.
Keep in mind that we do not know for sure that H0 is false. What we know is that one of the following is the
case:
• H0 is false
• H0 is true, but something very unusual just happened.
Unusual things can happen – but not very often. That’s what makes them unusual.
3
Creating Randomization Distributions
R Pruim
2014-09-16
Hugo
I once played a game with my kids expecting a ghost to be rolled 1 time in 6. It felt like the ghost (Hugo)
was coming up too often, so I collected some data. In 50 rolls of the die, there were 16 “Hugos”. What should
I conclude about the die?
Our null hypothesis is that the proportion is 1/6:
• H0 : p = 1/6
• Ha : p =
6 1/6
Let’s create a randomization distribution and use it to get a p-value. We can do this almost like we did with
the Lady Tasting Tea, but this time we need to simulate a proportion of 1/6 because our null hypothesis is
that Hugo is rolled one time in 6.
Here’s how to do it:
Hugo.Rand <- do(1000) * rflip( 50, prob=1/6)
## Loading required package: parallel
Density
histogram( ~ heads, data=Hugo.Rand, width=1, v=16)
0.10
0.05
0.00
0
5
10
15
heads
mean( ~heads, data=Hugo.Rand)
## [1] 8.402
1
tally( ~heads, data=Hugo.Rand )
##
##
##
1
3
2
4
3
11
4
42
5
6
7
8
9 10
76 121 127 142 130 130
11
89
12
62
13
35
14
18
15
4
16
4
17
1
18
1
tally( ~(heads >=16), data=Hugo.Rand, format="prop")
##
## TRUE FALSE
## 0.006 0.994
We could also choose to work with proportions instead of counts:
histogram( ~ prop, data=Hugo.Rand, width=0.02, v=16/50)
Density
6
4
2
0
0.0
0.1
0.2
0.3
prop
mean( ~prop, data=Hugo.Rand)
## [1] 0.168
tally( ~prop, data=Hugo.Rand )
##
## 0.02 0.04 0.06 0.08
##
3
4
11
42
## 0.32 0.34 0.36
##
4
1
1
0.1 0.12 0.14 0.16 0.18
76 121 127 142 130
0.2 0.22 0.24 0.26 0.28
130
89
62
35
18
tally( ~(prop>=16/50), data=Hugo.Rand, format="prop")
##
## TRUE FALSE
## 0.006 0.994
2
0.3
4
Caffeine and Finger Tapping
Our Null Hypothesis is that the mean finger tapping rate is the same for the caffeine and no caffeine groups.
• H0 : µC = µN C
• Ha : µC =
6 µN C
Now we must simulate a world where the two means are the same. One way to do this that reflects the design
of the study is to shuffle the Caffeine/No Caffeine labels. If there isn’t a relationship between caffeine and
finger tapping. Here’s how:
mean( Taps ~ Group, data=CaffeineTaps)
##
##
Caffeine No Caffeine
248.3
244.8
diff( mean( Taps ~ Group, data=CaffeineTaps) )
## No Caffeine
##
-3.5
diffmean( Taps ~ Group, data=CaffeineTaps)
## diffmean
##
-3.5
diffmean( Taps ~ shuffle(Group), data=CaffeineTaps )
## diffmean
##
-0.7
Tap.Rand <- do(10000) *
diffmean( Taps ~ shuffle(Group), data=CaffeineTaps )
histogram( ~ diffmean, data=Tap.Rand, v=-3.5)
Density
0.3
0.2
0.1
0.0
−4
−2
0
2
diffmean
3
4
mean( ~diffmean, data=Tap.Rand)
## [1] 0.01296
# one-sided
tally( ~(diffmean <= -3.5), data=Tap.Rand,
format="prop")
##
##
TRUE FALSE
## 0.0022 0.9978
# both sides:
tally( ~(abs(diffmean) >= 3.5), data=Tap.Rand,
format="prop")
##
##
TRUE FALSE
## 0.0054 0.9946
Survey says
Next time we will look at how the version of our survey questions affected the response. This situation will
be almost identical the the one above except that the response variable is categorical instead of quantititative.
So our null hypothesis is about proportions instead of about means:
• H0 : p1 = p2
• Ha : p1 =
6 p2
If we replace diffmean() with diffprop() to deal with proportions instead of means, we can follow the
outline above. We’ll do that next time.
4
Randomization Review
R Pruim
2014-09-19
Overview of Randomization Distributions we have seen
We have used randomization distributions in 5 situations so far. The procedure for each is very similar:
1. State the Null and Alternative Hypotheses
Remember these will be about a parameter or parameters.
2. Compute a Test Statistic
Often the test statistic uses “data versions” of the parameters. If the hypothesis is about population means,
the test statistics will involve sample means, etc.
3. Determine the p-value (from a simulated randomization distribution)
We need to simulate a world
a. where the null hypothesis is true,
b. that reflects the design of our study, and
c. takes advantage of information the data provides about the population.
4. Interpret/Draw a conclusion
Small p-values provide evidence against the null hypothesis.
When the p-value is small enough we say the results are statistically significant. (The threshold for
smallness depends on context.)
Creating Randomization Distributions
Here is a side by side comparision of the ways we have created randomization distributions in our 5 key
examples:
# rflip can simulate any proportion
Lady.Rand <- do(10000) * rflip(20)
# p = 0.50 by default
Hugo.Rand <- do(10000) * rflip(50, prob=1/6)
These three involve two variables and shuffle the explantory variable.
Taps.Rand <- do(10000) *
diffmean( Taps ~ shuffle(Caffeine), data=CaffeineTaps)
Ticket.Rand <-do(10000) *
diffprop(ticket ~ shuffle(ticketVer), data=Survey)
Ticket.Rand <- do(10000) *
diffprop(ticket ~ shuffle(ticketVer), data=Survey)
1
Testing a mean is a little bit different because we want to shift the disribution to make the mean match the
null hypothesis value. This maintains the shape of the distribution. Note the use of resample() here. This
is sampling with replacement.
Temp.Rand <- do(10000) *
mean( ~ (BodyTemp + .34), data= resample(BodyTemp50))
Full details for each example appear below.
Testing a Proprtion:
Lady Tasting Tea
H0 : p = 0.50; n = 20
Test Stat: x = 18 or p̂ = 18/20 = 0.9
Lady.Rand <- do(10000) * rflip(20)
## Loading required package: parallel
histogram ( ~ heads, data=Lady.Rand, v = 18)
0.25
Density
0.20
0.15
0.10
0.05
0.00
5
10
15
heads
# one-sided p-values
tally( ~ heads >= 18, data=Lady.Rand, format="prop")
##
##
TRUE FALSE
## 0.0004 0.9996
tally( ~ prop >= 0.9, data=Lady.Rand, format="prop")
##
##
TRUE FALSE
## 0.0004 0.9996
2
20
Hugo
H0 : p = 1/6; n = 50
Test Stat: x = 16 or p̂ = 16/50 = 0.32
Hugo.Rand <- do(10000) * rflip(50, prob=1/6)
histogram ( ~ prop, data=Hugo.Rand, v = .32)
10
Density
8
6
4
2
0
0.0
0.1
0.2
0.3
0.4
prop
# one-sided p-values
tally( ~( heads >= 16), data=Hugo.Rand, format="prop")
##
##
TRUE FALSE
## 0.0056 0.9944
tally( ~( prop >= 0.32), data=Hugo.Rand, format="prop")
##
##
TRUE FALSE
## 0.0056 0.9944
Testing the Difference Between Two Means: Caffeine Taps
H0 : µ1 = µ2 (µ1 − µ2 = 0)
Test Stat: x1 − x2 (Use diffmean())
testStat = diffmean(Taps ~ Caffeine, data=CaffeineTaps); testStat
## diffmean
##
3.5
Taps.Rand <- do(10000) *
diffmean( Taps ~ shuffle(Caffeine), data=CaffeineTaps)
histogram( ~diffmean, data=Taps.Rand, v=testStat )
3
Density
0.3
0.2
0.1
0.0
−4
−2
0
2
4
diffmean
# two-sided
tally( ~(abs(diffmean) >= abs(testStat)), data=Taps.Rand, format="prop" )
##
##
TRUE FALSE
## 0.0035 0.9965
Testing the Difference Between Two Proportions: Survey Question
H0 : p1 = p2 (p1 − p2 = 0)
Test Stat: p̂1 = p̂2 = 0.071
testStat <- diffprop(ticket ~ ticketVer, data=Survey); testStat
## diffprop
## 0.07099
Ticket.Rand <- do(10000) *
diffprop(ticket ~ shuffle(ticketVer), data=Survey)
histogram(~diffprop, data=Ticket.Rand, v=testStat)
2.5
Density
2.0
1.5
1.0
0.5
0.0
−0.5
0.0
0.5
diffprop
4
# one-sided
tally(~ (diffprop >= testStat), data=Ticket.Rand, format="prop")
##
##
TRUE FALSE
## 0.4366 0.5634
# one-side, alternative method
prop(~ (diffprop >= testStat), data=Ticket.Rand)
##
TRUE
## 0.4366
Testing for Associaition Between Two Quantitative Variables: NFL Uniforms
H0 : ρ = 0
Test Stat: r = ‘cor(ZP enY ds N F LM alevolence, data = M alevolentU nif ormsN F L)‘
head(MalevolentUniformsNFL)
##
##
##
##
##
##
##
NFLTeam NFL_Malevolence ZPenYds
1 LA Raiders
5.10
1.19
2 Pittsburgh
5.00
0.48
3 Cincinnati
4.97
0.27
4 New Orleans
4.83
0.10
5
Chicago
4.68
0.29
6 Kansas City
4.58
-0.19
xyplot( ZPenYds ~ NFL_Malevolence, data=MalevolentUniformsNFL)
ZPenYds
1.0
0.5
0.0
−0.5
−1.0
−1.5
3.0
3.5
4.0
4.5
5.0
NFL_Malevolence
testStat <cor( ZPenYds ~ NFL_Malevolence, data=MalevolentUniformsNFL)
testStat
5
## [1] 0.4298
NFL.Rand <- do(10000) *
cor(ZPenYds ~ shuffle(NFL_Malevolence), data=MalevolentUniformsNFL)
head(NFL.Rand)
##
##
##
##
##
##
##
result
1 0.08512
2 0.27475
3 -0.14868
4 0.25148
5 0.22910
6 0.25933
histogram( ~ result, data=NFL.Rand)
2.0
Density
1.5
1.0
0.5
0.0
−0.5
0.0
0.5
result
# one-sided
tally( ~ (result >= testStat), data=NFL.Rand, format="prop")
##
##
TRUE FALSE
## 0.0098 0.9902
# two-sided
tally( ~ (abs(result) >= testStat), data=NFL.Rand, format="prop")
##
##
TRUE FALSE
## 0.0195 0.9805
Testing a Mean: Body Temperature
H0 : µ = 98.6
Test Stat: x = 98.26
6
mean(~ BodyTemp, data=BodyTemp50)
## [1] 98.26
histogram(~ BodyTemp, data=BodyTemp50)
0.5
Density
0.4
0.3
0.2
0.1
0.0
96
97
98
99
100
101
BodyTemp
98.6 - 98.26
## [1] 0.34
# shift by 0.34 so the mean is like in the null hypothesis
Temp.Rand <- do(10000) *
mean(~ (BodyTemp + .34), data= resample(BodyTemp50))
histogram( ~ result, data=Temp.Rand, v=98.26)
Density
3
2
1
0
98.2
98.4
98.6
98.8
99.0
result
tally( ~ result > 98.26, data=Temp.Rand, format="prop")
##
##
TRUE FALSE
## 0.9994 0.0006
7
# a short cut
prop(~ result > 98.26, data=Temp.Rand)
##
TRUE
## 0.9994
8
Paired Designs
R Pruim
2014-09-23
Setting the Random Seed
Before we do our example, as a reminder, let’s set the random seed so we get the “same random randomizations”
every time we knit this file.
set.seed(12345)
# we can choose any number here.
Bowls of M and M’s
A book by Tintle et al includes a data set based on an experiment to see whether univeristy students take
more M and M’s if they are presented with a larger bowl. Each subject was “tested” twice (on different days),
once with a large bowl and once with a small bowl. Half of the students had the large bowl the first time,
the other half had the small bowl the first time.
require(Tintle1)
## Loading required package: Tintle1
##
## Attaching package: 'Tintle1'
##
## The following objects are masked from 'package:Lock5withR':
##
##
BodyFat, MarriageAges
head(BowlsMMs)
##
##
##
##
##
##
##
1
2
3
4
5
6
Small Large
33
41
24
92
35
61
24
19
40
21
33
35
As we see, each case has two varialbes, the number of M & M’s chosen form the large bowl and the number
chosen from the smaller bowl. Our null hypothesis is that on average, the difference between these two
measurements is 0:
• H0 : µdif f = 0.
Our test statistic is the mean difference:
1
mean( ~( Large - Small), data=BowlsMMs)
## [1] 10.88
We can think about the randomization distribution at least two ways.
Converting this to a problem about one mean
Since our null hypothesis is about one mean (the mean difference), we could treat it just like the Body
Temperature example. But first we need to compute all those differences.
MM2 <- mutate( BowlsMMs, diff = Large - Small )
head(MM2)
##
##
##
##
##
##
##
1
2
3
4
5
6
Small Large diff
33
41
8
24
92
68
35
61
26
24
19
-5
40
21 -19
33
35
2
From this point on it is just like before
testStat <- mean( ~diff, data=MM2 ); testStat
## [1] 10.88
MM2.Rand <- do(10000) *
mean( diff - testStat, data=resample(MM2))
## Loading required package: parallel
histogram( ~ result, MM2.Rand, v=testStat)
Density
0.04
0.03
0.02
0.01
0.00
−20
0
20
result
2
# 1-sided
tally( ~(result >= testStat), data=MM2.Rand, format="prop")
##
##
TRUE FALSE
## 0.1022 0.8978
A 2-sided p-value would be about twice as large.
Swapping the Small/Large labels
Another way to randomize – a way that matches the design of the study well – is to flip a coin to decide
whether or not we randomly swap the value of Small with the value of Large for each subject.
Notice how some of the Small/Large values have been swapped in the example below.
BowlsMMs
##
##
##
##
##
##
##
##
##
##
##
##
##
##
##
##
##
##
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
Small Large
33
41
24
92
35
61
24
19
40
21
33
35
88
42
36
50
65
11
38
104
28
97
50
36
26
43
34
62
51
33
25
62
26
32
# let's randomly swap some
swap( BowlsMMs, Small ~ Large )
##
##
##
##
##
##
##
##
##
##
##
1
2
3
4
5
6
7
8
9
10
Small Large
33
41
24
92
61
35
24
19
21
40
35
33
88
42
36
50
65
11
38
104
3
##
##
##
##
##
##
##
11
12
13
14
15
16
17
97
50
43
34
33
62
32
28
36
26
62
51
25
26
# here is a different randomization
swap( BowlsMMs, Small ~ Large )
##
##
##
##
##
##
##
##
##
##
##
##
##
##
##
##
##
##
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
Small Large
41
33
92
24
61
35
19
24
21
40
35
33
42
88
50
36
11
65
104
38
97
28
36
50
26
43
34
62
51
33
62
25
26
32
Now we are all set to create a randomization distribution.
BowlsMMs.Rand <- do(1000) *
mean( Large - Small, data=swap(BowlsMMs, Small ~ Large))
histogram(~result, BowlsMMs.Rand, v=testStat)
Density
0.04
0.03
0.02
0.01
0.00
−20
−10
0
10
result
4
20
tally( ~(result >= testStat), data=BowlsMMs.Rand, format="prop")
##
## TRUE FALSE
## 0.119 0.881
5