Distribution-free testing
If the data are normally distributed, we may apply a ztest or t-test when the parameter of interest is . But
what if this is the normal probability plot:
Or this:
1
Distribution-free testing
The Box-plots also indicate non-normality: skewness
in the first case and more outliers than expected under
normality in the second case:
2
Distribution-free testing
When the data are clearly non-normal, we need an
alternative: distribution-free testing. Also known under
the terms: nonparametric testing or rank tests.
Characteristic of such tests: they can be applied
irrespective of what the true probability distribution of
the data is (which we prove later).
Disadvantage: the power of these tests is (slightly)
smaller than those of normal-based tests if the data are
normally distributed.
Two of the most widely applied tests:
Wilcoxon rank sum test: test H0 : 1 = 2 for two
unpaired samples.
Wilcoxon signed-rank test: Either one sample and
test: H0 :  = 0 or two samples, paired, test H0 :
d = 0, with d = 1 -2.
3
Hypothesis testing, once again
Steps of hypothesis testing
1. Parameter of interest (, 2, p)? Assumptions?
Normal distribution, yes/no?
2. Hypotheses. One- or two-sided?
3. Testing with?
a) Computer: p-values
b) Table of critical values
c) Asymptotic distribution of test statistic
4.
Reject null hypothesis if
a) p-value smaller or equal to 
b) Value of test statistic lies in critical region.
c) Asymptotic p-value is smaller or equal to 
4
Wilcoxon rank sum test
1.
Parameter of interest:  = 1 - 2
2.
H0 :  = 0 (or 1 = 2). Alternative is either oneor two-sided
3.
Computation of exact p- value with software
(StatXact, Mathematica), tables of critical values
are available in Statistisch Compendium (Dutch).
Asymptotic normal distribution of the test statistic
will be proven.
4.
Reject null hypothesis if
a) p-value smaller or equal to 
b) Value of test statistic lies in critical region.
c) Asymptotic p-value is smaller or equal to 
5
Wilcoxon rank sum test
Test statistic
1: Rank the observations from small to large. Equal
observations (ties) are assigned an average rank
number (so, if the 4th, 5th and 6th observations are tied
then they all correspond to rank (4+5+6) / 3 = 5.
Prothesis 1 Prothesis 2
3.2
2.3
4.3
4.3
5.6
1.3
1.2
2.2
0.6
0.7
0.4
Ranks 1
8
9.5
11
4
2
Ranks 2
7
9.5
5
5
3
1
Sum = 34.5
2: Test statistic W: sum of ranks of the smallest
sample, 34.5 in the example.
3: Find critical region for type I error .
Note: for one-sided testing, first multiply  by 2 to
obtain the critical value if the table is based on twosided testing.
6
Wilcoxon rank sum test
Smallest sample, sample ‘1’. The other is sample ‘2’.
If
a) H1: 1 < 2 , use the left-critical value only
b) H1: 1 > 2 , use right-crititical value only
c) H1: 1  2 , use both.
In the example (n = 5 and m = 6) for  = 0.05:
a) look at  = 0.1: WL = 20, so reject H0 if W  20
b) WR = n(m + n + 1) – WL = 5*12 – 20 = 40. So, reject
H0 if W  40
c) look at  = 0.05. WL = 18, WR = 42. Reject H0 if W  18
or W  42.
7
Wilcoxon rank sum test
Prothese 1 Prothese 2 Rangnrs 1
Rangnrs 2
3.2
2.3
8
4.3
4.3
9.5
5.6
1.3
11
1.2
2.2
4
0.6
0.7
2
0.4
Som = 34.5
7
9.5
5
5
3
1
4:Compare test statistic with critical region. One-sided
testing (first protesis is a new design) : H1: 1 > 2 ,
because you want to show that the new protesis is
better than the old one.
W = 34.5 < 40 so do not reject H0: we cannot say that
the new protesis is better than the old one.
m
Later we show that for n  , m  ,  c,
N
where c > 0,
z0 
W  n(m  n  1) / 2
mn(m  n  1) / 12
is standard normal distributed, so use this for values
not in the Wilcoxon table.
8
Wilcoxon signed-rank test
Situation:
a) One sample and test: H0 :  = 0
b) Two samples, paired, test H0 : d = 0, with
d = 1 -2.
In both cases we create one sample of which the mean
should be approximately equal to ‘0’ if H0 holds.
a) Subtract specified 0 from each observation
Data: 3.9, 2.3, 4.0, 4.5, 1.5, 2.2, 1.7, 3.6, 6.1, 1.2, 5.3,
3.3, -0.6, 5.2, 0.2, 0.9, 2.6, 2.2, 3.4, 2.8
Hypothesis: H0:  = 2. Sample to which we will
apply the test:
Data:1.9, 0.3, 2.0, 2.5, -0.5, 0.2, -0.3, 1.6, 4.1, -0.8, 3.3,
1.3, -2.6, 3.2, -1.8, -1.1, 0.6, 0.2, 1.4, 0.8
b) Compute pairwise differences. These differences are
the sample to which we will apply the test .
9
Wilcoxon signed-rank test
Assumption: density f (x) of one-sample data is
symmetric.
Data: Before introducing a new beer on the market, the
brewery wants to know whether people appreciate it
more than an existing beer. Fifteen people give marks
to both beers (blind test).
The null hypothesis is H0 : d = 0, with d = nieuw –
bestaand and the alternative hypothesis is H1 : d > 0.
Person
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Mark, new
6
8
4
8
9
6
7
5
8
8
8
7
9
5
6
Mark, old
4
3
7
6
5
8
4
5
6
5
8
5
7
4
5
Difference
2
5
-3
2
4
-2
3
0
2
3
0
2
2
1
1
10
Wilcoxon signed-rank test
Data and ranks:
Person New Old Differ. Abs. Rank Abs Positive
1
6
4
2
2
7.5
yes
2
8
3
5
5
15
yes
3
4
7
-3
3
12
no
4
8
6
2
2
7.5
yes
5
9
5
4
4
14
yes
6
6
8
-2
2
7.5
no
7
7
4
3
3
12
yes
8
5
5
0
0
1.5
zero
9
8
6
2
2
7.5
yes
10
8
5
3
3
12
yes
11
8
8
0
0
1.5
zero
12
7
5
2
2
7.5
yes
13
9
7
2
2
7.5
yes
14
5
4
1
1
3.5
yes
15
6
5
1
1
3.5
yes
Test statistic W+: sum of ranks corresponding to
positive observations plus half of the sum of ranks
corresponding to ‘0’. Ties: average the ranks. In the
example: W+ = 99.
11
Wilcoxon signed-rank test
Example Alternative hypothesis was H1 : d > 0, so
one-sided test for  = 0.05. Right critical value WR =
n(n+1) / 2 – WL = 120 – 30 = 90.
W+ = 99 > WR = 90, so reject H0: d = 0: new beer is
significantly nicer than the old beer.
Normal approximation
W   n(n  1) / 4
99  n(n  1) / 4
P(W  99)  P(

)
n(n  1)( 2n  1) / 24
n(n  1)( 2n  1) / 24

 P( Z  2.15)  0.015.
12