S6880 #10
Sampling Methods
Sampling With Replacement
Sampling Without Replacement
Outline
1
Sampling With Replacement
Sampling with Replacement
2
Sampling Without Replacement
Nonsequential Sampling
Sequential Sampling
(WMU)
S6880 #10
S6880, Class Notes #10
2 / 10
Sampling With Replacement
Sampling Without Replacement
Sampling with Replacement
Want: Select a sample of size n with replacement from
{x1 , x2 , · · · , xN }.
Method: Generate discrete uniform from the indices {1, 2, · · · , N}
and index these indices.
Algorithm: Generate U1 , · · · , Un ∼ U(0, 1) and select
xdNUi e , i = 1, 2, · · · , n.
R: sample(x, n, replace=T),
sample(x,repl=T) # bootstrap sample
sample(x,n,repl=T,prob=p)
# length(x)==length(p)
(WMU)
S6880 #10
S6880, Class Notes #10
3 / 10
Sampling With Replacement
Sampling Without Replacement
Outline
1
Sampling With Replacement
Sampling with Replacement
2
Sampling Without Replacement
Nonsequential Sampling
Sequential Sampling
(WMU)
S6880 #10
S6880, Class Notes #10
4 / 10
Sampling With Replacement
Sampling Without Replacement
Nonsequential Sampling
i.e., access to data items is non-sequential
Want: a sample of size n without replacement from {x1 , · · · , xN }.
Algorithm:
1
Form (i, xi ), i = 1, · · · , N.
2
Set size = 0.
3
Repeat
Generate j ∼ U{1, 2, · · · , N − size} (index to x’s).
Change (j, xj ) to (N − size, xj ) and
(N − size, xN−size ) to (j, xN−size ).
Increment size (i.e., size ←− size + 1)
Deliver xj as sizeth sample item.
Until size = n.
(WMU)
S6880 #10
S6880, Class Notes #10
5 / 10
Sampling With Replacement
Sampling Without Replacement
Nonsequential Sampling
i.e., access to data items is non-sequential
Want: a sample of size n without replacement from {x1 , · · · , xN }.
Algorithm:
1
Form (i, xi ), i = 1, · · · , N.
2
Set size = 0.
3
Repeat
Generate j ∼ U{1, 2, · · · , N − size} (index to x’s).
Change (j, xj ) to (N − size, xj ) and
(N − size, xN−size ) to (j, xN−size ).
Increment size (i.e., size ←− size + 1)
Deliver xj as sizeth sample item.
Until size = n.
(WMU)
S6880 #10
S6880, Class Notes #10
5 / 10
Sampling With Replacement
Sampling Without Replacement
Nonsequential Sampling
i.e., access to data items is non-sequential
Want: a sample of size n without replacement from {x1 , · · · , xN }.
Algorithm:
1
Form (i, xi ), i = 1, · · · , N.
2
Set size = 0.
3
Repeat
Generate j ∼ U{1, 2, · · · , N − size} (index to x’s).
Change (j, xj ) to (N − size, xj ) and
(N − size, xN−size ) to (j, xN−size ).
Increment size (i.e., size ←− size + 1)
Deliver xj as sizeth sample item.
Until size = n.
(WMU)
S6880 #10
S6880, Class Notes #10
5 / 10
Sampling With Replacement
Sampling Without Replacement
Nonsequential Sampling
i.e., access to data items is non-sequential
Want: a sample of size n without replacement from {x1 , · · · , xN }.
Algorithm:
1
Form (i, xi ), i = 1, · · · , N.
2
Set size = 0.
3
Repeat
Generate j ∼ U{1, 2, · · · , N − size} (index to x’s).
Change (j, xj ) to (N − size, xj ) and
(N − size, xN−size ) to (j, xN−size ).
Increment size (i.e., size ←− size + 1)
Deliver xj as sizeth sample item.
Until size = n.
(WMU)
S6880 #10
S6880, Class Notes #10
5 / 10
Sampling With Replacement
Sampling Without Replacement
Nonsequential Sampling in R
sample(x,n)
sample(x)
sample(n)
(WMU)
# sample n items from x w/o replacement
# random permutation of Xi’s
# random permutation of 1,2,· · ·,n.
S6880 #10
S6880, Class Notes #10
6 / 10
Sampling With Replacement
Sampling Without Replacement
Sequential Sampling, N is Known
1
Generate Ui ∼ U(0, 1) and associate it with i for i = 1, 2, · · · N.
That is, (U1 , 1), (U2 , 2), · · · , (UN , N).
2
Sort Ui ’s.
3
Select the first n individuals according to first n samllest Ui ’s.
For example, (.7,1), (.3,2), (.4,3), (.4,4), N = 4.
=⇒ (.3,2), (.4,3), (.4,4), (.7,1) (sorted)
=⇒ If n = 2, then individuals 2, 3 are selected. That is, x2 and x3 are
selected.
(WMU)
S6880 #10
S6880, Class Notes #10
7 / 10
Sampling With Replacement
Sampling Without Replacement
Sequential Sampling, N is Known
1
Generate Ui ∼ U(0, 1) and associate it with i for i = 1, 2, · · · N.
That is, (U1 , 1), (U2 , 2), · · · , (UN , N).
2
Sort Ui ’s.
3
Select the first n individuals according to first n samllest Ui ’s.
For example, (.7,1), (.3,2), (.4,3), (.4,4), N = 4.
=⇒ (.3,2), (.4,3), (.4,4), (.7,1) (sorted)
=⇒ If n = 2, then individuals 2, 3 are selected. That is, x2 and x3 are
selected.
(WMU)
S6880 #10
S6880, Class Notes #10
7 / 10
Sampling With Replacement
Sampling Without Replacement
Sequential Sampling, N is Known
1
Generate Ui ∼ U(0, 1) and associate it with i for i = 1, 2, · · · N.
That is, (U1 , 1), (U2 , 2), · · · , (UN , N).
2
Sort Ui ’s.
3
Select the first n individuals according to first n samllest Ui ’s.
For example, (.7,1), (.3,2), (.4,3), (.4,4), N = 4.
=⇒ (.3,2), (.4,3), (.4,4), (.7,1) (sorted)
=⇒ If n = 2, then individuals 2, 3 are selected. That is, x2 and x3 are
selected.
(WMU)
S6880 #10
S6880, Class Notes #10
7 / 10
Sampling With Replacement
Sampling Without Replacement
Sequential Sampling, N is Known
1
Generate Ui ∼ U(0, 1) and associate it with i for i = 1, 2, · · · N.
That is, (U1 , 1), (U2 , 2), · · · , (UN , N).
2
Sort Ui ’s.
3
Select the first n individuals according to first n samllest Ui ’s.
For example, (.7,1), (.3,2), (.4,3), (.4,4), N = 4.
=⇒ (.3,2), (.4,3), (.4,4), (.7,1) (sorted)
=⇒ If n = 2, then individuals 2, 3 are selected. That is, x2 and x3 are
selected.
(WMU)
S6880 #10
S6880, Class Notes #10
7 / 10
Sampling With Replacement
Sampling Without Replacement
Sequential Sampling, N is Known, a better algorithm
Suppose ni = #{xj included in the sample: 1 ≤ j < i} which in turn = #
of population items included prior to encountering the i th population
item. Note that population items are encountered sequentially.
Then
(
n−ni
N−i+1 , if ni < n,
P(xi is included in the sample) =
0,
otherwise.
(WMU)
S6880 #10
S6880, Class Notes #10
8 / 10
Sampling With Replacement
Sampling Without Replacement
Sequential Sampling, N is Known, a better algorithm
continued
1
Set i = 0, ni = 0.
2
Repeat
i ←− i + 1
Generate Ui ∼ U(0, 1)
If Ui ≤ (n − ni )/(N − i + 1) then
include xi and ni ←− ni + 1
else skip xi
Until (ni = n) or (i = N)
(WMU)
S6880 #10
S6880, Class Notes #10
9 / 10
Sampling With Replacement
Sampling Without Replacement
Sequential Sampling, N is Unknown
1
Set i = n. Generate Ui ∼ U(0, 1), 1 ≤ i ≤ n. Read the first n
records and form a table of (Ui , yi ), 1 ≤ i ≤ n where yi = xi .
2
While (exists more records)
i ←− i + 1
Generate Ui ∼ U(0, 1).
Read record i (i.e., get xi )
Let Uk = max1≤j≤n Uj . (In case of ties, Uk is chosen
randomly from ties.)
If (Ui ≤ Uk ) or (Ui = Uk and (V ∼ U(0, 1)) ≤ 21 )
then replace (Uk , yk ) in the table by (Ui , xi ).
3
Return y1 , y2 , · · · , yn .
(WMU)
S6880 #10
S6880, Class Notes #10
10 / 10
Sampling With Replacement
Sampling Without Replacement
Sequential Sampling, N is Unknown
1
Set i = n. Generate Ui ∼ U(0, 1), 1 ≤ i ≤ n. Read the first n
records and form a table of (Ui , yi ), 1 ≤ i ≤ n where yi = xi .
2
While (exists more records)
i ←− i + 1
Generate Ui ∼ U(0, 1).
Read record i (i.e., get xi )
Let Uk = max1≤j≤n Uj . (In case of ties, Uk is chosen
randomly from ties.)
If (Ui ≤ Uk ) or (Ui = Uk and (V ∼ U(0, 1)) ≤ 21 )
then replace (Uk , yk ) in the table by (Ui , xi ).
3
Return y1 , y2 , · · · , yn .
(WMU)
S6880 #10
S6880, Class Notes #10
10 / 10
Sampling With Replacement
Sampling Without Replacement
Sequential Sampling, N is Unknown
1
Set i = n. Generate Ui ∼ U(0, 1), 1 ≤ i ≤ n. Read the first n
records and form a table of (Ui , yi ), 1 ≤ i ≤ n where yi = xi .
2
While (exists more records)
i ←− i + 1
Generate Ui ∼ U(0, 1).
Read record i (i.e., get xi )
Let Uk = max1≤j≤n Uj . (In case of ties, Uk is chosen
randomly from ties.)
If (Ui ≤ Uk ) or (Ui = Uk and (V ∼ U(0, 1)) ≤ 21 )
then replace (Uk , yk ) in the table by (Ui , xi ).
3
Return y1 , y2 , · · · , yn .
(WMU)
S6880 #10
S6880, Class Notes #10
10 / 10