Statistics and Probability Letters 125 (2017) 99–103
Contents lists available at ScienceDirect
Statistics and Probability Letters
journal homepage: www.elsevier.com/locate/stapro
On randomization-based causal inference for matched-pair
factorial designs
Jiannan Lu ∗ , Alex Deng
Analysis and Experimentation, Microsoft Corporation, United States
article
info
Article history:
Received 13 January 2017
Received in revised form 26 January 2017
Accepted 2 February 2017
Available online 10 February 2017
abstract
Under the potential outcomes framework, we introduce matched-pair factorial designs,
and propose the matched-pair estimator of the factorial effects. We also calculate the
randomization-based covariance matrix of the matched-pair estimator, and provide the
‘‘Neymanian’’ estimator of the covariance matrix.
© 2017 Elsevier B.V. All rights reserved.
Keywords:
Experimental design
Factorial effect
Precision
Potential outcome
1. Introduction
Randomization is widely regarded as the gold standard of causal inference (Rubin, 2008). Under the potential outcomes
framework (Neyman, 1923; Rubin, 1974), for a two-level factor, we define the causal effect as the linear contrast of
the potential outcomes under treatment and control. To investigate multiple factors simultaneously, 2K factorial designs
(Fisher, 1935; Yates, 1937) can be employed. Randomization-based causal inference for factorial designs has deep roots in
the experimental design literature (e.g., Kempthrone, 1952), and was recently presented using the language of potential
outcomes (Dasgupta et al., 2015; Mukerjee et al., 2016).
Pair-matching (Cochran, 1953), as a special form of stratification, has been widely adopted by researchers and
practitioners (e.g., Grossarth-Maticek and Ziegler, 2008). For treatment-control studies (i.e., 21 factorial designs), pairmatching has been extensively investigated by the causal inference community (Rosenbaum, 2002; Imai, 2008; Imai
et al., 2009; Ding, in press; Fogarty, 2016a,b). Unfortunately, similar discussion appears to be missing for general factorial
designs. In this paper, we fill this theoretical gap by extending Imai’s (2008) analysis to matched-pair factorial designs. We
restrict the experimental units to be a fixed finite population, for a two-fold reason. First, as shown in Imai (2008), it is
straightforward to generalize the finite-population analyses to infinite populations. Second, for some practical examples, it
might be unreasonable to view the experimental units as a random sample from an infinite population.
The paper proceeds as follows. Section 2 reviews the randomization-based causal inference framework for completely
randomized factorial designs. Section 3 introduces matched-pair factorial designs, proposes the matched-pair estimator
for the factorial effects, calculates its covariance matrix and the corresponding estimator. Section 4 briefly discusses the
precision gains by pair-matching in factorial designs, and concludes.
∗
Correspondence to: One Microsoft Way, Redmond, WA 98052-6399, USA.
E-mail address: [email protected] (J. Lu).
http://dx.doi.org/10.1016/j.spl.2017.02.007
0167-7152/© 2017 Elsevier B.V. All rights reserved.
100
J. Lu, A. Deng / Statistics and Probability Letters 125 (2017) 99–103
2. Causal inference for completely randomized factorial designs
To ensure self-containment, we first review the randomization-based causal inference framework for completely
randomized factorial designs. Although most materials are adapted from Dasgupta et al. (2015) and Lu (2016a,b), some
are refined for better clarity. For more detailed discussions on factorial designs, see, e.g., Wu and Hamada (2009).
2.1. Factorial designs
A 2K factorial design consists of K two-level (coded −1 and +1) factors. We represent it by the corresponding model
matrix (Wu and Hamada, 2009), a 2K × 2K matrix HK = (h0 , . . . , h2K −1 ) that can be constructed as follows:
1. Let h0 = 12K ;
2. For k = 1, . . . , K , construct hk by letting its first 2K −k entries be −1, the next 2K −k entries be +1, and repeating 2k−1
times;
3. If K ≥ 2, order all subsets of {1, . . . , K } with at least two
by cardinality and then lexicography. For
 elements, first
k = 1, . . . 2K − K − 1, let σk be the kth subset and hK +k = l∈σk hl , where ‘‘ ’’ stands for entry-wise product.
The use of the constructed HK is two-fold:
1. h0 corresponds to the null effect; h1 to hK correspond to the main effects of the K factors; hK +1 to hK +(K ) correspond to
the two-way interactions; . . .; h2K −1 corresponds to the K -way interaction;
2. The jth row of (h1 , . . . , hK ) corresponds to the jth treatment combination zj .
2
For j = 1, . . . , 2K , let λj denote the jth row of HK .
Example 1. For 22 factorial designs, the model matrix is:
h0
λ1
H2 =
λ2
λ3
λ4

+1
 +1
 +1
+1
h1
h2
−1
−1
+1
+1
−1
+1
−1
+1
h
3

+1
−1 
.
−1 
+1
The four treatment combinations are z1 = (−1, −1), z2 = (−1, +1), z3 = (+1, −1) and z4 = (+1, +1). We represent
the main effects of factors 1 and 2 by h1 = (−1, −1, +1, +1)′ and h2 = (−1, +1, −1, +1)′ respectively, and the two-way
interaction by h3 = (+1, −1, −1, +1)′ .
2.2. Randomization-based causal inference
We consider a 2K factorial design with N = 2K r units. By invoking the Stable Unit Treatment Value Assumption (Rubin,
1980), for i = 1, . . . , N and l = 1, . . . , 2K , let the potential outcome of unit i under zl be Yi (zl ), the average potential outcome
N
for zl be Ȳ (zl ) = N −1 i=1 Yi (zl ), and Yi = {Yi (z1 ), . . . , Yi (z2K )}′ . Define the individual and population-level factorial effect
vectors as
τi =
1
HK′ Yi
2K −1
(i = 1, . . . , N );
τ=
N
1 
N i=1
τi,
(1)
respectively. Our interest lies in τ .
We denote the treatment assignment mechanism by
Wi (zl ) =
1,
0,

if unit i is assigned treatment zl ,
otherwise.
(i = 1, . . . , N ; l = 1, . . . , 2K ).
We impose the following restrictions on the treatment assignment mechanism:
2K

Wi (zl ) = 1 (i = 1, . . . , N );
l =1
N

Wi (zl ) = r
(l = 1, . . . , 2K ).
i =1
In other words, we assign r units to each treatment, and one treatment to each unit. Therefore, the observed outcome of
2K

obs
unit i is Yiobs =
(zl ) = r −1 Ni=1 Wi (zl )Yi (zl ).
l=1 Wi (zl )Yi (zl ), and the average observed outcome for treatment zl is Ȳ
Under complete randomization, Dasgupta et al. (2015) estimated τ by
τ̂ C = 2−(K −1) HK′ Ȳ obs ,
Ȳ obs = {Ȳ obs (z1 ), . . . , Ȳ obs (z2K )}′ .
The sole source of randomness of τ̂ C is the treatment assignment. Dasgupta et al. (2015) and Lu (2016b) derived the
covariance matrix of this estimator, and the ‘‘Neymanian’’ estimator of the covariance matrix. We summarize their main
results in the following lemmas.
J. Lu, A. Deng / Statistics and Probability Letters 125 (2017) 99–103
101
Lemma 1. τ̂ C is unbiased, and its covariance matrix is
Cov(τ̂ C ) =
1
2K

22(K −1) r l=1
λ′l λl
N

{Yi (zl ) − Ȳ (zl )}2 −
1
N − 1 i =1


1
N

(τ i − τ)(τ i − τ)′ .
N (N − 1) i=1
(2)

S 2 (zl )
Moreover, the ‘‘Neymanian’’ estimator of the covariance matrix is
 (τ̂ C ) =
Cov
1
2K

22(K −1) r l=1
λ′l λl
N

1
r − 1 i=1
Wi (zl ){Yiobs − Ȳ obs (zl )}2 ,


whose bias is
N
i=1

s2 (zl )
(τ i − τ)(τ i − τ)′ /(N 2 − N ).
 (τ̂ C ) is ‘‘conservative,’’ because its diagonal entries, i.e., the variance estimators of
The covariance matrix estimator Cov
the components of τ̂ C , have non-negative biases.
3. Causal inference for matched-pair randomized factorial designs
3.1. Matched-pair designs and causal parameters
As pointed out by Imai (2008), they key idea behind matched-pair designs is that ‘‘experimental units are paired based
on their pre-treatment characteristics and the randomization of treatment is subsequently conducted within each matched
pair’’. To apply this idea to factorial designs, we group the N experimental units into r ‘‘pairs’’ of 2K units, and within each
pair randomly assign one unit to each treatment. Let ψj be the set of indices of the units in pair j, such that
| ψj |= 2K (j = 1, . . . , r );
ψj ∩ ψj′ = ∅ (∀j ̸= j′ );
∪rj=1 ψj = {1, . . . , N }.

′
For pair j, denote the average outcomes for treatment zl as Ȳj· (zl ) = 21K
i∈ψj Yi (zl ), and Ȳj· = {Ȳj· (z1 ), . . . , Ȳj· (z2K )} , and
the factorial effect vector as τ j· = 2−(K −1) HK′ Ȳj· . It is apparent
r
1
r j =1
Ȳj· (zl ) = Ȳ (zl )
r
1
(l = 1, . . . , 2k );
r j =1
τ j· = τ.
Within each
one unit to each treatment. Let the observed outcome of treatment zl in pair j be
 pair, we randomly assign
obs
Yjobs (zl ) =
= {Yjobs (z1 ), . . . , Yjobs (z2K )}′ . We estimate τ j· by τ̂ j· = 2−(K −1) HK′ Yjobs . The matchedi∈ψj Yi (zl )Wi (zl ), and Yj
pair estimator for τ is
τ̂ M =
r
1
r j =1
τ̂ j· .
(3)
3.2. Randomization-based inference
We now present the main results of this paper.
Proposition 1. τ̂ M is an unbiased estimator of τ , and its covariance matrix is
Cov(τ̂ M ) =
1
2K

22(K −1) r 2 l=1
1
λ′l λl ∆l −
2K (2K − 1)r 2
Σ,
(4)
where
∆l =
1
2K − 1

(N − 1)S (zl ) − 2
2
K
r



2
Ȳj· (zl ) − Ȳ (zl )
j=1
and
Σ=
N
r


(τ i − τ)(τ i − τ)′ − 2K
(τ j· − τ)(τ j· − τ)′ .
i=1
j =1
(l = 1, . . . , 2K ),
102
J. Lu, A. Deng / Statistics and Probability Letters 125 (2017) 99–103
Proof. To prove the first part, note that τ̂ j· is an unbiased estimator of τ j· , for j = 1, . . . , r. This fact combined with (3)
completes the proof.
To prove the second part, let Wj = {Wi (zl )}i∈ψj ,l=1,...,2K denote the treatment assignment for pair j. By definition, Wj ’s are
independently and identically distributed, implying the (joint) independence of τ̂ j· ’s. Consequently, we can treat each pair
as a completely randomized factorial design with 2K units. Therefore by Lemma 1,
2K

1
Cov(τ̂ j· ) =
22(K −1) r 2
λ′l λl
l=1
1
2K
−1

{Yi (zl ) − Ȳj· (zl )}2 −
i∈ψj


1
2K
(
2K
− 1)
r2

(τ i − τ j· )(τ i − τ j· )′ .
i∈ψj

Sj2 (zl )
This implies that
r
1 
Cov(τ̂ M ) =
r 2 j =1
Cov(τ̂ j· )
2K

1
=
22(K −1) r 2
λ′l λl
l =1
r

Sj2 (zl ) −
j =1
1
2K
(
2K
− 1)
r2
r 

(τ i − τ j· )(τ i − τ j· )′ .
(5)
j=1 i∈ψj
To prove the equivalence between (4) and (5), simply note that
(2K − 1)
r

Sj2 (zl ) + 2K
j =1
r

{Ȳj· (zl ) − Ȳ (zl )}2 = (N − 1)S 2 (zl )
j=1
and
N
r 
r



(τ i − τ j· )(τ i − τ j· )′ + 2K
(τ j· − τ)(τ j· − τ)′ =
(τ i − τ)(τ i − τ)′ .
j=1 i∈ψj
j =1
The proof is complete.
i =1
We discuss a special case before moving forward. When K = 1, we have the classic treatment-control studies, and label
the treatment and control as +1 and −1, respectively. We are interested in the difference-in-mean estimator
τ̂MP =
r
1
r j =1
{Yjobs (+1) − Yjobs (−1)}.
Denote ψj = {j1 , j2 }. Imai (2008) (p. 4861, Eq. (8)) derived the variance of τ̂MP as
Var(τ̂MP ) =
r
1 
4r 2 j=1
{Yj1 (+1) − Yj2 (−1) − Yj2 (+1) + Yj1 (−1)}2 .
(6)
As a validity check, Proposition 1 reduces to (6) when K = 1. We leave the proof to the readers.
We discuss the estimation of Cov(τ̂ M ), because Lemma 1 does not apply for matched-pair factorial designs. Inspired by
Imai (2008), we propose the following estimator:
 (τ̂ M ) =
Cov
1
r

(τ̂ j· − τ̂ M )(τ̂ j· − τ̂ M )′ .
r (r − 1) j=1
(7)
Proposition 2. The bias of the covariance estimator in (7) is
 (τ̂ M ) − Cov(τ̂ M ) =
E Cov


1
r

(τ j· − τ)(τ j· − τ)′ .
r (r − 1) j=1
Proof. The proof is a basic maneuver of the expectation and covariance operators. First, by (3) and the joint independence
of τ̂ j· ’s,
Cov(τ̂ M ) = r −2
r

j=1
Cov(τ̂ j· ).
J. Lu, A. Deng / Statistics and Probability Letters 125 (2017) 99–103
103
Therefore by (7),
 (τ̂ M ) =
r (r − 1)E Cov


r

E(τ̂ j· τ̂ j· ) − rE(τ̂ M τ̂ M )
′
′
j =1
=
r

Cov(τ̂ j· ) +
j =1
r

τ j· τ ′j· − rCov(τ̂ M ) − r ττ ′
j=1
= r (r − 1)Cov(τ̂ M ) +
r

(τ j· − τ)(τ j· − τ)′ . j =1
Proposition 2 implies that the estimator of Cov(τ̂ M ) is also ‘‘conservative’’. We leave it to the readers to prove that for
treatment-control studies, Proposition 2 reduces to the corresponding results in Imai (2008) (p. 4862, Prop. 2, Part 1).
4. Discussions and concluding remarks
For treatment-control studies, Imai (2008) compared the variance formulas for the complete-randomization and
matched-pair estimators, and derived the condition under which pair-matching leads to precision gains. For general factorial
designs, analogous comparisons can be made between (2) and (4). However, to our best knowledge, intuitive closed-form
expressions might not be available without additional assumptions on the potential outcomes.
There are multiple future directions based on our current work. First, we may compare the precisions of the completerandomization and matched-pair estimators under certain mild restrictions on the potential outcomes. Second, it is possible
to unify the randomization-based and regression-based inference frameworks, as pointed out by Samii and Aronow (2012)
and Lu (2016b). Third, additional pre-treatment covariates may shed light on the pair-matching mechanism, and help
sharpen our current analysis.
Acknowledgments
The first author thanks Professor Tirthankar Dasgupta at Rutgers University and Professor Peng Ding at University
of California at Berkeley, for their early educations on causal inference and experimental design. The authors thank the
Co-Editor-in-Chief and an anonymous reviewer for their thoughtful comments, which have substantially improved the
presentation of this paper.
References
Cochran, W.G., 1953. Matching in analytical studies. Am. J. Public Health 43, 684–691.
Dasgupta, T., Pillai, N., Rubin, D.B., 2015. Causal inference from 2k factorial designs using the potential outcomes model. J. R. Stat. Soc. Ser. B Stat. Methodol.
77, 727–753.
Ding, P., 2016. A paradox from randomization-based causal inference (with discussion). Statist. Sci., in press. arXiv: 1402.0142.
Fisher, R.A., 1935. The Design of Experiments. Oliver and Boyd, Edinburgh.
Fogarty, C.B., 2016a. Regression assisted inference for the average treatment effect in paired experiments. arXiv:1612.05179.
Fogarty, C.B., 2016b. Sensitivity analysis for the average treatment effect in paired observational studies. arXiv:1609.02112.
Grossarth-Maticek, R., Ziegler, R., 2008. Randomized and non-randomized prospective controlled cohort studies in matched pair design for the long-term
therapy of corpus uteri cancer patients with a mistletoe preparation. Eur. J. Med. Res. 13, 107–120.
Imai, K., 2008. Variance identification and efficiency analysis in randomized experiments under the matched-pair design. Stat. Med. 27, 4857–4873.
Imai, K., King, G., Nall, C., 2009. The essential role of pair matching in cluster-randomized experiments, with application to the mexican universal health
insurance evaluation (with discussion). Statist. Sci. 24, 29–53.
Kempthrone, O., 1952. The Design and Analysis of Experiments. Wiley, New York.
Lu, J., 2016a. Covariate adjustment in randomization-based causal inference for 2k factorial designs. Statist. Probab. Lett. 119, 11–20.
Lu, J., 2016b. On randomization-based and regression-based inferences for 2k factorial designs. Statist. Probab. Lett. 112, 72–78.
Mukerjee, R., Dasgupta, T., Rubin, D.B., 2016. Causal inference in rebuilding and extending the recondite bridge between finite population sampling and
experimental design. arXiv:1606.05279.
Neyman, J.S., 1923. On the application of probability theory to agricultural experiments. essay on principles (with discussion). section 9 (translated).
reprinted ed.. Statist. Sci. 5, 465–472. 1990.
Rosenbaum, P.R., 2002. Observational Studies, second ed. Springer.
Rubin, D.B., 1974. Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 66, 688–701.
Rubin, D.B., 1980. Comment on Randomized analysis of experimental data: The Fisher randomization test by D. Basu. J. Amer. Statist. Assoc. 75, 591–593.
Rubin, D.B., 2008. For objective causal inference, design trumps analysis. Ann. Appl. Stat. 808–840.
Samii, C., Aronow, P.M., 2012. On equivalencies between design-based and regression-based variance estimators for randomized experiments. Statist.
Probab. Lett. 82, 365–370.
Wu, C.F.J., Hamada, M.S., 2009. Experiments: Planning, Analysis, and Optimization. Wiley, New York.
Yates, F., 1937. The Design and Analysis of Factorial Experiments. Technical Communication, Vol. 35, Imperial Bureau of Soil Science, London.