The Effects of Montessori Education: Evidence
from Admission Lotteries
Nienke Ruijs
TIER WORKING PAPER SERIES
TIER WP 14/22
The Effects of Montessori Education: Evidence
from Admission Lotteries
Nienke Ruijs∗
November 4, 2014
Abstract
Little is known about the effects of Montessori education, even though
many students in many countries are educated in Montessori classrooms. This
study investigates the causal effects of Montessori secondary education by exploiting admission lotteries in Dutch Montessori schools. The results indicate
that Montessori education provides an alternative way to attain similar outcomes. Montessori students obtain their secondary school degree without
delay at the same rate and with similar grades as non-Montessori students,
although the route towards the exams is somewhat different. Further, Montessori students show similar levels of motivation and do not score better on various measures of independence, even though these are the main characteristics
Montessori education claims to foster.
∗
The author is affiliated with the University of Amsterdam, the Tinbergen Institute and TIER.
Contact: [email protected]. The author would like to thank school A, school B and DUO for
providing the data for this study. I am indebted to Hessel Oosterbeek and Sjoerd Karsten for
valuable comments and suggestions and help with obtaining the data. Further thanks go to
Thomas Buser, Nadine Ketel, Henriette Maassen van den Brink, Noemi Peter, Erik Plug and
seminar participants at UvA for helpful comments on earlier drafts.
1
1
Introduction
This study investigates the causal effects of Montessori secondary education by exploiting school admission lotteries. Currently, over 22,000 schools in 117 countries
are educating their students using the Montessori pedagogy (Association Montessori Internationale, 2013). Notwithstanding the large number of students involved,
Montessori methods have passionate advocates and strong opponents.
The advocates claim that Montessori education has positive effects on students’
independence and motivation. By providing students with a large amount of choice
in their school tasks, independence would be fostered, while motivation is enhanced
by minimizing external rewards (Lillard, 2005). Critics, on the other hand, argue
that Montessori education can harm academic achievement by a lack of structure
and academic standards (Chattin-McNichols, 1992).
For parents, students and policy makers, it is important to know which claims are
valid. When Montessori education has positive effects on students, schools may be
improved by a broader implementation of Montessori methods. Yet, if Montessori
education negatively affects students, students and parents could better opt for
regular education schools. When there are both positive and negative effects, this
is highly relevant information for parents in order to make an informed decision
about their child’s education. This paper therefore investigates the effects of being
exposed to Montessori education.
Despite its popular character, little is known about the effectiveness of Montessori methods. The reason is that students in Montessori schools cannot be directly
compared to regular education students. Students and parents select themselves
into Montessori education. Parents opting for Montessori schools are, for example, generally of higher socioeconomic status and less often from ethnic minorities
(Dohrmann et al., 2007; Rathunde and Csikszentmihalyi, 2005a).
While this study is the first to investigate the causal effects of Montessori sec-
2
ondary education, there is a handful of empirical studies for primary education.
Lillard and Else-Quest (2006) study students who won and lost an admission lottery for a Montessori primary school in Wisconsin. They find that 5 and 12 year
old Montessori students are doing just as well or better than their non-Montessori
counterparts on cognitive and social measures. In two older studies, Miller et al.
(1975) and Karnes et al. (1970) randomly assigned preschool students to Montessori or non-Montessori Head Start programs, and found no or little initial effects of
Montessori education compared to other preschool programs. In the longer run, one
study finds higher achievement and IQ for Montessori educated boys, but not for
girls (Miller and Bizzell, 1984). The second study finds that Montessori preschoolers
more often graduate from high school and experience less grade retention (Karnes
et al., 1983). Both studies (and their follow-ups), however, implemented compromised versions of Montessori programs in terms of age grouping, teacher training
and day routine and suffer from significant attrition and small samples.
This paper circumvents the selection issues described earlier by exploiting school
admission lotteries. The admission lotteries basically create a series of randomized
experiments: they determine by chance which students are educated at Montessori
schools and thereby create valid treatment and control groups. By using administrative data on academic achievement and questionnaire data on socio-emotional
functioning, this study investigates a wide range of potential outcomes of Montessori
education.
The results show that the academic achievement of Montessori students is akin
to the academic achievement of students in regular secondary education. Montessori
students obtain their secondary school degree without delay at the same rate and
with similar grades as non-Montessori students. The socio-emotional functioning
of Montessori and non-Montessori students turns out to be comparable as well.
Students show similar levels of motivation and Montessori students do not score
better on measures of independence. These results imply that neither the claims of
3
the opponents, nor the claims of the critics are supported by the evidence.
The paper is structured as follows. Section 2 discusses the characteristics of
Montessori education. Section 3 provides additional background information on
secondary education in the Netherlands. Section 4 and 5 describe the data and empirical strategy. Section 6 presents and discusses the results and Section 7 concludes.
2
Montessori education
2.1
Montessori primary education
Montessori education was developed by Maria Montessori, starting with the ’Casa
dei Bambini’ in a poor neighborhood in Rome in 1907, and spreading over the world
ever since (Whitescarver and Cossentino, 2008). As Montessori education is most
renown for primary education, this section starts with a description of Montessori
primary schools.
When walking into a Montessori classroom, there are several features that distinguish it from traditional primary education classes. While students in traditional
education mainly learn from texts, students in Montessori classrooms are often
working with their hands using Montessori materials (Lillard, 2005). Montessori
materials are largely self-correcting educational objects designed to teach subjects
and concepts via repeated use and in a structured sequence. To teach writing skills,
for example, pupils practice the phonetic sounds of letters while tracing letters made
from sandpaper with their fingers. In the same period, they develop the motor skills
needed for writing with drawing tasks using metal frames. Later, the two activities
come together (Lillard, 2005).
Other distinctive features of Montessori classrooms are that classes are organized in three-year-age groupings, such that younger students can learn from older
students (Montessori, 1972). Students are allowed to choose their own activities
4
from a young age onwards (Mooney, 2000) and generally work and get instructed
on their own or in small groups (Whitescarver and Cossentino, 2008). Further,
there is an absence of formal grades and tests.
One of the main goals of Montessori education is to help students to become
independent. Maria Montessori believed that children need to acquire independence
in order to grow and develop (Montessori, 1989b). In early childhood education,
for example, independence is fostered through practical life exercises such as table
washing (Lillard, 2005). Moreover, children in Montessori classrooms can freely
choose their work, which gives them experience in making choices (Montessori,
1989a).
Another aim of the changes in classroom environment is to create an environment where students are intrinsically motivated to learn. The Montessori materials
are designed to sparkle children’s interest. Students actively manipulate the objects
and learn through experience, which is claimed to enhance learning and motivation
(Lillard, 2005). Increasing motivation is also the argument for the absence of formal grades and tests. Traditional schools offer constant feedback on achievement by
providing grades and rewards, thereby replacing intrinsic motivation towards learning by the need for external rewards. Maria Montessori believed that such extrinsic
rewards are not necessary and even disrupt students’ learning (Lillard, 2005).
2.2
Montessori secondary education
While Montessori education is often associated with primary education, it also has
a lively and international secondary education community. An important part of
the history of Montessori secondary education lies in the Netherlands. Just 7 years
after the opening of the ’Casa dei Bambini’, the first Dutch Montessori primary
school opened. As soon as the first students finished primary school, demand for
a Montessori secondary school developed. In 1928, the first Montessori secondary
school in the world was opened in the Netherlands (Calff, 1980). Today, the school
5
is still in existence and it is one of the schools considered in this study.1
Montessori secondary schools adhere to the famous Montessori adagio: “help me
to do it alone”. Similar to primary education, Montessori methods in secondary
education are organized towards fostering independence (Montessori, 1973) and
creating an environment for intrinsic motivation (Rathunde and Csikszentmihalyi,
2005b).
Modern2 Montessori secondary schools differ from regular schools in a number
of ways. Most salient is the amount of choice students are allowed in their schoolwork: part of the hours at school are specifically allocated for free choice of activities (Rathunde and Csikszentmihalyi, 2005a). Students are allowed to choose which
teacher they want to join during these hours. Further, students can choose when
they want to take tests in the lower grades of secondary education3 and Montessori
secondary schools focus more broadly than academic achievement. Field projects,
such as internships, are organized to promote social development and to learn students how to function in society (Seldin and Epstein, 2003; Rubinstein, 2008).
3
Secondary education in the Netherlands
Dutch students and their parents have free school choice, they are not restricted
by catchment areas or school fees. Moreover, virtually all primary and secondary
schools are completely publicly funded. This includes the religious and special
program schools, such as Montessori schools. The government funding is nationally
determined and largely dependent on student numbers. Schools can get additional
1
Nowadays there are 16 Montessori secondary schools in the Netherlands. All schools participate in the Dutch national exams, so the content of the curriculum has large overlap with regular
secondary education. The Dutch Montessori secondary schools conduct visitations amongst each
other to guard the Montessori quality of the schools.
2
As Maria Montessori passed away while developing Montessori methods for adolescents Lillard
(2013), Montessori secondary education is largely developed by the Montessori community. Maria
Montessori’s writings on secondary education include a radical boarding school program called
’Erdkinder’. This program has been implemented very sparsely, and is not the focus of this paper.
3
At some schools, students can also determine the timing of (part of) the tests in the higher
grades of secondary school.
6
Figure 1: The Dutch secondary school system
funding for students from disadvantaged neighborhoods.
Dutch secondary education starts at age 12, and lasts four to six years. The
school system is highly tracked, Figure 1 gives a graphic description. Not all secondary schools offer all school tracks and schools can educate children of different
tracks together. The highest track (pre-university education, vwo) takes six years,
and gives access to university education. The intermediate track (senior general secondary education, havo) takes five years, and gives access to universities of applied
sciences, also called higher professional education. The lowest tracks (pre-vocational
education, vmbo) last four years, and give access to vocational education programs.
Within the pre-vocational track, there are four different levels. Only the highest
pre-vocational level (tl, from hereon referred to as pre-vocational IV) and the senior
general secondary and pre-university tracks are offered at the Montessori schools in
this study.
The initial school track is based on the school track advice of the primary school
teacher and on a standardized high stakes test (in most cases a nationwide exit test
called the “citotoets”) at the end of primary school. Dependent on student achievement and school policies, students can change track during secondary education.
7
Further, students can follow a higher track after finishing a lower track.4 Subject
to some conditions, students can choose their own courses in the second half of secondary education. Secondary schools have to follow national curriculum guidelines,
and all students take centrally determined national exams at the end of secondary
school. The national exams count for 50% of the final grade, the other 50% is determined by school specific exams taken in the last two or three years of secondary
education.
4
Data
4.1
Admission lotteries in Montessori schools
In this paper, data from school admission lotteries in two Dutch Montessori secondary schools are used. The names of the schools are anonymized to school A and
school B. The schools generally receive more applications than they can enroll and
conduct lotteries to allocate the available places. In both schools, the lotteries are
organized for each school year separately and are executed by a notary.
School A conducts school admission lotteries since 2003. The lotteries are divided into a lottery for students of the two highest school tracks (senior general
secondary/pre-university) and for the highest pre-vocational track (pre-vocational
IV). Not all applying students participate in the lottery. Students with a brother or
sister at the school, children from employees and students applying from a Montessori primary school get placed with priority.5 Table 1 shows the number of lottery
participating students and the number of priority students per year and school
4
In that case, students enter the higher track in the year before the final exams. When doing
pre-university education after senior general secondary education, for example, students enter in
year 5 of pre-university education. When graduating at once, they have their pre-university degree
after 7 years of secondary education instead of 6.
5
Since the majority of the lottery participating students subscribed to school A, this implies
that the results are largely identified on a sample of students who did not attend Montessori
primary schools. Section 6.4 describes this issue in more detail. Of all students receiving priority
to school A, 90.2% gets priority because they were educated at a Montessori primary school.
8
Table 1: Distribution of lottery participating and priority students over
years
School year
Lottery winner
Lottery loser
Priority
Total
School A: Senior general secondary/pre-university education
2003/2004
50
46
169
265
2004/2005
16
59
203
278
2005/2006
47
9
196
252
2009/2010
37
42
163
242
2010/2011
29
17
178
224
Total
179
173
909
1261
School A: Pre-vocational IV education
2003/2004
17
45
53
115
2004/2005
10
30
60
100
2007/2008
10
29
62
101
2008/2009
29
3
42
74
2009/2010
5
10
43
58
Total
71
117
260
448
School B: Senior general secondary/pre-university education
2008/2009
52
15
60
127
2009/2010
83
4
16
103
2010/2011
38
9
43
90
2011/2012
56
3
34
93
Total
229
31
153
413
Grand Total
479
321
1322
2122
Note: This table reports the number of lottery participating students and the number of priority
students per school, year and school track.
track. In total, 540 students participated in a school admission lottery for school
A.6
In school B, students from Montessori primary schools do not get priority (brothers and sisters of current students and children of employees do). Instead, the lottery
is based on residence: the school proportionally takes up students from 3 regions.
When there is over-subscription from one region, students from this region partici6
The actual number of students applying to school A for years and school tracks that conducted
lotteries is slightly higher: 1759 instead of 1709. However, 44 students (2.5%) could not be merged
at the Dutch educational administration. All of these students lost the lottery, mainly in the
earlier years, but there is no selection on gender, citoscore and school advice. Other background
characteristics are not available for these students. Further, 6 students (0.3%) are dropped because
of missing information on ethnicity or living in a disadvantaged neighborhood.
9
pate in a lottery. School B does not offer the pre-vocational IV track. For school B,
260 students participated in a school admission lottery, bringing the total number
of lottery participating students to 800.7
Since the effects of Montessori education are studied by comparing students
winning admission lotteries to students who lose the lottery and therefore attend
another school, it is important to know the characteristics of the alternative schools.
The majority of the lottery losing students (80.1%) are enrolled in schools with
regular education programs. 8.5%8 still attend Montessori schools and 11.4% attend
a Dalton school, a school with a pedagogy comparable to Montessori. With respect
to school tracks, most lottery losing students are enrolled in schools similar to
school A and school B. 0.63% of the lottery losing students attend schools that only
offer the pre-university school track, while 22.8% go to schools that also offer the
lower pre-vocational education tracks or attend schools that do not offer the highest
track(s).
Table 2 describes differences in school characteristics between school A and
school B and their alternative schools attended by lottery losers in the first year
of secondary education. In general, the standard deviations of the average school
characteristics of the alternative schools are large, especially for school A, indicating
that lottery losing students attend a wide range of alternative schools. For most
characteristics, the alternative schools are significantly different from school A and
B. The first seven rows of Table 2 pertain to school quality measures of the Dutch
education inspectorate. The patterns indicate that the Montessori schools have less
delay in the higher grades and higher average exam scores, although grade progression in the lower grades is lower. It is important to note that these differences
7
202 students from regions without lotteries in a certain year are dropped from the sample.
Comparable students at school A, in years for which a certain school track did not have a lottery,
are not available for analysis. 1 student (0.2%) could not be merged at the Dutch educational
administration. 20 students (4.6%, of whom 13 participated in the lottery) are dropped from the
sample because of missing information on primary school advice.
8
17 losing students (5.4%) are admitted to School A or school B from a waiting list, details are
in Section 5. The other 10 students (3.2%) attend another Montessori secondary school.
10
cannot be interpreted as effects of Montessori education, as they reflect average
school characteristics that are potentially created by differences in school populations. The four bottom rows report more general school characteristics. It turns out
that school A is significantly larger and school B is significantly smaller than their
alternative schools. With respect to the percentage of students from disadvantaged
neighborhoods, school A has more students from disadvantaged neighborhoods in
the pre-vocational III/IV and pre-university tracks, but fewer in the senior general
secondary track. There are no significant differences in the percentage of students
from disadvantaged neighborhoods for school B compared to the alternative schools
attended by its lottery losers.
4.2
9
Data on academic achievement
The information on students’ academic achievement is provided by the Dutch educational administration (Dienst Uitvoering Onderwijs, DUO). The most recent data
available are exam scores of the exams administered in May 2012 and educational
positions in October 2012. Students’ final exam results are used to create an indicator that equals one if the student obtained the degree of the advised school
track (or higher) on time. Otherwise the indicator equals zero. Simple exam grades
are not a good outcome measure, since students can change school track during
secondary school and exams are made for each school track separately. When a
student changes to a lower track, it is likely that his grades will be higher than they
would have been at the higher school track. Therefore, the school track advice of
the primary school teacher is used as the school track the student is expected to
obtain by the end of secondary school. This advice is compared to the obtained
9
The high percentage of students from disadvantaged neighborhoods is somewhat misleading
for school A and its alternatives. It is based on a policy taking into account the characteristics
of students’ residential neighborhoods. In the city of school A, this caused some peculiarities in
which ’elite’ schools were suddenly identified as schools with many disadvantaged students. The
old policy based on individual student characteristics was still in effect in the first years considered
in this study. According to that definition, the percentage of disadvantaged students was nearly
zero for school A, and around 4% for its alternatives.
11
Table 2: Characteristics of alternative schools attended by lottery losers
School A
Index grade progression lower grades
School B
Mean
Mean (SD)
Mean
Mean (SD)
school A
alternatives
school B
alternatives
(1)
(2)
(3)
(4)
94.07
100.96***
99.03
101.08**
(13.79)
% no delay pre-voc. III/IV
81.19
(5.05)
74.70***
(10.68)
% no delay senior gen.
67.55
45.65***
75.26
(13.47)
% no delay pre-uni.
67.57
51.06***
(4.91)
60.66
(15.07)
Mean grade pre-voc. III/IV
6.36
62.19***
68.27***
(9.01)
6.13***
(0.25)
Mean grade senior gen.
6.44
6.03***
6.52
(0.26)
Mean grade pre-uni.
6.57
6.08***
(0.13)
6.62
(0.33)
Number of students
1579.97
823.49***
61.83
6.35***
(0.15)
788.02
(304.67)
% disadvantaged pre-voc. III/IV
6.16***
1079.96***
(301.03)
33.43***
(39.18)
% disadvantaged senior gen.
53.15
60.28**
7.31
(24.74)
% disadvantaged pre-uni.
56.21
50.49**
6.97
(14.34)
3.49
4.73
(21.51)
(12.40)
Note: Columns (1) and (3) report school characteristics of school A and school B. Columns (2)
and (4) report the means and standard deviations of each characteristic for the alternative schools
attended by lottery losers in the first year of secondary education for school A and B separately.
The information in the table is derived from the Quality Cards 2002-2010 of the Dutch education
inspectorate. The education inspectorate publishes secondary school information on a yearly
basis. The measures published in the year of school choice are used for each lottery participating
student. This creates an average weighted by the number of winning students in each year for the
Montessori schools. For the alternative schools, this creates an average weighted by the number
of attending lottery losers in each year. The first seven rows describe school quality measures.
The index for grade progression in the lower grades is a measure of the percentage of students
getting to the third grade without grade retention, correcting for students who attend higher or
lower school tracks than their primary school advice. The other measures pertain to the higher
grades, and report on the percentage of students passing through the higher grades without delay
and the average grade on the final exams. The stars refer to the p-values of one-sample t-tests
testing whether the means of the alternative schools equal the means for school A or school B,
where the means of school A and B are the constants. ***p<0.01 **p<0.05 *p<0.10.
12
degree. When a student gets a degree from a lower track or when he is grade retained, he is coded as not obtaining his degree on time. When a student follows the
advised or a higher school track and obtains his degree without delay, he is counted
as obtaining his degree on time.10 For students with combined school track advices
(e.g. senior general secondary/pre-university education) delays or lower degrees are
counted from the lowest of the two tracks.11
To study whether Montessori education has an effect on gaining high grades,
an indicator variable for obtaining a degree on time with at least a grade point
average of 7.5 (on a 10 point scale) is used. Further, indicators for obtaining a
degree on time with at least a passing grade for Dutch, English and mathematics
are constructed.12
Since the youngest cohort of students started secondary education in 2011/2012,
exam outcomes are only available for 47.9% of the lottery participating students
in the sample. To investigate academic outcomes for a larger group of students, I
consider students’ educational positions measured four years after participating in
an admission lottery. The fourth year is chosen because students are clearly tracked
by the time they are in 4th grade.13 In lower grades, students can be educated in
combined classes, such as senior general secondary/pre-university education.
Two different outcome measures are used for the fourth year. The first one is
10
It can also be that a student gets a higher degree with some delay. In that case, he is coded
as obtaining his degree on time when he gets the higher degree at the same time as he could have
gotten that degree if he first followed the school track of his primary school advice. For example,
a student with a senior general secondary education advice is considered to obtain his degree on
time if he passes his pre-university exams in seven years.
11
Counting from the highest of the two tracks does not change the results for the exam measures.
For the 4th year, there is a slight difference, indicating that students at Montessori schools are
more often at the lower track of the combined advice in 4th grade.
12
To ensure that grades are comparable across schools, the national exam grades are used. A
grade of 5.5 or higher (on a 10 point scale) is considered a passing grade. It is possible for Dutch
students to fail courses and still obtain their degree. Dutch secondary education has different
levels of mathematics within the same school track. This distinction is not taken into account in
the mathematics variable. Alternative specifications (computing the indicator for each math level
separately and correcting math grades for difficulty following Leuven et al. (2010)) yield similar
results.
13
In this paper, 4th grade refers to the 4th grade of Dutch secondary education, in which
students are approximately 16 years old.
13
whether a student is in 4th grade at the advised school track or higher after four
years of secondary education. Basically, this measure captures the students who
are on track to obtain their degree on time: grade retained students are coded as
zeros. Since parents and students might care more about the school track a student
eventually obtains instead of whether he is grade retained or not, grade retainers
are not taken into account in the second measure. This measure indicates whether
students who are in 4th grade after four years are educated at their advised school
track or higher.
As described in Section 3, students can choose a course program in the second
half of secondary education. The science program is considered the more difficult
program in the senior general secondary and pre-university school tracks (Buser
et al., 2014). Since (not) choosing a science program strongly affects the options
for tertiary education, it is interesting to see whether Montessori students are more
or less often enrolled in the science program in the higher grades of the academic
school tracks.
4.3
Questionnaire on socio-emotional functioning
To measure the effects of Montessori education on socio-emotional functioning, a
questionnaire was used. The content of the questionnaire is described in Table 3.
The table includes information on the instruments used to measure each outcome,
the reliability of these instruments and sample questions.
As independence is one of the main characteristics that Montessori education
is claimed to foster, the questionnaire includes two measures of independence. Independence at school measures the degree to which students are able to effectively
work on school tasks with classmates or on their own. As no existing instrument
was available, it was developed for this study. The scale includes questions on taking initiative, working together and evaluating the learning process. Independence
out of school focusses more broadly on functioning as an autonomous individual.
14
15
Developed for this study
Interest in society
iii
7
16
8
23
22
questions
Number of
I consider it very important to vote at elections
I get along well with most of my teachers
I would like to learn a lot at school
I think about how my actions will affect others
an answer myself, before turning to the teacher
If there is something I don’t understand, I first look for
Sample question
0.58
0.84
0.83
0.75
0.68
in sample
Cronbach’s α
Parental support
Developed for this study
3
My parents often help me with my homework
0.57
Note: Column 5 reports Cronbach’s α statistics which are computed using the students who responded to the questionnaire. Cronbach’s α is a measure
for reliability. It is a coefficient of internal consistency, indicating the extent to which items on the same scale are measuring one common
characteristic.i Some items adapted from L&S (Vorst, 1993), ii Beckert (2007), iii Anderson et al. (1994), iv Smits (2008).
SVL 2008: PS, RL
Well-being at school
iv
SVL 2008: LG
iv
CASE: ET, CV , WAS: BA
Motivation towards learning tasks
Independence out of school
ii
Developed for this study
Independence at school
i
Instrument
Outcome
Table 3: Content and reliability of the questionnaire
It includes sub-scales from existing questionnaires on evaluative thinking, need for
confirmation (comparative validation) and behavioral autonomy.
To study whether Montessori students are indeed more motivated for school, a
scale on motivation towards learning tasks is included. To get a broader view on
students’ socio-emotional functioning, the questionnaire also includes questions on
well-being at school, interest in society and on problematic behavior like truancy
and use of alcohol and drugs. Further, the questionnaire includes items on parental
support to school matters and participating in commercial school support to investigate whether selective parental compensation diminishes potential differences in
academic achievement.
For budgetary reasons, a sample of 910 students was invited to fill out the
questionnaire. The details of the sampling procedure are described in Appendix
A. The students received a letter on university stationery, explaining that they
applied for a Montessori secondary school some time ago and that the University
of Amsterdam is curious to see how their secondary school time is going or went.
The letter contained a reference to a website, a personal login code and password,
and students received a reward of €10 when they completed the questionnaire.
609 students (67%), including 415 lottery participators, responded to the questionnaire. Table A2 in Appendix A describes the characteristics of students who
did and did not respond to the questionnaire. It turns out that nonrespondents are
older, more often lottery losers and have lower school advices. Further, they are
more often males and non-western migrants. While it is unfortunate that there is
selective nonresponse, the pattern of nonresponse is not uncommon (e.g. Van Loon
et al., 2003; Søgaard et al., 2004). Moreover, the balancing tests reported in Table
4 indicate that nonresponse does not create differences between lottery winners and
lottery losers in terms of background characteristics.
16
5
Empirical strategy
The effects of Montessori education are hard to identify, since the choice for a
Montessori school is endogenous. Different parents choose for Montessori and regular education, and parental characteristics determining Montessori choice are likely
to be related to student outcomes. The admission lotteries to school A and B,
however, basically create a series of randomized experiments on Montessori school
attendance. To investigate the effects of Montessori education, lottery participating
students attending and not attending school A and B are compared using:
0
yil = Xil β + γM ON Til + νl + εil
(1)
where yil indicates outcomes y of student i participating in lottery l. Xil0 is a
vector of student characteristics added for precision and includes gender, being a
nonwestern or western migrant, student age, living in a disadvantaged neighborhood, cito testscore, making another test than cito, no information on primary
school test-score and primary school advice. ν l is a fixed effect for lottery l, capturing the different lotteries in different years and schools. When a school has multiple
lotteries in a single year (for different school tracks or regions), a lottery fixed effect
is included for each lottery. The lottery fixed effects make sure that students within
the same lottery are compared, instead of comparing students in lotteries with potentially slightly different characteristics. Since the lotteries are year and school
specific, there are no year or school fixed effects. εil is the error term. M ON Til
is an indicator variable for ever attending Montessori school A or B, making γ the
parameter of interest.
Even in the sample of lottery participating students, ever attending a Montessori
school may be endogenous. Over time, some students who initially lose the lottery
are offered a place on the Montessori schools. To be specific, 43 students are present
17
at school A or B in some point of time after they lost the lottery, of whom 17 attend
the schools from the first year onwards, the always-takers.14 These students were
on the waiting list, and were placed when a slot came free. Not all students on the
waiting list decide to go to the Montessori schools when a slot comes free, some
decide to stay at their alternative secondary school. This choice may depend on
student characteristics, making the indicator for ever attending a Montessori school
potentially endogenous, which would bias γ. Therefore, an instrumental variables
framework is used, using the outcome of the lottery as an instrument for ever
attending a Montessori school. The first stage is defined as:
M ON Til = Xil0 π + λW IN il + υl + ηil
(2)
where W IN il is an indicator for whether student i is a lottery winner. In this
framework, the instrument, W IN il , should be both relevant and valid. For the
first condition, winning the lottery should predict Montessori school attendance.
Appendix B shows first stage results confirming that the lottery is indeed a relevant
instrument. For the instrument to be valid, it should be uncorrelated with the
error term εil . It is assumed that winning the lottery has no effect on student
outcomes except from the effect via Montessori school attendance. This implies, for
example, that disappointment about losing the lottery should not lead to a decrease
in motivation for school. Further, the outcome of the lottery should be randomly
determined, which is very likely since the lotteries are executed by a notary.
To show that the lotteries are indeed conducted fairly, Table 4 shows balancing
tests testing whether there are differences in observable characteristics between lottery winners and lottery losers. Columns 1 and 2 show the means and standard deviations for students losing and winning a school admission lottery. Columns 3 and 4
14
Reversely, no never-takers are found: all students who won the lottery are present at the
Montessori schools for at least one year.
18
show the actual balancing tests, regressing the dependent variables denoted in each
row on an indicator for winning the school admission lottery. The balancing tests
show that for nearly all characteristics, there are no significant differences between
winning and losing students. The exception is age: winning students are somewhat
older than losing students, although the coefficient is small. Table 4 reports separate
regression coefficients to facilitate interpretation. To test whether the coefficients
are jointly significantly different from zero, a joint balancing test was conducted.
Regressing the indicator for winning the lottery on all background characteristics, it
turns out that the null hypothesis cannot be rejected (F(10,774)=0.781, p=0.647),
supporting the conclusion that the lotteries were indeed conducted fairly.
Since not every lottery participating student participated in the questionnaire,
columns 5 and 6 show similar balancing tests for winning and losing students who
responded to the questionnaire. The results are comparable to the results of the previous balancing tests15 , indicating that winning and losing students who responded
to the questionnaire are similar in terms of background characteristics.
6
Results
6.1
The effects of Montessori education on academic outcomes
The results in Table 5 indicate that there is little impact of Montessori education
on students’ academic achievement. The first row shows that after four years of
secondary education, 75.2% of the students is in 4th grade of the advised track or
better. On this outcome, there is no significant difference between Montessori and
non-Montessori students. When a student is not in the advised track in 4th grade
15
The null hypothesis of the joint balancing test cannot be rejected either (F(10,389)=1.001,
p=0.441).
19
20
(Table continues on next page)
Western migrant
Non-western migrant
Dutch
Boy
primary school advice
Pre-uni.
primary school advice
Senior gen./pre-uni.
primary school advice
Senior gen.
primary school advice
Pre-voc. IV/senior gen.
primary school advice
Pre-voc. IV
Citoscore
variable
Dependent
0.102
(0.303)
(0.308)
(0.338)
(0.407)
0.106
0.132
(0.424)
(0.465)
0.209
0.766
(0.483)
(0.465)
0.685
0.367
(0.452)
0.286
(0.462)
0.307
(0.445)
0.271
(0.223)
0.052
(0.277)
0.315
(0.405)
0.206
(0.418)
0.224
(0.422)
0.231
(0.312)
0.109
(0.422)
0.084
(4.380)
(5.229)
0.231
541.517
(2)
(1)
540.438
lottery winners
lottery losers
Mean (SD) for
(0.026)
0.019
(0.030)
-0.003
(0.037)
-0.016
(0.042)
0.028
(0.034)
-0.012
(0.036)
0.008
(0.038)
0.005
(0.019)
-0.011
(0.020)
0.010
(0.293)
0.240
(3)
tests lottery
Balancing
0.450
0.914
0.662
0.504
0.730
0.825
0.900
0.567
0.620
0.413
(4)
P-values
Table 4: Descriptive statistics and lottery balance
Mean (SD) for
(0.031)
0.008
(0.037)
0.047
(0.045)
-0.055
(0.052)
0.016
(0.044)
-0.026
(0.045)
-0.032
(0.047)
0.067
(0.024)
-0.015
(0.025)
0.006
(0.376)
0.217
(5)
respondents
questionnaire
lottery for
Balancing tests
0.802
0.200
0.218
0.756
0.564
0.482
0.152
0.527
0.805
0.565
(6)
P-values
21
(0.499)
0.542
(0.446)
(0.465)
0.315
(0.415)
12.109
(2)
(1)
12.087
lottery winners
Mean (SD) for
lottery losers
Mean (SD) for
(0.039)
-0.023
(0.040)
0.079
(3)
tests lottery
Balancing
0.559
0.047
(4)
P-values
(0.049)
-0.020
(0.048)
0.082
(5)
respondents
questionnaire
lottery for
Balancing tests
0.683
0.085
(6)
P-values
Number of students
321
479
800
415
Note: Columns (1) and (2) display the means and standard deviations for students losing and winning a lottery for a Montessori school. Columns (3) and
(4) report separate regression coefficients and the p-values of the variables indicated in each row on an indicator variable equalling 0 if the student lost the
lottery and equalling 1 if the student won the lottery. The sample is restricted to students participating on the lottery. Columns (5) and (6) report similar
regression coefficients for the sample of lottery participating students who responded to the questionnaire. All regressions include lottery fixed effects.
Robust standard errors are reported in parentheses.
neighborhood
Disadvantaged
Age at application
variable
Dependent
Table 4: (continued)
after four years of secondary education, this can have two different causes: it can
be that the student is grade retained, or that he has moved to a lower school track.
The second line of results indicates that Montessori and non-Montessori students
differ in these causes: when grade retained students are dropped from the sample, it
turns out that Montessori students who are in 4th grade after four years more often
attend a lower school track than can be expected from their primary school advice.
This indicates that students at the Montessori schools are more often sent to a lower
school track, while similar students in traditional schools are grade retained.16
More important than fourth year outcomes are the final exam results. It turns
out that Montessori students obtain the degree of their advised school track on time
at the same rate as non-Montessori students, which matches the results on the first
row of Table 5. Since Montessori students are more often educated at lower school
tracks in 4th grade, one might expect fewer Montessori students to obtain the degree
of their advised school track with one year delay: students at a lower track may
finalize their secondary education at that lower school track, while grade retainers
obtain the degree of their initial track with one year delay. This conjecture is not
seconded by the results: Montessori education has no significant effect on obtaining
the degree of the advised track with one year delay. It might be that Montessori
students catch up by doing the higher school track after their first exams, which
also yields a degree with one year delay.
With respect to grades, it turns out that Montessori education has no significant
impact on obtaining a degree with high average grades, nor on obtaining a degree
with passing grades for Dutch, English and Math. Further, Montessori education
16
Both Montessori schools indeed have an avoiding policy towards grade retention. Another
potential explanation is that students who lose the lottery go to schools that only offer higher
school tracks. To avoid changing schools, students at these schools might prefer grade retention
over moving down a track. Indeed, 41.8% of the lottery losing students at school A attend a
school that does not offer the pre-vocational IV school track. School B, however, only offers the
two highest tracks, and 77.4% of their lottery losing students attend schools with the same school
tracks, while 22.6% attend schools that also offer lower, pre-vocational, tracks, implying that this
mechanism should not operate in school B. As shown in Table C1, the fourth year outcomes are not
significantly different for school A and school B, suggesting that school policies are more relevant
than avoiding to change schools.
22
Table 5: Results on academic outcomes
Outcome
In 4th grade of advised track or better
after four years
Mean (SD)
N
(1)
(2)
0.752
625
-0.049
-0.022
(0.048)
(0.047)
-0.079**
-0.080**
(0.038)
(0.037)
-0.049
0.000
(0.064)
(0.060)
-0.017
0.008
(0.058)
(0.057)
-0.035
-0.036
(0.030)
(0.030)
-0.047
-0.015
(0.069)
(0.068)
0.022
0.057
(0.069)
(0.068)
-0.035
0.029
(0.078)
(0.075)
-0.069
-0.030
(0.053)
(0.052)
0.441
0.406
(0.302)
(0.308)
0.115*
0.112*
(0.064)
(0.066)
(0.432)
In 4th grade of advised track or better
0.895
after four years (cond. on being in 4th grade)
Obtained degree on time
(0.307)
0.671
on advised track or higher
525
383
(0.470)
Obtained degree on advised track or higher
0.777
with at most one year delay
376
(0.417)
Obtained degree on time on advised track
0.047
with at least a 7.5 on average
381
(0.212)
Obtained degree on time on advised track
0.517
with passing grade for Dutch
377
(0.500)
Obtained degree on time on advised track
0.561
with passing grade for English
378
(0.497)
Obtained degree on time on advised track
0.500
with passing grade for Math
308
(0.501)
Enrolled in science program
0.184
483
(0.388)
Parental support
8.664
408
(2.406)
Participated in commercial
0.491
school support
(0.501)
407
!
Controls
Note: Each row reports two IV regressions with winning the lottery as an IV for ever attending
a Montessori school. The sample is restricted to students participating on the lottery. The
first column displays the means and standard deviations for these lottery participating students.
Columns (1) and (2) report IV estimates without and with controls. Robust standard errors are
reported in parentheses. Alternative specifications of the standard errors (e.g. clustering on school
attended in the first year or clustering on school attended in the first year*school advise*year)
yield similar results. All regressions include lottery fixed effects. Controls include gender, being a
nonwestern or western migrant, student age, living in a disadvantaged neighborhood, cito testscore,
making another test than cito, no information on primary school test-score and primary school
advice. Exams are graded on a 10-point scale, where grades above 5.5 are considered a pass.
***p<0.01 **p<0.05 *p<0.10.
23
does not affect the probability of being enrolled in a science program.
It is conceivable that the limited differences in academic outcomes in either
direction are created by selective parental compensation. Parents might send badly
achieving students to commercial school support activities (such as private tutoring)
or decide to invest more time themselves, while they do not interfere when a student
is doing well. The results in Table 5 show that there is no significant difference in
the amount of parental support between Montessori and non-Montessori students.
Montessori students do, however, receive more commercial school support, although
the coefficient is only significant at the 10% level.
An alternative explanation is that the coefficients are imprecisely estimated.
Looking at the signs of the coefficients, it should be noted that all coefficients on
additional support are positive, while 13 out of the 18 coefficients for the academic
outcomes are negative. Since the standard errors are rather large, the possibility of
substantial negative effects cannot be excluded.
Overall, the results indicate that Montessori students achieve just as well as
students attending regular secondary education.
6.2
The effects of Montessori education on socio-emotional
outcomes
While the previous section focussed on academic achievement, the results in Table
6 focus on socio-emotional functioning. Even though Montessori education aims to
improve students’ independence and motivation, the results show no positive effects
of Montessori education on these outcomes. There are no significant differences between Montessori and non-Montessori students on the measures of independence
at school and motivation towards learning tasks. Contrary to the expectations,
Montessori education has a negative impact on independence out of school. Exploring the sub-scales (results not shown), it turns out that this result is mainly caused
by the sub-scale of evaluative thinking. Evaluative thinking measures the extent to
24
which students think about their actions and decisions before acting.
On the other measures of socio-emotional functioning, there are few significant effects of Montessori education. The exception is well-being at school, which
indicates that Montessori students enjoy their school better and have a better relationship with their teachers. Montessori education does not significantly affect
problematic behavior nor students’ interest in society.
Including the academic outcomes, 20 different variables were tested in the analyses above. At a 5% significance level, you would expect one variable to turn out
significantly by chance: a Type I error. When applying the Bonferroni and the
Bonferroni-Holm method to control for multiple testing, none of the null hypotheses can be rejected. Even though the Bonferroni and Bonferroni-Holm methods
are very conservative, the results should be interpreted with caution. The main
message to take out of the results is that the outcomes of Montessori students are
similar to the outcomes of non-Montessori students.
6.3
Heterogeneous effects
The results presented so far are estimates based on two Montessori secondary
schools. Although both schools are Montessori schools, it might be that one school
attains better outcomes than the other. Table C1 in Appendix C shows heterogeneous effects for school A and school B on the main outcomes considered in this
study.17 It turns out that the effects on academic achievement after four years
of secondary education are not significantly different between the two schools.18
The finding that Montessori students more often participate in commercial school
support, however, is driven by students at school B.
17
The interaction effects for the other outcome measures (on grades and problematic behavior)
are insignificant for school A and school B and for boys and girls. For the different school tracks, it
turns out that compared to senior general secondary/pre-university students, the pre-vocational
IV students in Montessori education more often obtain a passing grade on English and Math.
Further, these students are less often late, truant or sent out of class.
18
Differential effects on exam measures are not available because the students participating in
admission lotteries for school B are too young for secondary school exams.
25
Table 6: Results on socio-emotional outcomes
Outcome
Mean (SD)
N
(1)
(2)
73.520
408
-0.008
0.095
(0.917)
(0.938)
-2.253**
-1.923**
(0.975)
(0.974)
-0.369
-0.269
(0.552)
(0.555)
1.302
1.462*
(0.823)
(0.830)
0.770
0.660
(0.484)
(0.506)
-3.022
-5.833
(4.019)
(4.297)
0.084
0.082
(0.052)
(0.054)
0.048
0.014
(0.057)
(0.058)
0.041
0.024
(0.054)
(0.051)
Independence at school
(7.619)
Independence out of school
83.475
408
(7.936)
Motivation towards learning tasks
17.895
409
(4.494)
Well-being at school
37.875
409
(6.570)
Interest in society
23.627
408
(3.994)
Number of times being late, truant or sent
21.593
out of class in the past 6 months
407
(39.622)
Smoked cigarettes or drank alcohol
0.597
in the past month
407
(0.491)
Used drugs of any kind
0.373
in the past 12 months
407
(0.484)
Ever got a fine or was suspected of a
0.270
criminal offense by the police
(0.445)
407
!
Controls
Note: Each row reports two IV regressions with winning the lottery as an IV for ever attending
a Montessori school. The sample is restricted to students participating on the lottery. The
first column displays the means and standard deviations for these lottery participating students.
Columns (1) and (2) report IV estimates without and with controls. Robust standard errors are
reported in parentheses. Alternative specifications of the standard errors (e.g. clustering on school
attended in the first year or clustering on school attended in the first year*school advise*year)
yield similar results. All regressions include lottery fixed effects. Controls include gender, being a
nonwestern or western migrant, student age, living in a disadvantaged neighborhood, cito testscore,
making another test than cito, no information on primary school test-score and primary school
advice. ***p<0.01 **p<0.05 *p<0.10.
26
With respect to socio-emotional functioning, independence at school has the
only significant interaction effect. It indicates that students educated at school B
score significantly lower on independence at school than their non-Montessori counterparts while no such effect is found for students participating in school admission
lotteries for school A. Since 12 out of 14 interaction effects are insignificant, the
general conclusion is that the effects of Montessori education are similar for school
A and school B.
Next to differences between schools, there might be differential effects of Montessori education for different groups of students. For example, it could be that girls
do better at Montessori schools, since girls are generally more able to work independently than boys. For the same reason, it might also be that students in the
higher school tracks do better in Montessori education than students in the lower
school tracks.
Tables C2 and C3 of Appendix C show additional differential effects of Montessori education for boys and girls and for different school tracks. Regarding gender,
Table C2 shows that none of the interaction effects is significant at the 5% level.
Therefore, the null-hypothesis of equal effects of Montessori education for boys and
girls cannot be rejected, which implies that the effects of Montessori education are
similar for boys and girls.
For the different school tracks, some of the interaction effects are significant.
The effects indicate that Montessori education has a more positive impact on prevocational IV students than on students at the academic tracks. Compared to senior
general secondary/pre-university students, pre-vocational IV students more often
obtain their degree on time on the advised track or higher. The main effects on this
variable are not significant. Further, the positive effect of Montessori education on
well-being at school seems to be driven by pre-vocational IV students.
27
6.4
Difference between lottery participants and students with
priority
As mentioned in Section 4.1, not every student attending the Montessori schools
participated in a lottery. Both schools yield priority to students having brothers
or sisters at the school and to children of employees. Moreover, school B conducts
lotteries based on students’ residence, while school A yields priority to students
from Montessori primary schools.
It might be that students who got placed with priority are different from lottery
participating students. Table 7 shows a-priori differences between students who
participated in the lottery and students who got placed with priority. The table
shows that on most background characteristics, there are no differences between
lottery participators and students with priority. The differences that are found
indicate that compared to students with priority, students who participate in the
lottery are less often boy and more often have a non-western migrant background.
More important than background characteristics is the fact that school A yields
priority to students from Montessori primary schools. As the majority of the lottery
participating students subscribed for school A, the effects of Montessori education
are mainly identified on a sample of students who did not attend Montessori primary schools. It is possible that the effects of Montessori secondary education are
more positive for students who were educated in Montessori primary schools: students from Montessori primary schools will have more experience with independent
working and choice in their school tasks.
Table 8 reports differences in the main outcomes between Montessori students
who won the lottery and students who were placed with priority. It focusses on
winning students as students who lost the lottery are generally educated at regular
education schools. Although the results are descriptive, the results in the table
indicate that the outcomes of students with priority are not significantly different
28
Table 7: Differences in background characteristics between students participating on the lottery and students being placed with priority
Dependent
variable
Citoscore
Pre-voc. IV
primary school advice
Pre-voc. IV/senior gen.
primary school advice
Senior gen.
primary school advice
Senior gen./pre-uni.
primary school advice
Pre-uni.
primary school advice
Boy
Dutch
Non-western migrant
Western migrant
Age at application
Disadvantaged
neighborhood
Mean (SD) for
Balancing test
students with priority
lottery participation
(1)
(2)
(3)
541.0
0.128
0.497
(5.421)
(0.189)
0.107
0.012
(0.310)
(0.012)
0.082
-0.019
(0.274)
(0.012)
0.248
0.031
(0.432)
(0.021)
0.280
0.011
(0.449)
(0.020)
0.283
-0.034
(0.451)
(0.020)
0.408
-0.081
(0.492)
(0.024)
0.744
-0.029
(0.437)
(0.022)
0.132
0.039
(0.338)
(0.018)
0.125
-0.010
(0.331)
(0.016)
12.086
-0.003
(0.439)
(0.021)
0.501
0.018
(0.500)
(0.023)
P-values
0.328
0.094
0.127
0.607
0.082
0.001
0.184
0.031
0.502
0.900
0.432
Number of students
1322
2122
Note: Column (1) displays the means and standard deviations for students who got placed with
priority. Columns (2) and (3) report separate regression coefficients and the p-values of the
dependent variables indicated in each row on an indicator variable equalling 0 if the student was
placed with priority and equalling 1 if the student participated in the lottery. The regressions are
on the full sample, including all students who participated in the lottery and all students who
got placed with priority. All regressions include lottery fixed effects. Robust standard errors are
reported in parentheses.
29
from the outcomes of students who won a school admission lottery.19
As the priority rules for school A are likely to create a larger difference between
priority and non-priority students, it is interesting to note that the results are similar
when only focussing on students at school A (results not shown). This implies that
Montessori students who were educated at regular primary schools achieve similar to
their secondary education peers who were educated at Montessori primary schools.
Since students from Montessori primary schools do not get priority in school B,
further support on this observation is provided by Table C1. This table shows that
the effects of Montessori education are similar for students in school A and school
B.
Overall, the results in this subsection indicate that it is unlikely that the results
on the effects of Montessori secondary education are driven by the priority rules.
7
Conclusions
In this study, the causal effects of Montessori education are investigated by exploiting school admission lotteries for two Montessori secondary schools. It is found that
Montessori education has little impact on students’ academic achievement. Montessori students obtain their secondary school degree without delay at the same rate
and with similar grades as non-Montessori students, although the route towards the
exams is somewhat different. After four years of secondary education, Montessori
students more often attend a lower school track while non-Montessori students are
more often grade-retained.
On most socio-emotional outcomes, there is no significant effect of Montessori
education. Most notably, there are no positive effects of Montessori education on
independence and motivation, although these are the main characteristics Montessori education claims to foster. Regarding independence out of school, Montessori
19
The results for the other outcome measures (on grades and problematic behavior) are insignificant as well.
30
Table 8: Differences in outcomes between students winning the lottery
and students being placed with priority
Outcome
(1)
Achievement
In 4th grade of advised track or better after four years
-0.050*
(0.029)
In 4th grade of advised track or better after four years (cond. on being in 4th grade)
-0.029
(0.024)
Obtained degree on time on advised track or higher
-0.031
(0.039)
Obtained degree on advised track or higher with at most one year delay
-0.027
(0.034)
Enrolled in science program
0.020
(0.028)
Parental support
0.263
(0.257)
Participated in commercial school support
0.064
(0.054)
Socio-emotional functioning
Independence at school
-0.992
(0.856)
Independence out of school
-0.195
(0.855)
Motivation towards learning tasks
-0.531
(0.474)
Well-being at school
-0.123
(0.704)
Interest in society
-0.164
(0.441)
Note: Each coefficient reports an OLS regression regressing the dependent variable in that row
on an indicator equalling 1 if the student participated in the lottery. The regressions include the
students who got placed with priority and the students who won the lottery for the Montessori
school. The number of students varies from 807 to 1381 for the academic outcomes and from 401
to 403 on the variables measured by the questionnaire. Robust standard errors are reported in
parentheses. All regressions include lottery fixed effects and student controls. Controls include
gender, being a nonwestern or western migrant, student age, living in a disadvantaged neighborhood, cito testscore, making another test than cito, no information on primary school test-score
and primary school advice. Exams are graded on a 10-point scale, where grades above 5.5 are
considered a pass. ***p<0.01 **p<0.05 *p<0.10.
31
students even score worse than their non-Montessori counterparts.
As the results in this study are based on two Montessori schools, one might
worry that the results cannot be generalized to other Montessori secondary schools.
It is reassuring that both schools in this study yield similar outcomes. Still, the
results are based on Dutch Montessori secondary schools that conduct school admission lotteries. Since the demand for these schools is high, it might be that
these are especially high quality Montessori programs. In that case, the results
can be interpreted as the effects of Montessori education in high quality Montessori
programs.
Montessori secondary education programs are less standardized than Montessori
primary education programs, and the results of this study should not be generalized
to the small number of rather radical Montessori boarding schools with ’Erdkinder’
programs. The schools considered in this study are, however, part of a long tradition
of Montessori secondary education. Montessori secondary schools in other countries
are based on the same philosophy and have similar approaches with respect to choice
of activities and field projects.
Another issue is that the results are based on students applying for Montessori
schools. It is likely that this is a selective sample of students, as students at Montessori schools are generally of higher socioeconomic status and less often from ethnic
minorities (Dohrmann et al., 2007; Rathunde and Csikszentmihalyi, 2005a). It is
possible that the effects of Montessori education are different for less advantaged
groups of students. Further, the results for one of the schools are based on students
who did not attend Montessori primary schools. From a critic’s point of view, you
could argue that if Montessori education has a negative impact, the effects might
be especially negative for students who did not attend Montessori primary schools.
These students are unfamiliar with the Montessori education system and are probably not used to a large amount of choice and independent working. The outcomes
of these students, however, turn out to be similar to the outcomes of the Montessori
32
secondary education students who were educated at a Montessori primary school.
Despite its limitations, this is the first study to investigate the causal effects
of Montessori secondary education. Overall, the results of this study indicate that
Montessori education provides an alternative route to similar student outcomes.
This is important information for parents, students and policy makers, as it does
not support popular claims on the effects of Montessori education. Critics are not
supported in their concern that Montessori education has a negative impact on
student achievement. For advocates, on the other hand, there is no evidence that
Montessori education has a positive effect on independence and motivation.
33
References
Anderson, R. A., Worthington, L., Anderson, W. T., and Jennings, G. (1994). The
development of an autonomy scale. Contemporary Family Therapy, 16(4):329–
345.
Association Montessori Internationale (2013). Montessori movement. Retrieved
12-08-2013 from http://www.montessori-ami.org/.
Beckert, T. E. (2007). Cognitive autonomy and self-evaluation in adolescence: A
conceptual investigation and instrument development. North American Journal
of Psychology, 9(3):579–594.
Buser, T., Niederle, M., and Oosterbeek, H. (2014). Gender, competitiveness and
career choices*. The Quarterly Journal of Economics.
Calff, J. (1980). Van pionier tot mammoet: Het Amsterdams Montessori Lyceum
1930-1980. Amsterdam: Stichting Montessori 50.
Chattin-McNichols, J. (1992). The Montessori controversy. New York: Delmar.
Dohrmann, K. R., Nishida, T. K., Gartner, A., Lipsky, D. K., and Grimm, K. J.
(2007). High school outcomes for students in a public Montessori program. Journal
of Research in Childhood Education, 22(2):205–217.
Karnes, M. B., Shwedel, A. M., and Williams, M. B. (1983). A comparison of five
approaches for educating young children from low-income homes. In Consortium
for Longitudinal Studies, editor, As the twig is bent: Lasting effects of preschool
programs, pages 133–170. Hillsdale: Lawrence Erlbaum Associates.
Karnes, M. B., Teska, J. A., and Hodgins, A. S. (1970). The effects of four programs
of classroom intervention on the intellectual and language development of 4-yearold disadvantaged children. American Journal of Orthopsychiatry, 40(1):58–76.
34
Leuven, E., Oosterbeek, H., and Klaauw, B. (2010). The effect of financial rewards
on students’ achievement: Evidence from a randomized experiment. Journal of
the European Economic Association, 8(6):1243–1265.
Lillard, A. and Else-Quest, N. (2006). The early years: Evaluating Montessori
education. Science, 313(5795):1893–1894.
Lillard, A. S. (2005). Montessori: The science behind the genius. New York: Oxford
University Press.
Lillard, A. S. (2013). Playful learning and Montessori education. American Journal
of Play, 5(2):157–186.
Miller, L. B. and Bizzell, R. P. (1984). Long-term effects of four preschool programs:
Ninth- and tenth-grade results. Child Development, 55(4):1570–1587.
Miller, L. B., Dyer, J. L., Stevenson, H., and White, S. H. (1975). Four preschool
programs: Their dimensions and effects. Monographs of the Society for Research
in Child Development, 40(5-6):1–170.
Montessori, M. (1972). The discovery of the child. New York: Ballantine Books.
Montessori, M. (1973). From childhood to adolescence. Including "Erdkinder" and
the functions of the university. Second Edition. New York: Shocken Books.
Montessori, M. (1989a). The child, society and the world. Unpublished speeches and
writings. The Clio Montessori series volume 7. Oxford: Clio Press.
Montessori, M. (1989b). What you should know about your child. Based on lectures
delivered by Maria Montessori. The Clio Montessori series volume 4. Oxford:
Clio Press.
Mooney, C. G. (2000). Theories of childhood: An introduction to Dewey, Montessori, Erikson, Piaget, and Vygotsky. St. Paul: Redleaf Press.
35
Rathunde, K. and Csikszentmihalyi, M. (2005a). Middle school students’ motivation
and quality of experience: A comparison of Montessori and traditional school
environments. American Journal of Education, 111(3):341–371.
Rathunde, K. and Csikszentmihalyi, M. (2005b). The social context of middle
school: Teachers, friends, and activities in Montessori and traditional school environments. The elementary school journal, 106(1):59–79.
Rubinstein, M. (2008). Karakteristieken van scholen voor voortgezet Montessorionderwijs [Characteristics of schools for secondary Montessori education]. Apeldoorn: Garant.
Seldin, T. and Epstein, P. (2003). The Montessori Way. Sarasota: The Montessori
Foundation Press.
Smits, J.A.E. & Vorst, H. (2008). Schoolvragenlijst: Herziene uitgave 2008, Voortgezet Onderwijs [School questionnaire: revised edition 2008, secondary education].
Amsterdam: Pearson.
Søgaard, A. J., Selmer, R., Bjertness, E., and Thelle, D. (2004). The Oslo Health
Study: The impact of self-selection in a large, population-based survey. International journal for equity in health, 3(1):3.
Van Loon, A. J. M., Tijhuis, M., Picavet, H. S. J., Surtees, P. G., and Ormel, J.
(2003). Survey non-response in the Netherlands: Effects on prevalence estimates
and associations. Annals of Epidemiology, 13(2):105 – 110.
Vorst, H. C. M. (1993). Leren en studeren: Vragenboekje voortgezet onderwijs en
middelbaar beroepsonderwijs [Learning and studying: Questions for secondary and
vocational education]. Nijmegen: Berkhout B.V.
Whitescarver, K. and Cossentino, J. (2008). Montessori and the mainstream: A
36
century of reform on the margins. The Teachers College Record, 110(12):2571–
2600.
37
Appendix A
Details on questionnaire sampling and
response
In this appendix, the sampling procedure for the questionnaire and the questionnaire
response are described. Because of the large difference in the number of lottery
losing students, a different sampling procedure was used for school A and school
B. Table A1 reports the number of lottery winners, losers and priority students
selected for the questionnaire in each year, school and school track.
Table A1: Number of students selected for the questionnaire
No questionnaire
School year
Lottery winner
Questionnaire
Priority
Lottery winner
Lottery loser
Priority
Total
School A: Senior general secondary/pre-university education
2003/2004
114
50
46
55
265
2004/2005
184
16
59
19
278
2005/2006
141
47
9
55
252
2009/2010
117
37
42
46
242
2010/2011
145
29
17
33
224
Total
701
179
173
208
1261
School A: Pre-vocational IV education
2003/2004
34
17
45
19
115
2004/2005
47
10
30
13
100
2007/2008
52
10
29
10
101
2008/2009
9
29
3
33
74
2009/2010
36
5
10
7
58
Total
178
71
117
82
448
School B: Senior general secondary/pre-university education
2008/2009
28
60
24
15
127
2009/2010
77
16
6
4
103
2010/2011
25
43
13
9
90
2011/2012
50
34
6
3
93
Total
180
153
49
31
413
Grand Total
180
1032
299
321
290
2122
Note: This table reports the number of students selected for the questionnaire by school, school
track, year and lottery status.
For school A, all 540 students participating in the school admission lotteries
38
were selected to receive the questionnaire. Besides that, a stratified random sample
of 290 priority students was drawn. This sample is stratified by school year and
school track, weighted on the number of lottery winners plus 10%.
Since the number of students losing the lottery is much lower in school B, the
questionnaire sample at school B was restricted to lottery participating students.
Because the number of winning students is much larger than the number of losing
students, a weighted random sample of winning students was used. This sample
is weighted on the yearly number of lottery losers plus 40%, with a minimum of 6
students. This way, a total of 80 students applying to school B were selected for the
questionnaire, bringing the total number of students selected for the questionnaire
to 910.20
To protect the privacy of the students, the invitations for the questionnaire were
sent out via the Dutch educational administration. Four different letters were sent:
for Montessori and non-Montessori students and for students above and under 16.
The last distinction was made because parental permission is required for research
participation by students under 16 years of age. 505 students were sent the letter for
Montessori students of 16 years and older, 98 students got the letter for Montessori
students under 16. 268 students got the letter for lottery losing students of 16 years
and older, 31 students got the letter for lottery losing students under 16.
The online questionnaire was filled out by 609 students (67%), including 415
lottery participators. As described in Section 4.3, there was selective nonresponse to
the questionnaire. Characteristics of responders and non responders are summarized
in Table A2.
20
In fact, the original number of questionnaires requested to be sent out was 915 instead of
910. 5 students were dropped from the sample later on because of missing values on ethnicity and
primary school advice. It turned out that 13 students did not have accurate address information,
so 902 letters were sent. Which students have inaccurate address information is unknown.
39
Table A2: Selectivity in nonresponse to the questionnaire
Non-responders
Responders
P-value
School B
0.053
0.105
0.009
Admission for year 2003/2004
0.359
0.204
0.000
Admission for year 2004/2005
0.216
0.135
0.002
Admission for year 2005/2006
0.116
0.125
0.712
Admission for year 2007/2008
0.057
0.053
0.805
Admission for year 2008/2009
0.083
0.130
0.037
Admission for year 2009/2010
0.093
0.212
0.000
Admission for year 2010/2011
0.076
0.128
0.020
Admission for year 2011/2012
0.000
0.015
0.034
Lottery participant
0.681
0.681
0.991
Lottery loser
0.405
0.327
0.020
Citoscore
540.2
540.8
0.115
Pre-voc. IV primary school advice
0.263
0.130
0.000
Pre-voc. IV/senior gen. primary school advice
0.100
0.113
0.535
Senior gen. primary school advice
0.249
0.227
0.450
Senior gen./pre-uni. primary school advice
0.223
0.271
0.116
Pre-uni. primary school advice
0.166
0.259
0.002
Boy
0.392
0.322
0.036
Dutch
0.651
0.723
0.027
Non-western migrant
0.219
0.164
0.043
Western migrant
0.130
0.113
0.476
Age at application
12.166
12.071
0.002
Disadvantaged neighborhood
0.535
0.498
0.290
Number of students
301
609
Note: The first two columns report the mean of questionnaire non-responders and questionnaire
responders for the characteristic mentioned in that row. The p-values refer to two-group mean
comparison t-tests comparing students who did and did not respond on the questionnaire.
40
Appendix B
First stage coefficients
Tables B1 and B2 report the first stage coefficients belonging to the IV regressions
in Tables 5 and 6. For all variables, the first stage coefficients are large (between
0.826 and 0.858) and significant at the 1% level. The partial F-statistics are high
as well, ranging from 492.78 to 1083.18, well above the rule of thumb of a minimum
of 10. These results confirm that the lottery is indeed a relevant instrument for
presence at the Montessori secondary school.
41
Table B1: First stages achievement
Outcome
(1)
(2)
0.844***
0.843***
(0.026)
(0.026)
0.848***
0.848***
(0.027)
(0.028)
0.857***
0.858***
(0.029)
(0.031)
0.867***
0.869***
(0.027)
(0.030)
0.856***
0.858***
(0.029)
(0.031)
0.856***
0.857***
(0.029)
(0.031)
0.856***
0.857***
(0.029)
(0.031)
0.837***
0.838***
(0.034)
(0.038)
0.845***
0.846***
(0.029)
(0.030)
Parental support
0.835***
0.827***
(0.030)
(0.031)
Participated in commercial
0.834***
0.826***
(0.030)
(0.031)
In 4th grade of advised track or better
after four years
In 4th grade of advised track or better
after four years (cond. on being in 4th grade)
Obtained degree on time
on advised track or higher
Obtained degree on advised track or higher
with at most one year delay
Obtained degree on time on advised track
with at least a 7.5 on average
Obtained degree on time on advised track
with passing grade for Dutch
Obtained degree on time on advised track
with passing grade for English
Obtained degree on time on advised track
with passing grade for Math
Enrolled in science program
school support
!
Controls
Note: Each row reports two sets of first stages of IV regressions with winning the lottery as an IV
for ever attending a Montessori school. The sample is restricted to students participating on the
lottery. Robust standard errors are reported in parentheses. All regressions include lottery fixed
effects. Controls include gender, being a nonwestern or western migrant, student age, living in
a disadvantaged neighborhood, cito testscore, making another test than cito, no information on
primary school test-score and primary school advice. ***p<0.01 **p<0.05 *p<0.10.
42
Table B2: First stages socio-emotional functioning
Outcome
(1)
(2)
0.835***
0.827***
(0.030)
(0.031)
0.835***
0.827***
(0.030)
(0.031)
0.835***
0.827***
(0.030)
(0.031)
0.835***
0.827***
(0.030)
(0.031)
0.835***
0.827***
(0.030)
(0.031)
0.834***
0.826***
(0.030)
(0.031)
0.834***
0.826***
(0.030)
(0.031)
0.834***
0.826***
(0.030)
(0.031)
Ever got a fine or was suspected of a
0.834***
0.826***
criminal offense by the police
(0.030)
(0.031)
Independence at school
Independence out of school
Motivation towards learning tasks
Well-being at school
Interest in society
Number of times being late, truant or sent
out of class in the past 6 months
Smoked cigarettes or drank alcohol
in the past month
Used drugs of any kind
in the past 12 months
!
Controls
Note: Each row reports two sets of first stages of IV regressions with winning the lottery as an IV
for ever attending a Montessori school. The sample is restricted to students participating on the
lottery. Robust standard errors are reported in parentheses. All regressions include lottery fixed
effects. Controls include gender, being a nonwestern or western migrant, student age, living in
a disadvantaged neighborhood, cito testscore, making another test than cito, no information on
primary school test-score and primary school advice. ***p<0.01 **p<0.05 *p<0.10.
43
Appendix C
Heterogeneous effects
Table C1: Differential results for school A and school B
Outcome
Treatment
Interaction
Treatment
effect school A
effect
effect school B
(1)
(2)
(3)
-0.027
0.036
0.010
(0.049)
(0.136)
(0.131)
-0.076*
-0.038
-0.114***
(0.041)
(0.055)
(0.044)
-0.018
-0.082
-0.100
(0.052)
(0.161)
(0.155)
0.329
0.473
0.801
(0.343)
(0.716)
(0.642)
0.048
0.386***
0.435***
(0.072)
(0.147)
(0.132)
Achievement
In 4th grade of advised track or better
after four years
In 4th grade of advised track or better
after four years (cond. on being in 4th grade)
Enrolled in science program
Parental support
Participated in commercial
school support
(Table continues on next page)
44
Table C1: (continued)
Outcome
Treatment
Interaction
Treatment
effect school A
effect
effect school B
(1)
(2)
(3)
0.879
-4.778**
-3.899**
(1.048)
(2.165)
(1.913)
-1.623
-1.828
-3.451*
(1.082)
(2.186)
(1.948)
-0.094
-1.061
-1.155
(0.627)
(1.220)
(1.054)
1.827*
-2.223
-0.396
(0.941)
(1.757)
(1.503)
0.763
-0.628
0.135
(0.557)
(1.169)
(1.058)
-7.131
7.880
0.749
(5.008)
(6.700)
(4.655)
0.062
0.121
0.183
(0.060)
(0.129)
(0.116)
0.016
-0.012
0.004
(0.067)
(0.099)
(0.075)
0.025
-0.008
0.017
Socio-emotional functioning
Independence at school
Independence out of school
Motivation towards learning tasks
Well-being at school
Interest in society
Number of times being late, truant or sent
out of class in the past 6 months
Smoked cigarettes or drank alcohol
in the past month
Used drugs of any kind
in the past 12 months
Ever got a fine or was suspected of a
criminal offense by the police
(0.059)
(0.097)
(0.076)
Note: Each row reports two IV regressions with winning the lottery as an IV for ever attending
a Montessori school. Columns (1) and (2) display the treatment effect for school A and the
interaction effect on school B. Column (3) is another IV regression describing the treatment effect
for school B. The sample is restricted to students participating on the lottery. Differential effects
on exam measures are not available because the students participating in admission lotteries for
school B are too young to have done secondary school exams. Robust standard errors are reported
in parentheses. All regressions include lottery fixed effects and controls. Controls include gender,
being a nonwestern or western migrant, student age, living in a disadvantaged neighborhood, cito
testscore, making another test than cito, no information on primary school test-score and primary
school advice. Exams are graded on a 10-point scale, where grades above 5.5 are considered a
pass. ***p<0.01 **p<0.05 *p<0.10.
45
Table C2: Differential results for boys and girls
Outcome
Treatment
Interaction
Treatment
effect girls
effect
effect boys
(1)
(2)
(3)
0.016
-0.115
-0.099
(0.054)
(0.083)
(0.074)
-0.058
-0.073
-0.131**
(0.042)
(0.064)
(0.057)
0.010
-0.030
-0.020
(0.071)
(0.118)
(0.101)
0.017
-0.029
-0.012
(0.064)
(0.110)
(0.098)
-0.061
0.095
0.034
(0.053)
(0.095)
(0.092)
0.478
-0.215
0.264
(0.366)
(0.591)
(0.500)
0.164**
-0.153
0.011
(0.080)
(0.125)
(0.103)
0.383
-0.858
-0.476
(1.132)
(1.884)
(1.559)
-1.148
-2.308
-3.456**
(1.143)
(2.031)
(1.721)
0.404
-2.003*
-1.599*
(0.657)
(1.083)
(0.910)
1.815*
-1.051
0.764
(1.044)
(1.616)
(1.271)
0.320
1.011
1.331
Achievement
In 4th grade of advised track or better
after four years
In 4th grade of advised track or better
after four years (cond. on being in 4th grade)
Obtained degree on time
on advised track or higher
Obtained degree on advised track or higher
with at most one year delay
Enrolled in science program
Parental support
Participated in commercial
school support
Socio-emotional functioning
Independence at school
Independence out of school
Motivation towards learning tasks
Well-being at school
Interest in society
(0.593)
(1.006)
(0.866)
Note: Each row reports two IV regressions with winning the lottery as an IV for ever attending a
Montessori school. Columns (1) and (2) display the treatment effect for girls and the interaction
effect on boys. Column (3) is another IV regression describing the treatment effect for boys. The
sample is restricted to students participating on the lottery. Robust standard errors are reported
in parentheses. All regressions include lottery fixed effects and controls. Controls include gender,
being a nonwestern or western migrant, student age, living in a disadvantaged neighborhood, cito
testscore, making another test than cito, no information on primary school test-score and primary
school advice. Exams are graded on a 10-point scale, where grades above 5.5 are considered a
pass. ***p<0.01 **p<0.05 *p<0.10.
46
Table C3: Differential results for pre-vocational IV and academic students
Outcome
Treatment
Interaction
Treatment
effect
effect
effect
pre-voc. IV
senior gen./
pre-uni.
(1)
(2)
(3)
0.048
-0.103
-0.054
(0.084)
(0.101)
(0.057)
-0.039
-0.060
-0.099**
(0.073)
(0.085)
(0.043)
0.142
-0.248**
-0.106
(0.092)
(0.120)
(0.079)
0.079
-0.124
-0.044
(0.091)
(0.117)
(0.073)
0.015
-0.054
-0.039
(0.131)
(0.144)
(0.057)
0.293
0.153
0.445
(0.684)
(0.760)
(0.339)
-0.054
0.224
0.170**
(0.131)
(0.152)
(0.076)
-0.142
0.318
0.176
(1.856)
(2.111)
(1.067)
-2.006
0.112
-1.894*
(2.107)
(2.381)
(1.095)
0.599
-1.166
-0.567
(1.215)
(1.366)
(0.621)
4.324**
-3.847*
0.477
(1.781)
(1.996)
(0.915)
1.070
-0.551
0.519
Achievement
In 4th grade of advised track or better
after four years
In 4th grade of advised track or better
after four years (cond. on being in 4th grade)
Obtained degree on time
on advised track or higher
Obtained degree on advised track or higher
with at most one year delay
Enrolled in science program
Parental support
Participated in commercial
school support
Socio-emotional functioning
Independence at school
Independence out of school
Motivation towards learning tasks
Well-being at school
Interest in society
(0.967)
(1.139)
(0.596)
Note: Each row reports two IV regressions with winning the lottery as an IV for ever attending a
Montessori school. Columns (1) and (2) display the treatment effect for pre-vocational IV students
and the interaction effect on senior general/pre-university students. Column (3) is another IV
regression describing the treatment effect for senior general/pre-university students. The sample
is restricted to students participating on the lottery. Robust standard errors are reported in
parentheses. All regressions include lottery fixed effects and controls. Controls include gender,
being a nonwestern or western migrant, student age, living in a disadvantaged neighborhood, cito
testscore, making another test than cito, no information on primary school test-score and primary
school advice. Exams are graded on a 10-point scale, where grades above 5.5 are considered a
pass. ***p<0.01 **p<0.05 *p<0.10.
47
TIER WORKING PAPER SERIES
TIER WP 14/22
© TIER 2014
ISBN 978-94-003-0091-0