MTLE Passing Score-Setting
Board of Teaching
October 15, 2010
Passing Score-Setting Agenda
•
•
•
•
Regulatory overview
Review of test development activities
Standards & Rules input
Score-setting discussion and action
▫ Panel-based passing score recommendations
▫ Additional considerations: all fields
▫ Field-specific considerations
Minnesota Law
• 122A.09, Subdivision 4(b) and (e) – requirements for basic skills
testing in reading, writing, mathematics; requirement for pedagogy
and content knowledge tests; requirement for reading standards
• 122A.18, Subdivision 2(b), (c), (d) – requirement for basic skills
tests; requirement for remediation; requirement for legislative
reports; requirements /conditions for issuing one-year licenses to
candidates who have not passed the basic skills tests; requirement
for making the basic skills a requirement to earn a first continuing
license
▫ Note: The legislative history suggests that this does not mean that
all teachers must take the basic skills, but rather that people
renewing a license under Subdivision 2(c) would not be able to
earn a continuing license until they had passed the basic skills.
(Session Law 1994: Chapter 647, Article 6, Section 14)
• 122A.23, Subdivision 2 (b), (d), (e) – requires out-of-state
applicants to pass Minnesota’s licensure examinations
Minnesota Law
• 122A.18, Subdivision 2a.Reading strategies.
(a) All colleges and universities approved by the Board of
Teaching to prepare persons for classroom teacher
licensure must include in their teacher preparation
programs research-based best practices in reading,
consistent with section 122A.06, subdivision 4, that
enable the licensure candidate to know how to teach
reading in the candidate's content areas. These colleges
and universities also must prepare candidates for initial
licenses to teach prekindergarten or elementary students
for the assessment of reading instruction portion of the
examination of licensure-specific teaching skills under
section 122A.09, subdivision 4, paragraph (e).
Minnesota Law
• 122A.09, Subdivision 4(e)
The board must adopt rules requiring candidates for initial licenses to successfully
complete an examination of general pedagogical knowledge and examinations of licensurespecific teaching skills. The rules shall be effective by September 1, 2001. The rules
under this paragraph also must require candidates for initial licenses to teach
prekindergarten or elementary students to successfully complete, as part of
the examination of licensure-specific teaching skills, test items assessing the
candidates' knowledge, skill, and ability in comprehensive, scientifically based
reading instruction under section 122A.06, subdivision 4, and their knowledge
and understanding of the foundations of reading development, the
development of reading comprehension, and reading assessment and
instruction, and their ability to integrate that knowledge and understanding.
• NOTE: The amendment to subdivision 4, by Laws 2009, chapter 96, article 2, section 17, is
effective May 17, 2009, and applies to teacher candidates beginning February 1,
2012. Laws 2009, chapter 96, article 2, section 17, the effective date.
Minnesota Rule 8710.0500:
Examinations for Teacher Licenses
• Subpart 1A – requirement for basic skills testing in reading, writing,
mathematics
• Subpart 1B – requirement for pedagogy and content knowledge tests;
requires tests to be adopted by the BOT; requires content test for additional
fields of licensure, but new fields do not require another pedagogy test
• Subpart 2 – requirements for selection and adoption of a testing system
• Subpart 3(1) – requires BOT to set passing scores; requires candidates to
achieve passing scores
• Subpart 4 – requires passing scores to be published in the State Register
and BOT to notify MN colleges and universities of passing scores
Minnesota Rule 8710.0500:
Examinations for Teacher Licenses
• Subpart 5 – allows MN approved programs to attest to
the passing status of applicants
▫ Translation: The folks in Educator Licensing do not
receive score reports to verify passing the tests for MN
completers; the recommendation from an approved
program includes this as a requirement.
• Subpart 6 – provides guidance on test administration,
scoring, and reporting
• Subpart 7 – allows candidates to be charged a fee for
testing
Minnesota Rule 8710.0500:
Examinations for Teacher Licenses
• Subpart 8 – requires candidates to attempt the basic
skills tests prior to beginning their teacher preparation
program
• Subpart 10 – allows for retakes of tests
• Subpart 11 – requires out-of-state applicants to meet
MN’s testing requirements
• Subpart 12 – requires BOT to periodically review the
testing system and to publish changes in the State
Register
MTLE Design: RIGOR
• History: Social Studies broad-field licensure test
• MTLE sub-test model inherently increases the
rigor and accountability for more narrow bands
of content within a licensure field
• For example:
▫ Elementary content knowledge
▫ 5-12 Mathematics content knowledge
Test Development Activities
• Summer 2009: Program planning
• Fall 2009
▫ Advisory Committees established
 Bias Review Committee (BRC) – 35 participants
 37 Content Advisory Committees (CAC) - 300 participants
 Targeted a 70/30 split: licensed teachers/higher education
faculty members
▫ Test frameworks developed
 Reading stakeholder meeting
 Content Advisory Committee preliminary online framework
review
 Bias Review Conference for framework review
Test Development Activities
• Fall-Winter 2009-2010: Item banks developed
• February 2010: Test Materials Review Conference
▫ Bias Review Committee (4 days)
 24 participants
 Item review
 9,200+ items reviewed!!
▫ Content Advisory Committees (8 days)
 Framework review AND item review
 260 participants; 22 institutions represented
• Spring 2010: Content Validation Surveys
Test Development Activities
• April 2010
▫ Content Validation online surveys
 200 randomly selected teachers per licensure field
 100 randomly selected faculty members per licensure field
 Survey analysis
▫ Field Testing
 22 institutions participating
▫ Statistical analysis of results to finalize item banks
 Out of 5,550 items field tested, 22 items were deleted
June Passing Score Activities
• Composition of panels
▫
▫
▫
▫
Content-specific
Targeted 70/30 split of teachers and faculty members
Average of 10 members per panel
Many, but not all, had been part of earlier test
development work
• Types of panels
▫ Multiple choice item banks
▫ Marker response selection for constructed response
items
June Passing Score-Setting Activities
• Training provided to panel members
▫ Recognition of performance: “would” vs. “should”
 Reasonable expectation to begin practice
 Reasonable expectation to maintain public trust
• 2 rounds of passing score judgments:
“Imagine a hypothetical group of individuals who are just at
the level of knowledge and skills required to perform the
job of an educator receiving an initial license in Minnesota.
What percent of this group would answer the item
correctly?”
June Passing Score-Setting Activities
• The result:
▫ An item-by-item review by Minnesota experts
▫ Typically, panelist item judgments fell within a
range of 3-5 rating scale points (on a scale of 1-10)
▫ A median of all ratings computed to provide a
measure of expected performance of acceptably
qualified for each item
▫ A formula-based computation applied to the item
ratings to generate a recommended score
BOT Passing Score-Setting
• Information available by subtest *
▫
▫
▫
▫
▫
# of scorable items
Standard Error of Measurement
Panel-based passing score recommendation
Score points at -2 SEM, -1 SEM, +1 SEM, +2 SEM
Relationship between recommended raw score
performance and # of scorable items
* Remember: The subtest model inherently
increases rigor and accountability for
content.
BOT Passing Score-Setting
• Additional information / considerations
▫ Impact data where n>10 in the September
administration
▫ Test objectives aligned to BOT standards
▫ New test structure, format for candidates
▫ Expanded study guides available for K-6, Early
Childhood, Basic Skills; limited preparation
materials in other fields
▫ Structure of multiple subtests increases content
accountability
BOT Passing Score-Setting
• Field-specific considerations
▫ Basic Skills
▫ New reading standards
 Elementary and Early Childhood
 19 content areas
▫ Revised standards
 Reading Teacher
 Library Media
 Computers, Keyboarding, and Related Technology
Applications
▫ Vocal and Instrumental Music test
▫ World Language and Culture tests
MTLE Test Fields
• Basic Skills
▫ Reading Subtest
▫ Writing Subtest
▫ Mathematics Subtest
• Pedagogy Tests:
▫ Early Childhood (Birth to Grade 3)
▫ Elementary (Grades K-6)
▫ Secondary (Grades 5-12)
MTLE Test Fields
Content Area Tests:
Agricultural Education (Grades 5-12)
Business (Grades 5-12)
Chemistry (Grades 9-12)
Communication Arts/Literature (Grades 5-12)
NEW Computer, Keyboarding, and Related Technology Endorsement (Grades K-12)
Early Childhood Education (Birth to Grade 3)
Earth and Space Science (Grades 9-12)
Elementary Education (Grades K-6)
English as a Second Language (Grades K-12)
Dance (Grades K-12)
Family and Consumer Sciences (Grades 5-12)
Health (Grades 5-12)
Instrumental and Vocal Classroom Music (Grades K-12)
Library Media Specialist (Grades K-12)
Life Science (Grades 9-12)
MTLE Test Fields
Content Area Tests (continued):
Mathematics (Grades 5-12)
Middle Level Endorsements (Grades 5-8)
Communication Arts/Literature
Mathematics
Science
Social Studies
Physical Education (Grades K-12)
Physics (Grades 9-12)
NEW PrePrimary Endorsement (Age 3 to Prekindergarten)
Reading Teacher (Grades K-12)
Social Studies (Grades 5-12)
Special Education Core Skills (Birth to Age 21)
Technology (Grades 5-12)
Theatre (Grades K-12)
Visual Arts (Grades K-12)
World Language and Culture (Grades K-12): NEW Chinese (Mandarin), French,
German, Spanish