Annual Assessment Report to the College 2010-2011
College: Michael D. Eisner College of Education
Department: Special Education
Program: Education Specialist Preliminary Credential Programs
Note: Please submit report to your department chair or program coordinator and to the Associate Dean of your College by September 30, 2010.
You may submit a separate report for each program which conducted assessment activities.
Liaison: Kathy Peckham-Hardin
1. Overview of Annual Assessment Project(s)
1a. Assessment Process Overview: Provide a brief overview of the intended plan to assess the program this year. Is assessment under the
oversight of one person or a committee? The Associate Dean and college Assessment Coordinator oversee assessment throughout the
college. In each department, the department chair and department assessment committee are responsible for that department’s assessment
tasks, such as making sure that data are collected on a timely basis, that data collection is complete, designing instruments, etc. The
department chair and department faculty appointed by the chair, are responsible for writing assessment reports as required by the AAPR, or
accreditation agencies. These reports are reviewed by the Associate Dean prior to submission to the AAPR or accreditation agencies. The
College Assessment Coordinator is responsible for data analysis as well as managing the college data warehouse and electronic data collection
system. The Coordinator ensures that each department receives their program data tables. In addition there is a Unit Assessment Committee
(UAC) consisting of department chairs or their designees, department assessment committee members. The UAC is convened by the
Associate Dean and Assessment Coordinator.
1b. Implementation and Modifications: Did the actual assessment process deviate from what was intended? If so, please describe any
modification to your assessment process and why it occurred. While the general assessment process is fixed (see attached program
assessment matrix), departments revise assessment instruments to reflect analyses of prior data, curricular changes, and changes in
accreditation standards.
SPED Annual Assessment Report Fall 11 (data for 2010-11)
1
SPED Annual Assessment Report Fall 11 (data for 2010-11)
2
STATE CHANGES, IF ANY
The Department of Special Education implemented new preliminary and clear credential programs to meet new standards. Assessment
instruments have been modified to address the new standards.
2. Student Learning Outcome Assessment Project: Answer questions according to the individual SLO assessed this year. If you assessed an
additional SLO, report in the next chart below.
2a. Which Student Learning Outcome was measured this year? All SLOs were measured this year.
1. Engage and Support All Students in Learning
2. Create and Maintain Effective Environments for Student Learning
3. Make Subject Matter Comprehensible for Student Learning
4. Plan Instruction and Design Learning Experiences for All Students
5. Assess Student Learning
6. Develop as a Professional Educator
2b. What assessment instrument(s) were used to measure this SLO?
•
Teaching Evaluation completed by university supervisors (all pathways; traditional, intern, ACT, ITEP) and cooperating teachers (traditional, ACT,
ITEP) for beginning and final fieldwork assignments. Four-point rating scale: 1=Inconsistent practice, 2=Developing practice, 3=Maturing practice,
4=Exemplary practice.
•
Exit survey completed by candidates regarding program preparation.
2c. Describe the participants sampled to assess this SLO:
All credential candidates in fieldwork classes were assessed on the measures listed in 2b.
2d. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points)
or was a cross-sectional comparison used (comparing freshmen with seniors)? If so, describe the assessment points used.
Assessment was a cross-sectional comparison (comparing early fieldwork candidates and final fieldwork candidates). The methodology
also provides cross-sectional comparisons related to credential pathways (traditional, intern, ACT and ITEP)
2e. Assessment Results & Analysis of this SLO: Provide a summary of how the data were analyzed and highlight important findings from the
data collected.
SPED Annual Assessment Report Fall 11 (data for 2010-11)
3
FINDINGS FOR MILD/MODERATE DISABILITIES
Table 1: Traditional Pathway – Fieldwork Teaching Evaluation Fall 2010 and Spring 2011
TRADITIONAL
SPED 403MM
Domain
Fall 2010
University
Cooperating
Supervisor
Teacher
15
22
Spring 2011
University
Cooperating
Supervisor
Teacher
8
4
Domain 1: Engaging and Supporting All
Students in Learning
2.89
3.52
2.92
3.75
Domain 2: Creating and Maintaining An
Effective Environment for Students
2.73
3.73
3.13
3.75
Domain 3: Understanding and Organizing
Subject Matter Knowledge for Student
Learning
2.96
3.63
2.90
3.69
Domain 4: Planning Instruction and Designing
Learning Experiences for Students
2.95
3.59
2.96
3.73
3.08
3.75
3.03
3.75
3.00
3.74
2.82
3.68
Domain 5: Assessing Student Learning
Domain 6: Developing as a Professional
2.86
3.76
Educator
Mean
2.87
3.65
Rating scale: 1=Inconsistent practice, 2=Developing practice, 3=Maturing practice, 4=Exemplary practice
Mean ratings were consistent across SLO domains and semesters with the university supervisor rating candidates consistently lower on each SLO
than the collaborating teacher (university supervisor 2.87 and 3.0, collaborating teacher 3.65 and 3.74). Candidates were rated as expected by
the university supervisors, indicating developing practice, but rated inappropriately high by the cooperating teachers.
SPED Annual Assessment Report Fall 11 (data for 2010-11)
4
FINDINGS FOR MILD/MODERATE DISABILITIES
Table 2: Traditional Pathway – Student Teaching Evaluation Fall 2010 and Spring 2011
TRADITIONAL
SPED 580MM
Domain
University
Supervisor
N = 15
Fall 2010
Cooperating
Teacher
N = 10
Spring 2011
University
Supervisor
N = 25
Cooperating
Teacher
N = 24
Domain 1: Engaging and Supporting All
Students in Learning
3.43
3.4
3.62
3.62
Domain 2: Creating and Maintaining An
Effective Environment for Students
3.35
3.47
3.50
3.53
Domain 3: Understanding and Organizing
Subject Matter Knowledge for Student
Learning
3.48
3.50
3.67
3.61
Domain 4: Planning Instruction and Designing
Learning Experiences for Students
3.55
3.64
3.68
3.63
3.81
3.65
3.51
3.55
Domain 5: Assessing Student Learning
Domain 6: Developing as a Professional
3.69
3.65
Educator
Mean
3.50
3.54
Rating scale: 1=Inconsistent practice, 2=Developing practice, 3=Maturing practice, 4=Exemplary practice
3.8
3.75
3.68
3.63
Mean ratings were consistent across SLO domains, semesters, and supervisors (university supervisor 3.50 and 3.68 and cooperating teacher 3.54
and 3.63). As expected, ratings were higher than in early fieldwork that reflected developing and maturing practices. By the end of student
teaching, mean ratings of candidates reflected maturing and exemplary practice.
SPED Annual Assessment Report Fall 11 (data for 2010-11)
5
FINDINGS FOR MILD/MODERATE DISABILITIES
Table 3: Intern Pathway – Teaching Evaluation Fall 2010 for Semesters 1-4; Ratings by University Supervisors
INTERN FALL 2010
SPED 506
First
N = 12
Second
N=0
Third
N = 20
Fourth
N=8
Domain 1: Engaging and Supporting All
Students in Learning
2.85
0
3.59
3.48
Domain 2: Creating and Maintaining An
Effective Environment for Students
2.68
0
3.49
3.53
Domain 3: Understanding and Organizing
Subject Matter Knowledge for Student
Learning
2.87
0
3.54
3.67
Domain 4: Planning Instruction and Designing
Learning Experiences for Students
2.87
0
3.69
3.81
3.78
3.86
3.66
3.78
3.63
3.69
3.00
0
Domain 5: Assessing Student Learning
Domain 6: Developing as a Professional
3.04
0
Educator
Mean
2.89
0.00
Rating scale: 1=Inconsistent practice, 2=Developing practice, 3=Maturing practice, 4=Exemplary practice
Interns work in their own classrooms and therefore do not have cooperating teachers assigned to them
Mean ratings by university supervisors were consistent across SLO domains and were lower in year 1 (first semester 2.89) when interns were
beginning the program than in year two (3.63 and 3.69) when interns were completing the program. The ratings reflect development of interns
as they progress through the program. Data is missing for semester two.
SPED Annual Assessment Report Fall 11 (data for 2010-11)
6
FINDINGS FOR MILD/MODERATE DISABILITIES
Table 4: Intern Pathway – Teaching Evaluation Spring 2011 for Semesters 1-4; Ratings by University Supervisors
INTERN SPRING 2011
SPED 506
First
N=0
Second
N=9
Third
N=0
Fourth
N = 22
Domain 1: Engaging and Supporting All
Students in Learning
0
3.33
0
3.69
Domain 2: Creating and Maintaining An
Effective Environment for Students
0
3.07
0
3.55
Domain 3: Understanding and Organizing
Subject Matter Knowledge for Student
Learning
0
3.21
0
3.61
Domain 4: Planning Instruction and Designing
Learning Experiences for Students
0
3.21
0
3.72
0
3.81
0
3.79
0.00
3.70
0
3.14
Domain 5: Assessing Student Learning
Domain 6: Developing as a Professional
0
3.57
Educator
Mean
0.00
3.26
Rating scale: 1=Inconsistent practice, 2=Developing practice, 3=Maturing practice, 4=Exemplary practice
Interns work in their own classrooms and therefore do not have cooperating teachers assigned to them
Mean ratings by university supervisors (interns work in their own classrooms and therefore do not have cooperating teachers assigned to them)
were consistent across SLO domains and were lower in year 1 (second semester 3.26) when interns were beginning the program than in year
two (3.70) when interns were completing the program. The ratings reflect development of interns as they progress through the program. There
were no interns enrolled in the first semester and data is missing for third semester interns.
SPED Annual Assessment Report Fall 11 (data for 2010-11)
7
FINDINGS FOR MILD/MODERATE DISABILITIES
Table 5: ACT Pathway – Fieldwork Teaching Evaluation Fall 2010
ACT FALL 2010
SPED 579ACT
University Supervisor
N=9
Cooperating Teacher
N=9
Domain 1: Engaging and Supporting All Students in Learning
2.39
2.85
Domain 2: Creating and Maintaining An Effective Environment for
Students
2.89
2.89
Domain 3: Understanding and Organizing Subject Matter Knowledge for
Student Learning
2.67
2.95
Domain 4: Planning Instruction and Designing Learning Experiences for
Students
2.74
3.01
Domain 5: Assessing Student Learning
2.71
3.07
2.83
Domain 6: Developing as a Professional Educator
Mean
2.71
Rating scale: 1=Inconsistent practice, 2=Developing practice, 3=Maturing practice, 4=Exemplary practice
3.19
2.99
ACT Candidates complete fieldwork in the fall semester only.
Mean ratings were consistent across SLO domains, semesters, and supervisor (university supervisor 2.71 and collaborating teacher 2.99).
Candidates were rated as expected, indicating developing practice.
SPED Annual Assessment Report Fall 11 (data for 2010-11)
8
FINDINGS FOR MILD/MODERATE DISABILITIES
Table 6: ACT Pathway – Fieldwork Teaching Evaluation Spring 2011
ACT SPRING 2011
SPED 580 ACT
University Supervisor
N=9
Cooperating Teacher
N=9
Domain 1: Engaging and Supporting All Students in Learning
3.80
3.71
Domain 2: Creating and Maintaining An Effective Environment for
Students
3.36
3.61
Domain 3: Understanding and Organizing Subject Matter Knowledge for
Student Learning
3.62
3.71
Domain 4: Planning Instruction and Designing Learning Experiences for
Students
3.63
3.73
Domain 5: Assessing Student Learning
3.69
3.69
3.68
Domain 6: Developing as a Professional Educator
Mean
3.63
Rating scale: 1=Inconsistent practice, 2=Developing practice, 3=Maturing practice, 4=Exemplary practice
3.75
3.70
ACT Candidates complete student teaching in the spring semester only.
Mean ratings were consistent across SLO domains, semesters, and supervisor (university supervisor 2.63 and collaborating teacher 3.70).
Candidates were rated as expected, indicating developing practice.
SPED Annual Assessment Report Fall 11 (data for 2010-11)
9
FINDINGS FOR MILD/MODERATE DISABILITIES
Table 7: ITEP Pathway – Fieldwork Teaching Evaluation Fall 2010 and Spring 2011
ITEP
SPED 403MM
Domain
University
Supervisor
N=6
Fall 2010
Cooperating
Teacher
N=8
Spring 2011
University
Cooperating
Supervisor
Teacher
N=1
N=2
Domain 1: Engaging and Supporting All
Students in Learning
3.00
3.29
3.00
3.83
Domain 2: Creating and Maintaining An
Effective Environment for Students
3.00
3.50
3.00
3.50
Domain 3: Understanding and Organizing
Subject Matter Knowledge for Student
Learning
3.19
3.44
3.27
3.53
Domain 4: Planning Instruction and Designing
Learning Experiences for Students
3.04
3.43
3.17
3.79
3.00
3.63
3.11
3.70
Domain 5: Assessing Student Learning
Domain 6: Developing as a Professional
3.22
3.69
Educator
Mean
3.09
3.51
Rating scale: 1=Inconsistent practice, 2=Developing practice, 3=Maturing practice, 4=Exemplary practice
3.38
3.75
3.14
3.67
Mean ratings were consistent across SLO domains and semesters with the university supervisor rating candidates consistently lower on each SLO
than the collaborating teacher (university supervisor 3.09 and 3.14, collaborating teacher 3.14 and 3.67). Candidates were rated as expected by
the university supervisor, indicating developing/maturing practice, but rated inappropriately high by the cooperating teachers.
SPED Annual Assessment Report Fall 11 (data for 2010-11)
10
FINDINGS FOR MILD/MODERATE DISABILITIES
Table 8: ITEP Pathway – Student Teaching Evaluation Fall 2010 and Spring 2011
ITEP
SPED 580MM
Domain
Fall 2010
Cooperating
University
Teacher
Supervisor
N=4
N=3
Spring 2011
Cooperating
Cooperating
Teacher
Teacher
N=5
N=3
Domain 1: Engaging and Supporting All
Students in Learning
3.63
3.40
3.20
3.47
Domain 2: Creating and Maintaining An
Effective Environment for Students
3.61
3.33
3.24
3.36
Domain 3: Understanding and Organizing
Subject Matter Knowledge for Student
Learning
3.71
3.34
3.59
3.56
Domain 4: Planning Instruction and Designing
Learning Experiences for Students
3.78
3.38
3.57
3.50
3.6
3.92
3.69
3.5
Domain 5: Assessing Student Learning
Domain 6: Developing as a Professional
3.97
3.48
Educator
Mean
3.73
3.41
Rating scale: 1=Inconsistent practice, 2=Developing practice, 3=Maturing practice, 4=Exemplary practice
3.92
3.68
3.52
3.58
Mean ratings were consistent across SLO domains, semesters, and supervisor (university supervisor 3.73 and 3.41) and cooperating teacher
(3.52 and 3.58). As expected, ratings were higher than in early fieldwork that reflected developing and maturing practices. By the end of
student teaching, mean ratings of candidates reflected maturing and exemplary practice.
SPED Annual Assessment Report Fall 11 (data for 2010-11)
11
FINDINGS FOR MILD/MODERATE DISABILITIES
Exit Survey
In addition to the performance based teaching evaluation completed each semester for credential candidates, the CSU exit survey was
completed by candidates regarding their program preparation. Candidates exiting the credential program in 2010-11 rated items that relate to
the SLOs on a 4-point rating scale (well prepared, adequately prepared, somewhat prepared and not at all prepared) with the following
directions.
Please rate how well the CSUN program prepares candidates to: 6 items
Please rate your experiences with the CSUN program: 6 items
As a new teacher I am prepared to (level of preparation): 50 items
Instruction in your Teaching Credential Program: 15 items
Rate the value of these elements of your credential program: 9 items
The Department has established 80% as a benchmark (calculated by combining well prepared and adequately prepared), and examines items
that fall below the benchmark. Items below the benchmark for each Mild/Moderate Disability Pathway are shown below.
Of the 71 items on the exit survey, Table 9 shows 12 items that were rated below the 80% benchmark for at least one pathway. Five items were
rated below 80% (7%) for the traditional program, one item for the intern and ACT programs (1%), and 9 items for the ITEP program (13%). The
results in the ITEP program should be interpreted with caution given the small number of candidates. Of the items below the benchmark, two
relate to students of diverse backgrounds (English learners and students from diverse cultural backgrounds) and one item on resources in the
school and community for at-risk students and families (Traditional and ITEP pathways); three items relate to computer technology (all
pathways).
In summary, candidates who completed the Education Specialist Program in Mild/Moderate Disabilities met competencies as indicated by the
performance based student teaching measure. The data suggest that candidates increase in competency from early fieldwork to student
teaching. There’s consistency in ratings between university supervisors and cooperating teachers; the one exception is in early fieldwork for the
traditional and ITEP pathways where ratings are higher by cooperating teacher. Findings from the exit survey indicate that candidates feel
adequately to well prepared as beginning teachers. Three areas that may need program revision, rated below the benchmark of 80% in at least
two pathways are preparation in serving students from diverse cultural and language backgrounds, use of computer technology, and
familiarizing with resources regarding students at-risk and their families.
SPED Annual Assessment Report Fall 11 (data for 2010-11)
12
Table 9 – Exit Survey: Items Related to SLOs Below the Benchmark of 80%
Exit Survey 2010-11
Survey Item
N
Traditional
N = 21
...to meet the instructional needs of students who are English language learners.
21
76%
...to meet the instructional needs of students from diverse cultural backgrounds.
7
Intern
N = 22
ACT
N = 27
Pathway
ITEP
N=7
71%
...to use computer-based technology to help students learn subjects of the curriculum.
21
71%
57%
...to use computer-based technology for instruction, research, and record keeping..
21
71%
71%
Instruction in using computer technology for classroom instruction.
77%
74%
...to know about resources in the school & community for at-risk students and families.
21
67%
...to develop and implement transition plans for special education students.
19
79%
...to understand child development, human learning and the purposes of schools.
...to assist students in making sound ethical judgments.
7
7
71%
71%
...to assist students in decision-making, problem-solving, and critical thinking.
7
71%
...to use language so pupils at different levels understand oral and written English.
7
71%
...to know and understand federal and state laws that govern special education.
7
71%
SPED Annual Assessment Report Fall 11 (data for 2010-11)
58%
13
2f. Use of Assessment Results of this SLO: Think about all the different ways the results were or will be used. For example, to recommend
changes to course content/topics covered, course sequence, addition/deletion of courses in program, student support services, revisions to
program SLO’s, assessment instruments, academic programmatic changes, assessment plan changes, etc. Please provide a clear and detailed
description of how the assessment results were or will be used.
Assessment results are reviewed at Department Meetings and with faculty in each specialization area (MM, MS, DHH, ECSE). Faculty discuss
areas of concern and make recommendations to improve program practices.
3. How do your assessment activities connect with your program’s strategic plan?
4. Overall, if this year’s program assessment evidence indicates that new resources are needed in order to improve and support student
learning, please discuss here.
5. Other information, assessment or reflective activities not captured above.
SPED Annual Assessment Report Fall 11 (data for 2010-11)
14
6. Has someone in your program completed, submitted or published a manuscript which uses or describes assessment activities in your
program? Please provide citation or discuss.
SPED Annual Assessment Report Fall 11 (data for 2010-11)
15