BUILDING EXPERTISE 2014 Annual Interna+onal Marzano Conference June 18-­‐20, 2014 CONJUNCTIVE SCORING Penny Sell What Do You Think? •  Summa+ve scoring is an average of all scores. •  Formal evalua+ons are weighted more than informal evalua+ons in the summa+ve score. •  Growth scores and evalua+ve scores are calculated separately. •  Scores at the beginning and not using are harmful to a teacher’s overall ra+ng. •  It is not possible to use the same observa+on tool for teacher growth and summa+ve evalua+on. Development or Measurement? What should be the focus of your teacher evalua+on system? 1. 
2. 
3. 
4. 
5. 
Purely to develop teachers Emphasize development but also measure Equal emphasis on measurement and development Emphasize measurement but also develop Purely to measure teachers ©2014 Learning Sciences Interna+onal 1 BUILDING EXPERTISE 2014 Annual Interna+onal Marzano Conference June 18-­‐20, 2014 What is ConjuncMve Scoring? •  Conjunc+ve: Joined together; combined •  Conjunc+ve Scoring allows for development and measurement to work together in one framework Dual Purposes of Teacher EvaluaMon Development Measurement •  Areas of weakness for development •  Reliability – consistently and systema+cally measuring the value the same way •  Professional development targe+ng growth areas •  Monitor growth progress •  Feedback •  Valid – accurate to the value being measured What will support conjuncMve scoring? •  In many buildings it will require a culture shiZ. •  Some low scores will not harm teachers. No low scores will harm teachers. ©2014 Learning Sciences Interna+onal 2 BUILDING EXPERTISE 2014 Annual Interna+onal Marzano Conference June 18-­‐20, 2014 CALCULATING THE INSTRUCTIONAL PRACTICE SCORE Two Components of the
Evaluation System
Status Score Student Growth InstrucMonal PracMce Deliberate PracMce Score InstrucMonal PracMce Score •  Reflects teachers’ performance across all elements within the framework (Domains 1-­‐4) •  Accounts for teachers’ experience levels •  Assigns weight to the domain with greatest impact on student achievement (Domain 1) •  Acknowledges teachers’ focus on deliberate prac+ce by measuring teacher improvement over +me on specific elements within the framework ©2014 Learning Sciences Interna+onal 3 June 18-­‐20, 2014 BUILDING EXPERTISE 2014 Annual Interna+onal Marzano Conference RaMng Scale for Domain Elements Forma&ve Ra&ngs Used for Each Domain Element Ra&ngs Used for Each Domain Element 4 3 2 1 0 InnovaMng Applying Developing Beginning Not Using 4 3 2 1 EffecMve Needs Improvement/ Developing UnsaMsfactory/ IneffecMve Highly EffecMve Example -­‐ Count the number of raMngs at each level for Domain 1 Domain 1:
Elements with
Data from
Observations
Frequency at
Level 4
Frequency at
Level 3
Frequency at
Level 2
Frequency at
Level 1
Frequency at
Level 0
11 Proficiency Scale for Category I Teachers CI Highly EffecMve (4) EffecMve (3) At least 65% at Level 4 At least 65% at Level 3 or higher Developing (2) UnsaMsfactory (1) D1: D2: D3: Less than 65% at Level 3 or Greater than or equal to higher and Less than 50% 50% at Level 1, 0 at Level 1, 0 D4: Note: Scores are based on the Marzano scale: Innova+ng (4), Applying (3), Developing (2), Beginning (1) and Not Using (0) 12 ©2014 Learning Sciences Interna+onal 4 June 18-­‐20, 2014 BUILDING EXPERTISE 2014 Annual Interna+onal Marzano Conference Status Score WeighMng System Recommended weight for each domain (60 Total Elements) Domain 3 10% Domain 4 10% Domain 2 20% Domain 1 60% Percentages can be adjusted by the district InstrucMonal PracMce Status Score •  The Instruc+onal Prac+ce Score is computed by finding the weighted average of the 4 domain proficiency scores, and the resul+ng number on the scale. Highly EffecMve EffecMve Need Improvement or Developing UnsaMsfactory/ IneffecMve 3.5 – 4.0 2.5 – 3.4 1.5 – 2.4 1.0 – 1.4 UnsaMsfactory Score Sample 45 Score 40 Innova+ng (4) 0 0.00% 35 Applying (3) Count 10 15.15% % 30 Developing (2) 17 25.76% 25 Beginning (1) 24 36.36% 20 Not Using (0) 15 22.73% 15 24 10 17 15 10 5 0 0 Innova+ng -­‐ 4 Applying -­‐ 3 Developing -­‐2 Beginning -­‐ 1 Not Using -­‐ 0 59.09% At Beginning -­‐ 1 or Not Using -­‐ 0 Matches the Rule for Unsa+sfactory (1): 50% or more at Beginning -­‐1 or Lower” Score if Averaged: 1.4 ©2014 Learning Sciences Interna+onal 5 June 18-­‐20, 2014 BUILDING EXPERTISE 2014 Annual Interna+onal Marzano Conference Needs Improvement Score Sample 45 40 35 30 25 20 Score Count % Innova+ng -­‐ 4 1 1.52% Applying -­‐ 3 15 22.73% Developing -­‐2 22 33.33% Beginning -­‐ 1 20 30.30% Not Using -­‐ 0 8 12.12% 15 22 10 20 15 5 8 1 0 Innova+ng -­‐ 4 Applying -­‐ 3 Developing -­‐2 Beginning -­‐ 1 Not Using -­‐ 0 24.25% At Applying -­‐ 3 or Higher AND 42.42% At Beginning -­‐ 1 or Lower Matches the Rule for Needs Improvement (2): “Less than 65% at Applying -­‐ 3 or Higher and Less than 50% at Beginning -­‐1 or Lower” Score if Averaged: 1.8 EffecMve Score Sample 45 Score Count % Innova+ng -­‐ 4 10 15.15% 35 Applying -­‐ 3 36 54.55% 30 Developing -­‐2 12 18.18% 25 Beginning -­‐ 1 7 10.61% 20 Not Using -­‐ 0 1 1.52% 40 36 15 10 12 10 5 7 1 0 Innova+ng -­‐ 4 Applying -­‐ 3 Developing -­‐2 Beginning -­‐ 1 Not Using -­‐ 0 69.7% At Applying -­‐ 3 or InnovaMng -­‐ 4 Matches the Rule for Effec+ve (3): “At Least 65% at Applying -­‐ 3 or Higher” Score if Averaged: 2.7 Highly EffecMve Score Sample 45 40 35 30 25 45 20 Score Count % Innova+ng -­‐ 4 45 68.18% Applying -­‐ 3 11 16.67% Developing -­‐2 3 4.55% Beginning -­‐ 1 5 7.58% Innova+ng -­‐ 4 45 68.18% 15 10 11 5 3 0 Innova+ng -­‐ 4 Applying -­‐ 3 Developing -­‐2 5 Beginning -­‐ 1 2 Not Using -­‐ 0 68.18% At InnovaMng -­‐ 4 Matches the Rule for Highly Effec+ve (4): “At Least 65% at Innova+ng -­‐ 4” Score if Averaged: 3.4 ©2014 Learning Sciences Interna+onal 6 June 18-­‐20, 2014 BUILDING EXPERTISE 2014 Annual Interna+onal Marzano Conference Score for a Category II Teacher CalculaMng the Deliberate PracMce Score Use the scale to obtain the Deliberate Prac+ce Score for each target element Highly EffecMve (4) EffecMve (3) Needs Improvement or Developing (2) UnsaMsfactory (1) Grows 3 levels Grows 2 levels Grows 1 level Achieves no growth Or grows to Level 4 Or grows to Level 3 Or grows to Level 2 Or scores at Level 1 MulMple Deliberate PracMce Elements Average the growth levels across target elements October Data Point 1 March Data Point 2 Growth Deliberate PracMce Score Domain 1: #1 Providing clear learning goals and scales (rubrics) 2 3 1 3 Domain 1: #2 Tracking student progress 1 3 2 3 0 3 3 Domain 1: #3 Celebra+ng success Final Deliberate PracMce Score: ©2014 Learning Sciences Interna+onal 4 3.33 7 BUILDING EXPERTISE 2014 Annual Interna+onal Marzano Conference June 18-­‐20, 2014 Example InstrucMonal PracMce CalculaMon The Addi+ve method puts the teacher in the Highly Effec+ve range rather than the middle of the Effec+ve range. AddiMve Score
Score
Status
3.0
Deliberate Prac+ce
3.33
WeighMng
Final
3.00
0.20
0.67
Instruc+onal Prac+ce Score
3.67
Instruc+onal Prac+ce Level
Highly EffecMve
Score
WeighMng
Final
Status
ConjuncMve Score
3.0
60%
1.80
Deliberate Prac+ce
3.33
40%
1.33
Instruc+onal Prac+ce Score
3.13
Instruc+onal Prac+ce Level
EffecMve
Paradigm Shigs Old School “ProtecMon” •  Limit the number of observa+ons (1 formal observa+on for “dog and pony” show with best lesson) •  High scores protect teachers (unstated is that IP needs to be higher to offset the unknown and distrust of the VAM) New Reality “ProtecMon” •  More observa+ons = less measurement error and more feedback opportuni+es for teachers’ growth •  Some honest low scores protect teachers by iden+fying growth opportuni+es necessary to raise student achievement TO LEARN MORE: MarzanoCenter.com LearningSciences.com 1.877.411.7114 ©2014 Learning Sciences Interna+onal 8