Kodu Trials
Celebration
11 December 2009
Evaluation of the Trials
Ian Phillips, I & J Management Services
Why evaluate?
• Evidence base for making decisions
 Does this program make a difference?
 Should we invest further?
 How can we do it better?
• Feedback to stakeholders
 Participants: students, teachers
 Sponsors: school, DEECD, Minister.
• So what do we want to know?
 Impact on kids
 Impact on teachers
 Impact on schools and the education system.
Kodu
Degree of Difficulty
Pre-Trial
64%
Mid-Trial
32%
62%
Post-Trial
4%
38%
58%
0%
38%
4%
28%
Pre-Trial
56%
Mid-Trial
54%
Post-Trial
16%
29%
37%
17%
42%
21%
Pre-Trial
28%
56%
Mid-Trial
46%
Post-Trial
46%
16%
37%
17%
42%
12%
.
IT Skills
.
Resources
.
Integration
Teacher impact
Time
Pre-Trial
8%
Mid-Trial
64%
21%
46%
12%
Post-Trial
0%
28%
33%
42%
20%
46%
40%
Easy
Medium
60%
Hard
80%
100%
What did you say?
“I am still trying to work with the school technician to successfully
install Kodu on school computers. I have loaded the program only to
find that the graphics aren't compatible. I then download appropriate
graphics and Kodu gets wiped from the computers. The technician is
having issues with the program as well. It is not user friendly to set up
and install, and can be extremely frustrating. It is resource hungry.
There needs to be a seamless installation process so that the
program finds what it needs – checking installation capability before
going through the whole process only to find it doesn't work.
On the positive side, the kids love Kodu.”
What did you say?
“Integration and resources are no problem, as this has been made
easy through the school’s amazing ICT coordinator.”
“The students have clicked with how Kodu works more quickly than I
have due to their experience with XBox and PlayStation games.”
“It is obvious that I, as the teacher, need to know more about Kodu so
that I can take my students' learning to a deeper level.”
Student impact
Kodu
Student Impact
Average Score (5 = High to -1 = Negative)
5
4
3
2
1
3.0
3.0
3.1
3.1
3.5
3.6
3.4
3.7
Student Achievement
Student Attitudes
Pre-Trial
Mid-Trial
Post-Trial
Student Behaviour
3.8
What did you say?
“Many of the students are more self motivated to problem solve and
more focused on achieving goals.”
“All of the boys in my class have shown an increase in motivation,
especially for literacy activities that require them to reflect on or write
about Kodu.”
“There has been a dramatic, HUGE improvement in all children's
willingness and interest in using ICT.”
“XXX has totally turned around in his attitude to school as he can
identify with the XBox controllers, plays XBox games often and is not
feeling threatened.”
What did you say?
“This project is particularly having an impact on two Year 3 boys selected to
be a part of the 'expert group' to teach skills to others. These boys are not
particularly academic. They can be hard to motivate when asked to do tasks
that they do not enjoy. However, through this project, not only are they totally
engaged in what they are doing, but the responsibility of teaching others has
had a huge impact on their self-esteem – they are beginning to see
themselves as 'experts' and learners for the first time.”
“I noticed a difference in the behaviour of some of my difficult students. They
loved Kodu and wanted any opportunity to use it, so they controlled their
behaviour more. XX1 became a class leader peer tutoring others. XX2 (a
very difficult student), was highly motivated and XX3 (who rarely attends
school) has improved his attendance greatly this term. I believe Kodu excited
the boys more, though my girls were very interested in using the XBox
controllers and being leaders within the school.”
What did you say?
“Responses ranged from … ‘I just want to play the games’ to ‘I will
make the best game ever’.”
“I found that the students who did not have the required knowledge
and skills to use this do not enjoy Kodu.”
“One student is anxious that we will be wasting valuable learning time
on playing games.”
“Many of my enthusiastic students are losing interest in Kodu and
consequently losing interest in completing work in class.”
(Secondary)
Kodu
Scope of Application
Pre-Trial
Mid-Trial
Post-Trial
Pre-Trial
Mid-Trial
Post-Trial
Recognition .
Adaptation
.
Practice
.
Concept
System impact
Pre-Trial
Mid-Trial
Post-Trial
Pre-Trial
Mid-Trial
Post-Trial
0%
20%
My Classroom
40%
My School
60%
My Education System
80%
A Global System
100%
What did you say?
“Please don't release this to schools
until all these issues are sorted out!!!!!!!”
What did you say?
“The more my students explore Kodu and the more I see their
abilities develop, the more I think that Kodu could be used in a Prep
classroom to increase critical and creative thinking skills in students.”
“I have seen real potential for learning through this software: metacognition, thinking mathematically and problem-solving strategies.”
“The knowledge spreads like wildfire – as soon as one figures out
how to do it, they are all adding it to their own games. Kodu is
developing a learning community!!!”
“The level of integration of Kodu into the curriculum is dependant on
the ability of the teacher to match pedagogy with the software.”
What have we learned?
“Throughout this trial I have seen students who are hard to motivate grow in
confidence and change their attitudes towards school. They have taken on
leadership roles in participating in an ‘expert' group. They were able to step
up and teach their peers something, a role they previously would never have
had the opportunity to do. Students have really had to think hard about
problem solving, and learn persistence in not giving up when things don't
work. They have had to be resilient when the software crashed. They have
had to learn and negotiate teamwork skills, and they have had to recognise
that there are no longer 2 or 3 teachers in the room, but 51. We have hardly
scratched the surface with what our students have been able to do creating a
game during this time, as the learning opportunities for negotiation,
teamwork, engagement, peer teaching, communication, collaboration and
problem solving have taken over. We expected to learn how to create a
game and we ended up learning how to be 21st century learners.”
What have we learned?
•
Learning that is authentic, student-centred, engaging and
fun contributes to improved student attitudes to learning and
behaviour.
 All of this contributes to improved learning outcomes
•
•
The ability to use technology effectively as part of teaching and
learning requires IT skills, resources and most importantly time.
The more familiar we become with technology and how to use it,
the more we can see the potential implications for the wider
curriculum, our peers, other schools and the education system.