Integrated assessment of disciplinary,
personal and interpersonal skills
in a design-build course
Kristina Edström, Khalid El Gaidi, Stefan Hallström
and Jakob Kuttenkeuler
June 8, 2005
Integrated assessment of disciplinary,
personal and interpersonal skills
in a design-build course
Kristina Edström, Khalid El Gaidi, Stefan Hallström
and Jakob Kuttenkeuler
June 8, 2005
Integrated assessment of disciplinary,
personal and interpersonal skills
in a design-build course
Kristina Edström, Khalid El Gaidi, Stefan Hallström
and Jakob Kuttenkeuler
June 8, 2005
The design-build course
• Idea to realisation (verifiable!)
• Large project groups (8-15 students)
• Final year
• Full year - 1/4 time
• Open-ended project specifications
Project task example
Bring 2 kg of payload up in the air and keep it there
using the sun as the only source of power
+ a few constraints
Project examples
2001-2002
Solar powered aircraft
2002-2003
Waterbike
2003-2004
Subskimmer
Solar powered aircraft
The course format
• Project based
• Some activities predefined and fixed – others to be defined
• "A smörgåsbord syllabus" – different students do different things:
- Conceptual analysis
- Deep analysis
- Project management
- Manufacturing
- Experiments
- PR & sponsoring
- Planning and follow-up
- …
Typically all students do most of the things
but no student does everything
Course goals
After the course the participant is expected to be able to
work efficiently as an engineer in a project environment, i.e.:
• take personal responsibility for small tasks and the project
as a whole
• analyse technical problems from a holistic point of view
• handle technical problems which are incompletely stated and
subjects to multiple constraints
• develop strategies for systematic choice and use of available
engineering methods and tools
• make estimations and appreciate their value and limitations
• make decisions based on acquired knowledge
• communicate engineering – orally, in writing and graphically
Integrated learning
What does it mean to
•
communicate engineering – orally, in writing and graphically ?
 to describe and present ideas
 to develop ideas through collaborative sketching and engineering reasoning
 if necessary, to argue for or against conceptual ideas and solutions
 to communicate technically, both with experts and laypersons
 to nurture confidence in expressing oneself within a certain field of work
Communicating engineering is contextual
Assessment challenges and approach
We want to assess individual achievements in a group setting
How?!
As personal and interpersonal skills are explicitly addressed
personal and interpersonal skills should be assessed!
We could only assess what we could observe or measure but here
the main part of the work is carried out behind the scenes!
In order to assess the work behind the scenes one has to
either be there or ask those who are!
The students’ trust in us is sometimes worrying…
Assessment strategy
Introduction
Faculty
• communicate course goals
• instruct students to collect evidence (portfolios)
• instruct students to log own and others’ activities
Students
• express personal goals
• plan own time
• declare anticipated working experiences
START
END
Assessment strategy
Mid-course poll
Faculty
• recap course goals
• teach giving/receiving feedback
Students
• write narratives
• read narratives, write feedback, suggest peer grades
• read feedback, write reflective documents
• revisit/revise personal goals
• follow up with reflective exercises
START
END
Assessment strategy
Final assessment
Faculty
• recap course goals
• recap reflections from poll
Students
• write narratives
• write feedback, suggest grades
• follow up with discussions on the
contribution from each student
• follow up with reflective exercises
• write reflective documents
START
END
Grading
The grades are finally set based on a holistic assessment of
• portfolios (reports, protocols, presentations,
sketches, hardware, …)
- with respect to course goals
• given feedback
• received feedback + recommended grades
• reflective documents
• participation
and guided by
• logged time
• continuous observations
Some general observations
• Students’ attitudes to knowledge are challenged
• Applied knowledge differs from theoretical knowledge
• Decision making is new to students
• Distinction between project goals and course goals is
essential
• Peer assessment is delicate to compose – not enough
to use the right instruments, one has to play them
careful too
Conclusions and final remarks
Open-ended project implies limited number of predetermined activities
 assessment scheme should be generic
In order for peer assessment to be useful and constructive
• students must be comfortable in the process
• students must rely on the process
The experience from developing the peer assessment scheme is that
• it is not only what you do that matters but also
• how you do it and
• how you present it to the students