The impact of a set of design based engineering activities on
first grade students’ problem solving abilities
Investigator: Merredith Portsmore
PhD Thesis Proposal Draft 1.0
5/2/05
1
Abstract
The inclusion of engineering design activities in K-12 classrooms has been
purported to help teach students problem solving skills. However, the assertion
is not directly supported by research that looks specifically at students problem
solving evaluated quantitatively as a result of an engineering design curriculum.
The research proposed in this paper will evaluate first grade students’ ability to
solve both simple and complex problem solving tasks before and after
participating in an set of engineering design activities for 13 weeks in a typical
classroom setting.
2
Introduction:
The Center for Engineering Educational Outreach (CEEO) at Tufts University has
been engaged in promoting engineering education in K-12 for over 10 years.
The CEEO’s efforts have focused on creating tools and supporting curriculum
that allow students to engage in hands-on design problems. The LEGO
materials have been an important part of the toolset that has supported activities
in kindergarten through college level classes (Rogers & Portsmore, 2004). In
addition to the goal of teaching math and science in an engaging hands-on way,
the CEEO also hopes to achieve higher order learning goals such as:
1. Curiosity
2. Enthusiasm for Learning
3. Problem Solving
4. Testing Validity of Answers
5. Self Confidence
to help promote life long learning. These aspects have been informally observed
and reported as a product of doing hands-on LEGO based engineering activities
in the classroom (Rogers et all, 2001; Murray & Bartelmay 2005). The goal of
this paper is to outline a plan for research that would help to generate evidence
to support (or disprove) these anecdotal observations.
Research Questions:
The main hypothesis for this research is:
Participation in classroom based engineering design activities improves first
3
grade students’ performance on general and complex problem solving tasks
The hypothesis centers around the notion that by participating in a problem
solving based environment students will improve specific as well as general
problem solving skills. The quantitative methods in the research project will
focus on obtaining data to support or invalidate this hypothesis.. The qualitative
methods will focus on understanding how the engineering design environment
impacts students problem solving skills. Including:
1) How do students think about engineering challenges (problems) posed
to them?
2) How do students approach engineering challenges (problems) ?
3) Does students’ approach to engineering problem solving change over
the course of the intervention?
Background:
Engineering
Engineers design products or solutions that meet a need or solve a problem.
Using their knowledge of math and sciences as well as the use of technological
tools, engineers take their designs from concept to prototype to marketing.
Recently, there has been a push to make engineering part of K-12 education,
particularly in Massachusetts (NAE & NRC 2002; Massachusetts Department of
Education, 2001). The motivations for adding engineering in K-12 classrooms
include:
4
i. the teaching of math and science (NAE & NRC 2002)
ii. technological literacy (Lewis 2004; NAE & NRC 2002. ISTE,
2000)
iii. the teaching of problem solving and design skills (Rogers &
Portsmore, 2004, NRC & NAE N2004)
Knight & Cunningham (2004) found that students have little understanding of
what engineering is and what engineers do. However, little systematic research
currently exists that supports the effectiveness engineering in supporting any of
the aforementioned motivations.
Problem Solving
Researchers struggle to define problem solving because of the number of
different contexts it can be applied to – from writing a book, to gaining a parents
attention, to building a bridge or car. A commonly accepted definition poses
problem solving as “ the development of a path through a problem space.” (Hunt
1994). There is significant research that indicates that problem solving ability is
likely highly domain dependent and that within the domain conceptual (content)
knowledge and procedural knowledge impact problem solving abilities (Hegarty,
1991, Mayer 1998).
Researchers have also investigated the developmental aspects of problem
solving. Deloache (1987) looked at children’s ability to use representational
models to solve problems, finding that younger children (2 ½ year olds) had less
success that slightly older children (3 year olds). As language increase, so do
5
children’s ability to reason, collaborate, and their use of planning strategies (Ellis
& Siegler 1994). Design problems have been established among the most
complex and ill structured problems (Jonassen 2000). While research has been
done evaluating how experts solve design problems, no research has been done
that looks at whether engaging in design problems actually develops problem
solving abilities.
Methods:
Sample
The sample for this study will be approximately 80 students from 4 first grade
classrooms in Lincoln, MA, an upper class suburb of Boston, MA. Lincoln has a
minority population of approximately 20% and 27 % of the students are eligible
for free or reduced lunch.
First grade students were selected because they have had minimal exposure to
school and other experiences with engineering that might influence their problem
solving abilities. However, they are old enough to manipulate most of the LEGO
materials without assistance, which allows for them to engage in complex design
tasks.
Research Design
This study will employ a classic pre and post test experimental design. The 4
first grade classrooms will be randomly placed in either the engineering
intervention group or the control group.
6
Engineering
Intervention
(40 students)
Control
Pre Test
Engineering
Activities
Post-Test
Pre-Test
No Engineering
Activities
Post-Test
(40 students)
Students will tested before and after the intervention for problem solving skills
using standardized measures as well as newly designed measures.
Instruments
The pre and post test instruments were selected on their basis to evaluate
students problem solving abilities. Both tests will be comprised of two
components
1) Standardized elements from the Wechsler Intelligence Test for Children
(WISC III) Block Design Subtest
2) Trapped Toys – hands-on, complex design tasks designed by the
investigator.
The WISC III addresses ages 6 to age 16. The test is designed to generate 3 IQ
scores (Verbal, Performance, and Full Scale). The Performance Scale has a
Block Design Subtest (where students complete different patterns based on 2-D
and 3-D blocks). (Kaufman & Lichtenberger 2000) Elements 1,3,5,7, & 9 would
be used in the pre test and elements 2,4,6,8,&10 would be used in the post test.
Performance on the elements would be scored according to the WISC IIII guide
(which evaluates time to completion, orientation of the blocks, planning, and
problem solving strategies)
7
Trapped Toys are two comparable design challenges designed by the
investigator to evaluate how students approach a complex problem solving task .
The tasks use materials that all students in both groups should be familiar with.
 Trapped Toy I has a small toy “trapped” in the bottom of a bottle Students
can’t use their hands to touch the bottle but may use any of the materials
provided (tape, straws, chopsticks, rubber bands, string etc..) to construct
a device to remove the toy (which they get to keep regardless of success).
 Trapped Toy II places the toy in the bottom of a box inside a small bucket
with a handle. Students are again tasked to create a method to remove
the toy from inside the box without using their hands. Assorted materials
are provided.
The students would be video taped during the Trapped toy test which would be
coded for problem solving behaviors.
Description of Technology/Intervention
The two first grade classrooms in the engineering intervention classroom will use
the LEGO Motorized Simple Machines kits (Figure 1) and basic supplementary
pieces to engage in 13 weeks (1 – 1.5 hours per week) of engineering activities.
8
Figure 1: The Motorize Simple Machines Kit has a range of LEGO Technic
pieces
This kit focuses on construction, simple machines, and basic automation
(through the use of an electric motor and battery box). Pieces range from basic
building components (bricks and plates) to more complex construction elements
(tires, axles, beams). (Table 1).
9
BRICK
PLATE
BEAM
TIRES
HUBS
AXLE
Table 1: Sample of LEGO pieces initially presented to students
Each pair of students receives 1 kit to work with. Pairs also have access to
additional pieces and are allowed to borrow pieces from other groups.
The 13 weeks of the intervention will be based on activities in Engineering by
Design (Green et all, 2005),a curriculum written by Lincoln first grade teachers
and CEEO staff and students in 2001/2002. Engineering by Design (Figure 2)
focuses on introducing students to the LEGO components, the concept of
engineering, and the design process.
10
Figure 2: Engineering by Design is a PDF download available from
ROBOLAB@CEEO (http://www.ceeo.tufts.edu/robolabatceeo)
11
Week Lesson Title
Concepts
Assessment
1.
2.
Introduction to LEGO pieces &
Engineering
Build a Sturdy Wall
Names of LEGO
Pieces
Design constraints
Sturdy Construction
3.
Building a Chair for Mr. Bear
Design Constraints
Sturdy Construction
LEGO Matching
Sheet
Can your wall
withstand the flick
test?
The drop test
Can your chair hold
Mr. Bear?
4.
Introduction to Pulleys
5.
Introduction to motors
Pulleys
Belts
Pulley Ratio
Motors
Wires
6.
Build a sturdy car
Motor attachment to
car
7.
8.
Build a sturdy car (continued)
Introduction to Gears
Gears
Gear rations
Gear spacing &
meshing
9.
Build a snowplow
Design with pulleys or
gears
10.
11.
Build a snowplow (continued)
Transportation Invention
Define your own
problem to solve
12.
Transportation Invention
(cont’d)
Transportation Invention
(cont’d)
-
Can you explain how
your pulley wall
works?
Can you attach a
color wheel to the
motor and make it
spin?
Can you build a car
that drives forward
using a pulley?
Can you explain how
your gear wall works?
Can you tell me
which gear
combination will go
faster or slower?
Can you choose the
appropriate
combination of
pulleys or gears to
push heavy snow?
Can you identify your
own design
constraints?
-
-
-
13.
Table 2: Sequence of Intervention (yellow represent content based lessons, blue
represents design/problem solving based lessons, green indicates the final
project)
The investigator will lead the classes in the two Engineering Intervention group
12
classrooms with support from the teacher, parent volunteers, and undergraduate
engineering students. Four of the thirteen lessons (#1,4,5,8) will focus on
providing students with content and material knowledge (Lego piece names and
uses, how gears and pulleys work, how motors work). The remaining lessons
will be structured as design challenges with semi structured problems. The final
lessons will be an open ended design challenge where students are able to
define their own transportation based problem with which to design a solution for.
Data Collection
The investigator and any of those assisting with data collection will spend time in
all the classrooms (control and intervention) getting to know the children as a
classroom aid. It is anticipated that 2 other teachers or graduate students will be
trained in administering the pre and post test measures due to the large number
of students. Pre Test measures will be implemented in the two weeks prior to the
beginning of the intervention and Post Test measures will be implemented in the
two weeks following the conclusion of the intervention. During the intervention,
qualitative data in the form of journals
Baseline Information
Basic information will be collected about the students via survey and teacher
feedback. Student data collection will include a survey that is sent home to be
completed with the parents that includes identifies gender, age, number of
siblings, attitudes toward school, and favorite toys. Teachers will be asked to
13
rate each child’s performance in reading and math as low, middle, or high
performing.
Pre & Post Test
Each student participating in the research study will participate in a 30 minutes
pre and post test interview. The interview will be conducted in the corner of the
classroom with all efforts made to keep distractions to a minimum. Fifteen
minutes will be allocated for working on the WISC III block test elements and
fifteen minutes will be allocated for working on the Trapped Toy challenge (I or
II). The Pre and Post Test will be video taped to assist with coding and scoring.
Students will be scored on how WISC component according to the number of
block patterns they complete and the correctness of each block pattern. Students
will also be observed for problem solving behaviors (trial and error, haphazard
experimentation). The Trapped Toy challenge will also be coded for the problem
solving behaviors that are displayed (trial and error, haphazard).
Qualitative Data
In the engineering intervention classrooms, efforts will be made to document how
students engage in problem solving. Students will keep engineering journals with
photographs and hand drawn pictures of their designs and final creations. They
will be asked to identify successful and unsuccessful efforts using pictures and
signs. Students will be invited to ask the instructor, teacher or volunteers to take
a picture of both types of efforts. This picture will be printed on a photo printer in
14
the classroom. It is hoped that the novelty of this effort will encourage students
to record both types of efforts. These journals will help the investigator
understand how students are progressing and their attitudes. The journals will
also serve as a catalyst for informal video taped interviews to ask students how
they went about solving design challenges. Students will also be video taped
while they are working to look at problem solving strategies.
Data Analysis
Quantitative & Baseline Data
The quantitative data will be analyzed with the baseline data to see if there are
differences between the mean scores of groups on the WISC III block test portion
of the pre & post test. The following tests are anticipated:
 Independent T tests (pre and post test WISC III block test scores for control
and intervention)
o Pre Test: Is there a difference between the groups due to individual
differences?
o Post Test: Is there a difference between the groups due to the
intervention?
 Matched T test (pre and post WISC block test) for students in the
o control group (is there any developmental or re-test effect?)
o intervention group (Is there an intervention effect?)
 Two way ANOVA (gender and intervention) for mean WISC block test
15
scores (with Neumann Keuls Post Hoc)
o Is there a relationship between gender and intervention?
 Correlation of WISC III block test scores and age
o Is there a relationship between test scores and age?
Quantitative Data
The Trapped Toy video data will be coded for
1) Problem Solving Behaviors
a. Trial and error
b. Planning
c. Haphazard
2) Dialog about problem solving
a. Dialog about materials
b. Dialog about overall design
These cases analyzed will be across interventions based on baseline data (How
do girls in the control group compare to girls in the intervention group?) As well
as within the groups (How do boys in the intervention group compare to girls in
the intervention group?)
The student journals will be coded for their display of problem solving strategies.
1) Is there evidence of trying multiple ideas?
2) Is there evidence of systematic trial of ideas?
3) Does students success or failure reflect their attitude?
16
The coding about the journals will be supplemented by the video interviews. The
video obtained from the informal interviews about the building process will also
be coded for design and problem solving techniques.
Deliverables
There will be three main deliverables for this research project:
1) The PhD Thesis of the investigator
2) A journal article publishing the results (if significant)
3) Presentations at education and engineering (ASEE) conferences
4) A meeting/seminar for parents and teachers at the participating school to
share the final results and conclusions.
17
References:
DeLoache, J.S. (1987). Rapid change in the symbolic functioning of young
children. Science, 238, 1556-1557.
Ellis, S., Siegler, R.S., (1994) Development of Problem Solving. In Thinking and
Problem Solving San Diego, CA: Academic Press, Inc. p.333-367
Green, T., Kearns, S.A., Rosenbloom, J., McMullon, L., Leary, J., Cejka, E.,
Portsmore, M., (2005) Engineering by Design. Retrieved April 30, 2005 from
http://www.ceeo.tufts.edu/robolabatceeo/k12/curriculum_units
Hegarty, M., (1991) Knowledge and Processes in Mechanical Problem Solving.
In Complex Problem Solving Hillsdale, NJ: Lawrence Erlbaum associates, Inc. p
253-285.
Hunt, E. (1994) Problem Solving. In Thinking and Problem Solving San Diego,
CA: Academic Press, Inc. p.215-232.
Gillespie, C., Beisser S., (2001) Developmentally Appropriate LOGO Computer
Programming with Young Children. Information Technology in Childhood
Education, 229-245
International Society for Technology in Education. (2000). National educational
technology standards for students: Connecting curriculum and technology:
International Society for Technology in Education.
Jonassen, David H. (2000)Toward a design theory of problem solving.
Educational Technology Research & Development. Vol 48(463-85. Assn for
Educational Communications & Technology, US
Kaufman, A., Lichtenberger, E., (2000) Essentials of the WISC III and WPPSI-R
Assessment. New York, NY. John Wiley and Sons.
Knight, M. Cunningham, C.. (2004) Draw an Engineer Test (DAET):
Development of a Tool to Investigate Students' Ideas about Engineers and
Engineering.. in American Society of Engineering Education. 2004. Salt Lake
City, Utah.
Lewis, T., (2004) A Turn to Engineering: The Continuing Struggle of Technology
Education for Legitimization as a School Subject. Journal of Technology
Education, Fall 2004, Volume 16, Number 1
Massachusetts Department of Education. (2001). Science and
Technology/Engineering Curriculum Framework. Malden, MA: Massachusetts
Department of Education.
18
Mayer, R. (1998) Cognitive, metacognitive, and motivational aspects of problem
Solving. Instructional Science 26: 49–63
Murray, J. & Bartelmay, K. (2005) Inventors in the Making. Science & Children
(Pending Publication)
National Academy of Engineering, & National Research Council. (2002)
Technically speaking: Why all Americans need to know more about technology.
Washington, D.C.: National Academy Press.
Rogers, C., Portsmore, M., (2004) Bringing Engineering to Elementary School.
The Journal of STEM Education: Innovations and Research Vol 5.
http://www.auburn.edu/research/litee/jstem/index.php
Rogers, C.B, Kearns, S.A., Rogers, C., Barsosky, J., Portsmore, M., (2001)
Successful methods for introducing engineering into the first grade classroom.
Proceedings of the American Society for Engineering Education, 2001.
Sperling’s Best Place (2005) Lincoln Schools. Retrieved April 30, 2005 from
http://www.bestplaces.net/school
19