Evaluating Student
Computer-Based
Products
Presentation by
Deb Greaney
Presentation Acknowledgements: G. Oggero,
D. Greaney, V. DeWitt
Evaluating Student Computer-based Products Copyrighted B. Porter
Introductions & Thank You’s
Presenter - Deb Greaney
ROE Consultants:


Mary Ann Quivey
Ron Rull
Thanks to


Vicki DeWitt LTC 5
Director
Scott WalthesTech Support
To Make You Comfortable
Please turn off your cell phone/pagers
Take care of yourself
One voice “in the air” at a time
Parking Lot
Purpose
Overview of the Workshop
Handbook & Handouts
 Evaluating Student Computer-based Products
 Handouts
Warm-up
1. Use your unit template and the description
of your Final Team Performance.
2. Find a partner
3. Each pair take 10 minutes each to describe
their projects and products to each
other.…jot a few thoughts or impressions on
the orange worksheet.
4. Hold on to those thoughts…
Do something,
Do anything…..
Just use it!
Do what you can
Do what you want
Just use it!
If You DON’T Use It…
We’ll Lose IT!
Educators need to collectively organize for
the types of learning use that return the
highest value of their technology
resources.
Changing the Pedagogical
Focus for Student Technology
Use
From this…..
Textbook centered
Teacher centered
Direct Instruction
Facts/Answers
Literacy/Adaptation
Prescribed Curriculum
To This…
Resource centered
Student centered
Constructivism, Problembased, Inquiry
Questions
Re-culturing/Transforming
Authentic Learning &
Learning to Learn
The Need for
Technology Use Assessment
Need to Answer the “So What” question
 Federal NCLB & Grant Requirements
 State Accountability
 Local Accountability
Is the technology worth the money & how do
we prove it?
Starting at the Bottom Line
Three Levels of Use
 Literacy Uses
 Adapting Uses
 Transforming Uses
Literacy Uses
Technology Focus:
Acquiring Technology Skills—“just-in-case”
skill learning acquired for possible future
needs
Literacy classes
Learning hardware and software
Students projects are technology focused
rather than expecting standards to
intentionally drive the use of technology for
learning
Literacy Uses
Instructional Focus
Technology-centered pedagogy
Teacher talk is "technology talk” rather than
"learning talk."
Technology uses are organized for their own
sake
Acquiring and assessing technical skills
Offered as separate and/or optional
experiences/programs
Allowed when “real work” is completed or
considered alternative/”reward” activities
Literacy Uses
Staff Development Focus:
Designated “experts” tend to be
self-initiating in learning on their
own.
Other interested staff mostly learn
on their own time and own dime.
Examples of Literacy Uses
Activities include:
 Keyboarding
 Using computers when “real work” is
finished
 Computer literacy classes
 Computer programming
 Computer application classes
 Doing technology for its own sake
 Using curriculum to provide “topics” for
technology use
Adapting Uses
Technology Focus:
Optional/Adaptive Learning Tasks—integrating
is seen as “use it for something,
anything…just use it”
Drill and practice with content software
Instructional games
Productivity tools used to adapt assignments
tasks given without technology in the past
Curriculum provides “topics” for tech uses
Adapting Uses
Instructional Focus
Teacher-centered, direct instruction
pedagogy
Teacher talk is “same stories with new tools”
–there is confusion that new tools make new
instructional stories.
Technology uses are adapted/ provided but
still optional with traditional curriculum
goals.
Teacher and student roles remain the same
Adapting Uses (cont.)
Technology viewed as interesting but
optional and not necessary to achieve
present curriculum goals
Assessment practices are unchanged
Student experiences depend upon
teacher directed assignments
Adapting Uses
Staff Development Focus:


Participation and support while encouraged is
still optional as well as unfocused.
Funding is inadequate – less than 30% of
total technology budget.
Examples of Adapting Uses
Activities include:





Drill and practice software
Instructional games
Integrated learning systems
Using word processing to write themes
Content-specific software
Transforming Uses
Technology Focus:
Essential/Re-culturing Learning Tasks —
integrating is “just-in-time” technology skills
as needed for learning content
standards/projects
Complex learning and thinking tools
Community learning tools
Assessment tools such as rubrics
Productivity tools used to conduct inquiries,
construct meaning, and produce information
Transforming Uses
Instructional Focus
Student-centered, Constructivist pedagogy—
Teacher talk is “new stories with new tools.”
Technology uses enables new learning tasks not
possible without technology
Student roles expand to include explorers,
producers of knowledge, communicators,
and self-directed learners
Transforming Uses (cont.)
Teacher roles expand to include facilitators,
designers, learners, and researchers
Learning and assessment practices are
changed
Students initiate technology uses as they
create their own learning experiences
Development of HOTS fluency through
accessing, processing and communicating
information
Transforming Uses
Staff Development Focus:
Essential skills and practices are
articulated, expected, supported and
measured for all teachers.
Adequate funding of at least 30% of
technology budget.
Examples of Transforming
Uses
Activities include:
 Using technologies as complex learning and
thinking tools
 Student in different schools collaborating to
gather, process and report on a common project
 Students working together to solve real-world
problems with real-world technology tools
 Students using groupware that facilitates
collaborative decisions, writing, brainstorming or
other technology-based, group activities
 Students constructing or organizing, knowledge
useful to others.
Turn to Page 9 & 10
Exercise #1: Where’s The
Flashlight?
Gallery Walk
Pick a partner
Please take 15-20 minutes to discuss and
classify the 20 students tasks on the walls
around the room using colored dots to classify
by level



Literacy – Red dots
Adapting – Blue dots
Transforming – Green dots
Please keep track of one or two tasks that that
provoked more discussion
Student Computer-Based
Artifacts
Definition of Computer-Based—





Multi-media products
Text with graphics
Web-published products
Stand-a-lone slide shows
Video Presentations
http://www.ncrtech.org
Looking at Other Student
Computer-Based Artifacts
Grades 3-12
www.thinkquest.org
Scoring Guides vs. Rubrics
Scoring Guides

Scoring Guide is design to help set and raise
the bar of student achievement
Nationally created
Predetermined standards beyond local
expectations
Completely objective
Scoring Guides vs. Rubrics
Rubric

Rubrics are designed to measure each
student’s performance vis-à-vis the bar set by
the scoring guides
Local assessment
Locally created by teachers and/or
students
Guide student learning as determined
locally
NCREL
Five Uses for Scoring Guides
External Evaluators

Assess the accumulated impact of technology on student
achievement
Internal Evaluators

Internal designated district staff would take a role similar to
external evaluator
Staff Training

Acquire skills on how to use tools and processes to evaluate
their own student products
Peer Collaborative Scoring – Critical Friends

Teachers/Students use Scoring Guides for peer coaching on
quality of products
Teacher/Student Rubric Creation

Scoring Guides used to guide selection of criteria for rubric
creation
Why Scoring Guides
Increase need for developing exemplar skills in
accessing, processing, and communicating
information
Ways to share and learn from each other
Increase our own capacity to deal with the
exponential growth of information
Ability to communicate or demonstrate that
information to others effectively
Introduction to Scoring
Guides—Page 19
Scoring Guides are meant to assess the
quality of:
 students content understanding
 and the technical skills of
communication
So we will be looking at two parts of every
student computer-based product
I. Content Communication
II. Craftsmanship of Communication
Steps to Evaluating Student
Computer Based Artifacts
Step 1: Content Communication: Determine
Type of Communication of Student
Product—Page 19-24




Narrative
Information/Expository
Persuasive
Environments
Step 1: Choosing Your Scoring
Guide for Content Appendix A
Part I—Content Communication—28
Scoring Guides: two for each type-Analytical & Holistic
 Personal expression—Page 40
 Myth/Folk Tales---Page 42
 Additional scoring guides through Page
67
Step 2: Choosing Your Scoring
Guide for Content Appendix A
Part II:
Craftsmanship of
Communication
 One Scoring
Guide--- page 68
(analytic)
Nine Traits of Student Computerbased Product Scoring Guides
Part I: Content Communication– 3 Sections
 Preparation Process – Pg 73
 Content Knowledge – Pg 74-76
 Format/Organization – Pg 77
Part II: Craftsmanship of Communication-- 6 Sections
 Text Communication
 Voice/Sound Communication
 Image Communication
 Design of Communication
 Presentation Communication
 Interactivity of Communication
Note: Do not do ALL 6 on a single product
 Focus on one or two at a time
Exercise 1: Poster Activity
In your group read the assigned Scoring
Guide trait for Part II
Create a poster and be prepared to
present the significant points of that trait to
the group
You will have 15 minutes to prepare your
5 minute (maximum) presentation
Text Communication
Revised role of text in
communication




Amount of text
Word Choice
Fluency
Mechanics
Voice/Sound Communication
Can include sound effects,
music, transition sounds,
recorded speech



Contribution and Relevancy
Emotional Tone/Impact
Technical Quality
Image Communication
Increases the speed and retention of
knowledge






Relevancy
Appropriate Use
Tone/Style/Theme
Charts and Graphs
Technical Quality
Notes: Original artwork, chart colors, data
distortion
Design of Communication
Overall effect of intentional use of all elements
4 Goals
 Get audience’s attention
 Draw audience into information
 Keep attention
 Make a lasting impression
Elements
 Theme
 Layout
 Fonts
 White Space/Color
 Transition & Flow
 Special Effects
 Overall Design
Presentation of Communication
Ranks elements of speaker’s presentation
skills






Eye Contact
Voice
Visual and Auditory Aides
Body and Gestures
Poise and Style
Engagement in Reflection
Interactivity of Communication
Provides users choices in exploring and
learning through the product






User Control
User Orientation
Navigation
Resources
Repetitive Use
Functionality
Exercise #2
Let’s try to rate a product



Choose the analytical scoring guide for type
of communication
Use the applicable sections of the analytical
guide for craftsman ship of communication
As a group, view and discuss the product and
rate its quality
Rate This Product
Emily Dickinson’s
“Hope”
Rate This Product
They Were the Children
of Our Future
By: Megan Clemente
Scoring Guides Online
Demo of Online Scoring
Guide at
http://www.ncrtec.org
Drop down arrow to
Scoring Guide for
Student Products
Click on link to Scoring
Guide for Student
Products
Sample Template Forms
Appendix C





Teacher lesson profile
Student lesson profile
Sample agendas
Rotation Evaluation dialogue
Group Meta-summary Report
Student Product Analysis
Bottom Line…Where is your flashlight
now?



Literacy Uses
Adapting Uses
Transforming Uses
Have we raised your expectations for
student computer-based products?
Questions & Evaluations
Please fill-out your evaluations sheets
•What has worked well with today’s workshop
•What needs to work better
[email protected]
Thank you!