Evaluating Open
Educational Resource
(OER) Objects
Rubric V: Quality of Technological
Interactivity
CC BY Achieve 2013
Open Educational Resources:
[With Webster’s Definitions]
O
Open: Containing an open license - no restrictions on remixing or reusing
E
Educational: Used for teaching and learning
R
Resource: Object used to support an effort or task
[A source of support or aid, especially one that can be readily
[Having no enclosing or confining barrier; not restricted to a particular group
or category of participants]
[Pertaining to the action or process of educating or being
educated]
drawn upon when needed]
2
Open Educational Resources (OER)
What are OER?
• OER are teaching, learning, and research resources that contain an open
license.
• They provide extraordinary opportunities for educators to freely share
knowledge and resources.
• They offer great potential for instructional innovation and networks for
sharing best practices.
• There are millions of OER objects available online.
3
Open Educational Resources (OER)
OER objects may include (but are not limited to):
• Images
• Applets
• Games
• Worksheets
• Lesson plans
• Original-source texts
• Assessments
• Units
• Textbooks
• Teacher/learner support materials
o NOTE: Any smaller component of a more complex object, that can exist as a stand-alone,
alsoqualifies as an “object” (e.g. a unit in a textbook, a lesson in a unit, or an activity in a lesson).
4
The Achieve OER Evaluation Rubrics
How and why were the Achieve OER Rubrics created?
• Educators using the vast system of sharing need a method for filtering OER to meet
the needs of their students.
• The purpose of the rubrics is to provide a structure for evaluating an online resource
in a systematic, purposeful and comprehensive way.
• There are two ways of approaching the evaluation of a resource:
o Holistically evaluating the object with a single rating, (similar to the star-rating
systems used on Netflix, Yelp and Amazon)
o Separately evaluating each of the components of quality, as with the Achieve
OER Rubrics.
• The rubric criteria are based on Achieve protocols used to assist states in alignment
and quality review studies.
• The Achieve OER Evaluation Tool, hosting the rubrics on OERCommons.org, allows
a user to identify, evaluate and sort objects based on the specified essential
elements of quality.
5
Applying the Achieve OER Rubrics
How do the Achieve OER Rubrics work?
• The rubrics represent an evaluation system for objects found within Open
Educational Resources.
• They are hosted as an online evaluation tool on the repository,
OERCommons.org, but can also be used independently.
• They are applied to any content area.
o
NOTE: At this stage only Common Core Standards for English Language Arts/Literacy
and Mathematics are available in the Achieve OER Evaluation Tool. Content standards
for History, Social Studies, Sciences, and technical subjects are not yet available.
• Each rubric is applied independently to an object.
• They rate the potential, not the actual, effectiveness of an object.
• They use a five-point scoring system to describe levels of quality.
6
The Achieve OER Evaluation Tool Rubrics
This presentation will focus on Rubric V:
Rubric I. Degree of Alignment to Standards
Rubric II. Quality of Explanation of the Subject Matter
Rubric III. Utility of Materials Designed to Support Teaching
Rubric IV. Quality of Assessment
Rubric V. Quality of Technological Interactivity
Rubric VI. Quality of Instructional Tasks and Practice Exercises
Rubric VII. Opportunities for Deeper Learning
o NOTE: Sometimes one or more of the rubrics may not be relevant to a particular
object. In those cases, a rating of N/A is appropriate.
7
Rubric V:
Quality of Technological Interactivity
How and Why is Rubric V Applied?
• “Interactivity” is used broadly to mean that the object responds to the user,
or one that behaves differently based on what the user does.
• Applied to objects that are designed with a technology-based interactive
component,
• Used to rate the degree and quality of the interactivity of that component
(Not a rating for technology in general but for technological interactivity),
• For interactive objects designed for student use, the rubric is applied to the
entire object.
• For objects designed for teacher use, for example lesson plans, which
may include interactive components for use by students, the rubric is
applied only to those interactive parts.
8
Rubric V:
Quality of Technological Interactivity
When does Rubric V not apply?
• Rubric V is not applicable if an object does not have an interactive
element.
• This is not a rating for technology in general, but only for technological
interactivity between the user and the object.
• The rubric does not apply to how students interact with each other, but
rather how the technology responds to an individual student user.
• Does not apply to objects where interaction with the object is limited to, for
example, opening a PDF attachment.
9
Rubric V:
Quality of Technological Interactivity
3: An object is rated superior only if all of the following are true:
• The object, or interactive component of an object, is responsive to student
input in a way that creates an individualized learning experience. This
means the object adapts to the user based on what s/he does, or the
object allows the user some flexibility or individual control during the
learning experience.
• The interactive component is purposeful and directly related to learning.
• The interactive component of the object is well designed and easy to use,
encouraging learner use.
• The interactive component of the object appears to function flawlessly on
the intended platform.
For example an interactive student-directed game is considered superior for Rubric V because it
responds to individual students by increasing the level of challenge when students successfully
complete sections of questions and offers hints when incorrect answers are given and moves the
student to intervention questions when they are unable to give the correct answer.
10
Rubric V:
Quality of Technological Interactivity
2: An object, or interactive component of an object, is rated strong if it has
an interactive feature that is purposeful and directly related to learning, even if
it does not provide an individualized learning experience. Similarly to the
superior objects, strong interactive objects must be well designed, easy to
use, and function flawlessly on the intended platform.
For example an object has a component that reads aloud sections of text,
making it easier for some students to stay focused or to hear the voice in a
passage of literature. Even though this interactive feature is well designed,
purposeful, and directly related to the learning, there is no personal feedback
or individualized response. This object is considered strong for Rubric V.
11
Rubric V:
Quality of Technological Interactivity
1: An object, or interactive component of an object, is rated limited if its
interactive element neither relates to nor detracts from learning.
This kind of interactive element might be designed to increase student
motivation and to build content understanding by rewarding or entertaining
the learner and extending the time the user engages with the content.
For example even though the interactive element of a student-directed game
may motivate the student to maintain the connection with the object, it is still
considered limited for Rubric V. The game’s interactive feature includes
earning points, leading to achieving levels, as a reward for correct answers.
The interactivity component does not connect directly to the subject matter,
provides no opportunity to learn from incorrect answers, or, in some cases,
even to recognize when an incorrect answer was given.
12
Rubric V:
Quality of Technological Interactivity
0: An object, or interactive component of an object, is rated very weak or no
value if it has interactive features that are poorly conceived and/or executed.
The interactive features might fail to operate as intended, distract the user, or
unnecessarily take up user time.
For example in a student-directed game a group of clowns tumble across the
screen with each correct answer, distracting the student and slowing their
progress. This interactive feature is considered very weak or of no value.
13
Rubric V:
Quality of Technological Interactivity
N/A: This rubric is not applicable (N/A) for an object that does not have an
interactive technological element.
Rubric V is not applicable if there is no interactive feature in the object or if
technological interaction with the object is limited to, for example, opening a
user-selected PDF.
14
Using the Achieve OER Evaluation Tool On
OERCommons.org
Rubric V
Achieve OER Evaluation Tool
on OERCommons.org
OERCommons.org hosts the rubrics as the Achieve OER Evaluation Tool
o NOTE: Specific instructions for Using the OERCommons Website to search for and evaluate
objects, using all of the rubrics, can be found in another slide presentation of this training
package.
16
Rubric V:
Quality of Technological
Interactivity
Achieve OER Evaluation Tool
Using OERCommons.org to
evaluate an object for its quality
of technological interactivity:
After you have evaluated an
object using Rubrics I through
IV, you should be ready to apply
Rubric V.
Begin by clicking on
at the end of Rubric IV or on the
title line for Rubric V.
17
Rubric V:
Quality of Technological
Interactivity
Achieve OER Evaluation Tool
To view a video about Rubric V
click here…
18
Rubric V:
Quality of Technological
Interactivity
Achieve OER Evaluation Tool
Hover over the rating numbers
to see an abbreviated version of
the rating descriptors.
Review the object with your
focus on the quality of the
technologically interactive
feature(s) included in the object
and select the appropriate
rating.
19
Rubric V:
Quality of Technological
Interactivity
Achieve OER Evaluation Tool
To download a full text version of
the rubrics click here:
20
Rubric V:
Quality of Technological
Interactivity
Achieve OER Evaluation Tool
Across the bottom of Rubric V you
see options to:
• Leave a [+ Comment],
• Clear rating,
• [Save & Go to the Next
Rubric] (You can also click on
any rubric title to leave Rubric V,
but your ratings will not be
saved.)
21
Achieve OER Evaluation Tool
on OERCommons.org: Rubric V
Now you are ready to rate the object using the Achieve OER
Evaluation Tool’s other rubrics. Rubrics can be used in any order
and can be selected by simply clicking on the rubric title.
Remember to click
that your work is saved.
after each rubric to ensure
You can tell if your work was saved by the symbol at the
beginning of each rubric title:
indicates not yet rated or
rating was not saved
indicates your rating has been saved
22
Slides developed by Achieve:
www.achieve.org
1400 16th Street, NW / Suite 510
Washington, DC 20036