Reliability
Module 6
Activity 5
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
Overview
In this presentation, you will learn about reliability. The following
topics will be addressed:
1. What is reliability?
2. Different types of reliability
3. Calculating internal consistency
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 2 of 29
What is reliability?
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
Reliability
Reliability means repeatability or consistency.
A measure is considered reliable if it gives us the same result
over and over again.
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 4 of 29
Reliability
Reliability is a necessary but not sufficient condition for validity.
Remember, validity is the extent to which a test measures what
it is intended to measure.
For a test to be valid, its results must be reliable. But it must also
meet other conditions as well.
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 5 of 29
Reliability
Reliability is associated with statistics.
It is a number that can range from 0.00 (unreliable) to 1.00
(absolutely reliable).
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 6 of 29
Reliability
Reliability refers to the results we get from a test or
measurement instrument, not to the measurement instrument
itself.
For example, imagine that I give a reading test to a class of
children and the reliability is .23 (very low). I give the same test to
a class of teenagers and the reliability is .98 (high).
I would say that the results of the test are not reliable for the
class of children, but they are very reliable for the class of
teenagers.
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 7 of 29
Different types of reliability
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
Different types of reliability
There are three types of reliability:
Consistency over time
Consistency across equivalent/parallel forms
Internal consistency
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 9 of 29
Consistency over time
Consistency over time is also known as test-retest reliability.
It answers the following questions:
Are test scores reliable over time?
Would students get the same score if I gave them the same test
in a week?
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 10 of 29
Consistency over time
To examine test-retest reliability, you can give the same
students the same test at two different times.
Their scores from the first time and the second time should be
about the same.
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 11 of 29
Consistency over equivalent
forms
Consistency over equivalent forms is also known as parallel form
reliability. It answers the following questions:
Are the test scores reliable over different forms of the same test?
Would the student get the same score if given a different form of the
test?
When we say form we are referring to a physical test. A different form
of the test means that the two tests are assessing the same
information, but the items are different.
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 12 of 29
Consistency over equivalent
forms
To examine parallel form reliability, you can do the following:
1. Make two tests that are the same difficulty.
2. Give both tests to the same group of students.
3. Compare their scores. Their scores on both tests should be
about the same.
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 13 of 29
Internal consistency
Internal consistency refers to how well the test items work
together to provide a consistent measurement. It answers the
following questions:
Is the test internally consistent?
Are the items consistent with each other?
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 14 of 29
Internal consistency
There are different ways to measure internal consistency.
The most commonly-used method is called Cronbach’s Alpha.
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 15 of 29
Calculating internal
consistency
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
Internal consistency
In the next several slides, you will learn how to calculate internal
consistency using Cronbach’s Alpha.
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 17 of 29
Cronbach’s Alpha
Go to languagetesting.info
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 18 of 29
Cronbach’s Alpha
Click on “statistics”
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 19 of 29
Cronbach’s Alpha
Scroll down until
you see
“Cronbach’s
Alpha” on the left.
Click on “Excel
Download”
Open the file.
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 20 of 29
Cronbach’s Alpha
You will see an Excel
spreadsheet that looks
like this.
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 21 of 29
Cronbach’s Alpha
Now, you can enter your data. Enter your students’ names.
Enter a “1” if they got the first item right. Enter a “0” if they got
the item wrong.
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 22 of 29
Cronbach’s Alpha
Now, look at the bottom of the screen. You should see this:
Click on “Results”
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 23 of 29
Cronbach’s Alpha
You will see something that looks like this:
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 24 of 29
Cronbach’s Alpha
This will tell you the average (mean)
of the test. It will tell you the standard
deviation (SD). And, it will tell you
Cronbach’s Alpha (alpha).
Cronbach’s Alpha for this test is 0.75.
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 25 of 29
Cronbach’s Alpha
So, how high should reliability be? Remember, the range is
between 0.00 and 1.00.
For teacher-made tests, reliability should be about 0.60 – 0.85.
For high-stakes tests, reliability should be higher.
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 26 of 29
Assignment #4
Write a one to two paragraph response to the following
questions. This is an off-line, non-graded assignment.
Why is reliability important?
Why should we expect to see estimates of reliability?
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 27 of 29
Key Terms & Concepts
New Terms:
Reliability
Test-retest reliability
Parallel form reliability
Internal consistency
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 28 of 29
Next Steps
Now you should do assignment #5. In this assignment, you will
update your assessment glossary.
This is a program of the U.S. Department of State administered by
FHI 360 and delivered by Northern Arizona University.
PARTNER
LOGO
Slide 29 of 29