MAT 1000
Mathematics in Today's World
Last Time
1. Collecting data with experiments
2. Practical problems with experiments
Today
Measurement
Validity
Error
Bias
Variability
Measurement
Recall structure of data: individuals and
variables
Measurement is the assignment of a value
to a variable.
Measurement
Example
Unemployment rate
Individuals: working age American adults
Variable: employment status
Values: employed or unemployed.
To measure, ask an individual: “are you
employed?”
Measurement
Example
Heights of MAT 1000 students
Individuals: MAT 1000 students
Variable: height
Values: numeric
To measure, use a measuring tape, yard stick,
etc.
Measurement
An instrument is a device or method for taking
a measurement
Measuring height: different possible
instruments, like measuring tape or yard sticks
Unemployment: the instrument is the question
asked
Measurement
Some instruments give measurements in
“units.”
Tape measure could give height measured in
inches, or in centimeters.
Validity
Are we really measuring the attribute we are
interested in?
Example
University admission boards want to measure
how prepared applicants are for college.
One way this can be measured is with
standardized test scores (SAT or ACT score)
Validity
A valid measurement is an appropriate
representation of the property being
measured.
An extreme example: measure college
readiness by height. Clearly not valid.
Validity
But are ACT test scores even a valid
measurement of college readiness?
Some people argue that they are not—that ACT
scores don’t tell us if someone is ready for
college.
Validity
This is a common scenario. The question we
are interested in is somewhat vague, and we
have to try and find an appropriate (valid) way
to measure.
How else could we measure “college
readiness”?
Predictive Validity
Can we get evidence that a particular
measurement like ACT scores is valid?
Yes. We look if ACT scores can predict how well
an applicant will do in college.
If a measurement can predict success on a
related task, it has predicitive validity.
Predictive Validity
If applicants with ACT scores tend to do well in
college (i.e. high GPA), this gives evidence that
the ACT is a valid measurement of college
readiness.
Conclusion: one way to decide if a
measurement is valid is to ask if it has
predictive validity.
How else can we decide?
Rates
Measuring highway safety.
One method is to count deaths from car
accidents.
In 1994 there were 40,676 traffic fatalities in
the U.S.
In 2003 there were 42,643.
Was driving becoming less safe?
Rates
What else changed from 1994 to 2004?
Higher population: 260 million to 290 million
More drivers: 175 million to 196 million
More miles driven: 2,359 billion to 2,890 billion
So is it valid to compare highway safety from
1993 to 2003 using the number of traffic
fatalities?
No.
Rates
A more valid measurement is to use a rate, like
traffic fatalities per number of drivers, or traffic
fatalities per mile drive.
1993: 23.23 fatalities per 100,000 drivers
2003: 21.74 fatalities per 100,000 drivers
1993: 1.7 fatalities per 100,000,000 miles
2003: 1.48 fatalities per 100,000,000 miles
Rates
2012 (most recent year available):
33,561 fatalities, compared to 42,643 in 2004.
Are we safer on the road?
Yes, but we have to use rates to see this
Even though the count is lower in 2012, that
doesn’t make it a valid measurement.
Rates
Rates from 2003 to 2012:
2003: 21.74 fatalities per 100,000 drivers
2012: 15.84 fatalities per 100,000 drivers
2003: 1.48 fatalities per 100,000,000 miles
2012: 1.13 fatalities per 100,000,000 miles
Validity
Is a measurement valid?
Does it have predictive validity?
If we use a count to measure, would a rate
perhaps be better?
But, even if we believe our measurement is
valid, there are still potential sources of error in
the measurement process.
Sources of error
An error in a measurement is a discrepancy
between the measured value and the true
value.
“Error” does not mean mistake.
Error is inevitable in any measurement.
Sources of error
Two different sources of errors in
measurements:
1. Bias
2. Variability
Sources of error
Bias: comes from the way we measure.
Systematically wrong in the same direction.
Example
Using police reports to measure public safety.
Problem: not all crimes are reported to the
police.
Result: a biased measurement.
Sources of error
How can we reduce bias?
Bias comes from instruments or measuring
procedures.
Conclusion: the only way to reduce bias is to
improve the instrument or use a better
procedure.
Sources of error
Variability: repeated measurements of the
same individual give different values.
Example
Analog bathroom scale: each time you step on
you may get a slightly different measurement
of your weight.
Sources of error
All measurements will have some variability.
Example
Measuring time with an atomic clock.
Every 10 days the NIST reports the amount of
error in time measurements.
Some recent values: 6.0 ns, 5.9 ns, 5.5 ns, 5.2
ns
Note that 1 ns = 0.000000001 seconds
Sources of error
If a measurement has small variability, then
repeated measurements of the same individual
will be close to each other.
If a measurement has small variability, we say
it is reliable.
Sources of error
One way to reduce variability (make a
measurement more reliable):
Take several measurements, and then take the
average.
Variability causes measurements to be
randomly scattered around the true value—
some too small, some too large.
Averaging several measurements will reduce
the affect of this scattering.
Sources of error
The two sources of error—bias and
variability—are independent of each other.
A measurement could have high bias but low
variability, or vice versa.
The following picture (which we have seen
before) is helpful in understanding the two
sources of error:
Sources of error
Consider shooting arrows (measurements) at a target (the true value):
Bias means the archer systematically misses in the same direction.
Variability means that the arrows are scattered.
Sources of error
Whenever we collect data, we make
measurements.
1. Make sure measurements are valid (look
for predictive validity, or use a rate
instead of a count)
2. Measurement = true value + bias +
variability
3. Averaging several measurements can
reduce variability
Sample surveys are measurements
A key example of a measurement is a
sample survey.
We attempt to measure a parameter using
a statistic, but our measurement may have
some bias, and will certainly have variability
statistic = parameter + bias + variability
Bias, Variability, and Validity
Keep in mind that bias and variability are
both independent of validity.
We can have a measurement that has very
low bias and low variability, but if it isn’t
valid, it’s useless.
We can measure height very accurately,
but that doesn’t make it a valid way to
measure college readiness.