A Better Way to Measure Community
College Performance: An Achieving the
Dream Cross-State Data Initiative
2009 SHEEO/NCES Network Conference and IPEDS Workshop
May 21, 2009
J. Keith Brown
North Carolina Community College System
Success is what counts.
Presentation Summary:
 Overview of Achieving the Dream and the CrossState Data Workgroup
 Recommendations for an Alternative Set of Measures
for Community College Performance
 Introduction of Intermediate Milestones and Final
Measures of Student Performance
 Implications for Policy and Practice
Success is what counts.
Achieving the Dream Overview
 National initiative to help more community
college students succeed (earn degrees, earn
certificates, or transfer)
 Particularly concerned about student groups
that have faced the most significant barriers
to success, including low-income students
and students of color
Success is what counts.
82 Institutions in 15 States
AR, CT, FL, HI, MA, MI, NC, NM, OH, OK, PA, SC, TX, VA, WA
Success is what counts.
Achieving the Dream Values
 Student-centered
 Equity and excellence
 Culture of evidence, inquiry,
accountability, and shared responsibility
Success is what counts.
Cross-State Data Workgroup
Initial States
Connecticut
Florida
North Carolina
Ohio
Texas
Virginia
Success is what counts.
States Joining
Arkansas
Massachusetts
New Mexico
Oklahoma
South Carolina
Washington
Data Workgroup Goals
Develop a set of indicators to:
 More effectively track student performance
 Evaluate the effectiveness of interventions
 Learn from the strengths of other community
college systems
Success is what counts.
Test Drive:
Six States Pilot Better
Ways to Measure and
Compare Community
College Performance
Success is what counts.
Recommended Changes:
Prior Enrollment
Current Federal Method
ATD Work Group Method
First-time-in-college students only Same as federal method
Enrollment Status
Current Federal Method
ATD Work Group Method
Full-time students only
Full-time and part-time students
Intent at time of Enrollment
Current Federal Method
ATD Work Group Method
Only students seeking a
certificate or degree
Same as federal method
Success is what counts.
Recommended Changes:
Timeframe
Current Federal Method
ATD Work Group Method
Three years (150% of
“normal time” to completion)
Six years
Success Outcomes
Current Federal Method
ATD Work Group Method
Earned degree or certificate
- Earned degree or certificate
- Transferred without an award
- Enrolled in year 6 with at least 30 hrs
Success is what counts.
Recommended Changes:
Tracking transfer students within the 2-year sector
Current Federal Method
ATD Work Group Method
Reporting is based on
individual college; doesn’t track
outcomes of students who
transfer to another college;
simply reported as transferred
out
Reporting is based on statewide
community college system;
tracks outcomes within the
system (therefore across
colleges)
Success is what counts.
Recommended Changes:
Controlling for factors associated with success
Current Federal Method
ATD Work Group Method
Part-time students excluded
Disaggregated results by partfrom analysis; no disaggregation time and full-time status and age
of results by age at initial
at initial enrollment
enrollment
Success is what counts.
Results of Cross-State Comparison
Success is what counts.
Results of Cross-State Comparison
AGE DISTRIBUTION OF ENROLLMENT
100%
2%
3%
2%
7%
2%
2%
15%
90%
23%
22%
22%
23%
80%
43%
PERCENT SHARE
70%
60%
50%
40%
83%
74%
76%
75%
74%
30%
51%
20%
10%
0%
Connecticut
Florida
Success is what counts.
North Carolina
22<
23-45
Ohio
46>
Texas
Virginia
Results of Cross-State Comparison
GENDER DISTRIBUTION OF ENROLLMENT
100%
90%
80%
PERCENT SHARE
70%
55%
54%
56%
45%
46%
44%
Connecticut
Florida
North Carolina
53%
54%
56%
47%
46%
44%
Ohio
Texas
Virginia
60%
50%
40%
30%
20%
10%
0%
Success is what counts.
Male
Female
Results of Cross-State Comparison
Success is what counts.
Results of Cross-State Comparison
Distribution of Awards and Transfer w/o Award Outcomes
(all students)
50%
40%
30%
25%
23%
22%
19%
20%
16%
14%
14%
10%
9%
10%
13%
12%
7%
6%
6%
5%
3%
2%
1%
0%
CT
FL
Success
is what counts.
Award of Less Than AA Degree
NC
OH
Associate's Degree
TX
Transfer w/o Award
VA
Results of Cross-State Comparison
Students Who Earned an Award or Transferred OR
Who Are Still Enrolled with 30+ Credits (all students)
100%
90%
80%
70%
60%
50%
9%
8%
40%
30%
8%
9%
20%
10%
3%
5%
36%
42%
40%
35%
30%
23%
0%
CT
FL
Success is what counts.
Total Earned Award or Transfer
NC
OH
TX
Still Enrolled w/o Award with 30+ Credits
VA
Results of Cross-State Comparison
Younger Students (< or = 22) Who Earned an Award or Transferred and Who Are
Still Enrolled with 30+ Credits
100%
90%
80%
Percentage
70%
60%
50%
10%
8%
40%
30%
4%
5%
10%
20%
10%
10%
39%
40%
FL
NC
45%
35%
38%
25%
0%
CT
Success is what counts.
Total Earned Award or Transfer
OH
TX
Still Enrolled w/o Award with 30+ Credits
VA
Results of Cross-State Comparison
PERCENTAGE OF STUDENTS EARNING AN AWARD WHO
SUBSEQUENTLY TRANSFER TO A 4-YEAR INSTITUTION
(FULL-TIME STUDENTS)
100%
90%
80%
72%
PERCENTAGE
70%
60%
47%
50%
40%
36%
30%
20%
54%
54%
22%
17%
15%
12%
10%
5%
7%
7%
0%
Connecticut
Florida
Success is what counts.
North
Carolina
% of Certificate Earners
Ohio
Texas
% of Associate's Degree Earners
Virginia
Results of Cross-State Comparison
PERCENTAGE OF STUDENTS EARNING AN AWARD WHO
SUBSEQUENTLY TRANSFER TO A 4-YEAR INSTITUTION
(PART-TIME STUDENTS)
100%
90%
80%
PERCENTAGE
70%
59%
60%
50%
43%
43%
40%
34%
29%
30%
23%
20%
10%
8%
8%
7%
6%
4%
3%
0%
Connecticut
Florida
Success is what counts.
% of Certificate Earners
North
Carolina
Ohio
Texas
% of Associate's Degree Earners
Virginia
Next Steps: Developing Intermediate
Benchmarks to Measure Student
Progress
Success is what counts.
Intermediate milestones to track students
First-Year Milestones
• Persisted fall to spring
• Passed 80% or more of attempted hours
• Earned 24 or more hours
Second-Year Milestones
• Persisted fall to fall
• Completed developmental math by year 2
• Earned 48 or more hours
Third-Year Milestones
• Passed gatekeeper English or higher by year 3
• Passed gatekeeper math or higher by year 3
Success is what counts.
First-Year Milestones
 Returned spring semester
 16% increase in final success outcomes for
full-time students; 28% for part-time
students
 Earned 24 credits (full-time) or 18 credits
(part-time) by the end of the first year
25% increase in final success outcomes for
full-time students; 66% for part-time
students
 Passed 80% of credits attempted


12% increase in final success outcomes for
full-time students; 46% for part-time
students
Success is what counts.
Second-Year Milestones
 Returned in fall of second year

23% increase in final success outcomes for fulltime students; 53% for part-time students
 Returned and earned 42 credits (full-time) or 24
credits (part-time) by the end of the second year
 32% increase in final success outcomes for fulltime students; 49% increase for part-time
students
Success is what counts.
Second-Year Milestones
 Passed developmental mathematics course by
the end of the second year

84% increase in final success outcomes for fulltime students; 110% increase for part-time
students
 Passed developmental English course by the
end of the third year
 17% increase in final success outcomes for fulltime students; 39% increase for part-time
students
Success is what counts.
Third-Year Milestones
 Passed “gatekeeper” mathematics course by
the end of the third year

45% increase in final success outcomes for fulltime students; 147% increase for part-time
students
 Passed “gatekeeper” English course by the end
of the third year
 17% increase in final success outcomes for fulltime students; 59% increase for part-time
students
Success is what counts.
Tracking toward final success measures
Fourth- and Sixth-Year Measures
• Award of less than associate’s degree w/o transfer
• Award of associate’s degree or higher w/o transfer
• Award of less than associate’s degree and transferred
• Award of associate’s degree or higher and transferred
• Transferred w/o an award
• Still enrolled with 30 or more college hours
• Total success rate
Success is what counts.
Next Steps…finishing data runs
• Run benchmarks at state and institutional levels
• Disaggregate and analyze performance by:
o academic readiness
o income
o ethnicity
o gender
• Identify and document promising interventions
Success is what counts.
What does all this mean and what are
the policy implications?
Success is what counts.
Context of the Data
 Reflects the mission of colleges/state
systems
Examples: age distribution, award
distribution
 Reflects differences in state
priorities/policies
 Example: college transfer policy
 Not all measures pertain to all students
 Example: gatekeeper math course

Success is what counts.
AGE DISTRIBUTION OF ENROLLMENT
100%
2%
3%
2%
7%
2%
2%
15%
90%
23%
22%
22%
23%
80%
43%
PERCENT SHARE
70%
60%
50%
40%
83%
74%
76%
75%
74%
30%
51%
20%
10%
0%
Connecticut
Florida
Success is what counts.
North Carolina
22<
23-45
Ohio
46>
Texas
Virginia
Distribution of Awards and Transfer w/o Award Outcomes
(all students)
50%
40%
30%
25%
23%
22%
19%
20%
16%
14%
14%
10%
9%
10%
13%
12%
7%
6%
6%
5%
3%
2%
1%
0%
CT
FL
Award of Less Than AA Degree
Success is what counts.
NC
OH
Associate's Degree
TX
Transfer w/o Award
VA
Context of the Data
 Reflects the mission of colleges/state
systems
Examples: age distribution, award
distribution
 Reflects differences in state
priorities/policies
 Example: college transfer policy
 Not all measures pertain to all students
 Example: gatekeeper math course

Success is what counts.
Impact of state policy on outcomes
Differences in transfer patterns reflect policy differences
o Encouraging transfer after earning a degree
• e.g. FL – 69% transfer after degree & 7% before
o Encouraging transfer without a degree
• e.g. TX – 25% transfer without degree
o Absence of strong transfer policies
• e.g. OH – 22% transfer after degree & 6% before
o Balanced approach to transfer
• e.g. NC – 16% transfer after degree & 14% before
Success is what counts.
Context of the Data
 Reflects the mission of colleges/state
systems
Examples: age distribution, award
distribution
 Reflects differences in state
priorities/policies
 Example: college transfer policy
 Not all measures pertain to all students
 Example: gatekeeper math course

Success is what counts.
Implications of the Data: College
Perspective
 Identification of at-risk students
 Student advising
 Review of policies/practices
 Examples:

Course taking sequence/timing

Drop/add policies
Success is what counts.
Implications of the Data:
State/System Perspective
 Policy development to improve student
success
 Review of regulations
 Performance indicators
 Development/refinement of student
database
 Benchmarking
Success is what counts.
Implications of the Data: National
Perspective
 More appropriate measure of student
success: accountability
 Financial aid policy
 Expand the body of knowledge on
successful community colleges/practices
Success is what counts.
Acknowledgements
The following individuals are gratefully acknowledged for
their contributions to this presentation:
Chris Baldwin: Jobs for the Future
Pat Windham: Florida
Donna Jovanovich: Virginia
Corby Coperthwaite: Connecticut
Success is what counts.
Achieving the Dream
Success is what counts.
ww.achievingthedream.org
Success is what counts.