VFA Year Two: Show & Tell
September 30, 2014
 Dawn Cullity
 Bernadette Ferro
 Kent Phillippe
VFA Overview
Then
The Voluntary Framework of Accountability
is the first national system of accountability
SPECIFICALLY
for community colleges
by community colleges.
Now
AACC’s Objectives for the VFA
 Improve the user experience
 Increase utility (so we can meet all the goals listed here)
User Experience
• Value-added metrics for accountability (using data to
measure performance and make better decisions)
• National benchmarking for good practices
• Opportunity to streamline reporting / relieve IR
capacity
• Ability to tell a better, more robust story
• Better data to inform public policy and advocacy
Utility
Objective: Improve Data Input
 Feedback on the user set-up and data collection
process/data tool
 Stability of, and access to, the online collection and
display (dashboards)
feedback
 Tool is clunky, especially the online forms, and needs
better validation processes
 Workflow is not intuitive and progress bars does not
tell us where we are in data collection process
improvements
Workflow: Data Configuration
Cohort
Bulk Upload
Online Data
Entry
Raw Data File
Upload
Not Reporting
2008 Cohort (6 year)
O
O
O
O
2012 Cohort (2 year)
O
O
O
O
CTE
O
O
Not Available
O
ABE
O
O
Not Available
O
Submit
Workflow: Managing Data Collection
WORKFLOW ACTIONS
STATUS / NOTES
Configure Data
“Data last configured on…”– with a list data configured (6 year, 2 year, CTE,
and/or ABE) or
“Data needs to be configured”
Upload Data File(s)
If no files to be uploaded show nothing.
If files have been uploaded “data last uploaded on .../../..” and lists each file
type that has been configured and the date uploaded
Edit/View Data
Dimmed if data not configured – “Data needs to be configured”
Dimmed if data upload files are in the queue – “Data in the queue to be
uploaded”
If all data is Processed – allow user to select.
Data Validation Checks
Mark Data for Review
If Data configured – allow for the user to click Mark for Review – ‘Display the
last notes stored by the users of the system for review’
Publish Data
Dashboard and Exports
Takes user to Exports page
Public Outcomes Reports
Dimmed if data has not been made public yet
VFA Manual Data Entry
VFA Cohorts Disaggregated
Objective: Increase Utility of VFA Exports
 Feedback on the use of the VFA and its data
 Dashboards need improvement to be useful and
understandable
feedback
 Colleges need to be able to download final values
including data and calculated percentages
 Benchmarking parameters and policies
 Dashboards need a lot of interpreting before passing to
senior leadership/board
improvements
VFA Export: Outcomes Dashboard
VFA Export: Tabular Data
VFA Benchmarking Dashboard: Benchmarking Selectors
VFA Benchmarking Dashboard: Six-Year Outcomes
VFA Export: Public Dashboard Report
Objective: Improve Data Collection Resources
 Feedback on the resources (instructions)
 Clarity: make the Metrics Manual user friendly,
understandable
feedback
 Consistency
 Authority: be definitive with methodology and how
metrics are to be reported
improvements
Reporting VFA
Six Year Cohort
Two Year Cohort
Main
Developmental
Education Progress
Measures
Cohorts
Two-Year Progress
Measures
Credential Seeking
First Time
In College
Six-Year Outcomes
Measures
Measures
CTE Cohort
Career & Technical
Education Measures
ABE Cohort
Adult Basic Education
Measures
VFA Metrics Manual
VFA Metrics Manual (cont.)
VFA Overview & Training Webinars
http://vfa.aacc.nche.edu/webinars/Pages/default.aspx
Close-Out
 What do you think should be our priorities going forward?
 Questions & Feedback
 Contact VFA at [email protected]
Y2 , Y3, Y4…
Reminders
 Sign-ups open October 1st
 Data collection opens November 3rd
principle
accountability
system
Questions During the Webinar
Data Dashboards
 Have the dashboards been tested with IE, Firefox, and Chrome?
 The VFA dashboards are supported on the following Internet browsers:
 Internet Explorer (IE 8 – IE 10)
 Firefox (16.x – 22.x)
 Google Chrome (22.x – 28.x)
Questions
&
Answers
 Please clarify public vs private dashboards.
 Upon publishing your VFA data, the VFA makes a link available on your College Profile page that takes the public
user to your Outcomes data. The Outcomes data are presented to the user in a packaged PDF report, again made
available on a link from your College Profile page. All information that is contained in your Outcomes Dashboard is
packaged into this public PDF report. The Outcomes data includes the “All Students” (not disaggregated) counts of
the following: Developmental Education Progress, Two-Year Progress, and Six-Year Outcomes Measures; CTE Profile
& Completer/Leaver Outcomes, and ABE Cohort Outcomes. Raw data, disaggregated data, and the benchmarking
tool and data will not be available to the public. We will send the user group a sample in the next few weeks.
 Will the new dashboards show data from previous years’ data collections?
 Yes. The new Outcomes and Benchmarking dashboards will render all data from Year One (2013-14 data collection)
onward.
• The Public Dashboard Report – a PDF report showing the graphs or your published Outcomes data –is only
available for VFA Year Two (2014-15 data collection) ONWARD. To the public, the link to this report will always
be for the most recent year.
 Can we view the names of benchmarking institutions selected?
 Currently, the tool does not return a list of the colleges that are in the benchmarking group. AACC is working with
the VFA Oversight Board and the dashboard developers on both the policy and technical changes needed to make
this happen. We have receive this request and are working toward fulfilling it.
•
Please note that the benchmarking tool will always use selector criterion to establish the colleges in the
benchmarking group; colleges will not be able to pick individual colleges against which to benchmark.
Questions During the Webinar (cont.)
Raw File Upload Process
 More info about the "raw data" upload
 The Raw Data upload process requires the college to upload 5 files:
1. Student demographics,
2. Student developmental education placement,
3. Student course data,
4. Completions at two years, and
5. Completions at six years.
 The data will be extracted, translated and loaded into the VFA database.
 After the ETL is completed, the data from the five files will be permanently
deleted from the system. The VFA does not keep student-unit record data.
 If the raw data upload method will become the "method of choice," will the bulk
upload method eventually be abandoned?
 The VFA has no plan to abandon any of the three data input methods at this
time. We anticipate that many users will move toward the raw upload method
because it is a process where data extraction is streamlined and methodology is
preserved. Like all data initiatives, though, we understand that users work
differently and have preferences for how to enter data.
Questions
&
Answers
Questions During the Webinar (cont.)
Misc.
 What data can be downloaded to Excel?
 All data that you entered for your college can be downloaded to an Excel document,
so that the college may manipulate their own data. Downloading benchmarking data
requires more development and is not available for Year Two.
How is the VFA handling the "English/Reading" combination topic going forward?
 For Year One, the VFA allowed colleges who offered English and reading under a
combined topic to report the progress and outcomes of these students under an
English/Reading combination subject. There was confusion and errors in reporting, so
for Year Two we ask all colleges who offer the combined subject to report it under the
English subject. For these colleges, the reading subject area will be null.
When will the new versions of the VFA Metrics Manual and User Guide be released?
 There are aspects of the Metrics Manual and User Guide that require all other data
collection development to be completed. Therefore, the VFA Metrics Manual will be
released on November 3rd, when data collection opens. The User Guide will be released
soon after.
Is there an easy way to report SAM (Student Achievement Measure) using VFA?
 The data reported for SAM’s associate/certificate model is a roll-up of the VFA’s SixYear Outcomes for the full-time and part-time, credential-seeking cohort. If a college
reports the VFA, it can easily report SAM. The SAM folks have been working with
their developers on a file that helps with getting VFA data into SAM. We will update
you on this post-October 20 when SAM launches. See the Resources section of the
SAM website for more information:
http://www.studentachievementmeasure.org/resources#
Questions
&
Answers