Using the Degree Qualifications Profile:
Institutional Examples of
Promising Practices
Jillian Kinzie, Indiana University & NILOA
Pat Hutchings, Gonzaga University & NILOA
Margaret Bailey, Point Loma Nazarene University
Jennifer Helm, American Public University System
Karan Powell, American Public University System
The Plan
 Brief Introduction to DQP
 NILOA’s role and overview of findings
 Three campus examples
> Point Loma Nazarene University
> American Public University System
> Georgia State University and Georgia
Perimeter College
 Conversation and Questions
What is the DQP?
• What have
you heard?
• Are you using
the DQP on
your campus?
What is the DQP?
• Framework that illustrates
what students should be
expected to know and do -regardless of major
• Specifies five learning
outcomes critical for all
academic programs
•
•
•
•
•
Applied learning
Intellectual skills
Specialized knowledge
Broad knowledge
Civic learning
• Knowledge, skills and
abilities in 5 areas grow,
like the spiral threads of a
spider web
NILOA
NILOA’s mission is to discover and
disseminate effective use of assessment data
to strengthen undergraduate education and
support institutions in their assessment efforts.
SURVEYS ● WEB SCANS ● CASE STUDIES ● FOCUS GROUPS ● OCCASIONAL
PAPERS ● WEBSITE ● RESOURCES ● NEWSLETTER ● LISTSERV ●
PRESENTATIONS ● TRANSPARENCY FRAMEWORK ● FEATURED WEBSITES ●
ACCREDITATION RESOURCES ● ASSESSMENT EVENT CALENDAR ● ASSESSMENT
NEWS ● MEASURING QUALITY INVENTORY ● POLICY ANALYSIS ● ENVIRONMENTAL
SCAN ● DEGREE QUALIFICATIONS PROFILE
www.learningoutcomesassessment.org
NILOA’s partnership with the DQP
NILOA is serving as
information harvester for
all of the funded and
unfunded work currently
going on with the DQP
NILOA and the DQP
• Document what is being done, by whom,
and lessons learned;
• Identify synergies and cross-cutting
issues;
• Provide support to campuses;
• Gather information to help guide the
upcoming revision of the DQP.
www.learningou
DQP Corner
http://www.learningoutcomesassessment.org/DQPCorner.html
NILOA’s DQP Corner
Resources
 New to the DQP
 DQP in Practice
 DQP Resource Kit
 DQP Event Calendar
 DQP Community Forum
 DQP Webinar Series
What We’re Learning
Institutional Control of Institutions Working with the DQP
20 7
142
225
Public
Private not-for-profit
Private for-profit
N/A
403 institutions working with the DQP:
▪ 238 institutions are unfunded
▪ 165 institutions in Lumina funded projects
What is the nature of this work?
Discussion of DQP
Outcome Review
Curriculum Mapping
Transfer
Program Development
Accreditation
Strategic Planning
Assessment
Other
0
50
100
150
No. of institutions
200
250
• DQP as a conversation starter, and a
common vocabulary for talking about
outcomes
• Curricular mapping—where are we teaching
these things, where are the gaps?
• The importance of faculty engagement,
which takes time.
• The challenge of a new vision of
assessment—beyond “exo-skeleton,” built
into classroom assignments, for all
students.
DQP has compelled faculty “to think
collectively about the course sequence
and program as well as degree in ways
we never had before.”
Point Loma
Nazarene University
Degree Qualification Profile
CIC & WASC pilot program 2011-2013
Point Loma Nazarene University, San Diego
DQP Team Leaders:
Dr. Kerry Fulcher, Provost and Chief Academic Officer
Dr. Maggie Bailey, Vice Provost, Program Development and Accreditation
Dr. Maria Zack, Chair Mathematical, Information and Computer Sciences
PLNU – Scope of the pilot program
– Piloted in undergraduate academic programs with a culminating experience for
the major (e.g. capstone, senior seminar, action research, senior exhibition, etc.).
– The DQP Task Force surveyed department chairs to identify what majors had
strong culminating experiences used for degree program summative assessment.
– Seven program directors volunteered their capstone courses for participation in
the DQP pilot in Spring 2013.
– DQP Task Force, identified a subset of DQP learning outcomes they would assess
during the pilot.
– The Task Force agreed to use AAC&U Value Rubrics for assessment (e.g. written
communication, quantitative literacy, critical thinking, information literacy).
– Compared the results of the DQP pilot with institutional results from the ETS
Proficiency Profile and NSSE .
PLNU – Scope of the pilot program
PLNU academic programs participating in the pilot:
•
•
•
•
•
•
Art & Design (dropped out of pilot)
School of Business
School of Education
Literature, Journalism and Modern Languages (2 programs)
Mathematical, Information, and Computer Sciences
School of Theology and Christian Ministry
All DQP reports, assignments, surveys and supporting materials are on
PLNU’s public website:
https://portal.pointloma.edu/web/institutional-effectiveness/dqp
https://portal.pointloma.edu/web/institutional-effectiveness/home
DQP pilot program Goals (original design)
•
Develop a set of GE learning outcomes that can be externally benchmarked and also
incorporate the unique aspects of institutional mission. (NOTE: shifted to Program Learning
Outcomes)
•
Provide additional accountability for progress on student learning outcomes
•
Provide a record of the challenges a that a mission driven university faces when engaging in a
national conversation around basic learning expectations for an undergraduate education
while maintain the mission aspects in its learning outcomes
•
Provide a clear set of reference points for continuous improvement of the General Education
program
•
Develop a common language around student learning for curricular improvements in the
institution
•
Expedite the academic programmatic approval and implementation process
•
Form a basis for developing a system of external benchmarking with participating institutions
with similar faith-based mission
Accomplishments & findings
• Compared three sets of data:
1. NSSE
2. ETS Proficiency Profile
3.
Pilot DQP outcomes and assessments
• Movement towards culminating experience in all majors
• Use of AAC&U Value Rubrics (adopted these for WASC Core
Competencies)
• DQP Intellectual Skills include the WASC Core Competencies
• Meaning, value, and integrity of the degree
• Key conversation about linking Program and GE Student Learning
Outcomes (begins with the mapping of the core competencies)
Challenges
1. The complex language of the DQP outcomes
2. Cross disciplinary comparisons
3. Faculty engagement was not uniform
4. In some cases faculty viewed DQP as an “add on” rather
than an enhancement
5. Academic leadership in the participating programs was key
Going forward
During the DQP pilot program PLNU was engaged in important curricular
conversations and work to reframe the PLNU vision around the meaning
and value of the degree in the context of student success. These
conversations will continue as the university commits to curricular and
co-curricular continuous improvement in the midst of disruptive changes.
– Examination of DQP Pilot, NSSE, and ETS results for continuous
improvement
– GE revision of Student Learning Outcomes and program review
– Culminating experience for every major (an issue addressed in program
review)
– DQP Learning areas for culminating/ capstone experience
– Cross-disciplinary comparisons and linking to GE SLOs
– WASC core competencies (Intellectual Skills), AAC&U Value Rubrics, and
mapping that links GE and Program Outcomes
American Public
University System
American Public University System
American Public University System
has been dedicated to educating
those who serve since our doors
first opened in 1991. The
American Public University System
(APUS) is a fully online system,
encompassing both American
Military University and American
Public University. We are
regionally accredited by the Higher
Learning Commission (HLC) of the
North Central Association.
About APUS
• AMU was founded in 1991 to serve active duty and retired military members,
became APUS in 2001, comprising AMU and APU.
• Initially accredited by DETC in 1995, now regionally accredited through NCA/HLC
since 2006
• Currently over 100,000 students worldwide
• Approximately 60% military, average age 35, open enrollment
• Over 90 degree programs; 21 associate’s, 40 bachelor’s, and 30 master’s level,
plus certificates
• Monthly course starts of 8 or 16 weeks
• Courses offered are 100% online in an asynchronous environment
• Headquarters and executive offices – Charles Town, WV
• Administrative offices – Manassas, VA
Implementing the DQP Framework at APUS
•
•
•
•
•
•
•
•
Strengthen the overall university and discipline level
curriculum.
Further enhance the curricular mapping of courses and
programs.
Enhance our readiness to implement revisions to our core
learning curriculum from a strategic perspective.
Ensure that APUS curriculum is current and relevant with
industry standards.
Assist in our focus on the discipline and the integration of
the field of study.
Provide transparency to the student by establishing
expectations of the student life cycle, helping them learn
what leads to a career and the next level of study.
Assist students with taking responsibility and ownership
of their own learning.
Ensure that our students are equipped with the
knowledge, skills, and abilities they need for success in
work, citizenship, global participation, and life.
Dialogue and Planning
•
•
•
•
Assembled leadership team
Paul Gaston visit
Introduced DQP framework
Stakeholder input
2012
The DQP leadership team sought
input from stakeholders, defined
assumptions, developed a
timeline of action, defined
project goals, benchmarked
schools, and created sample
maps
Implementation
• Core learning mapping
• DQP resources developed
• Incorporate into new program and
• DQP review meetings conducted
review processes
• Program Directors mapped programs
to the DQP framework to determine • Develop learning communities to
share DQP related assignments and
alignment and identify gaps
assessments
• Working sessions
2013
Decisions on initial program changes
recommended by year end 2013 for
implementation in 2014
Six new institutional learning outcomes
developed – five DQP areas of learning
plus digital information literacy.
2014
Student learning data
evaluated to determine impact
of DQP framework on student
learning and success
Support and Resources Important to Success
APUS DQP Leadership
Committee and Review Teams
Trends and Themes Across Programs
• The current process of identifying gaps in the programs is serving as a
tool/framework for a deep dive analysis of the curriculum and providing a
roadmap for future program development.
• The overall curriculum is being enhanced to address the civic skill area of
learning which encourages students to take an active role in the community
(work, service, co-curricular activities) and examines civic issues encountered
and insights gained.
• The overall curriculum is being enhanced to address the broad integrative skill
area of learning which encourages instructional methods to expand beyond the
discipline or field of study to ensure an inter-disciplinary approach for students.
• Instructions on assignments and syllabi are being clarified by instructors to
increase student understanding of expectations.
Trends and Themes Across Programs
• Program objectives are being modified and enhanced to ensure they align
with core course learning objectives and assignments.
• Course assignments are being revised and enhanced to ensure the
assessment methods accommodate various learning styles to enhance the
student learning experience.
• Programs are continuing to recognize the need to communicate in multiple
media formats in providing students with enhanced options for responding to
forums and creating assignments.
• The process is providing a means for faculty to gain a holistic picture on how
their courses fit into the overall progression and sequencing of curriculum.
• The process is providing a solid framework for the progression and
sequencing of courses through the AA, BA, and MA levels from the student
perspective.
Georgia State University
and
Georgia Perimeter College
AASCU DQP Project
• Exploring student transfer - AA to BA
• Leveraging characteristics of the DQP
to provide a better means of predicting
and aiding student success in the
transition from Associate’s- to
Bachelor’s level education than
traditional achievement
markers such as course
grade and cumulative GPA
Institutions Involved in AASCU DQP
Project
Georgia Perimeter College (GPC)
• 2-year, Associates granting
• Highly Diverse (60% non-white)
• 25,000 students on 6 campuses
Georgia State University (GSU)
• Research One institution
• Highly Diverse (60% non-white)
• 32,000 students
IMPORTANT: Between 6,000 and 7,000 current
GSU students began their studies at GPC
AASCU DQP Project: Developing the DQP
Step One
Establish a DQP for both Associate’s
and Bachelor’s students in each of the
three targeted disciplines (Bio, Psych,
Criminal Justice).
How would faculty members in specific
disciplines across institutions go about
developing a Degree Qualification Profile for
their fields? What are the obstacles? What
are the advantages?
36
AASCU DQP Example -- Degree Profile Matrix:
Psychology
Learning Goals
I. Knowledge, Skills, and Values
Consistent With the Science and
Application of Psychology
Goal 1: Knowledge Base of
Psychology
Students will demonstrate familiarity with the major concepts,
theoretical perspectives, empirical findings, and historical trends
in psychology.
Goal 2: Research Methods in Psychology
Students will understand and apply basic research methods in
psychology, including research design, data analysis, and
Interpretation.
37
AASCU DQP : Degree Profile Matrix:
Psychology GSU and GPC
Specialized Knowledge
(APA Goals 1, 10: Knowledge
Base, Career Planning &
Development)
GSU
Civic Learning
(APA Goals 5, 8, 9: Values,
Sociocultural
and Interna onal Awareness,
Personal Development)
Applied Learning
(APA Goals 2, 4: Research Methods, Applica on of Psychology)
GPC
Broad,
Integra ve
Knowledge
(APA Goal 6: Informa on &
Technological Literacy
Intellectual Skills
(APA Goals 3, 7:
Cri cal Thinking, Communica on Skills)
Figure 1: Degree Qualifications Profile for Psychology at Georgia Perimeter College and Georgia State University.
Lines indicate the degree to which APA-defined goals within the five cores are pursued within the psychology major
at the associate’s level (GPC, yellow) and the bachelor’s level (GSU, blue).
38
AASCU DQP: Measuring DQP Competencies
Step Two – The Assessment Question!!
Establish a means of determining student
DQP attainment in each of the selected
disciplines.
How to assess DQP competencies fairly and accurately?
Can these assessments reasonably be used on an
individual student level to gauge past achievement and to
predict future performance? Can they be used on a
program level?
39
AASCU DQP: Degree Profile Matrix: Psychology
First Day Assessment
Instructions. Without using a calculator, please find the solution to each of
the problems below. Show and label all of your work on a separate piece
of paper; put only the answers in the space provided below.
Your
performance on this assessment will NOT affect your grade in this course;
it will not be graded
Estimating Answers (Estimate whole-number answers to the problems)
1. (7.02)2
2. 1,989 X 8.01
Rounding Numbers (Round numbers to the nearest tenth)
3. 5.06
4. 0.46
5. 10.348
Decimals
Add:
6. 3.12 + 6.3 +
2.004
Subtract:
7. 3.2 – 1.135
Multiply:
8. 0.35 * 0.162
Divide:
9. 0.065 / 000038
Subtract:
11. 1 - 3
Multiply:
12. 4 * 3
Divide:
13. 7 ¸ 1
Fractions
Add:
10. 1 +
8
40
3
3
+
4
16
2
8
5
4
9
4
AASCU DQP – Some Preliminary Lessons
• The DQP is a positive “disrupter” in faculty
curricular discussions.
• The DQP is effective in
raising for faculty members
issues central to successful
transfer.
• The usefulness of the DQP to the transfer
process hinges on developing accurate,
trusted and easily admissible ways of
assessment. (Parallels to prior learning)
Contact Tim Renick, Vice Provost, Chief Enrollment Officer at
Georgia State University for more information.
41
Discussion
1. Familiar themes?
2. Questions?
3. Challenges you’re facing?