AAAS Evaluation Capacity Building Project with AGEP Leaders
& SBE Chairs/Faculty Leaders
The overall objective of this 3 year evaluation capacity building
project with the AGEP leaders and departments are to:
 Identify and collect quantitative indicators related to
recruitment & retention & progression to the PhD in SBE fields
 Assess changes in SBE departmental practices
 Suggest qualitative indicators for use in your evaluations
related to student skills development, departmental
transformation, and Alliance impact
 Help SBE Chairs and faculty leaders better understand how to
create an evaluation culture within the department related to
graduate student affairs and broadening participation and
success of URM
Quantitative Indicators
Recruitment
 New or first time graduate students enrollees in Masters
Programs and in PhD Programs
Retention
 Cohort tracking of new URM enrollees in SBE, starting with
2006/2007 cohort
2 to 3 year benchmark – earned Masters or passed qualifying
exams
Question
Should this data be collected for students who were in your
URP/REU and entered graduate school at another insititution?
Suggested Qualitative Indicators
Assessments/Perceptions about Graduate Students Performance,
Research Abilities, and Professional Behaviors
 Performance assessment – course performance, qualifying
examinations, and research
 Faculty mentor perceptions about graduate student --- students’
academic strengths/weaknesses, understanding of the research
process, analytical skills, critical thinking, science
communications (writing and oral), professional behavior
(research safety and ethics), interpersonal interactions, and
overall progression to the PhD
 Graduate student perceptions about their performance, research
abilities, professional behaviors, and progression to PhD
Perceptions about Research Abilities

Conduct literature searches and reviews

Use databases

Formulate a research question

Develop a research plan, including methods,
data to be collected

Use of equipment

Ability to analyze and display data

Ability to discuss experimental findings

Ability to write the research report
Perception about Professional Behavior
 Class attendance & participation
 On time arrivals
 Research lab attendance & time spent in research lab
 Departmental seminar attendance & participation & ability to
ask appropriate questions
 Attendance at departmental social functions
 Ability to read and understand journal articles
 Ability to prepare a poster abstract, poster, paper
 Ability to deliver an oral presentation
 Interpersonal interactions
Perceptions about Skills, Knowledge, & Environment
 Understanding of science research safety and ethics
 Use or research equipment
 Knowledge about contemporary issues in your field
 Understanding of how research impacts society
 Critical thinking skills
 Satisfaction with administrative issues (timely info about graduation
requirements, adequate financial aid and timely disbursements)
 Family Life/Work Balance (support from significant others)
 Departmental Climate
Institutional Transformation Indicators
 What are the sustainable changes at the administration level, in
regards to SBE recruitment and progression to the PhD?
 What are the sustainable changes in the SBE departmental
admission/selection process?
 What are the sustainable changes in SBE graduate school
academic curriculum and student academic support?
 What are sustainable changes as related to graduate student
retention?
Psychology

801 PSYCHOLOGY, COMBINED
42.01 PSYCHOLOGY, GENERAL

802 PSYCHOLOGY, EXCEPT CLINICAL
42.03 COGNITIVE PSYCHOLOGY AND PSYCHOLINGUISTICS
42.04 COMMUNITY PSYCHOLOGY
42.07 DEVELOPMENTAL AND CHILD PSYCHOLOGY
42.08 EXPERIMENTAL PSYCHOLOGY
42.09 INDUSTRIAL AND ORGANIZATIONAL PSYCHOLOGY
42.11 PHYSIOLOGICAL PSYCHOLOGY/PSYCHOBIOLOGY
42.16 SOCIAL PSYCHOLOGY
42.17 SCHOOL PSYCHOLOGY
42.99 PSYCHOLOGY, OTHER
51.2301 ART THERAPY
51.2303 HYPNOTHERAPY
51.2705 PSYCHOANALYSIS

803 CLINICAL PSYCHOLOGY
42.02 CLINICAL PSYCHOLOGY
Social Sciences

901 AGRICULTURAL ECONOMICS --01.0101 AGRICULTURAL BUSINESS AND
MANAGEMENT, GENERAL, 01.0102 AGRICULTURAL BUSINESS/AGRIBUSINESS
OPERATIONS, 01.0103 AGRICULTURAL ECONOMICS

902 ANTHROPOLOGY (CULTURAL AND SOCIAL) 45.02 ANTHROPOLOGY, 45.03
ARCHAEOLOGY

903 ECONOMICS -- 45.06 ECONOMICS, 52.06 BUSINESS/MANAGERIAL ECONOMICS

904 GEOGRAPHY-- 45.0701 GEOGRAPHY, 45.0702 CARTOGRAPHY

905 HISTORY AND PHILOSOPHY OF SCIENCE , 45.0804 HISTORY AND PHILOSOPHY
OF SCIENCE

906 LINGUISTICS, 16.0102 LINGUISTICS

907 POLITICAL SCIENCE/PUBLIC ADMINISTRATION -- 44.04 PUBLIC
ADMINISTRATION, 44.05 PUBLIC POLICY ANALYSIS, 44.99 PUBLIC ADMINISTRATION
AND SERVICES, OTHER
45.09 INTERNATIONAL RELATIONS AND AFFAIRS
45.10 POLITICAL SCIENCE AND GOVERNMENT
Social Sciences continued

908 SOCIOLOGY -- 45.05 DEMOGRAPHY AND POPULATION STUDIES, 45.11
SOCIOLOGY

909 SOCIOLOGY/ANTHROPOLOGY

910 SOCIAL SCIENCES, N.E.C. (Not elsewhere classified)
04.03 CITY/URBAN, COMMUNITY, AND REGIONAL PLANNING
04.07 ARCHITECTURAL URBAN DESIGN AND PLANNING
05 AREA, ETHNIC, AND CULTURAL STUDIES
30.11 GERONTOLOGY
43.01 CRIMINAL JUSTICE AND CORRECTIONS
44.02 COMMUNITY ORGANIZATION, RESOURCES, AND SERVICES
45.01 SOCIAL SCIENCES, GENERAL (Not elsewhere classified)
45.04 CRIMINOLOGY
45.12 URBAN AFFAIRS/STUDIES
45.99 SOCIAL SCIENCES, OTHER
During Years 1 & 3
A short online survey of SBE departments at the participating
AGEP institutions to determine practices related to graduate
student recruitment, retention, academic support, and data
collection and evaluation practices & uses.
Departments to be surveyed will be identified by SBE AGEPs.
Suggested Quantitative Data Collection Schedule
 URM new graduate student enrollment data
For 2006/2007 -- December 2006
For 2007/2008 -- December 2007
For 2008/2009 – December 2008
 Collection of data about 2006/2007 New Graduate
Student Cohort
October 2009
Where are they now? Completed course work,
earned Masters, passed qualifying
Suggested Qualitative Indicators
 SBE Related Questions incorporated into AGEP
Annual report
 Questions to be determined with AGEP/SBE
Program Directors with input from AGEP SBE
community
 Survey Schedule
August 2006?
August 2009?
Years 2 & 3 -- Site Visits to 2 AGEPs Each Year
(October 2006 to 2008)
The 2 day Alliance site visits will include meetings
with AGEP PIs, PDs, staff, and evaluators, and SBE
department Chairs and faculty.
Alliances to be site visited will be selected with the
NSF SBE and AGEP Program Directors.
 Site visits will be designed with Alliance Reps &
Could include TA
Evaluation Capacity Activities with SBE Faculty
Chair/Leaders
 Evaluation Capacity Building Workshop for SBE
Faculty Chairs/Leaders in Washington DC In
September 2006 (Combination focus group for
AAAS and opportunity for cross Alliance
chair/faculty exchanges)
 Identifying research gaps related to SBE graduate
education and research questions, particularly
questions related to progression to the SBE PhD
 Could bring in SBE professional societies to DC
meeting
 Use of AAAS AGEP web site for SBE Info briefs
about institution evaluations
Product --- On-line Evaluation & Assessment Guide
for SBE Department Chairs and Faculty Leaders
Development of the guide will begin in Year 2 and
by the end of Year 2 we will post a draft version for
comments by SBE Chairs and leaders in participating
AGEPs. (September 2007)
The final version of the on-line guide will be posted
by the end of Year 3. (September 2009)