Performance Assessment
Assessment of Organizational
Excellence
NSF Advisory Committee for
Business and Operations
May 5-6, 2005
Performance Assessment
Develop and use performance assessment
tools and measures to provide an
environment of continuous improvement
in NSF’s intellectual investments as well
as its management effectiveness.
(one of four areas—ALL now need to be
successful)
Expert Assessments
Integrated Throughout NSF
Components
NSF Mission
• Strategic or Long-Term
Planning
• Scientific Advisory Committee
Reviews
• NSF Performance Planning
OUTCOMES
STRATEGIC
• Advisory Committee for GPRA
Performance Assessment
(AC/GPA)
• Business & Operations
Advisory Committee
OUTPUT
OPERATIONAL
• Committees of Visitors (COVs)
• Merit Review
• Project Reports
INDIVIDUAL
ACTIVITY
• Program Assessment Ratings
Tool (PART)
• Individual Performance
Assessments Directly Linked to
Mission and Goals
In red, added 2001 or later
Performance Assessment:
FY 2005 Focus Areas
• PART Activities
– New Efficiency Metric
– Highest Ratings in All Programs
• AC/GPA Activities
– Refined processes
– Categorizing “Innovative High-Risk”
PART Activities I
Factors
New Efficiency
Measure: Percent of
award decisions made
available to applicants
within six months of
proposal receipt or
deadline date, while
maintaining a credible
and efficient competitive
merit review system, as
evaluated by external
experts.
Achievements
Reached consensus with OMB and
implemented change.
Results
PART Activities II
Factors
Completed PARTs in three
NSF Investment Categories
and one Priority Area:
- Institutions (People)
- Collaborations (People)
- Polar Tools, Logistics, and
Facilities (Tools)
- Biocomplexity in the
Environment (Priority
Area)
Achievements
100% of NSF PARTs (8 out of 8 in
total) received the highest rating of
“Effective” as compared with only
15% of 607 PARTs governmentwide.
Results
AC/GPA I
Factors
Achievements
Work on individual
indicators performed
through subgroup chairs
prior to annual meeting.
- More substantive discussions at
AC/GPA meeting
-Positive comments from AC/GPA
Results
AC/GPA II
Factors
Achievements
- Broad perspectives obtained on
why NSF-funded projects from
various disciplines fall into these
categories
Identifying “high-risk” and
“multidisciplinary” activities
Results
-Quality and quantity (~200 for
“high risk” and ~600 for
“multidisciplinary) of program
officer responses
A STARTING POINT FOR THE AC/GPA DISCUSSION
ON INNOVATIVE-HIGH RISK RESEARCH – FROM
PROGRAM OFFICER COMMENTS IN NUGGETS
“Innovative High-Risk”
Forefront, Novel or Transformative but Untried or Untested
High Reward but Significant Technical Challenges and/or High
Probability of Failure
Innovative and Contrary to Current Theory or Conventional
Paradigms
“Other High Risk” (few in number)
Risk from Not Succeeding
Risk to Principal Investigators/Others
Performance Assessment: FY 2005
Priority Initiatives
Why a Priority
Accomplishments
Indicators of
Success
Coming
Attractions/
Comments
Identify Appropriate
Efficiency Measure
Use Measure that
Replacement in PART Combines Quality
and Timeliness
PART Activities
Establishes agencywide evaluation
framework.
4 PARTs completed:
- Institutions,
- Collaborations,
- Polar Tools, and
- Biocomplexity
All received the
highest rating:
“effective”
2 PARTs underway:
- Fundamental
Science and
Engineering
- FFRDCs
Workload issues for
AC/GPA at annual
meeting
New process wellreceived
-More substantive
discussions at
AC/GPA meeting
Creating of separate
OE subgroup
Refining AC/GPA
Process
Reached Consensus
with OMB
Effective this year
-Comments from
AC/GPA
Identifying “high-risk”
and “multidisciplinary”
activities
AC/GPA
Recommendation
NAPA
Recommendation
Broad perspectives
obtained on why
NSF-funded projects
from various
disciplines fall into
these categories
-Quality and quantity
(~200 for “high risk”
and ~600 for
“multidisciplinary) of
program officer
responses
Definitions and
categories for
“innovative high risk”
Initiatives on the Horizon
Future Initiatives
1. PART
Comments
-Complete Fundamental Science and Engineering PART and Federally
Funded Research and Development Center PART
-Initiate five remaining PARTs for next year
2. “Innovative High
Risk Research”
-Work with AC/GPA to define categories for further use (e.g. by COVs, POs in
writing accomplishments (“nuggets”), input to the NSB Task Force on
Transformative Research)
Conclusion
NSF has demonstrated significant
achievement in Performance
Assessment.