NSF Business Analysis
Overview of Merit Review Activities
Presentation for
Business and Operations
Advisory Committee
Joseph F. Burt, Director
Division of Human Resource Management
May 6, 2005
NSF BUSINESS ANALYSIS
Business Analysis Approach: Concurrent consideration of human capital
and next-generation technology-enabled systems in an an analysis framed
around the Agency’s core business processes
Core Business Processes:
– Resource Allocation – setting the right priorities…
– Merit Review – identifying people, ideas and tools with the greatest potential for
impact…
– Award Management and Oversight – the award cycle, beginning to end…
– Knowledge Management – the right information, in the right places, at the right
time…
– Performance Assessment and Accountability – the highest standards of
excellence and integrity…
Major Tasks:
– Document Current Business Process Environment
– Identify Business Process Opportunities for Improvement
– Develop Business Cases and Implementation Plans for Each Opportunity
NSF BUSINESS ANALYSIS
1
NSF management and staff have been integrally involved in all aspects of
the Business Analysis
NSF Project Team - NSF project manager and leads for each of the technical areas of the project:
business processes, human capital management, and technologies and tools
Booz Allen Hamilton (BAH) Team - Contractor staff working in partnership with NSF to conduct
the Business Analysis
Integrated Process Teams - NSF-wide working groups focused on each of the core business
processes – worked in concert with the NSF and BAH teams to document current environment and
identify and develop opportunities for process improvement
NSF Management and Staff - Involved as resources for information, ideas, and reality checks –
over 600 NSF staff have participated in the Business Analysis to date, through interviews, focus
groups, or participation in working groups
Business Analysis Steering Committee - Composed of managers and staff from across NSF –
serves as sounding board for the NSF/BAH project team, validates project plans, and vets all
Business Analysis deliverables and products
Director, Deputy Director, and Assistant Directors – Provide high-level direction and oversight;
involved at key decision points throughout the project
NSF BUSINESS ANALYSIS
2
The Business Analysis has focused on NSF’s Merit Review process since the
inception of the project in FY02
Business Analysis Review of NSF’s Merit Review Process
FY2003
FY2004
FY2005
FY2002
Start:
6/26/02
FY2006
Baseline Analysis
Applicant
Survey
Effective
Practices
Review Process Designs
To-be Process Designs
Implementation
Planning
Implementation (Timeframe TBD)
Baseline Analysis and Applicant
Survey
 Identify key (mission-critical) processes
to analyze
 Map as-is process flows
 Document accompanying metrics on
cycle times, roles and responsibilities,
etc.
 Identify process bottlenecks and issues
Effective Practices Research
 Identify key organizations to research
 Document comparative organizational
information
 Identify effective practices that NSF
could incorporate in process designs
 Develop an understanding of why
specific strategies are successful
Preliminary Design Options and
Implementation Plans
 Clarify and justify the costs, benefits,
and impacts of adopting process
designs
 Identify risks and estimated costs of
process designs
 Identify high-level implementation
issues
 Survey applicant community (2003
NSF Applicant Survey)
NSF BUSINESS ANALYSIS
3
Of the staff that support MR and AM&O, the percentage of time that staff
spent in each of these activities varies, with Program Directors spending 55
percent on MR and 12 percent on AM&O
Directorates
Division Directors
Program Directors
Self-Reported % of
Annual Working
time
DGA
Program Assistants/
Lead Program Assistants
Science Assistants
Grants & Agreement
Specialists
MR Activities
29%
55%
66%
55%
10%
AM&O Activities
8%
12%
23%
8%
84%
63%
33%
11%
37%
6%
• Develop scientific
program
• Research, compile, and
produce reports
• Strategic planning
Other activities,
including:
• Outreach and community
relations
• Personnel management,
incl. recruitment
• Oversee and provide
advice regarding MR,
AM&O
• Participate in working
groups/task forces
• Participate in crossDirectorate and interagency
planning and/or special
initiative work
• Attend meetings/
workshops, make invited
presentations, or visit sites
to convey program
information
• Research, compile, and
produce statistics or
reports
• Research, compile, and
produce statistics or
reports
• Arrange non-panel
travel
• Manage program
website
• Manage award funding
obligations and budget
spreadsheets
• Prepare external
presentations
• Research, compile, and
produce reports
• Coordinate project
director quarterly
meetings
• General planning
• Orient new staff (IPAs)
• Normal "make the wheels
turn" activities (daily email,
phone calls, administrative
staff oversight)
• General administrative
management (letters,
correspondence not
related to proposals)
• Attend meetings
• Conduct special
initiatives
• Manage and coordinate
“nuggets”
• Compliance checking
and correcting
compliance Issues
• General administrative
(misc. email, phone calls,
queries, etc.)
Source: MR/AM&O Process Workload Analysis, March 14, 2003, Booz Allen Hamilton Analysis
NSF BUSINESS ANALYSIS
4
During 2003, the Business Analysis team surveyed over 6,000 FY02 NSF
applicants regarding NSF’s merit review and award monitoring and
oversight processes
Proposal Dwell Time – NSF currently processes >70% of proposals within 6 months;
74% of survey respondents would like to see their proposals processed in five months or less
Support during Proposal Preparation and Submission – Survey respondents
believe that NSF does a better job of proposal preparation and submission support than other
research funding agencies
– 88% satisfied with ease of FastLane use
– 84% satisfied with quality of process information
– 78% satisfied with help regarding proposal content
– 82% satisfied with help regarding proposal submission
Fairness and Quality of Review Process – Survey respondents indicated that NSF
could improve in terms of fairness and quality of merit review process
– 35% somewhat or very dissatisfied with fairness of process
– 35% somewhat or very dissatisfied with quality of process
NSF BUSINESS ANALYSIS
5
In the analysis of the Merit Review process, three strategic areas of
opportunity emerged that are being addressed in the current phase of the
Business Analysis project
Workload of Program Directors and Grants Staff
– Increase in the volume and complexity of proposals impacts the effectiveness of Program
Directors in performing their merit review responsibilities
– 48% increase in proposal workload from 2000 to 2004; workload survey data indicate that
Program Directors spend 55% of their time on merit review activities
Increase in Programs that Require Cross-Directorate Coordination
– The workload issue is compounded by the need for more formalized processes for
programs that require cross-Directorate coordination
– A sample of cross-Directorate programs indicates that the volume of proposals in such
programs has increased by more than 50% over a three year period
Increasing Demand on Reviewer Community
– The increase in volume of proposals is making recruiting and attracting reviewers more
difficult
NSF BUSINESS ANALYSIS
6
The Business Analysis has identified key opportunities for
improvement that address the major workflow issues facing NSF’s
merit review process
Workload of Program Directors and Grants Staff
Increase in Programs that Require Cross-Directorate Coordination
Approved:
– Improve the overall utility of the Project Reporting System
– Pilot an enhanced “customer service center” to address administrative inquiries
– Coordinate program due dates on a two-year out basis with windows for cross-NSF and
unsolicited proposals
Under Consideration:
– Introduce a “fast track” solicitation process for existing programs
– Provide PD flexibility to streamline review analysis
NSF BUSINESS ANALYSIS
7
Key opportunities for workflow improvement in NSF’s merit review
process (continued)
Increasing Demand on Reviewer Community
Under Consideration:
– Develop a centralized, searchable database that incorporates PI and reviewer
information
– Establish standard training that would be available to reviewers prior to
completing a review or serving on an NSF panel
– Establish a centralized process for online acceptance and internal assessment of
reviewer applications
– Consider targeted outreach efforts for recruiting new reviewers
NSF BUSINESS ANALYSIS
8