FDA and Pharmaceutical
Manufacturing Research Projects
Jeffrey T. Macher
Jackson A. Nickerson
Co-Principal Investigators
September 30, 2004
Presentation Overview
 FDA Research Project status report
History
Project Goals
Approach and Expected Outcomes
Current Status
 Pharmaceutical Manufacturing Research Project
(PMRP) status report
History
Project Goals
Approach and Expected Outcomes
Current Status
FDA Research Project
FDA Research Project History
 Research project idea emerged in Fall, 2001.
Observed significant increases in both number and severity of
cGMP violations.
 Approached FDA in Spring, 2002.
Dozens of meetings with CBER, CDER, ORA and district
offices.
 Formalized relationship with FDA in Fall, 2003.
Signed Material Transfer Agreement (MTA) in October, 2003.
 Received data September, 2004.
FDA Research Project Goals
 Risk-based assessment of FDA cGMP outcomes.
 Identify attributes (currently recorded by the FDA) that
impact inspection outcomes.
 Transfer “learning” to FDA.
FDA Project Approach
 Compile and link FDA databases.
 Estimate the likelihood of various outcomes:
NAI, VAI, OAI; Warning Letters; Field Alerts; Product Recalls.
 based on…
compound/product, facility, firm, FDA district, investigator and
training derived factors.
 in order to …
evaluate the allocation of investigational resources.
inform effectiveness of investigator training and management.
FDA Databases
COMIS (Supplement filings)
DQRS (Field alerts)
EES (Outsourcing)
FACTS (Inspections)
Product Listing
Product Recalls
Product Shortages
Registration
Warning letters
Training
Some basic “facts” about the FDA data
 Years covered: 1990-2003.
 Total number of facilities inspected: 3753.
 Total number of “Inspections”: 38,341.
 Total number of “Facility visits”: 14,162.
0
500
1,000
Number of FDA Facility Visits per Year: 1990-2003
1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003
0
1000
2000
3000
4000
5000
Days Between Facitlity Inspections: 1990-2003
0
2000
4000
6000
Frequency
8000
10000
FDA-related Performance Outcomes
 Inspection outcomes:
NAI
VAI
OAI
» Warning letters
 Field reports
 Product recalls
 Product availability
FDA-related Key Factors
 Product-level and Process-level variables
NDA/ANDA.
Prescription/non-prescription.
Product class.
Product subclass.
Process indicator code.
Supplement history.
Extent of vertical integration.
History of regulatory outcomes for the product.
History of regulatory actions for related products.
FDA-related Key Factors
 Facility-level variables
Age and Size.
Number of products produced.
Diversity of products produced:
» Prescription/non-prescription; Product class; Product subclass; Process
indicator code.
Increases/decreases in number of products and diversity of
production n years prior.
Time since last ownership change.
Regulatory history (e.g., inspectional outcomes, warning letters
and type).
FDA-related Key Factors
 Firm-level variables
Age and Size.
Number of manufacturing locations.
Number of products produced.
Diversity of products produced.
Number of products introduced in past n years.
Number of NAI, VAI, and OAI at non-focal facilities in the past n
years.
Number and type of warning letters at non-focal facilities in the
past n years.
FDA-related Key Factors
 FDA related variables
FDA District Identifier and annual allocated inspection hours.
Inspections
» Length of cGMP inspection; Number of investigators; Reason for
inspection; Time since last inspection.
Investigators
» Annual experience of Pharma versus non-Pharma inspections.
» Training: number and frequency of formal training classes.
Policy shifts (e.g., SUPACs)
0
10
20
30
40
Visits by Investigator in Prior Year: 1990-2003
0
2000
4000
Frequency
6000
Expected Outcomes of Analysis
 Statistical analysis will estimate the probability of various
outcomes based on counterfactuals.
Counterfactual analysis allows a risk-based assessment of
changes in at least some FDA-related oversight policies.
 Results will improve understanding of certain inspection
outcomes based on attributes examined.
 Risk assessments (in terms of increasing probability of
occurrence) will be used to:
Inform oversight choices of FDA.
Identify underlying quality of manufacturers’ production and
regulatory processes.
Status of FDA Research Project
 Phase 1: Exploratory pilot study.
Completed Summer 2003.
 Phase 2: Data collection
Complete for CDER and awaiting transfer.
Incomplete for CBER.
 Phase 3: Data analysis
Data received.
Analysis underway (~2-3 months for initial findings).
Pharmaceutical Manufacturing
Research Project
(PMRP)
PMRP Project History
 Research project idea emerged in Fall, 2001.
Explore whether cGMP violations related to managerial,
organizational and technical practices.
Attempt to translate lessons learned from similar research
study of manufacturing in semiconductor industry.
 Began interviewing manufacturers in Spring, 2002.
Dozens of interviews with pharmaceutical, biotechnology, API,
and contract manufacturers.
 Launched internet-based questionnaire in Fall, 2003.
Went live in November, 2003.
Ongoing marketing and participation solicitation since.
 First round closed but for a few firms finishing off the
survey.
PMRP Goals
 Develop standard set of benchmarks for measuring
manufacturing and regulatory performance.
 Identify managerial, organizational, and technical
practices underlying manufacturing and regulatory
performance.
 Provide confidential “scorecard” to manufacturing
facilities on how they perform against anonymous others.
PMRP Approach
 Develop focused questionnaire of potential factors that
influence manufacturing and regulatory performance.
 Administer questionnaire over internet on secure-site.
 Analyze data collected using a variety of econometric
techniques.
 Provide summary of findings and facility scorecard to
participating manufacturers.
PMRP Database
 Secured participation from cross section of U.S. and EU
manufacturers.
21 firms.
58 manufacturing facilities.
 Online survey with each manufacturing facility providing
detailed data on between 1 and 5 compounds.
PMRP Performance Outcomes
 Manufacturing Performance
Theoretical / Actual yield.
Batches started / failed.
Cycle time.
 Regulatory Performance.
Field alerts/Biologic deviation reports.
Warning letters.
Consent decrees.
Deviations.
Supplement approval.
PMRP-related Key Factors
 Company / SBU
Financial information.
Demographic information.
 Manufacturing Facility
Financial information.
Demographic information (size, location, age, employees, etc.).
Product information (number, type, etc.).
Regulatory inspection information.
Extent of Outsourcing (development, manufacturing, APIs,
etc.).
PMRP-related Key Factors
 Product and Process Development
Location.
Organization.
Timing.
 Human Resource Management
Appraisal, Promotion, Mobility, Demographics, Training, etc.
Extent and use of teams.
 Deviation and Supplement Management
Extent and use of information technology.
Extent and use of Process Analytic Technology (PAT).
Organization.
Status of PMRP Project
 Phase 1: Exploratory pilot study
Completed Summer, 2003.
 Phase 2: Data collection
First round closed (but for a few stragglers).
 Phase 3: Data analysis
Analysis will require 3-6 months.
Statistical and econometric analysis of data.
Final reports.