Using Tolerance Intervals for Setting Process
Validation Acceptance Criteria
Richard K. Burdick —Amgen, Inc. (CO)
Graybill Conference
June, 2008
Using Tolerance Intervals for Setting Process
Validation Acceptance Criteria
“A worn-out academician’s adventure in the ‘real
word’"
Outline
 Life at Amgen
 Nonclinical statistics
 Definitions for Process Characterization and
Validation
 Statistical Methods for Setting Process Validation
Acceptance Criteria
 Future Opportunities
Operational Excellence
3
Amgen: A Biotechnology Pioneer

Founded in 1980, Amgen was
one of the first biotechnology
companies to successfully
discover, develop and make
protein-based medicines

Today, we’re leading the
industry in its next wave of
innovation by:
– Developing therapies in
multiple modalities
– Driving cutting-edge research
and development
– Continuing to advance the
science of biotechnological
manufacturing
Operational Excellence
4
Research and Development at Amgen
Guiding Principles
 Focus on serious illness
 Be modality independent
 Assess efficacy in patients
 Seamless integration from
research through
commercialization
Therapeutic Areas
 Inflammation
 Oncology
 Hematology
 Metabolic and bone
disease
 Neuroscience
Operational Excellence
5
Nonclinical Statistics
 Chemistry, Manufacturing, Controls (CMC)
development establishes the process of
manufacturing drug product to meet clinical
requirements.
 Work in both research and development and
manufacturing.
Operational Excellence
6
Nonclinical statisticians involved in…
 R&D with
–
–
–
–
–
–
Assay validation
Process validation
Method transfer
Stability studies (storage conditions, shelf-life, expiry extensions)
DOE for process characterization
Establishment of specifications and process validation acceptance
limits.
 Manufacturing with
– Maximization of yields
– Control charting
– Support in non-conformance reports (identification of assignable
causes)
– Raw materials inspection
Operational Excellence
7
Timeframe of Characterization and Validation
Activities Relative to Clinical Trials
End of Phase II
Clinical Trial
Characterization
Operational Excellence
End of Phase III
Clinical Trial and
Commit to File
Validation
8
Update CV
documents
Very Simple Process Diagram
(Upstream)
Diafiltered Medium (DFM)
(Downstream)
Operational Excellence
9
Filtered Purified Bulk (FPB)
Process Characterization
 Process Characterization is a precursor to
process validation and is comprised of a set
of documented studies in which operating
parameters (inputs) are purposely varied to
determine the effect on product quality
attributes (outputs) and process performance.
 Employs Failure Modes and Effects Analysis
(FMEA) and Experimental Design
Operational Excellence
10
Process Validation
 Process validation provides the documented evidence
that the process, when operated within established
limits, can perform effectively and reproducibly to
produce an intermediate, active pharmaceutical
ingredient (API) or drug product meeting
predetermined criteria and quality attributes.
 Final drug product and API have specifications that
must be met based on standards mandated by safety
concerns and other factors.
 However, intermediate process steps (which do not
have mandated standards) have a number of
acceptance criteria that must be met to demonstrate
process consistency and the ability to meet final
specifications.
Operational Excellence
11
Process Validation Acceptance Criteria
 Process Validation Acceptance Criteria
(PVAC) A set of numerical limits that when
exceeded, signals a significant departure from
operating conditions or product quality.
 Set prior to initiation of the validation
campaign.
 Establishing PVAC is one of the greatest challenges in
the development of a commercial biopharmaceutical
manufacturing process.
Operational Excellence
12
Definitions
 Operating Parameter (OP): Parameter that can be
directly manipulated (input)
 Performance Parameter (PP): In-process parameter or
measurement used for process performance
evaluation (output)
 Normal Operating Range (NOR): A range for an
operating parameter that is listed in the
Manufacturing Procedure. Frequently based on
equipment and/or process capability.
Operational Excellence
13
Setting PVAC-A personal history
 My involvement with the ACO process development
(PD) group began as a discussion concerning
analysis of one-off studies conducted at 3 times
outside the NOR.
 Questions concerned how to determine the operating
parameters (OPs) that were most important in the
process.
 I helped them analyze the data in a manner they were
comfortable with, and gained their confidence so that
I could work with them on future projects.
Operational Excellence
14
Setting PVAC-A personal history
 My involvement with the ACO process development
(PD) group began as a discussion concerning
analysis of one-off studies conducted at 3 times
outside the normal operating range (NOR).
 Questions concerned how to determine the operating
parameters (OPs) that were most important in the
process.
 I helped them analyze the data in a manner they were
comfortable with, and gained their confidence so that
I could work with them on future projects.
Lesson 1: Sometimes it is best to answer the client’s
question instead of telling them what they are doing wrong.
Operational Excellence
15
 When the discussion of setting PVAC came up, I
researched the history of setting PVAC at ACO:
– There was some sentiment for “3 sigma” rules
– JMP Prediction Profiler at the extremes of the NOR had been
used with previous projects (these limits are actually the
confidence intervals on the average for a given value of the
OP).
– Data sets from robustness and edge of range studies were
not being combined. In some cases, only centerpoints were
being used to determine PVAC.
Operational Excellence
16
 When the discussion of setting PVAC came up, I
researched the history of setting PVAC at ACO:
– There was some sentiment for “3 sigma” rules
– JMP Prediction Profiler at the extremes of the NOR had been
used with previous projects (these limits are actually the
confidence intervals on the average for a given value of the
OP).
– Data sets from robustness and edge of range studies were
not being combined. In some cases, only centerpoints were
being used to determine PVAC.
Lesson 2: Find out why certain methods were used in the past.
Can you use these approaches as a starting point,
and demonstrate continuous improvement?
Operational Excellence
17
Construction of PVAC
 I suggested we use tolerance intervals for defining
PVAC because they describe the long range expected
behavior of the process.
 Bench data derived from process characterization
experimental design studies can be combined with
large-scale runs to compute tolerance intervals at setpoint conditions (or any other point in the NOR)
centered at either commercial or clinical scale.
Operational Excellence
18
TI Depends on OP
Assumed distribution
of PP for given OP is
normal.
PP
Regression
Line
99% of PP
values
in this range
when OP=+1
99% of PP
values
in this range
when OP=-1
OP=-1
Operational Excellence
OP=0
19
OP=+1
Type of TIs
 If all OPs are fixed effects, then exact one-sided
tolerance intervals can be constructed based on the
non-central t distribution
– See, e.g., Graybill (1976, pages 270-275)
 Exact two-sided tolerance intervals are available
(Eberhardt, Mee, and Reeve, 1989), but
computationally complex.
– Various two-sided approximations have been suggested
• Weissberg, A. and G. H. Beatty (Technometrics,1960)
• Lee, Y. and T. Mathew (JSPI, 2004)
• Liao, C. T., Lin, T. Y., and Iyer, H. (Technometrics, 2005).
Operational Excellence
20
One other refinement
 Many times, the PC models involve random effects
such as the raw materials that feed into a process
step.
 In this case, the fixed effect methods can not be
applied for computing tolerance intervals.
 Generalized Inference provides an approach for
computing tolerance intervals with a random effect.
• Liao, C. T., Lin, T. Y., and Iyer, H. (Technometrics, 2005)
• Based on generalized fiducial intervals
Operational Excellence
21
One other refinement
 Many times, the PC models involve random effects
such as the raw materials that feed into a process
step.
 In this case, the fixed effect methods can not be
applied for computing tolerance intervals.
 Generalized Inference provides an approach for
computing tolerance intervals with a random effect.
• Liao, C. T., Lin, T. Y., and Iyer, H. (Technometrics, 2005)
• Based on generalized fiducial intervals
Lesson 3: Continue to make improvements and
demonstrate you are willing to continually improve your work.
Operational Excellence
22
Example—Purification Column
 Purification is used in a biopharmaceutical product to
separate desired protein from unwanted materials.
 This example considers one such column where the
response is modeled as a function of a fixed OP
(coded -1 to +1) and the random effect feed material.
 Response is a purity measure in %.
Operational Excellence
23
Scatterplot of PP vs OP
93
92
91
PP
90
89
88
87
86
85
84
2
4
8
6
OP
Operational Excellence
24
10
12
Parameter Estimates
Summary of Fit
RSquare
RSquare Adj
Root Mean Square Error
Mean of Response
Observations (or Sum Wgts)
0.363075
0.354105
1.391341
88.86986
73
Term
Intercept
OP
Estimate Std Error DFDen t Ratio Prob>|t|
90.725619 0.826556
48.6 109.76 <.0001*
-0.201453 0.092223
68.09
-2.18 0.0324*
REML Variance Component Estimates
Random Effect
Column Feed Source Lot
Residual
Total
Var Ratio
0.4137555
Var
Com ponent Std Error 95% Low er 95% Upper Pct of Total
0.8009605 0.6543545
0.256919
11.161442
29.266
1.9358307
0.3336
1.4182415
2.8007496
70.734
2.7367912
100.000
-2 LogLikelihood = 263.85628974
Operational Excellence
25
 Using the GCI approach, the computed
tolerance interval for the OP=0 (setpoint
condition) is from 83.4-95%
Operational Excellence
26
Plot of Tolerance Intervals and Runs with
OP = 0
95.0
95
Response (%)
92.5
90.0
87.5
85.0
83.4
Operational Excellence
27
Future Opportunities
 FDA initiative for Quality by Design.
 ICH Q8 Appendix on movement within the proven
acceptable range (PAR)—also referred to as “Design
Space”.
Operational Excellence
28
Design Space (ICH Q8)
Risk Assessment to
Prioritize Investigation
Explored Space
Explored
Space
DOE
DOE
Modeling
Modeling
Prior Knowledge
Prior
Knowledge
First Principles
First
Principles
Unexplored Space
Unexplored Space
Knowledge Space
Knowledge Space
“ Design
Design ” Space
Space
Control Strategy
Specifications
Tolerances
PAR
PAR
(Proven Acceptable Range)
NOR
Explored with
NOR
Acceptable
Performance
NOR
(Normal Operating Range)
Operating Strategy
based on Business/Equipment Requirements
Operational Excellence
29
Operational Excellence
30