The Posterior Probability of Passing
a Compendial Test (Pa)
Dave LeBlond, Principle Research Statistician, Abbott
[email protected]
Linas Mockus, Research Scientist, Purdue University
[email protected]
May 10, 2012
Bayes 2012, Aachen
Outline
•  Process Qualification (PQ)
–  Compendial Tests
–  Test Pass Probability, Pa
•  ASTM E2709
•  USP<905> Dosage Uniformity
–  Hierarchical process model
•  Bayesian PQ Approach
•  Prior Calibration
•  Number of Batches for PQ?
–  Operating Characteristics
•  Application to Sample Data
–  Cost Estimation
•  Summary
18th Century Statistics
19th Century Regulations
+ 20th Century Computing
21st Century cGMPs
2
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
What is a Compendial Test?
Bad
•  “Bright line” standard of quality.
Good
–  batch should always pass.
•  Benchmark for setting batch
acceptance criteria.
•  May use multi-stage sampling.
–  USP<905> & <701> have 2 stages
–  USP<711> & <724> have 3 stages
Fail
•  May use complex limits
–  Indifference zones
–  Limits on means, individuals, RSDs, counts
–  Zero tolerance limits
3
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Pass
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
FDA Process Validation Guidance
•  Establishes 3 validation stages8
–  Process Design (QbD)
–  Process Qualification (PQ)
–  Continued Process Verification
•  “ [PQ] criteria … [should] … allow for a science- and risk-based
decision about the ability of the process to consistently produce quality
products…
•  … [and] include … statistical metrics defining both intra-batch and
inter-batch variability.”8
Ø Place acceptance limits on Pa, the probability that future
batches will pass the compendial test.
4
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Process Qualification
ASTM E2709
USP905
Bayesian Approach
Prior Calibration
Number of Batches?
Sample Data
Summary
ASTM E2709 Approach to PQ
•  Derive g: Pa >= g(θ )
•  Identify an acceptance region (AR) in parameter
space: g(θ ) >= LB
•  Identify θ est .
•  Choose a 100(1-α)% confidence region (CR)
method, given a sampling plan and data.
•  Generate an acceptance table (AT): θ est such that
CR is within AR.
•  Choose a sampling plan so that Prob(θ est is within AT) > some
desired value.
•  Obtain data. If θ est is within AT, there is 100(1-α)% confidence
that Pa >= LB on repeated sampling of that batch.
5
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
Praise for ASTM E2709…
•  “This paradigm shift fundamentally changes how our industry should
develop in-house specifications.”2
•  “[This new] concept of meeting specification … may begin to include
estimates of statistical confidence as part of cGMP.”3
•  “E2709 was highly effective in identifying nonconforming material.”4
•  Available for USP<905> & <711> as CUDAL, a validated SAS
program,10 and as an Excel Spreadsheet.11
•  Clearly ASTM E2709 is a very positive step… but ...
Is anything missing?
Can Bayesian tools make further improvement?
6
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
ASTM E2709 Limitations
•  Parameters are fixed (cannot have a distribution).
•  ∴ Pa is also denied a distribution.
•  Prediction of failure rates for future lots requires integration
over the uncertain Pa … Not allowed.
•  Confidence region approach
–  Conservative approximations à Biased predictions.
–  Confidence regions are not unique.
•  Qualifies 1 batch at a time. No inference about the process.
•  What about cost?
7
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
USP<905> Uniformity of Dosage Units: Quality
Stage
Sample Size
Requirements
8
Need stage 2
6
1
SD 10
4
Pass
2
10
0
70
100
X 10
130
8
6
2
SD 30
4
Fail
2
20 more
0
Pass
X 30 130
(+ individual limits met)
70
100
8
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
A Hierarchical Process Model
Process mean potency:
Process
β
Batches
1
2
…
B
Batch i mean potency:
iid
(
ui ~ N β , σ B2
)
Tablet j(i) observed:
Tablets
iid
(
yij ~ N ui , σ T2
9
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
)
[3]
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
Actual Production Data: Joint 95% CI
β σB
Product Batches LB
A
B
C
D
E
F
G
H
I
J
K
L
3
32
85
49
2
32
10
4
14
16
4
4
100.0
100.8
100.7
100.2
99.7
100.1
98.3
98.7
99.4
98.9
98.9
101.7
est
100.0
101.8
101.6
100.6
100.1
101.2
99.8
100.8
100.7
100.0
99.8
102.6
σT
UB
est
100.1
102.8
102.5
101.0
100.4
102.2
101.3
102.8
101.9
101.2
100.6
103.6
0.0
2.2
3.5
1.2
0.0
2.4
1.6
1.7
2.0
1.8
0.7
0.7
UB
0.0
3.1
4.2
1.5
9.9
3.3
2.3
7.4
3.4
3.0
3.0
3.4
•  Each batch passed USP<905>.
•  σB > 0 (Pvalue < 0.05)
10
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
est
3.1
3.0
3.3
1.5
2.5
2.5
2.1
1.1
2.6
3.5
0.8
1.7
UB
4.3
3.2
3.5
1.6
3.9
2.8
2.2
1.5
3.0
4.0
1.1
2.2
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
Pa is a deterministic function of model parameters
•  Use Monte-Carlo simulation to generate lookup table.
95
0.700
0.800
0.900
0.950
σB : 5
0.
0.3 200
0. 00
40
0
0.1
0.2 00
0.3 00
0.05.40 00
00 0
00
0 0.3
00.6.5.040000
000
0.1
σB : 3.5
0.500
0.600
0.700
0.800
00
0.2 00
0.3.400
0
σT
0.200
00
0.1 00
0.2.30000 0
0 .4 0 0
0
00.5.6.0700
0
6
0.800
0.900
0.950
0.990
105
00
0.1 00
0.2 .300 0
0 0 0
0.04.50600
0.
σB : 2
100
0.900
4
0.990
0.999
0.950
0.999
2
95
100
0.990
105
95
100
105
β
•  Based on 10K simulated batches for each of 18K grid points.
•  Use tri-linear interpolation to obtain Pa for any desired θ.
11
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
A Bayesian Approach to PQ
•  Generate (by simulation) an interpolation table for Pa = g(θ )
•  Choose PQ acceptance criteria (AC)
–  lower bound (LB) for Pa.
–  require that 100(1-α)% of the Pa posterior mass be >= LB.
•  Choose a sampling plan based on the simulated Operating
Characteristics (OC).
•  Obtain data.
–  Obtain a posterior sample of θ.
–  Obtain a posterior sample of Pa.
•  If Pa posterior is acceptable, there is at least 100(1-α)%
probability that Pa >= LB for the process.
•  Mean of Pa posterior = expected probability that future batches
will pass the compendial test.
12
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
Benefits of a Bayesian Approach to Qualification
•  Direct inference on the parameter of interest (Pa)
•  Posterior distribution of Pa
–  Quantitative risk assessment (i.e., ICH Q9)
–  Production planning (expected cost and throughput)
•  Leverage prior knowledge (if justified)
–  “[The qualification report should consider]… the entire compilation
of knowledge and information gained from the design stage through the
process qualification stage.”8
13
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
Proposed Bayesian PQ Acceptance Criterion
•  “The confidence level selected can be based on risk analysis as it relates to the
particular attribute under examination.”8
Ø Propose: median(Pa posterior) ≥ 0.9 (α = 0.5, LB = 0.9)
mean would be computationally simpler, more discriminating
•  Examples of simulated borderline cases…
= 105, σB = 2, σT = 4, 10 units/batch, Noninf Priors, 10K MCMC draws
Percent of Total
25
20
15
24 batches
30
Percent of Total
β
Median(Pa)=0.89
10
5
0
25
20
4 batches
Median(Pa)=0.92
15
10
5
0
0.2
0.4
0.6
0.8
0.2
Pa
0.4
0.6
Pa
14
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
0.8
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
Prior Calibration Using Simulated Data
σT
5
2
5
σB
2
102
107
σB
σT
β
2
5
2
5
2
5
2
5
2
2
5
5
2
2
5
5
102
102
102
102
107
107
107
107
Pa*
Ps*
1.00
0.98
0.97
0.78
1.00
0.87
0.49
0.50
0.00
0.03
0.27
0.44
0.02
0.17
0.72
0.67
* based on 100K simulated batches each.
β
•  Simulated qualification data from 3, 4, 6, 12, or 24 batches
•  4 weakly informative priors examined
15
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
Weakly Informative, Independent Priors Used
None
Least
More
Most
N(100,10002)
N(100,10002)
N(100,152)
N(100,102)
RIG(.001,.001)
Half Cauchy*
Half t*
Half t*
0-20
Undefined
mean
infinite mean
mean = 70.7,
mean = 14.1
df = 0
df = 1
df = 2
df = 2
RIG(0.5,8)
RIG(.001,.001)
Mean = 4
Undefined
mean
β
70-130
σΒ
σT
0-20
RIG(.001,.001) RIG(.001,.001)
Undefined
mean
Undefined
mean
df = 0
df = 0
df = 1
16
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
df = 0
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
% Bias in Pa Posterior Mean
More
Most
5
% Bias in Pa estimate
β : 102
σB : 5
σT : 2
None
10
15
20
15
Most
5
β : 107
σB : 5
σT : 2
0
-10
-20
-30
-40
β : 102
σB : 2
σT : 5
0
-10
-20
-30
-40
10
More
β : 102
σB : 5
σT : 5
β : 102
σB : 2
σT : 2
5
Least
% Bias in Pa estimate
Least
20
None
10
15
β : 107
σB : 5
σT : 5
β : 107
σB : 2
σT : 2
β : 107
σB : 2
σT : 5
0
-10
-20
-30
-40
5
10
Number of Batches
15
20
Number of Batches
•  All priors exhibit conservative (low) mean(Pa)
•  Non-informative prior is least conservative.
17
Posterior Probability of Passing a Compendial Test
Dave LeBlond
20
May 10, 2012
0
-10
-20
-30
-40
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
% RMSE in Pa Posterior Mean
More
Most
% RMSE in Pa estimate
5
β : 102
σB : 5
σT : 2
10
15
β : 102
σB : 2
σT : 5
15
Most
5
40
30
20
10
0
40
30
20
10
0
10
More
20
β : 102
σB : 5
σT : 5
β : 102
σB : 2
σT : 2
5
Least
None
% RMSE in Pa estimate
Least
β : 107
σB : 5
σT : 2
10
15
β : 107
σB : 2
σT : 5
40
30
20
10
0
10
15
20
Number of Batches
Number of Batches
•  In most cases Non-informative prior has lower RMSE.
18
Posterior Probability of Passing a Compendial Test
Dave LeBlond
20
β : 107
σB : 5
σT : 5
β : 107
σB : 2
σT : 2
5
20
None
May 10, 2012
40
30
20
10
0
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
90% Credible Interval Coverage of Pa
More
Most
90% Interval Coverage for Pa
5
β : 102
σB : 5
σT : 2
Least
None
10
15
0.95
0.90
0.85
0.80
0.75
β : 102
σB : 2
σT : 5
0.95
0.90
0.85
0.80
0.75
5
10
15
Most
5
β : 102
σB : 5
σT : 5
β : 102
σB : 2
σT : 2
More
20
20
90% Interval Coverage for Pa
Least
β : 107
σB : 5
σT : 2
None
10
15
β : 107
σB : 5
σT : 5
0.95
0.90
0.85
0.80
0.75
β : 107
σB : 2
σT : 2
β : 107
σB : 2
σT : 5
0.95
0.90
0.85
0.80
0.75
5
10
15
20
Number of Batches
Number of Batches
•  Non-informative prior has coverage closest to nominal
19
Posterior Probability of Passing a Compendial Test
20
Dave LeBlond
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
Summary and Final Choice of Prior
•  % Bias in Pa Posterior mean
–  Bias increases from -10% to 0% as #batches grows (3 to 24)
–  In all cases, Non-informative prior shows least bias
•  % RMSE in Pa Posterior mean
–  RMSE asymptotes from +10-30% to zero as #batches grows
–  Non-informative prior shows least RMSE
•  Coverage of 90% Credible Interval
–  Nominal coverage in most cases, some cases only 80%
–  Non-informative prior coverage closest to nominal
Ø Non-informative prior used here…
20
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
Establishing a PQ Sampling Plan
•  Operating Characteristic
Curve(OC)
•  Acceptable Quality Level
•  Rejectable Quality Level
Pa
0.4
0.95
1
Prob(Pass PQ)
•  Pa = quality metric
0.10
•  How many batches
RQL
(more/less) ?
•  Requires Monte-Carlo
simulation
Good
Quality
Quality
21
Dave LeBlond
AQL AQL
Poor
•  Does OC depend on θ ?
Posterior Probability of Passing a Compendial Test
RQL
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
OC of Proposed Approach
Number of Batches
σT 5
3
8
2
8
4
6
12
24
0.95
4
0.5
2
102
107
(Prob(median(Pa) ≥ 0.9))1
σB 5
112
β
•  27 grid points
•  1000 data sets per grid point
•  Vary number of batches
•  OC independent of θ ?
0.1
0.05
0
0.0
0.2
0.4
Population Pa
22
Posterior Probability of Passing a Compendial Test
0.6
Dave LeBlond
May 10, 2012
0.8
1.0
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
Using AQL and RQL to Set Number of Batches
Number of Batches
0.95
3
4
6
12
24
0.10
0.95
(Prob(median(Pa) ≥ 0.9))1
4
0.5
0.1
0.05
0
0.0
0.2
0.4
0.6
0.8
1.0
Population Pa
AQL
RQL
23
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
PQ Pass Rates(α=0.5, LB=0.90) : Bayes vs E2709
Popn.sig.B=5
Popn.sig.B=2
Popn.beta
β
Popn.beta
β
102
102
107
1
0.4
0.2
Pa=0.97
Pa=0.49
Y
0
2 4 6 8 10 12 2 4 6 8 10 12
Number of Batches Number of Batches
0.4
Pa=0.98
0.2
0
1
Pa=0.78
0.8
Pa=0.50
0.6
5
Y
0.6
5
5
0.8
Y
Popn.sig.T
Y
Popn.sig.T
1
0.6
0.4
0.2
0
2 4 6 8 10 12 2 4 6 8 10 12
Number of Batches Number of Batches
σΒ = 2
σΒ = 5
24
Posterior Probability of Passing a Compendial Test
Popn.sig.T
0
Pa=1.00
Pa=0.87
0.8
2
0.2
Pa=1.00
Popn.sig.T
σΤ
0.6
2
2
0.8
5
2 PQ
Probability
of Passing
1
0.4
107
Dave LeBlond
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
115
Product C
Product D
Potency
100
10
25
Lot
40
85
85
Potency
100
115
Actual Production Data
10
•  Note: Each lot passed USP<905>.
25
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
25
Lot
40
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
Joint Posterior Samples after 3, 6, and 20 batches
Product C
σB
σT
Product D
β
26
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
0.05, 0.50 & 0.95th Posterior Quantiles for Product D
Quantiles of Pa
0
0.5
1
•  Cumulative Analysis
(lots 3 to 45)
•  For very good process
3 may be enough.
REML Point Estimates
β = 100.6
σB = 1.2
σT = 1.5
10
25
40
# of lots
27
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
0.05, 0.50 & 0.95th Posterior Quantiles for Product C
Quantiles of Pa
0
0.5
1
•  Cumulative Analysis
(lots 3 to 45)
•  For challenging
processes >3 required.
REML Point Estimates
β = 101.6
σB = 3.5
σT = 3.3
10
25
# of lots
28
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
40
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
How Many Batches for a PQ?
2 approaches:
•  Prospectively: use simulated OC curves, or
•  Cumulative stability of Pa quantiles.
29
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
Cost Estimates
•  Not a regulatory consideration
•  Important for Manufacturer
–  Stage testing increases analytical costs
–  Failures risk supply of critical drugs
–  Resource planning
•  Need Pa to estimate cost
•  For multi-stage compendial tests, also need Ps, the probability
of stage testing.
30
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
USP<905> Uniformity of Dosage Units: Cost
Let
•  Ct = Cost of testing 10 tablets ( ~ $3K )
•  Cm = Cost of manufacturing 1 batch ( ~ $200K )
•  Ci = Cost of a failure investigation ( ~ $2K )
Then
E[total cost] = Ct + 2·Ct·Ps + Cm +Cm· (1-Pa) + Ci· (1-Pa)
•  Stage 2 triples testing cost
•  Failure doubles manufacturing cost and requires a
failure investigation.
31
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
Posterior Distribution of Future Costs for Product C
Pa Mean = 0.97
40
20
50
0
Ps Mean = 0.05
40
30
20
10
Percent of Total
60
60
Percent of Total
Percent of Total
•  Estimation after first 6 batches
40
20
0
0
0.6
0.7
0.8
0.9
0.1
Pa Posterior
Total Cost
Mean =
$209K
60
0.2
0.3
200
0.4
•  Production costs per batch (posterior expectation):
–  Testing: $3.3K
–  Manufacturing: $206K
–  Investigation: $60
32
Dave LeBlond
300
350
Total Cost ($1K) Posterior
Ps Posterior
Posterior Probability of Passing a Compendial Test
250
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
Future Work
•  Compare Stringency of Proposed Bayesian Criterion with
CUDAL.10, 11
•  Dependence of OC curve on θ ?
•  Non-normal populations?
•  Excel tool?
•  Extension to any compendial test
–  Same principles
–  Most will be trivial extensions, compatible with Excel
•  Dissolution & disintegration multi-stage tests
–  USP<711>, <724>, <701>
33
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Process Qualification
Prior Calibration
ASTM E2709
Number of Batches?
USP905
Sample Data
Bayesian Approach
Summary
Conclusions
•  Pa, is a key quality metric in PQ.
•  Ps may also be of interest to manufacturers.
•  ASTM E2709 is a breakthrough in PQ thinking.
•  PQ requires a model for between-batch variance.
•  Bayesian hierarchical modeling provides
– Direct inference on Pa and Ps
– Basis for sampling plan choice
– Manufacturing cost projections
34
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
References
1. 
2. 
American Society for Testing and Materials (ASTM) Standard E11 2709-10, May, 2010
Torbeck, LD (May 2, 2010) Statistical solutions: Bergum’s method recognized, Pharmaceutical
Technology.
3. 
Jon Clark (September 29, 2010) Confidence- critical to batch release: Application of ASTM E2709,
presented at QbD/PAT Conference, University of Heidelberg.
4. 
Lunney, P.D., Anderson, C.A., “Investigation of the Statistical Power of the Content Uniformity Tests
Using Simulation Studies”, Journal of Pharmaceutical Innovation, pp 24-35, 13March2009.
5. 
US Pharmacopoeia 34 (2011) General Chapter <905> Uniformity of Dosage Units (harmonized with JP
and EP).
6. 
LeBlond, DJ (Spring, 2005) Methodology for predicting batch manufacturing risk. MS Thesis, Colorado
State University.
7. 
LeBlond DJ (August, 2009) Risk Assessment of Drug Product Content Uniformity Release Failure: A
Bayesian Approach, Joint Statistical Meetings, Washington DC
8. 
FDA CDER, CBER, CVM (January 2011) Guidance for Industry, Process Valdiation: General Principles
and Practices, rev 1.
9. 
Gelman, A (2006) Prior distributions for variance parameters in hierarchical models, Bayesian Analysis
1(3), 515-533
10.  J.S. Bergum and L. Hua (October 2, 2007), Acceptance Limits for the New ICH USP 29 ContentUniformity Test, Pharm. Technol. Online http://pharmtech.findpharma.com/pharmtech/article/
articleDetail.jsp?id=463577), accessed Apr. 4, 2012, October 2007.
11. 
P. Cholayudth (2009), Establishing Acceptance Limits for Probability of Passing Multiple Stage Tests
in Proces Validation through a Process Capability Approach, Jrnl. of Validat. Technol. 15 (4), 77–90.
12.  Y. Hu and D. LeBlond (2011) Assessment of Large-Sample Unit-Dose Uniformity Tests, Pharmaceutical
Technology 35(10) 82-92.
35
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Backup Slides
36
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Full Conditionals for Gelman’s Half-t Prior9
(
p β | y, η, ξ , σ T2
)
B Ti
⎛
⎞
2
2
β
σ
+
σ
⎜
⎟
0 T
0 ∑∑ ( yij − ξηi )
2 2
σ 0 σ T ⎟
i =1 j =1
= N ⎜ β |
,
⎜
Nσ 02 + σ T2
Nσ 02 + σ T2 ⎟
⎜
⎟
⎝
⎠
N
S
⎛
⎞
p σ T2 | y, η, β , ξ = IG ⎜ σ T2 | + aT , + bT ⎟
2
2
⎝
⎠
where
(
)
B
N = ∑ Ti
i =1
B
2
Ti
S = ∑∑ ( yij − ui )
i =1 j =1
ui = β + ξηi
(
p ξ | y, η, β , σ
(
2
T
)
Ti
B
⎛
⎜ ∑ηi ∑ ( yij − β )
σ T2
i =1
j =1
= N ⎜ ξ | B
,
B
⎜
σ T2
σ T2
2
2
T
η
+
T
η
+
∑
∑
⎜
i i
i i
A2
A2
i =1
i =1
⎝
p ηi | y , β , ξ , σ η2 , σ T2
)
⎞
⎟
⎟
⎟
⎟
⎠
Ti
⎛
⎞
2
ξσ
2 2
⎜
⎟
η ∑ ( yij − β )
ση σ T
i =1
⎟
= N ⎜ηi |
,
σ T2 + Tiξ 2σ η2 σ T2 + Tiξ 2σ η2 ⎟
⎜
⎜
⎟
⎝
⎠
•  m
σ B2 = ξ 2 ⋅ ση2
B
⎛
⎞
η i2 + 1 ⎟
∑
⎜
B + 1 i =1
⎟
p σ η2 | η = IG ⎜ σ η2 |
,
2
2
⎜
⎟
⎜
⎟
⎝
⎠
(
)
37
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
%Bias in β posterior mean
More
Most
5
^
% Bias in β
β : 102
σB : 5
σT : 2
Least
None
10
15
β : 107
σB : 5
σT : 2
β : 102
σB : 2
σT : 5
0.0
15
20
β : 107
σB : 5
σT : 5
-0.5
-0.5
-1.0
β : 107
σB : 2
σT : 2
β : 107
σB : 2
σT : 5
0.0
-0.5
-1.0
-1.0
15
10
0.0
-0.5
10
None
0.0
-1.0
5
Most
5
β : 102
σB : 5
σT : 5
β : 102
σB : 2
σT : 2
More
20
^
% Bias in β
Least
5
20
10
15
20
Number of Batches
Number of Batches
•  Non-informative prior is least biased at lower number of batches
•  Bias may be induced by prior mean
38
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
%RMSE in β posterior mean
Least
More
Most
5
β : 102
σB : 5
σT : 2
None
10
15
20
β : 102
σB : 5
σT : 5
^
% RMSE in β
3
2
1
β : 102
σB : 2
σT : 2
β : 102
σB : 2
σT : 5
3
2
1
5
10
15
20
Number of Batches
•  Virtually identical for β=107
•  Non-informative prior has lowest loss based on MSE
39
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
90% Interval Coverage of beta
Least
More
Most
90% Interval Coverage for β
5
β : 102
σB : 5
σT : 2
None
10
15
β : 102
σB : 5
σT : 5
20
1.00
0.95
0.90
0.85
β : 102
σB : 2
σT : 2
1.00
β : 102
σB : 2
σT : 5
0.95
0.90
0.85
5
10
15
20
Number of Batches
•  Coverage is nearly nominal for Non-informative prior regardless
of number of batches.
40
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
% Bias in sB and sT Posterior Means
More
Most
5
^B
% Bias in σ
β : 102
σB : 5
σT : 2
Least
None
10
15
More
β : 102
σB : 5
σT : 2
β : 102
σB : 5
σT : 5
500
400
300
200
100
0
β : 102
σB : 2
σT : 2
Most
5
20
β : 102
σB : 2
σT : 5
500
400
300
200
100
0
None
10
15
20
β : 102
σB : 5
σT : 5
6
4
^T
% Bias in σ
Least
2
β : 102
σB : 2
σT : 2
β : 102
σB : 2
σT : 5
6
4
2
0
5
10
15
5
20
10
15
20
Number of Batches
Number of Batches
•  Virtually identical for beta = 107
•  Non-informative prior has lowest bias. Bias may be induced by prior
mean.
41
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
0
% RMSE in sB and sT Posterior Means
Least
More
Most
5
β : 102
σB : 5
σT : 2
None
10
15
Least
More
20
Most
5
β : 102
σB : 5
σT : 5
β : 102
σB : 5
σT : 2
None
10
15
20
β : 102
σB : 5
σT : 5
400
200
β : 102
σB : 2
σT : 2
0
β : 102
σB : 2
σT : 5
600
400
^T
% RMSE in σ
^B
% RMSE in σ
600
200
0
5
10
15
20
14
12
10
8
6
β : 102
σB : 2
σT : 2
β : 102
σB : 2
σT : 5
14
12
10
8
6
5
Number of Batches
10
15
20
Number of Batches
•  Virtually identical for beta = 107
•  For sigB, Non-informative prior has least RMSE. For sigT, prior choice
is irrelevant.
42
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Coverage of 90% interval estimates of sB and sT
More
Most
90% Interval Coverage for σB
5
β : 102
σB : 5
σT : 2
None
10
15
Least
0.95
0.90
0.85
0.80
β : 102
σB : 2
σT : 5
0.95
0.90
0.85
0.80
5
10
15
20
Most
5
β : 102
σB : 5
σT : 5
β : 102
σB : 2
σT : 2
More
20
90% Interval Coverage for σT
Least
β : 102
σB : 5
σT : 2
None
10
15
20
β : 102
σB : 5
σT : 5
0.92
0.91
0.90
0.89
0.88
β : 102
σB : 2
σT : 2
β : 102
σB : 2
σT : 5
0.92
0.91
0.90
0.89
0.88
5
Number of Batches
10
15
20
Number of Batches
•  Virtually identical for beta = 107
•  For sigB, Non-informative prior coverage is a little low in some cases.
For sigT, prior has little effect on coverage.
43
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
% Bias and % RMSE in the Ps Posterior Mean
More
Most
5
% Bias in Ps estimate
β : 102
σB : 5
σT : 2
None
10
15
Least
β : 102
σB : 5
σT : 5
β : 102
σB : 2
σT : 2
β : 107
σB : 5
σT : 2
β : 102
σB : 2
σT : 5
250
200
150
100
50
0
10
15
Most
5
250
200
150
100
50
0
5
More
20
% Bias in Ps estimate
Least
20
None
10
15
β : 107
σB : 5
σT : 5
250
200
150
100
50
0
β : 107
σB : 2
σT : 2
β : 107
σB : 2
σT : 5
250
200
150
100
50
0
5
10
Number of Batches
15
20
Number of Batches
•  The % bias for 102, 2, 2 was extremely high because the population Ps is so close to zero.
•  The % RMSE is nearly identical because the bias is the largest contributor to MSE
•  Non-informative prior has lowest % Bias
44
Posterior Probability of Passing a Compendial Test
20
Dave LeBlond
May 10, 2012
Coverage of the 90% Credible Interval for Ps
More
Most
90% Interval Coverage for Ps
5
β : 102
σB : 5
σT : 2
None
10
15
β : 102
σB : 2
σT : 5
15
20
Number of Batches
Most
5
0.9
0.8
0.7
0.6
0.5
0.4
0.9
0.8
0.7
0.6
0.5
0.4
10
More
20
β : 102
σB : 5
σT : 5
β : 102
σB : 2
σT : 2
5
Least
90% Interval Coverage for Ps
Least
β : 107
σB : 5
σT : 2
None
10
15
β : 107
σB : 5
σT : 5
β : 107
σB : 2
σT : 2
β : 107
σB : 2
σT : 5
0.9
0.8
0.7
0.6
0.5
0.4
5
10
15
20
Number of Batches
•  Non-informative prior has coverage closest to nominal
45
Posterior Probability of Passing a Compendial Test
Dave LeBlond
20
May 10, 2012
0.9
0.8
0.7
0.6
0.5
0.4
OC of Proposed Approach: Effect of model
parameter (one at a time, Non-informative prior)
Varied
beta
sigB
•  Population parameters varied
sigT
–  beta: 100-120 with
sigT=sigB=3
–  sigB: 2-15 with sigT=2 and
beta=102
–  sigT: 2-15 with sigB=2 and
beta=102
B:3
Prob[median(Pa) >= 0.9]
1.0
0.8
•  In principle, the median(Pa)
and its variability may depend
on:
0.6
0.4
–  Population Pa
–  Population Parameters.
0.2
•  This graph shows that model
0.0
0.0
0.2
0.4
0.6
0.8
1.0
Population Pa
parameter has a minor affect
on the OC curve, the OC
curve is controlled largely by
the population Pa.
46
Posterior Probability of Passing a Compendial Test
Dave LeBlond
May 10, 2012
Operating Characteristics of Proposed Approach:
Effect of prior, number of batches, choice of statistic
Number of Batches
3
4
6
12
Number of Batches
24
0.5
0.6
Prior : Most
0.7
0.8
3
4
0.9
6
12
1.0
0.5
Prior : None
0.6
Prior : Most
1.0
0.7
0.8
0.6
0.4
0.2
0.0
Prior : Least
Prior : More
1.0
0.8
0.6
0.4
0.8
0.6
0.4
0.2
0.0
Prior : Least
0.8
0.6
0.4
0.2
0.0
0.7
0.8
0.9
1.0
Prior : More
1.0
0.0
0.6
0.5
0.6
Population Pa
0.7
0.8
0.9
1.0
Population Pa
•  Non-Informative Prior least sensitive to sample size
•  Mean(Pa) is more conservative than median(Pa)
47
Posterior Probability of Passing a Compendial Test
1.0
1.0
0.2
0.5
0.9
Prior : None
0.8
Prob[mean(Posterior Pa)>0.9]
Prob[median(Posterior Pa)>0.9]
24
Dave LeBlond
May 10, 2012