Effect of an outlier on quantitative bioassays
Perceval Sondag, Lingmin Zeng, Binbing Yu, Rejane Rousseau, Bruno Boulanger, Harry Yang
Arlenda SA & MedImmune LLC.
Contact:
Perceval Sondag,
Statistician
[email protected]
Mobile: +32 491 22 17 56
Context
 Relative potency in bioassay design:
 The RP is estimated from a concentration or log(concentration)-response
function, as the horizontal difference between sample and standard
curves.
 Parallelism between functions is required to compute RP
 Occurrence of an outlier is common !
 Many work have been done on how to detect outlier
 But what is really the effect of an outlier on quantitative
bioassays?
 Effect on the Relative Potency estimation.
 Effect on Parallelism testing
 We focus here on Parallel-Curve Assays with 4PL curves.
Arlenda © 2015
2
Parallel curve design
Choosing a Non-Linear Model
 The four parameter logistic (4PL) model is a nonlinear function
characterized by 4 parameters:
 a = upper asymptote
b = slope at inflection point
 c = ec50 (inflection point)
d = lower asymptote
0.01
RP
Ref
Test
0.001
Response
𝑑−𝑎
𝑦=𝑎+
𝑥 𝑏
1+
𝑐
1
10
Parallel curve
0.1
1
10
Concentration
Arlenda © 2015
3
Example of format
Reference
Sample
Control or empty
Dilution
 Fit one curve for 3 replicates (3 rows).
 (Other formats possible).
Arlenda © 2015
4
What could possibly go wrong? [1]
 Single outlier value:
 A single measurement could be excessively distant from the other values
Arlenda © 2015
5
What could possibly go wrong? [2]
 Serial dilution outlier:
 If replicates come from different serial dilutions, one of them could fail
and the dilution factor could be affected for one of the replicates
Arlenda © 2015
6
What could possibly go wrong? [3]
 Location outlier:
 If replicates are in fact pseudo-replicates of the same serial dilution, a
location effect (row, plate,…) could affect the slope, an asymptote, or
induce a vertical shift of the curve.
Arlenda © 2015
7
Simulation material
 The effect of different types of outliers is assessed over a
simulation study.
 Every scenario is evaluated 1000 times.
 4PL curves are simulated with following parameters:
 a = 100

b=2
𝑦𝑅𝑒𝑓𝑒𝑟𝑒𝑛𝑐𝑒 = 100 +
c = 0.0625
0−100
1+
𝐶𝑜𝑛𝑐𝑒𝑛𝑡𝑟𝑎𝑡𝑖𝑜𝑛 2
0.0625
d=0
+ 𝑁 0, 𝜎
𝑦𝑇𝑒𝑠𝑡 = 100 +
0−100
1+
𝐶𝑜𝑛𝑐𝑒𝑛𝑡𝑟𝑎𝑡𝑖𝑜𝑛 2
0.0625∗𝑹𝑷
+ 𝑁(0, 𝜎)
 10 dilution steps: 𝐶𝑜𝑛𝑐𝑒𝑛𝑡𝑟𝑎𝑡𝑖𝑜𝑛 = 1, 0.5, 0.25, 0.125, … , 0.002
 Each curve is fitted across 3 replicates
 Simulation scenarios:
 Several Residual Standard Deviation: 𝜎 = 2, 5 or 10
 Several Relative Potencies: RP = 80%, 100%, 125%
Arlenda © 2015
8
Outlier Generation
 Single outlier
 A multiple of σ is added to a single observation. The multiplication factor
is a quantile of an F-distribution: ∆𝑜𝑢𝑡𝑙𝑖𝑒𝑟 = ±𝐹9,9 ∗ σ.
 Different quantile are tested: 0.8, 0.9, 0.99 (F =1.79, 2.44, 5.35).
 ∆𝑜𝑢𝑡𝑙𝑖𝑒𝑟 can be added or removed at any dilution step.
 Serial dilution outlier
 If everything works fine, concentration is divided by 2 at every step.
 Dilution factor is replaced by 1.75 or 2.5 (±12.5%) for one of the
replicates.
 Location outlier
 For one of the replicates, slope is replaced by 1 or 4 (±100%) or
 The upper asymptote is replaced by 90 or 110 (±10%) or
 A vertical shift of ±10 is added at every dilution step.
Arlenda © 2015
9
Effect of the outlier
 Bias on the RP estimation
 Bias in RP estimation is estimated by the geometric mean of
𝐸𝑠𝑡𝑖𝑚𝑎𝑡𝑒𝑑 𝑅𝑃
𝑇𝑟𝑢𝑒 𝑅𝑃
 Effect on parallelism test
 Equivalence test using Chi-squared metrics
 Equivalence test of the parameters (Yang et al., 2012)
 Equivalence margins for both test have been derived separately for each
scenarios and based on simulated data.
 Other parallelism tests exist.
Arlenda © 2015
10
Single Outlier – Bias in RP estimation
Arlenda © 2015
11
Single Outlier – Effect on parallelism test
Arlenda © 2015
12
Whole Curve Outlier – Bias in RP estimation
Arlenda © 2015
13
Whole Curve Outlier – Effect on parallelism test
Arlenda © 2015
14
Discussion
 Occurrence of outliers is common in bioassay setting.
 It is important to detect and reject an outlier when it has an effect
on the Relative Potency estimation.
 If an (undetected) outlier induces high bias of RP, it is preferable
that the parallelism test fails and reject the curve.
 Single outlier rarely induce parallelism rejection
 Whole curve outlier induce parallelism rejection if the assay
variability is small. But the probability of rejection is not
proportional to the bias included in the RP.
 This encourage outlier testing to detect and exclude outliers.
 Several outlier tests exist for single outliers, but nothing (yet)
exists for the detection of a whole curve outlier.
 To be continued…
Arlenda © 2015
15
Acknowledgment
 We would like to thank Tim Schofield and the MedImmune
Bioassay team for their help and support in this work.
 Thank you for your attention !
Arlenda © 2015
16