Statistical Recommendations for
Genotoxicity Assessment of
Pharmaceuticals: A Practical Approach
Helena Geys1, Fetene Tekle1, Maxim Nazarov2
June 13, 2017
1: Johnson & Johnson
2: Open Analytics
Objectives

Toxicology:


Prove the harmlessness of a test substance
Genotoxicology:

Detect compounds which induce genetic damage directly
or indirectly by various mechanisms

Positive compounds may induce

Cancer and/or

Heritable defects
1
Genotoxicity Testing


A standard genotox battery includes two invitro and one
invivo assay:

Test for gene mutations in bacteria (AMES)

Invitro test to detect chromosome aberrations (e.g Invitro
MNT)

Invivo micronucleus test (e.g Invivo MNT)
In the event of a positive event a COMET assay is often
considered
2
Invivo Micronucleus Test: Design
(slide: Bas-Jan Van der Leede)
Single dose/Multiple sampling
0h
24h
48h
Species: mouse/rat/….
Gender: 6 or 7 males in single gender
5 males/5 females
Samples: bone marrow/peripheral blood
Dose groups: VC, L, M, H, PC
3
Invivo Micronucleus Test
(slide: Bas-Jan Van der Leede)
bone marrow
X
Aneugenic
chemicals
Orthochromatic
Erythroblast
blood
Polychromatic
Erythrocyte
Normochromatic
Erythrocyte
PCE
NCE
Reticulocyte
RET
NCE
Single chromosome, 2N 2C
G1
Mitosis
G2
S
DNA Synthesis
Chromosome replication
X
Clastogenic
Doubled chromosome, 2N 4C
chemicals
4
Invivo Micronucleus Test
5
COMET Assay


Cells
•
From liver, stomach, kidney, duodenum, (blood)
•
Embedded in a thick layer of gel
•
Put in electrophoresis tank
Broken strands of DNA migrate out of the nucleus in a “comet tail”
(source: http://www.cellbiolabs.com/comet-assay-kits-and-slides)
6
Comet Assay


Advantages:

Quick

Sensitive

Cheap

Useful evaluation of local genotoxicity in organs which cannot
easily be evaluated with other standard tests
Optimal Experimental Design (Smith et al. 2008, Recommendations for
the design of the Comet Assay, Mutagenesis, 1-8)

V, L, M, H (+PC) dose groups

2-3 gels per tissue

50 nuclei per gel

5-6 rats per dose group
7
PSI SIG TOX


Title Explanation:
•
PSI: Statisticians in the
Pharmaceutical Industry
•
SIG: Special Interest Group
•
TOX: Toxicology
Forum for statisticians:
•
discuss,
•
review,
•
share examples
•
good statistical practice in
toxicology and safety assessment
Affiliates:
Pharmaceutical
Companies
Contract Research
Organizations
Academia
8
Invivo Micronucleus Test: Current
Statistical Analyses among PSI

Primary outcome: #micronucleated reticulocytes
(MNRET) per a certain number of reticulocytes (RET)

Analysis of V, L, M, H dose groups: wide variety of
approaches cross-company!

General Linear Model on transformed data (e.g square root
+ 1 or log)

Exact trend test (e.g one-sided JT)

Pairwise test: compare each dose group versus V (FE,
Chisquare)

…
9
Invivo Micronucleus Test: Current
Statistical Analyses among PSI
PC only used as check of study/equipment validity
(separate VC-PC comparison)
Historical Control Data:

Not formally used in stats analysis

Used to place statistical analysis into context
10
Invivo Micronucleus Test:
critical appraisal
Hothorn and Gerhard (2009):

What is the endpoint distribution?


What is the experimental unit?


Binomial proportion or count (Poisson data)
Clearly, the animal. Hence, variability between animals should be
taken into account, e.g using a quasi-Poisson model or quasibinomial model.
Confidence intervals or pvalues?

Pvalue is just a number between 0 and 1 (are “stars” appropriate?)

Conf intervals allow the claim for both significance and biological
relevance by its distance to the null-hypothesis value of one.
11
Invivo Micronucleus Test:
critical appraisal?

What is the objective of the evaluation?

To detect a possible effect = proof of hazard

To proof the harmlessness of a drug = proof of safety
Absence of Evidence is No Evidence of Absence!
A to-be-continued discussion or time for
action & implementation??
12
OECD Guidelines

OECD (2014), Test No. 474: Mammalian Erythrocyte
Micronucleus Test, OECD Publishing, Paris.
DOI: http://dx.doi.org/10.1787/9789264224292-en

More attention to statistics
Positive Chemical
 At least 1 of trts: stat sign
increase
 Increase = dose-related when
evaluated with appropriate
trend test
 Any of results outside of
historical negative control data
(e.g Poisson-based 95% control
limits)
Negative Chemical
 None of trts: stat sign increase
 No dose-related increase when
evaluated with appropriate trend
test
 All results inside historical
negative control data (e.g
Poisson-based 95% control limits)
13
Let’s do it!


Study Design:

V, L, M, H, SH dose groups

5 animals per group
Primary endpoint:


Binomial proportions MNRET/RET
Model:

Quasi-binomial model

Dunnett strategy (comparison versus V)

Simultaneous 95% confidence intervals

William’s trend test
14
Example: Summary Stats for
MN-RET percentages
15
Example

Maxim?
16
Example

Comparison versus Vehicle:

Trend test:
17
Example

Maxim?
18
Historical Control Data
19
Historical Control Data:
Summaries of MN counts per
20000 cells
20
Historical Control Data: QC
21
Historical Control Data (ind.)
22
Historical Control Data (aggr.)
23
Comet Assay: Nested Design

Three-level hierarchies with clustering at animal and
slide level
24
Comet Assay: Responses of
Interest



Tail Length (TL)
•
Length of tail
•
Criteria for determining the end of the tail
•
Not comparable across studies
Tail Intensity
•
Intensity of DNA fragments in the tail
•
Can be standardized across studies
•
Primary endpoint
Tail Moment (TLxTI)
25
Comet Assay: Statistical
Issues/Challenges


Non-Gaussian Outcomes (time-to-event like)

Asymmetric

Skewed

Positive

Bi- or multimodal

Mixture

…
Multi-level hierarchical structure
26
Comet Assay: Current
recommended Approach for dayto-day analyses (
)
Bright et al. 2011, Pharmaceutical Statistics

Analyse each tissue separately

Omit PC because variability is typically smaller here

Analysis strategy for V, L, M, H:

Log transform the outcome (+0.0001)

Picture the raw TI for individual cells: impression of
distribution of values and how these may have changed
wrt location and/or variability)

Hierarchical structure is partly or completely ignored

Summarize per gel or per animal through median and
mean

Central limit theorem: approximately normal

Analyse using ANOVA or repeated ANOVA
27
Comet Assay: Current
recommended Approach for dayto-day analyses (
)
Bright et al. 2011, Pharmaceutical Statistics

Recommend that confidence intervals and p-values
should be 1-sided (assuming, as is usual, that it is only
increases in TI that are of biological importance).

Typically p-values are not adjusted for multiple
comparisons but there is not a consensus and it remains
a point of discussion.

Again one might argue that focus should be on the
confidence intervals rather than p-values, since the
former immediately convey the sizes of effects
consistent with the study data (for a given level of
“confidence”).
28
Example
29
Example
30
Example
31
Comet Assay: Critical
Appraisal

Rather simple analysis approach for complex data

But:

Disadvantages:
THERE IS NO FREE
LUNCH !

Loss of information (150 cells summarized by e.g a single
value)

Averaging effect may have a major impact on parameter
estimation and corresponding inferences

Hierarchical nature is ignored:

Cells coming from the same rat could be more alike due to
biological reasons

Cells are grouped into three slides: could pose some clustering due
to uncontrolled differences (e.g. amount of gel being used)
32
Box Plots (TI & TL)
33
Tail Intensities
34
Tail Lenghts
35
Scatter Plots

Considerable variability at slide level

After adjustment for rat, variation shrinks most for TL

Rat effect more important for TL than for TI !
36
Alternative Approach
Needed?

Non-normal data are often modeled through
exponential family models

Notorious members are Bernoulli and Poisson

For time-to-event outcomes, the Weibull is described as
an appropriate choice
37
Incorporate Clustering and
Overdispersion
38
Three-level Hierarchy
39
Bayesian Generalized Frailty
Model

Ghebretinsae et al. (2012 JBS) published a paper on a
Bayesian Generalized Frailty Model for Comet assays that:
(1) uses the Weibull distribution
(2) deals with the complete hierarchical nature;
(3) uses all information instead of summary measures.


For TL (secondary endpoint):
•
Accounting for the hierarchical structure and inclusion of an
overdispersion parameter had a substantial impact on the
estimate (approx 3 times) and standard error (4 times)
•
Underscores the risk of using models that are too simple
For TI (primary endpoint!)
•
results in line with the simpler recommended traditional
approach! (slightly higher SE) => motivation!
40
General Discussion Points
Interpretation of Responses

Currently, proof of hazard is mostly implemented but
“absence of proof is no proof of absence”

Proof of safety through formal equivalence tests is
seldom adopted within the toxicology area!?

Informally it is assessed through historical control data,
e.g. if the combined sample distribution of the three
treated groups falls within the historical control
sampling distribution

Historical control mean and dispersion should be
stationary (use process control charts!)
41
Industry practice…
…a limitation?

Problem Setting:

Experiments run on a regular basis (compound screening)

Common experimental setup with little variation

Fixed structure of the statistical reports
42
(ou)R solution


Suite of R packages: reading, visualizing, analyzing data

invivoMNT

invitroMNT

cometAssays
Custom templates: automatic report generation
cometAssays
toxUtils
invivoMNT
Custom templates
invitroMNT
43
Acknowledgement
44