Supplementary Data
Determination of the strongest key and dose safety elimination rule
Let pj denote the true dose-limiting toxicity (DLT) rate of the current dose j. Assume that
at a decision time, nj patients have been treated at the current dose, and yj of them
experience DLT. We assume a standard beta-binomial model,
yj | pj ~ Binomial(pj)
pj ~ Beta(a, b),
where Beta(a, b) denotes a beta distribution with parameters a and b. We set a=b=1 to
obtain Jeffery’s non-informative prior. Given the observed data = (nj , yj), the posterior
distribution of pj follows a beta distribution,
pj |data ~ Beta(a+ yj, b+ nj - yj).
(1)
Given a key corresponding to interval (c1, c2), its posterior probability Pr⁡(𝑝𝑗 ∈
(𝑐1 , 𝑐2 )|𝑑𝑎𝑡𝑎), or the area under the posterior distribution curve of pj for that key, is easily
calculated as
Pr⁡(𝑝𝑗 ∈ (𝑐1 , 𝑐2 )|𝑑𝑎𝑡𝑎) = BETA(c2; a+ yj, b+ nj - yj)- BETA(c1; a+ yj, b+ nj - yj),
where BETA(c; a+ yj, b+ nj - yj) is the cumulative density function of the beta distribution
with parameters a+ yj and b+ nj - yj, evaluated at c. We calculate the posterior probability
of each key using the above formula. The key with the highest value of posterior
probability is the strongest key, which can be used to direct dose escalation and deescalation.
The statistical justification for using the strongest key to make the decision of dose
escalation and de-escalation is straightforward, noting that the strongest key represents the
most likely location of the true DLT rate of the current dose. If the strongest key is on the
left (or right) side of the target key, it means the observed data suggest that the current dose
is most likely to represent underdosing (or overdosing), and thus dose escalation (or deescalation) is needed. If the strongest key is the target key, the observed data support that
the current dose is most likely to be in the proper dosing interval, and thus it is desirable to
retain the current dose for treating the next patient.
At the boundaries that are close to 0 or 1, some values may not be covered by the
keys because they are not long enough to form a key. For example, given the proper dosing
interval or target key of (0.25, 0.35), on its left side, we form 2 keys of width 0.1, i.e.,
(0.15, 0.25) and (0.05, 0.15); and on its right side, we form 6 keys of 0.1, that is, (0.35,
0.45), (0.45, 0.55), (0.55, 0.65), (0.65, 0.75), (0.75, 0.85) and (0.85, 0.95). In other words,
(0, 0.05) and (0.95, 1) are not covered by the keys. This, however, does not cause any issue
because, given the non-informative prior beta(1, 1), the posterior distribution given by
equation (1) is either unimodal or monotonic, depending on the values of its two shaper
parameters  and . As the target DLT pT encountered in practice is mostly within (0.1,
0.5) and the width of the target key (i.e., proper dosing interval) << pT, there is at least one
key on each side of the target key. If the true toxicity probability of the current dose is
located in the boundaries that are not covered by the keys, e.g., pj  (0, 0.05) or (0.95, 1),
because the posterior distribution is unimodal or monotonic, the key closest to pj must have
the highest value and is also located on the same side of the target key (as pj). As a result,
the correct decision of dose escalation and de-escalation is guaranteed.
To determine whether the current dose should be eliminated from the trial, we
calculate the following posterior probability
Pr⁡(𝑝𝑗 > 𝑝𝑇 |𝑑𝑎𝑡𝑎) = 1-BETA(pT; a+ yj, b+ nj - yj),
where pT denotes the target DLT rate. If Pr⁡(𝑝𝑗 > 𝑝𝑇 |𝑑𝑎𝑡𝑎)>0.95, the dose is deemed
overly toxic and is eliminated from the trial. The dose elimination rule is evaluated after
every cohort, and a minimum of three patients must be treated at a dose before that dose
can be eliminated. When a dose is eliminated, all the doses higher than that dose are also
eliminated from the trial. If the first dose is eliminated, the trial is terminated early and no
dose should be selected as the maximum tolerated dose (MTD).
Statistical optimality and consistency of the keyboard design
Statistically, the location of the strongest key indicates the mode of the posterior
distribution of 𝑝𝑗 . It is well known that under Bayesian decision theory, decision making
based on the mode of the posterior distribution of 𝑝𝑗 is optimal under the 0-1 loss (Berger,
1993), which provides a formal statistical justification for the keyboard design. In addition,
it can be shown that the keyboard design is consistent.
Theorem 1. When the DLT rate of the MTD is within the proper dosing interval, under the
keyboard design, the dose allocation and selection converges to the MTD.
Proof: It is well known that asymptotically, the posterior distribution of pj under the betabinomial model converges to a normal distribution, and its mode is a consistent estimate of
pj (Gelman et al., 2013). As a result, under a large sample, the strongest key (with the
largest area under the posterior distribution) will have its center at pj. Therefore, if the dose
allocation settles on a dose j that is below the proper dosing interval (i.e., on the left side of
the MTD), the strongest key will eventually move to the left side of the target key (i.e., the
proper dosing interval). Consequently, dose escalation occurs and the dose allocation
leaves dose j. On the other hand, if the current dose j is on the right side of the proper
dosing interval, the strong key will eventually move to the right side of the target key.
Consequently, dose de-escalation occurs and the dose allocation leaves dose j. By the
argument of contradiction, in the long run, dose allocation can only settle on a dose within
the proper dosing interval. Consequently, dose selection converges to the MTD as the
isotonic estimate is consistent.
References
Berger, J.O. Statistical Decision Theory and Bayesian Analysis (Springer Series in
Statistics), 2nd edition, Springer, 1993
Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A., and Rubin, D.B. (2013)
Bayesian Data Analysis, 3rd edition, Chapman & Hall/CRC.
Software
The web-based application for implementing the keyboard design is freely available at
http://www.trialdesign.org, and the author’s website
http://odin.mdacc.tmc.edu/~yyuan/index_code.html
Table S1. Ten true toxicity scenarios for the target DLT rates of 0.2 and 0.3.
Scenario
1
1
2
3
4
5
6
7
8
9
10
0.2
0.2
0.1
0.08
0.04
0.01
0.05
0.02
0.01
0.01
1
0.3
2
0.3
3
0.08
4
0.13
5
0.04
6
0.01
7
0.06
8
0.02
9
0.01
10
0.06
Boldface indicates the MTD.
Dose level
2
3
4
Target DLT rate = 0.2
0.26
0.4
0.45
0.29
0.35
0.5
0.25
0.35
0.2
0.3
0.45
0.2
0.06
0.32
0.2
0.1
0.26
0.2
0.06
0.07
0.2
0.04
0.1
0.2
0.02
0.07
0.08
0.02
0.03
0.04
0.46
0.58
0.4
0.65
0.5
0.35
0.31
0.25
0.2
0.2
Target DLT rate = 0.3
0.36
0.42
0.45
0.4
0.55
0.6
0.38
0.42
0.3
0.42
0.5
0.3
0.07
0.35
0.3
0.12
0.41
0.3
0.07
0.12
0.3
0.05
0.16
0.3
0.02
0.04
0.06
0.07
0.08
0.12
0.46
0.7
0.52
0.8
0.42
0.55
0.4
0.36
0.3
0.3
5
Enroll 3
patients at
current dose
Escalate to
the next
higher dose
0 DLT
1 DLT
2 or 3 DLTs
Deescalate to
the next
lower dose
NO
Enroll 3 more
patients at the
same dose
Have 6 patients
been treated at
previous dose?
0 DLT in
6 patients
1 DLT in
6 patients
2 or more DLTs
In 6 patients
YES
Stop
MTD = current
dose
Stop
MTD =
previous dose
(a) 3+3L design
Enroll 3
patients at
current dose
Escalate to
the next
higher dose
0 DLT
1 DLT
2 or 3 DLTs
Deescalate to
the next
lower dose
NO
Enroll 3 more
patients at the
same dose
Have 6 patients
been treated at
previous dose?
0 or 1 DLT in
6 patients
2 DLTs in
6 patients
3 or more DLTs
In 6 patients
YES
Stop
MTD = current
dose
Stop
MTD =
previous dose
(b) 3+3H design
Figure S1. Schema of two variations of the 3+3 design: (a) 3+3L design and (b) 3+3H
design.
3000
2500
2000
1500
0
500
1000
Number of trials
3
6
9
12
15
18
21
24
27
30
Sample size
Figure S2. Sample size distribution of the 3+3H design when the true toxicity rates of 5
dose levels are 0.01, 0.12, 0.30, 0.41 and 0.55, respectively. The red vertical line indicates
the mean of the sample size.
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
DLT rate
1
2
3
4
5
Dose level
Figure S3. Ten dose-toxicity curves used in the simulation study for the target doselimiting toxicity (DLT) rate of 0.2. The horizontal dashed line indicates the target DLT
rate.
2
3
4
5
6
Scenario
7
8
9
10
5
10
15
20
25
0
1
0
Sample size
20
15
10
Keyboard
mTPI
3+3
5
Sample size
25
30
Target DLT rate = 30%
30
Target DLT rate = 20%
1
2
3
4
5
6
7
8
9
10
Scenario
Figure S4. Average sample size under the 3+3, mTPI and keyboard designs. DLT is doselimiting toxicity.
Comparison of the keyboard, CRM and BOIN designs
exp⁡(𝛼)
The CRM is based on the empirical model: 𝑝𝑗 = 𝑎𝑗
, where the skeleton aj is
generated using the method of Lee and Cheung (2009), with dose level 3 as the prior
estimate of the MTD. For the target DLT rate of 20%, we used the half indifferent interval
of 0.04, resulting in the skeleton (0.070, 0.127, 0.20, 0.285, 0.377); for the target DLT rate
of 30%, we used a slightly larger half indifferent interval of 0.05, resulting in the skeleton
(0.123, 0.204, 0.30, 0.402, 0.501). For the BOIN design, we used its default setting
provided by R package “BOIN”, available at https://cran.rproject.org/web/packages/BOIN/index.html.
80
70
60
50
40
40
50
60
Correct selection (%)
70
80
Keyboard
CRM
BOIN
20
30
30
Correct selection (%)
Target DLT rate = 30%
90
Target DLT rate = 20%
1
2
3
4
5
6
Scenario
7
8
9
10
1
2
3
4
5
6
7
8
9
10
Scenario
Figure S5. Percentage of correct selection of the MTD under the keyboard, CRM and
BOIN designs. A higher value is better. DLT is dose-limiting toxicity.
(a) Average number of patients treated at the MTD
Target DLT rate = 30%
70
60
50
40
10
20
30
Patients treated at MTD (%)
30
40
50
60
Keyboard
CRM
BOIN
20
Patients treated at MTD (%)
70
Target DLT rate = 20%
1
2
3
4
5
6
7
8
9
10
1
2
3
4
Scenario
5
6
7
8
9
10
9
10
Scenario
(b) Average number of patients treated above the MTD
50
40
30
20
10
0
10
20
30
40
50
Keyboard
CRM
BOIN
Patients treated above MTD (%)
Target DLT rate = 30%
0
Patients treated above MTD (%)
Target DLT rate = 20%
1
2
3
4
5
6
Scenario
7
8
9
10
1
2
3
4
5
6
7
8
Scenario
Figure S6. Average percentages of patients treated (a) at the maximum tolerated dose
(MTD) and (b) above the MTD under the keyboard, CRM and BOIN designs. A higher
value is better in (a) and a lower value is better in (b). DLT is dose-limiting toxicity.
(a) Risk of overdosing 60% or more of patients
30
20
10
0
10
20
30
Risk of overdosing (%)
40
Keyboard
CRM
BOIN
40
50
Target DLT rate = 30%
0
Risk of overdosing (%)
50
Target DLT rate = 20%
1
2
3
4
5
6
7
8
9
10
1
2
3
4
Scenario
5
6
7
8
9
10
9
10
Scenario
(b) Risk of overdosing 80% or more of patients
35
30
25
20
15
0
0
5
10
15
20
25
Risk of overdosing (%)
30
Keyboard
CRM
BOIN
5
Risk of overdosing (%)
Target DLT rate = 30%
10
35
Target DLT rate = 20%
1
2
3
4
5
6
Scenario
7
8
9
10
1
2
3
4
5
6
7
8
Scenario
Figure S7. Risk of overdosing (a) 60% or more and (b) 80% or more of the patients under
the keyboard, CRM and BOIN designs. A lower value is better. DLT is dose-limiting
toxicity.
Sensitivity Analysis
We investigated the sensitivity of the keyboard design in terms of the specification of the
keys. As the keys are automatically determined by the target key (i.e., proper dosing
interval), we considered four different target keys for 𝑝𝑇 = 0.2, namely, (0.17, 0.23),
(0.15, 0.23), (0.15, 0.25) and (0.13, 0.24). Figure S7 shows that the performance of the
keyboard design is very similar under different settings, suggesting that the design is
robust to the specification of the keys. The results for 𝑝𝑇 = 0.3 are similar (not shown).
Percent of patients treated at MTD
0
0
5
20
40
10
60
15
Percent of correct selection
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
Scenario
Scenario
Percent of patients treated above MTD
Risk of overdosing 60% patients
9
10
9
10
0
10
20
0 2 4 6 8
30
12
40
1
1
2
3
4
5
6
7
8
9
10
Scenario
1
2
3
4
5
6
7
8
Scenario
0
5
10 15 20 25
Risk of overdosing 80% patients
1
2
3
4
5
6
7
8
9
10
Scenario
Figure S8. Sensitivity analysis with the target dose-limiting toxicity rate of 0.2. Under
each scenario, the bars from left to right correspond to the target key (0.17, 0.23), (0.15,
0.23), (0.15, 0.25) and (0.13, 0.24). MTD is maximum tolerated dose.