---
title: "Utility-based dose finding: BOIN12 and its extensions"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Utility-based dose finding: BOIN12 and its extensions}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
library(simFastBOIN)
```

## A utility of the response and the toxicity

The BOIN12 design of Lin, Zhou, Yan, Li and Yuan (2020) looks for the optimal
biological dose (OBD) by scoring the four outcomes a patient can have. A
response without a dose-limiting toxicity (DLT) scores 100 and a DLT without a
response 0; a patient with neither scores `psi00` and a patient with both
`psi11`, 40 and 60 by default. The utility of a dose is the expected score of
its patients. After each cohort, the boundaries of the BOIN design rule out an
escalation from a dose that looks too toxic, and the next cohort goes to the
most desirable dose among the current dose and its neighbors, the
desirability being the posterior probability that the utility of the dose
exceeds a benchmark. At the end of the trial the OBD is the admissible dose,
up to the estimated MTD, with the highest estimated utility.

The package provides four designs of this family:

* **BOIN12** (Lin et al., 2020): `sim_boin12()` and `sim_boin12_multi()`;
* **TITE-BOIN12** (Zhou et al., 2022), which treats new patients while the
  DLT or the response of earlier ones is still pending: `sim_tite_boin12()`
  and `sim_tite_boin12_multi()`;
* **Comb-BOIN12** (Lu et al., 2025), for two drugs given together:
  `sim_comb_boin12()` and `sim_comb_boin12_multi()`;
* **TITE-Comb-BOIN12** (Lu et al., 2025), its time-to-event version:
  `sim_tite_comb_boin12()` and `sim_tite_comb_boin12_multi()`.

The trials and the selection of the OBD run in C++. In these functions
`target` is the highest acceptable DLT rate and `target_resp` the lowest
acceptable response rate. The `target_resp` of the BOIN-ET functions,
`vignette("simFastBOIN-boin-et")`, is a different quantity, the response rate
that BOIN-ET aims at.

## The desirability score table

`boin12_decision_table()` tabulates, for every number of patients, DLTs and
responses at a dose, whether the dose is admissible and the rank of its
desirability among the outcomes of the table, the score of Table 3 of Lin et
al. (2020).

```{r}
tab <- boin12_decision_table(target = 0.35, target_resp = 0.25)
tab_3 <- as.data.frame(tab)[tab$n == 3, ]
tab_3[, c("n", "n_tox", "n_resp", "admissible", "score")]
```

With the default numbers of patients, 0, 3, 6 and 9, the table has
`r nrow(tab)` rows, the cells of Table 3 of Lin et al. (2020); the lines
above are those for three patients. A higher score means a more desirable
dose, and only the order of the scores of the doses compared matters for the
next cohort. With three DLTs among three patients the dose is inadmissible
whatever the responses: under a uniform prior, the posterior probability that
its DLT rate exceeds 0.35 is then
`r round(stats::pbeta(0.35, 4, 1, lower.tail = FALSE), 3)`, above the cutoff
0.95.

## Operating characteristics for one scenario

Scenario 1 of Lin et al. (2020) has five doses, an upper DLT limit of 0.35, a
lower response limit of 0.25 and twelve cohorts of three patients.
`oc_scenarios()` gives the true utility of every dose and the true MTD and
OBD.

```{r}
p_true <- c(0.05, 0.12, 0.27, 0.35, 0.50)
p_resp <- c(0.20, 0.35, 0.36, 0.37, 0.38)

truth_1 <- oc_scenarios(list("Scenario 1" = list(p_true = p_true,
                                                 p_resp = p_resp)),
                        target = 0.35, target_resp = 0.25)
truth_1
```

The response rate hardly increases beyond dose 2 while the DLT rate does, so
the OBD, dose `r which(truth_1$true_obd)`, is well below the MTD, dose
`r which(truth_1$true_mtd)`.

```{r}
oc <- sim_boin12(target = 0.35, target_resp = 0.25, p_true = p_true,
                 p_resp = p_resp, n_cohort = 12, cohort_size = 3,
                 n_trials = 2000, seed = 123)
oc
```

Dose `r unname(which.max(oc$sel_percent))` is selected most often, in
`r round(max(oc$sel_percent), 1)` percent of the trials, and
`r round(oc$percent_no_obd, 1)` percent of the trials select no OBD.

## The rules of the simulations of the article

The defaults follow the BOIN12 software. Three arguments select the rules
under which Lin et al. (2020) simulated the design, where they differ:
`tie_break = "random"` breaks ties of desirability at random instead of
choosing the higher dose, `no_candidate = "stop"` ends the trial without an
OBD when no dose allowed for the next cohort is admissible instead of going to
the nearest admissible dose, and `explore_from = "admissible"` applies the
rule that escalates after 9 patients when the next dose is untried only at an
admissible dose. The article states the first; the other two are not written
in it and were inferred from the published results, which they reproduce
(see the last section).

```{r}
oc_article <- sim_boin12(target = 0.35, target_resp = 0.25, p_true = p_true,
                         p_resp = p_resp, n_cohort = 12, cohort_size = 3,
                         n_trials = 2000, tie_break = "random",
                         no_candidate = "stop", explore_from = "admissible",
                         seed = 123)
selection <- rbind(default = c(oc$sel_percent, oc$percent_no_obd),
                   article = c(oc_article$sel_percent,
                               oc_article$percent_no_obd))
colnames(selection) <- c(paste0("DL", 1:5), "None")
round(selection, 1)
```

## TITE-BOIN12

When the DLT and the response take long to assess, TITE-BOIN12 (Zhou et al.,
2022) treats new patients while earlier ones are still followed. A pending
outcome is replaced by its expectation given the time followed, as in
Equation (11) of the article (`pending_estimate = "conditional"`, the
default); `pending_estimate = "marginal"` replaces it by the estimated rate
itself, which reproduces the desirabilities of Table S.3 of their Supplement.
While fewer than half of the patients at the current dose have finished both
assessments, the next decision waits (`min_completed_share = 0.5`). The
simulations of the article have windows of 45 days for the DLT and 60 days
for the response and three patients a month; the unit of time is the day.

```{r}
tite <- sim_tite_boin12(target = 0.35, target_resp = 0.25, p_true = p_true,
                        p_resp = p_resp, n_cohort = 12, cohort_size = 3,
                        window = 45, resp_window = 60, accrual_rate = 3 / 30,
                        n_trials = 2000, seed = 123)
tite
```

When every decision waits for all the outcomes and the patients arrive at
fixed intervals, the trials are those of BOIN12 with the same seed, and only
the time differs.

```{r}
wait <- sim_tite_boin12(target = 0.35, target_resp = 0.25, p_true = p_true,
                        p_resp = p_resp, n_cohort = 12, cohort_size = 3,
                        window = 45, resp_window = 60, accrual_rate = 3 / 30,
                        accrual = "fixed", min_completed_share = 1,
                        n_trials = 2000, seed = 123)
all.equal(wait$sel_percent, oc$sel_percent)
round(c(TITE = tite$duration, wait = wait$duration) / 30, 1)
```

The trials last `r round(tite$duration / 30, 1)` months on average with
TITE-BOIN12 and `r round(wait$duration / 30, 1)` months when every decision
waits.

## Comb-BOIN12 and TITE-Comb-BOIN12

Comb-BOIN12 (Lu et al., 2025) applies the utility, the desirability and the
admissibility of BOIN12 to a matrix of combinations of two drugs, with the
levels of drug A in the rows and those of drug B in the columns. The next
cohort goes to a combination chosen by the desirability among the current one
and its neighbors along both drugs, as BOIN12 does along the doses. At the
end the response rates are estimated by a quadratic logistic model, and the
admissible combination with the highest utility up to the MTD combination is
selected. The utility of the article, `pE - wT * pT`, is that of BOIN12
with `psi00 = 100 * wT / (1 + wT)` and `psi11 = 100 / (1 + wT)`, and the
weight `w_u` of its benchmark is `benchmark_weight`. Scenario 16 of the
article has two levels of drug A and three of drug B.

```{r}
p_true_ab <- matrix(c(0.05, 0.10, 0.15, 0.15, 0.20, 0.45), nrow = 2)
p_resp_ab <- matrix(c(0.10, 0.20, 0.30, 0.40, 0.50, 0.60), nrow = 2)

comb_design <- list(target = 0.33, target_resp = 0.2, p_true = p_true_ab,
                    p_resp = p_resp_ab, n_cohort = 14, cohort_size = 3,
                    psi00 = 100 * 0.11 / 1.11, psi11 = 100 / 1.11,
                    benchmark_weight = 0.6, extrasafe = TRUE, offset = 0.10,
                    n_trials = 1000, seed = 123)
oc_comb <- do.call(sim_comb_boin12, comb_design)
oc_comb
```

The true OBD combination is
`r paste0("A", oc_comb$true_obd[, 1], "B", oc_comb$true_obd[, 2])`, and it
is selected in `r round(oc_comb$pct_true_obd_selected, 1)` percent of the
trials.

The article states that the OBD combination is chosen among the combinations
up to the levels of both drugs of the MTD combination
(`obd_bound = "rectangle"`, the default). The published selection
percentages agree better with candidates bounded, at every level of drug B,
by the MTD along drug A (`obd_bound = "contour"`), a rule inferred from the
published results. The same trials give

```{r}
oc_contour <- do.call(sim_comb_boin12,
                      c(comb_design, list(obd_bound = "contour")))
labels <- paste0("A", rep(1:2, 3), "B", rep(1:3, each = 2))
bounds <- rbind(rectangle = as.vector(oc_comb$sel_percent),
                contour = as.vector(oc_contour$sel_percent))
colnames(bounds) <- labels
round(bounds, 1)
```

TITE-Comb-BOIN12 applies the rules of Comb-BOIN12 at the decision times of
TITE-BOIN12. By default the pending outcomes count in the DLT rate of the
decision, in the admissibility and in the utility, as in TITE-BOIN12;
Sections 2.2 and 3.1 of Lu et al. (2025) state that they count in the utility
only, the decision and the admissibility using the observed outcomes
(`pending_use = "utility"`). The article does not describe the accrual or the
waiting rule, for which the package uses those of TITE-BOIN12. One of the
settings of Figure 2 of the article has windows of one month for the DLT and
two months for the response and two patients a month, here with uniform
times to the events, as Section 2.2 of the article assumes:

```{r}
tite_comb_design <- c(comb_design[names(comb_design) != "n_trials"],
                      list(window = 1, resp_window = 2, accrual_rate = 2,
                           event_time = "uniform", n_trials = 500))
oc_tc <- do.call(sim_tite_comb_boin12, tite_comb_design)
oc_tc_utility <- do.call(sim_tite_comb_boin12,
                         c(tite_comb_design, list(pending_use = "utility")))
pending <- rbind(all = c(as.vector(oc_tc$sel_percent), oc_tc$duration),
                 utility = c(as.vector(oc_tc_utility$sel_percent),
                             oc_tc_utility$duration))
colnames(pending) <- c(labels, "Months")
round(pending, 1)
```

## Comparing designs

`oc_scenarios()`, `oc_results()` and `oc_compare()` set the results of
different designs side by side, also for designs that look for the MTD.
Scenarios 1 to 3 of Lin et al. (2020) are run with BOIN, which selects the
MTD, and with BOIN-ET and BOIN12, which select an OBD. BOIN-ET needs the
response rate it aims at, set here to 0.40, and its default correlation of
the DLT and the response, 0.2, is set to 0, the default of BOIN12, so that
the designs are compared on the same outcomes.

```{r}
scenarios <- list(
  "Scenario 1" = list(p_true = c(0.05, 0.12, 0.27, 0.35, 0.50),
                      p_resp = c(0.20, 0.35, 0.36, 0.37, 0.38)),
  "Scenario 2" = list(p_true = c(0.05, 0.12, 0.27, 0.35, 0.50),
                      p_resp = c(0.01, 0.05, 0.30, 0.60, 0.60)),
  "Scenario 3" = list(p_true = c(0.03, 0.05, 0.20, 0.22, 0.45),
                      p_resp = c(0.05, 0.10, 0.50, 0.68, 0.70))
)
truth <- oc_scenarios(scenarios, target = 0.35, target_resp = 0.25)

boin <- sim_boin_multi(target = 0.35, scenarios = scenarios, n_cohort = 12,
                       cohort_size = 3, n_trials = 2000, seed = 123)
boin_et <- sim_boin_et_multi(target = 0.35, target_resp = 0.40,
                             scenarios = scenarios, n_cohort = 12,
                             cohort_size = 3, n_trials = 2000, resp_cor = 0,
                             seed = 123)
boin12 <- sim_boin12_multi(target = 0.35, target_resp = 0.25,
                           scenarios = scenarios, n_cohort = 12,
                           cohort_size = 3, n_trials = 2000, seed = 123)

comparison <- oc_compare(BOIN = boin, "BOIN-ET" = boin_et, BOIN12 = boin12,
                         truth = truth)
comparison
```

```{r, include = FALSE}
correct_1 <- function(design) {
  comparison$pct_correct[comparison$design == design &
                           comparison$scenario == "Scenario 1"]
}
```

The correct selection counts the true MTD for BOIN and the true OBD for the
other two designs; an overdose is a dose with a DLT rate above 0.35. In
Scenario 1, where the OBD lies below the MTD, BOIN12 selects the OBD in
`r round(correct_1("BOIN12"), 1)` percent of the trials and BOIN-ET in
`r round(correct_1("BOIN-ET"), 1)` percent. The figure shows the same table.

```{r compare-plot, fig.width = 7, fig.height = 6, eval = requireNamespace("ggplot2", quietly = TRUE)}
plot(comparison)
```

## How the implementation is checked

```{r checks, include = FALSE}
read_result <- function(dir, file) {
  utils::read.csv(system.file(dir, "results", file, package = "simFastBOIN"),
                  stringsAsFactors = FALSE)
}
n_pass <- function(x) sum(x$judgement == "PASS")
n_fail <- function(x) sum(x$judgement == "FAIL")
n_explained <- function(x) sum(x$judgement == "EXPLAINED")
n_within <- function(x) sum(grepl("within the tolerance", x$reason))

lin <- read_result("reproduce", "reproduce_Lin2020.csv")
lin_exact <- lin[lin$source %in% c("Lin 2020 Table 3", "Lin 2020 Figure 1",
                                   "Lin 2020 SI Section 6"), ]
lin_sim <- lin[lin$source %in% c("Lin 2020 Figure 3", "Lin 2020 SI Figure S3",
                                 "Lin 2020 SI Figure S4"), ]
lin_article <- lin_sim[!grepl("default rules", lin_sim$item), ]
lin_default <- lin_sim[grepl("default rules", lin_sim$item), ]

zhou <- read_result("reproduce", "reproduce_Zhou2022.csv")
zhou_s3 <- zhou[zhou$source == "Zhou 2022 Table S.3", ]
zhou_fig <- zhou[zhou$source == "Zhou 2022 Figure 3" &
                   zhou$judgement %in% c("PASS", "FAIL"), ]
zhou_fail <- zhou_fig[zhou_fig$judgement == "FAIL", ]
zhou_fail_scenario <- sub(",.*$", "", zhou_fail$item[1])

lu <- read_result("reproduce", "reproduce_Lu2025.csv")
lu_exact <- lu[lu$source %in% c("Lu 2025 Table 1", "Lu 2025 Table 2",
                                "Lu 2025 SI Table S2", "Lu 2025 SI Table S3",
                                "Lu 2025 SI Figure S1"), ]
lu_sim <- lu[!lu$source %in% lu_exact$source &
               lu$judgement %in% c("PASS", "FAIL"), ]
lu_fail_tite <- sum(lu_sim$judgement == "FAIL" &
                      grepl("TITE-Comb-BOIN12", lu_sim$item))
lu_info_select <- lu[lu$source %in% c("Lu 2025 Table 3", "Lu 2025 Table 4") &
                       lu$judgement == "INFO", ]

esc <- read_result("validation", "compare-with-escalation.csv")
```

The scripts in `inst/reproduce/` and `inst/validation/` compare the package
with the articles and with the 'escalation' package, and write tables of
judgments in their `results/` folders, which are summarized here. A
simulated value passes when it lies within three or four standard errors of
the difference between the two simulations, as each script states, plus the
rounding of the published value. Values listed for information are not
judged.

* **Lin et al. (2020).** `r n_pass(lin_exact)` of the `r nrow(lin_exact)`
  deterministic values (Table 3, the boundaries of Figure 1 and the example
  of Section 6 of the Supplement) agree to the printed digits. With the three
  arguments `tie_break = "random"`, `no_candidate = "stop"` and
  `explore_from = "admissible"`, `r n_pass(lin_article)` of the
  `r nrow(lin_article)` values of Figure 3 and of Figures S3 and S4 of the
  Supplement pass; the `r nrow(lin_article) - n_pass(lin_article)` others are
  listed for information, because no dose is an OBD there and the figure
  plots 0. With the defaults, `r n_within(lin_default)` of the
  `r nrow(lin_default)` values lie within the tolerance.
* **Zhou et al. (2022).** With `pending_estimate = "marginal"`,
  `r n_pass(zhou_s3)` of the `r nrow(zhou_s3)` desirabilities and estimates
  of Table S.3 of the Supplement agree to the printed digits, and the
  `r n_explained(zhou_s3)` others are explained by the table itself: one of
  its rows was computed from data other than those of Table 2 of the
  article, and one value repeats that of another dose. Of the
  `r nrow(zhou_fig)` values of Figure 3 that are judged,
  `r n_pass(zhou_fig)` pass and `r n_fail(zhou_fig)` fails, the number of
  patients at the OBD of TITE-BOIN12 in `r zhou_fail_scenario`,
  `r zhou_fail$published[1]` in the figure against
  `r zhou_fail$reproduced[1]`. The durations are not judged, because the
  article does not describe the accrual.
* **Lu et al. (2025).** `r n_pass(lu_exact)` of the `r nrow(lu_exact)`
  deterministic values (the utilities of Tables 1 and 2 and Supplementary
  Tables S2 and S3, and the tail probabilities of Supplementary Figure S1)
  agree to the printed digits. Of the `r nrow(lu_sim)` simulated values of
  Figure 2, Supplementary Figure S2 and Tables 3 and 4 that are judged, with
  `obd_bound = "contour"` and, for TITE-Comb-BOIN12,
  `pending_use = "utility"`, `r n_pass(lu_sim)` pass and `r n_fail(lu_sim)`
  fail:
  `r lu_fail_tite` bars of TITE-Comb-BOIN12, whose accrual and waiting rules
  the article does not describe, and `r n_fail(lu_sim) - lu_fail_tite`
  selection percentages of Comb-BOIN12. Their cause has not been found. The
  `r nrow(lu_info_select)` selection percentages of Tables 3 and 4 are listed
  for information: they differ from those of Figure 2 for the same trials,
  and no rule that was tried reproduces them.
* **'escalation'.** The operating characteristics of `sim_boin12()` are
  compared with those of `escalation::get_boin12()` in nine scenarios:
  `r n_pass(esc)` of the `r nrow(esc)` values pass and the
  `r n_explained(esc)` others are explained by the rules of 'escalation'
  when a dose becomes inadmissible, which 'escalation' checks only for the
  doses next to the current one.
* **Separate implementations.** The trials of the four engines agree
  exactly, trial by trial, with implementations written in Python and fed
  with the same random numbers, and so does the selection of Comb-BOIN12,
  whose logistic estimates agree with long Markov chain Monte Carlo runs
  within 0.01.

## References

Lin, R., Zhou, Y., Yan, F., Li, D. and Yuan, Y. (2020). BOIN12: Bayesian
optimal interval phase I/II trial design for utility-based dose finding in
immunotherapy and targeted therapies. *JCO Precision Oncology*, 4,
1393-1402.

Zhou, Y., Lin, R., Lee, J. J., Li, D., Wang, L., Li, R. and Yuan, Y. (2022).
TITE-BOIN12: A Bayesian phase I/II trial design to find the optimal
biological dose with late-onset toxicity and efficacy. *Statistics in
Medicine*, 41(11), 1918-1931.

Lu, M., Zhang, J., Yuan, Y. and Lin, R. (2025). Comb-BOIN12: A utility-based
Bayesian optimal interval design for dose optimization in cancer
drug-combination trials. *Statistics in Biopharmaceutical Research*, 17(2),
266-276.
