1.1 Basics of hybrid calibration

Hybrid calibration is requested via

calibration = "hybrid"

in design_singlearm_bf(). In this mode:

• Bayesian power is calibrated using the \(H_1\) design prior, i.e. averaged over \(p > p_0\) under a truncated Beta prior for directional tests.
• Frequentist type-I error is calibrated at \(p = p_0\), i.e. the probability of incorrectly rejecting \(H_0\) is controlled in the classical frequentist sense.

You must specify:

• target_power: the target Bayesian power,
• target_freq_type1: the target frequentist type-I error.

The frequentist power at a point alternative dp is computed and reported but does not drive feasibility of a design in the hybrid calibration mode. Thus, the frequentist power is solely computed and reported post-hoc for the isolated optimal hybrid design which meets the specified target constraints on Bayesian power and frequentist type-I-error.

1.2 Finding an optimal hybrid design

We start by constructing a hybrid-optimal design. Therefore, we consider the test of \[H_0:p\leq 0.2 \text{ versus }H_1:p>0.2\] for a clinical phase II trial with binary endpoints in the treatment group (success / failure). The novel treatment or drug is deemed effective if we find sufficient evidence in favour of \(H_1\), when the response probability to that novel treatment or drug is larger than 20%.

We specify slightly optimistic design priors via da1 = 2.5 and db1 = 2 under \(H_1\), and remain uninformative via the design prior parameters da0 = 1 and db0 = 1 under \(H_0\). The evidence threshold \(k\) is set to \(1/10\), which implies we use the standard threshold for strong evidence (Jeffreys 1961), whereas we use the less stringent threshold \(k_f=3\) to stop early for futility. The analysis prior parameters for the analysis prior used when computing the Bayes factor \(BF_{01}\) are specified via the arguments a = 1 and b = 1:

res_hybrid <- design_singlearm_bf(
  n1_min = 5,
  n2_max = 100,
  k      = 1/10,
  k_f    = 3,
  p0     = 0.2,
  a0     = 1,
  b0     = 1,
  a1     = 1,
  b1     = 1,
  dp     = 0.4,
  da0    = 1,
  db0    = 1,
  da1    = 2.5,
  db1    = 2,
  type   = "direction",
  calibration       = "hybrid",
  target_power      = 0.80,
  target_freq_type1 = 0.05,
  power_cushion = 0.025
)

We inspect the resulting output of the hybrid calibration algorithm as follows:

summary(res_hybrid)
#> Summary: Single-arm two-stage Bayes factor design
#> ---------------------------------------------------------
#> Feasible: TRUE
#> Calibration: hybrid
#> Design prior under H0: Beta(1, 1) truncated to [0, p0]
#> Design prior under H1: Beta(2.5, 2) truncated to (p0, 1]
#> 
#> Selected design: n1 = 7, n2 = 24
#> 
#> Bayesian operating characteristics
#>   Power: 0.8072
#>   Type-I: 0.0043
#>   CE H0: NA
#>   EN H0: 9.88
#>   EN H1: 22.45
#> 
#> Frequentist operating characteristics
#>   Power: 0.6195
#>   Type-I: 0.0316
#>   EN H0: 14.20
#>   EN H1: 21.30

The output shows:

• the selected interim and final sample sizes (n1, n2),
• Bayesian power and Bayesian type-I error,
• frequentist type-I error and frequentist power at dp,
• expected sample sizes under \(H_0\) and \(H_1\),
• feasibility and a status message.

The search results can be plotted, too:

plot(res_hybrid)

Figure 1: Output of the plot function for an optimal hybrid single-arm two-stage design using Bayes factors. The top left panel shows Bayesian and frequentist power, Bayesian type-I-error for varying interim sample sizes. The top right panel provides information about the optimal frequentist design found by the algorithm and its Bayesian and frequentist operating characteristics. The lower left and right panels visualize the analysis and design priors under the null and alternative hypothesis. For the frequentist operating characteristics, these are irrelevant. They influence only the Bayesian operating characteristics.

1.3 Interpretation of hybrid calibration

The result shown in Figure 1 visualizes the most relevant aspects about the isolated optimal trial design. In the hybrid mode:

• The Bayesian power target ensures that, on average under the \(H_1\) design prior, the design has at least the specified probability of producing evidence against \(H_0\). In this case, the plot shows both the Bayesian and frequentist power of the isolated optimal design. Note that while the Bayesian power meets the target constraint of 80% power, the frequentist power of the optimal hybrid design isolated by the algorithm is only about 59%.
• The frequentist type-I error target ensures that when \(p = p_0\), the probability of erroneously rejecting \(H_0\) is bounded in the usual sense. We see from the plot that the frequentist type-I-error rate is below the target threshold of 5%.
• The probabilities to stop early for futility are also shown as well as the expected sample sizes under \(H_0\) and \(H_1\) from a frequentist and Bayesian point of view. The next subsection explains why the expected sample sizes differ, in general, from a Bayesian and frequentist point of view.

1.4 Bayesian and frequentist expected sample sizes

For a fixed two-stage design with interim sample size \(n_1\) and final sample size \(n_2\), the expected sample size can be written in terms of the probability of early stopping for futility.

Let - \(N(p)\) be the total sample size when the true response probability is \(p\), - \(\pi_{\text{fut}}(p)\) be the probability of stopping early for futility at the interim analysis when the true response probability is \(p\).

By construction, \[ N(p) = n_1 \cdot \pi_{\text{fut}}(p) + n_2 \cdot \{1 - \pi_{\text{fut}}(p)\}, \] because the trial enrolls \(n_1\) patients if it stops early, and \(n_2\) patients otherwise.

1.5 Sensitivity to the \(H_1\) design prior

Hybrid calibration naturally depends on the \(H_1\) design prior because Bayesian power is a prior-predictive quantity. We can explore this by varying da1 and db1 while keeping other settings fixed.

prior_grid <- list(
  list(da1 = 1.5, db1 = 1.0, label = "weakly informative"),
  list(da1 = 2.5, db1 = 2.0, label = "moderately informative"),
  list(da1 = 4.0, db1 = 2.5, label = "more concentrated")
)

extract_row <- function(res, label) {
  if (is.null(res) || isFALSE(res$feasible)) {
    return(data.frame(
      prior_label = label,
      feasible    = FALSE,
      n1          = NA_integer_,
      n2          = NA_integer_,
      power       = NA_real_,
      type1       = NA_real_,
      freq_power  = NA_real_,
      freq_type1  = NA_real_,
      en_h0       = NA_real_,
      en_h1       = NA_real_
    ))
  }

  oc <- res$operating_characteristics
  data.frame(
    prior_label = label,
    feasible    = res$feasible,
    n1          = res$design["n1"],
    n2          = res$design["n2"],
    power       = oc$power,
    type1       = oc$type1,
    freq_power  = oc$freq_power,
    freq_type1  = oc$freq_type1,
    en_h0       = oc$en_h0,
    en_h1       = oc$en_h1
  )
}

hybrid_sensitivity <- lapply(prior_grid, function(pr) {
  fit <- tryCatch(
    design_singlearm_bf(
      n1_min = 5,
      n2_max = 100,
      k      = 1/10,
      k_f    = 3,
      p0     = 0.2,
      a0     = 1,
      b0     = 1,
      a1     = 1,
      b1     = 1,
      dp     = 0.4,
      da0    = 1,
      db0    = 1,
      da1    = pr$da1,
      db1    = pr$db1,
      type   = "direction",
      calibration       = "hybrid",
      target_power      = 0.80,
      target_freq_type1 = 0.05,
      power_cushion = 0.025
    ),
    error = function(e) NULL
  )

  extract_row(fit, pr$label)
})

hybrid_sensitivity <- do.call(rbind, hybrid_sensitivity)
knitr::kable(hybrid_sensitivity, digits = 3)

This table shows how the optimal design and its operating characteristics change with the \(H_1\) design prior. In particular:

• more concentrated priors around optimistic response rates can lead to smaller sample sizes and higher Bayesian power,
• priors with substantial mass near \(p_0\) may require larger sample sizes to reach the same power target.

1.6 Adding a compelling-evidence constraint under \(H_0\)

Hybrid calibration can also be combined with a constraint on the probability of compelling evidence in favour of \(H_0\). In formulas, this implies we require the additional target constraint \[Pr(BF_{01}<k_f)>f\] for the optimal hybrid two-stage design, next to our Bayesian power and frequentist type-I-error requirements. Here, \(f\in (0,1)\) is the threshold we set for the probability of compelling evidence for \(H_0\). In the implementation, this is tied to the futility threshold k_f.

res_hybrid_ce <- design_singlearm_bf(
  n1_min = 5,
  n2_max = 180,
  k      = 1/10,
  k_f    = 3,
  p0     = 0.2,
  a0     = 1,
  b0     = 1,
  a1     = 1,
  b1     = 1,
  dp     = 0.4,
  da0    = 1,
  db0    = 1,
  da1    = 2.5,
  db1    = 2,
  type   = "direction",
  calibration       = "hybrid",
  target_power      = 0.80,
  target_freq_type1 = 0.025,
  target_ce_h0      = 0.60
)

We inspect the results of the calibration algorithm:

summary(res_hybrid_ce)
#> Summary: Single-arm two-stage Bayes factor design
#> ---------------------------------------------------------
#> Feasible: TRUE
#> Calibration: hybrid
#> Design prior under H0: Beta(1, 1) truncated to [0, p0]
#> Design prior under H1: Beta(2.5, 2) truncated to (p0, 1]
#> 
#> Selected design: n1 = 8, n2 = 26
#> 
#> Bayesian operating characteristics
#>   Power: 0.8063
#>   Type-I: 0.0027
#>   CE H0: 0.9273
#>   EN H0: 11.71
#>   EN H1: 24.80
#> 
#> Frequentist operating characteristics
#>   Power: 0.6100
#>   Type-I: 0.0216
#>   EN H0: 16.94
#>   EN H1: 24.09

We see that the algorithm did not find an optimal hybrid two-stage design which fulfills all of our target constraints. The fixed-sample size isolated in the first step of the algorithm is \(n_2 = 25\), which is the sample size that achieves at least sufficient power of 80% in a one-stage design to yield reasonable candidate two-stage designs which have \(n_2 = 25\).

In some settings, no design can satisfy all three constraints (Bayesian power, frequentist type-I error, compelling evidence under \(H_0\)). In that case, the function returns a message explaining that no feasible design was found.

2.1 Interpretation of power_cushion

The argument power_cushion modifies only the first step of the optimization, that is, the fixed-sample anchor search. If target_power = 0.80 and power_cushion = 0.025, then the anchor search uses a power target of 0.825. The final feasibility check for the actual two-stage design still uses the original target power of 0.80.

Thus, power_cushion should not be interpreted as changing the scientific power requirement. Rather, it is a numerical planning device that protects against the expected power loss caused by introducing interim futility stopping after the anchor size has been chosen. It serves as a methodological airbag against isolating a final sample size which is too small in the first step of the calibration algorithm.

2.2 Practical guidance

In practice, the following strategy is often useful:

• Start with power_cushion = 0.
• If no feasible two-stage design is found, inspect the search results.
• If the corrected power is systematically just below the target, while the other constraints are satisfied, increase power_cushion modestly, for example to 0.01, 0.02, or 0.025.
• Refit the design and check whether a feasible two-stage solution appears.

A useful rule of thumb is that power_cushion is most helpful when the candidate two-stage designs miss the target only narrowly, as in the present example. If the power shortfall is substantial, increasing n2_max, changing the evidence thresholds, or revisiting the design prior under H1 may be more appropriate.

We can again visualize the final design like before:

plot(res_with_cushion)

Figure 2: Output of the plot function for an optimal hybrid single-arm two-stage design using Bayes factors after adding a power cushion to the calibration algorithm call. The top left panel shows Bayesian and frequentist power, Bayesian type-I-error for varying interim sample sizes. The top right panel provides information about the optimal frequentist design found by the algorithm and its Bayesian and frequentist operating characteristics. The lower left and right panels visualize the analysis and design priors under the null and alternative hypothesis. For the frequentist operating characteristics, these are irrelevant. They influence only the Bayesian operating characteristics.

The power_cushion argument is useful because the fixed-sample anchor found in the first step of the optimization may have little or no margin above the nominal target power. Once an interim futility analysis is introduced, corrected power can decrease, and a previously sufficient fixed-sample anchor may no longer support any feasible two-stage design. By requiring a slightly larger power in the anchor step, power_cushion creates a buffer that can preserve feasibility after early stopping is incorporated.

We return to the example with a constraint on the probability of compelling evidence above, where the algorithm found no optimal hybrid two-stage design. Now, we add a small power cushion and see whether the algorithm now isolates an optimal hybrid design for us:

res_hybrid_ce_with_power_cushion <- design_singlearm_bf(
  n1_min = 5,
  n2_max = 180,
  k      = 1/10,
  k_f    = 3,
  p0     = 0.2,
  a0     = 1,
  b0     = 1,
  a1     = 1,
  b1     = 1,
  dp     = 0.4,
  da0    = 1,
  db0    = 1,
  da1    = 2.5,
  db1    = 2,
  type   = "direction",
  calibration       = "hybrid",
  target_power      = 0.80,
  target_freq_type1 = 0.05,
  target_ce_h0      = 0.60,
  power_cushion = 0.025
)

We inspect the results of the calibration algorithm:

summary(res_hybrid_ce_with_power_cushion)
#> Summary: Single-arm two-stage Bayes factor design
#> ---------------------------------------------------------
#> Feasible: TRUE
#> Calibration: hybrid
#> Design prior under H0: Beta(1, 1) truncated to [0, p0]
#> Design prior under H1: Beta(2.5, 2) truncated to (p0, 1]
#> 
#> Selected design: n1 = 7, n2 = 24
#> 
#> Bayesian operating characteristics
#>   Power: 0.8072
#>   Type-I: 0.0043
#>   CE H0: 0.9119
#>   EN H0: 9.88
#>   EN H1: 22.45
#> 
#> Frequentist operating characteristics
#>   Power: 0.6195
#>   Type-I: 0.0316
#>   EN H0: 14.20
#>   EN H1: 21.30

We see that now an optimal design is found, and we can inspect the details of all two-stage designs analyzed by the algorithm further by calling

res_hybrid_ce_with_power_cushion

The example exemplifies that is is recommended to always use a small power cushion to isolate optimal designs reliably with the two-step algorithm underlying the function. For further details, see also (Kelter and Pawel 2025a) and (Kelter and Pawel 2025b).

2.3 Practical recommendations for hybrid calibration

When using the hybrid mode in practice:

• Choose the \(H_1\) design prior to reflect realistic expectations about the treatment effect; it directly impacts Bayesian power.
• Keep the frequentist type-I error target in line with regulatory or scientific standards for early-phase trials (often 0.1, 0.05 or 0.025).
• Use the optional target_ce_h0 only when a minimum probability of compelling evidence under \(H_0\) is truly required; otherwise, omit it to avoid overconstraining the problem.