Last updated on 2026-11-21 00:52:26 CET.
| Flavor | Version | Tinstall | Tcheck | Ttotal | Status | Flags |
|---|---|---|---|---|---|---|
| r-devel-linux-x86_64-debian-clang | 6.7.0 | 19.24 | 570.48 | 589.72 | ERROR | |
| r-devel-linux-x86_64-debian-gcc | 6.7.0 | 11.75 | 578.96 | 590.71 | ERROR | |
| r-devel-linux-x86_64-fedora-clang | 6.7.1 | 15.00 | 383.12 | 398.12 | OK | |
| r-devel-linux-x86_64-fedora-gcc | 6.7.1 | 14.00 | 386.72 | 400.72 | OK | |
| r-devel-windows-x86_64 | 6.7.0 | 25.00 | 535.00 | 560.00 | OK | |
| r-patched-linux-x86_64 | 6.7.0 | 21.34 | 565.47 | 586.81 | OK | |
| r-release-linux-x86_64 | 6.7.0 | 18.77 | 542.58 | 561.35 | OK | |
| r-release-macos-arm64 | 6.7.1 | 4.00 | 120.00 | 124.00 | OK | |
| r-release-macos-x86_64 | 6.7.1 | 13.00 | 376.00 | 389.00 | OK | |
| r-release-windows-x86_64 | 6.7.0 | 22.00 | 529.00 | 551.00 | OK | |
| r-oldrel-macos-arm64 | 6.7.1 | 5.00 | 154.00 | 159.00 | OK | |
| r-oldrel-macos-x86_64 | 6.7.1 | 14.00 | 420.00 | 434.00 | OK | |
| r-oldrel-windows-x86_64 | 6.7.0 | 31.00 | 756.00 | 787.00 | OK |
Version: 6.7.0
Check: tests
Result: ERROR
Running ‘testthat.R’ [6m/11m]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> library(testthat)
> library(PatientLevelPrediction)
> test_check("PatientLevelPrediction")
Internet: TRUE
attempting to download GiBleed
trying URL 'https://raw.githubusercontent.com/OHDSI/EunomiaDatasets/main/datasets/GiBleed/GiBleed_5.5.zip'
Content type 'application/zip' length 7414767 bytes (7.1 MB)
==================================================
downloaded 7.1 MB
attempting to extract and load: /home/hornik/tmp/scratch/Rtmpcl7TXb/GiBleed_5.5.zip to: /home/hornik/tmp/scratch/Rtmpcl7TXb/GiBleed_5.5.sqlite
Cohorts created in table main.cohort
No cdm database id entered so using cdmDatabaseSchema - if cdmDatabaseSchema is the same for multiple different databases, please use cdmDatabaseId to specify a unique identifier for the database and version
Connecting using SQLite driver
Constructing the at risk cohort
Executing SQL took 0.0306 secs
Fetching cohorts from server
Loading cohorts took 0.179 secs
Constructing features on server
Executing SQL took 0.174 secs
Fetching data from server
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
Fetching data took 2.71 secs
Fetching outcomes from server
Loading outcomes took 0.0687 secs
Use timeStamp: TRUE
Creating save directory at: /home/hornik/tmp/scratch/Rtmpcl7TXb/saveLoc388d6143aa4b12/Test
Currently in a tryCatch or withCallingHandlers block, so unable to add global calling handlers. ParallelLogger will not capture R messages, errors, and warnings, only explicit calls to ParallelLogger. (This message will not be shown again this R session)
Patient-Level Prediction Package version 6.7.0
Study started at: 2026-11-19 21:25:21.111424
AnalysisID: Test
AnalysisName: Testing analysis
TargetID: 1
OutcomeID: 3
Cohort size: 1800
Covariates: 75
Creating population
Outcome is 0 or 1
Population created with: 1800 observations, 1800 unique subjects and 355 outcomes
Population created in 0.129 secs
seed: 12
Creating a 25% test and 75% train (into 3 folds) random stratified split by class
Data split into 449 test cases and 1351 train cases (451, 450, 450)
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
Data split in 9.6 secs
Train Set:
Fold 1 451 patients with 89 outcomes - Fold 2 450 patients with 89 outcomes - Fold 3 450 patients with 89 outcomes
75 covariates in train data
Test Set:
449 patients with 88 outcomes
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
Removing 2 redundant covariates
Removing 0 infrequent covariates
Normalizing covariates
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
Tidying covariates took 11.2 secs
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
Train Set:
Fold 1 451 patients with 89 outcomes - Fold 2 450 patients with 89 outcomes - Fold 3 450 patients with 89 outcomes
73 covariates in train data
Test Set:
449 patients with 88 outcomes
Running Cyclops
Done.
GLM fit status: OK
Creating variable importance data frame
Prediction took 0.852 secs
Time to fit model: 4.74 secs
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
Removing infrequent and redundant covariates and normalizing
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
Removing infrequent and redundant covariates covariates and normalizing took 3.41 secs
Prediction took 1.01 secs
Prediction done in: 7.69 secs
Calculating Performance for Test
=============
AUC 71.48
95% lower AUC: 65.41
95% upper AUC: 77.54
AUPRC: 35.74
Brier: 0.14
Eavg: 0.02
Calibration in large- Mean predicted risk 0.1897 : observed risk 0.196
Calibration in large- Intercept 0.3216
Weak calibration intercept: 0.3216 - gradient:1.2182
Hosmer-Lemeshow calibration gradient: 1.23 intercept: -0.06
Average Precision: 0.37
Calculating Performance for Train
=============
AUC 72.43
95% lower AUC: 68.99
95% upper AUC: 75.86
AUPRC: 39.39
Brier: 0.14
Eavg: 0.02
Calibration in large- Mean predicted risk 0.1976 : observed risk 0.1976
Calibration in large- Intercept 0.174
Weak calibration intercept: 0.174 - gradient:1.1421
Hosmer-Lemeshow calibration gradient: 1.21 intercept: -0.04
Average Precision: 0.39
Calculating Performance for CV
=============
AUC 67.83
95% lower AUC: 64.15
95% upper AUC: 71.50
AUPRC: 31.83
Brier: 0.14
Eavg: 0.02
Calibration in large- Mean predicted risk 0.1979 : observed risk 0.1976
Calibration in large- Intercept 0.1826
Weak calibration intercept: 0.1826 - gradient:1.149
Hosmer-Lemeshow calibration gradient: 1.17 intercept: -0.03
Average Precision: 0.32
Time to calculate evaluation metrics: 0.768 secs
Calculating covariate summary @ 2026-11-19 21:25:56.953574
This can take a while...
Creating binary labels
Joining with strata
calculating subset of strata 1
calculating subset of strata 2
calculating subset of strata 3
calculating subset of strata 4
Restricting to subgroup
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
Calculating summary for subgroup TestWithNoOutcome
Restricting to subgroup
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/Rtmpcl7TXb/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
ℹ This message has been shown 60 times and will not be shown again this session.
Calculating summary for subgroup TrainWithNoOutcome
Restricting to subgroup
Calculating summary for subgroup TrainWithOutcome
Restricting to subgroup
Calculating summary for subgroup TestWithOutcome
Aggregating with labels and strata
Finished covariate summary @ 2026-11-19 21:26:09.928735
Time to calculate covariate summary: 13 secs
Run finished successfully.
Saving PlpResult
Creating directory to save model
plpResult saved to ..\/home/hornik/tmp/scratch/Rtmpcl7TXb/saveLoc388d6143aa4b12/Test\plpResult
runPlp time taken: 48.9 secs
Use timeStamp: TRUE
Diagnosing impact of minTimeAtRisk in populationSettings
Outcome is 0 or 1
Population created with: 1800 observations, 1800 unique subjects and 355 outcomes
Population created in 0.121 secs
Outcome is 0 or 1
Population created with: 1800 observations, 1800 unique subjects and 355 outcomes
Population created in 0.0963 secs
Outcome is 0 or 1
Population created with: 1800 observations, 1800 unique subjects and 355 outcomes
Population created in 0.132 secs
Outcome is 0 or 1
Population created with: 1800 observations, 1800 unique subjects and 355 outcomes
Population created in 0.165 secs
Outcome is 0 or 1
Population created with: 1800 observations, 1800 unique subjects and 355 outcomes
Population created in 0.235 secs
Outcome is 0 or 1
Population created with: 1800 observations, 1800 unique subjects and 355 outcomes
Population created in 0.207 secs
Outcome is 0 or 1
Population created with: 1800 observations, 1800 unique subjects and 355 outcomes
Population created in 0.0936 secs
Saving diagnosePlp to /home/hornik/tmp/scratch/Rtmpcl7TXb/saveLoc388d6143aa4b12/Test/diagnosePlp.rds
Outcome is 0 or 1
Population created with: 1800 observations, 1800 unique subjects and 355 outcomes
Population created in 0.0999 secs
seed: 12
Creating a 25% test and 75% train (into 3 folds) random stratified split by class
Data split into 449 test cases and 1351 train cases (451, 450, 450)
Data split in 9.63 secs
seed: 12
Creating a 25% test and 75% train (into 3 folds) random stratified split by class
Data split into 449 test cases and 1351 train cases (451, 450, 450)
Data split in 9.77 secs
Use timeStamp: TRUE
Creating save directory at: /home/hornik/tmp/scratch/Rtmpcl7TXb/saveLoc388d6143aa4b12/tinyResults/tinyFit
Patient-Level Prediction Package version 6.7.0
Study started at: 2026-11-19 21:26:51.366986
AnalysisID: tinyFit
AnalysisName: Study details
TargetID: 1
OutcomeID: 3
Cohort size: 865
Covariates: 2
Creating population
Outcome is 0 or 1
Population created with: 865 observations, 865 unique subjects and 262 outcomes
Population created in 0.0893 secs
seed: 123
Creating a 25% test and 75% train (into 3 folds) random stratified split by class
Data split into 215 test cases and 650 train cases (217, 217, 216)
Data split in 10 secs
Train Set:
Fold 1 217 patients with 66 outcomes - Fold 2 217 patients with 66 outcomes - Fold 3 216 patients with 65 outcomes
2 covariates in train data
Test Set:
215 patients with 65 outcomes
Removing 0 redundant covariates
Removing 0 infrequent covariates
Normalizing covariates
Tidying covariates took 8.63 secs
Train Set:
Fold 1 217 patients with 66 outcomes - Fold 2 217 patients with 66 outcomes - Fold 3 216 patients with 65 outcomes
2 covariates in train data
Test Set:
215 patients with 65 outcomes
Running Cyclops
Done.
GLM fit status: OK
Creating variable importance data frame
Prediction took 0.704 secs
Time to fit model: 3.96 secs
Removing infrequent and redundant covariates and normalizing
Removing infrequent and redundant covariates covariates and normalizing took 4.35 secs
Prediction took 0.622 secs
Prediction done in: 7.93 secs
Calculating Performance for Test
=============
AUC 63.68
95% lower AUC: 57.26
95% upper AUC: 70.10
AUPRC: 40.36
Brier: 0.20
Eavg: 0.01
Calibration in large- Mean predicted risk 0.3018 : observed risk 0.3023
Calibration in large- Intercept -0.118
Weak calibration intercept: -0.118 - gradient:0.8335
Hosmer-Lemeshow calibration gradient: 0.90 intercept: 0.03
Average Precision: 0.41
Calculating Performance for Train
=============
AUC 64.99
95% lower AUC: 61.50
95% upper AUC: 68.48
AUPRC: 41.44
Brier: 0.19
Eavg: 0.00
Calibration in large- Mean predicted risk 0.3031 : observed risk 0.3031
Calibration in large- Intercept 0.0121
Weak calibration intercept: 0.0121 - gradient:1.0174
Hosmer-Lemeshow calibration gradient: 1.02 intercept: -0.00
Average Precision: 0.41
Calculating Performance for CV
=============
AUC 63.91
95% lower AUC: 59.57
95% upper AUC: 68.24
AUPRC: 39.42
Brier: 0.20
Eavg: 0.04
Calibration in large- Mean predicted risk 0.3031 : observed risk 0.3031
Calibration in large- Intercept -0.0778
Weak calibration intercept: -0.0778 - gradient:0.8913
Hosmer-Lemeshow calibration gradient: 1.02 intercept: 0.01
Average Precision: 0.39
Time to calculate evaluation metrics: 1.28 secs
Run finished successfully.
Saving PlpResult
Creating directory to save model
plpResult saved to ..\/home/hornik/tmp/scratch/Rtmpcl7TXb/saveLoc388d6143aa4b12/tinyResults/tinyFit\plpResult
runPlp time taken: 33.9 secs
Warning: PredictionDistribution not available for survival models
No databaseRefId specified so using schema as unique database identifier
Warning: Unable to coerce hyperParamSearch to data.frame; storing empty table for compatibility.
Connecting using SQLite driver
All or some PLP result tables do not exist, tables being recreated
Deleting existing tables
Creating PLP results tables
Executing SQL took 0.0343 secs
PLP result migration being applied
Migrating data set
Migrator using SQL files in PatientLevelPrediction
Connecting using SQLite driver
Creating migrations table
Executing SQL took 0.0165 secs
Migrations table created
Executing migration: Migration_1-store_version.sql
Executing SQL took 0.00722 secs
Saving migration: Migration_1-store_version.sql
Executing SQL took 0.00649 secs
Migration complete Migration_1-store_version.sql
Executing migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.0448 secs
Saving migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.0129 secs
Migration complete Migration_2-add_hyperparameter_settings.sql
Executing migration: Migration_3-add_model_name.sql
Executing SQL took 0.0113 secs
Saving migration: Migration_3-add_model_name.sql
Executing SQL took 0.00641 secs
Migration complete Migration_3-add_model_name.sql
Closing database connection
Updating version number
Connecting using SQLite driver
Executing SQL took 0.007 secs
Connecting using SQLite driver
Adding new model settings
Inserting data took 0.0267 secs
Connecting using SQLite driver
All or some PLP result tables do not exist, tables being recreated
Deleting existing tables
Creating PLP results tables
Executing SQL took 0.0354 secs
PLP result migration being applied
Migrating data set
Migrator using SQL files in PatientLevelPrediction
Connecting using SQLite driver
Creating migrations table
Executing SQL took 0.0161 secs
Migrations table created
Executing migration: Migration_1-store_version.sql
Executing SQL took 0.00777 secs
Saving migration: Migration_1-store_version.sql
Executing SQL took 0.00689 secs
Migration complete Migration_1-store_version.sql
Executing migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.0133 secs
Saving migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.00828 secs
Migration complete Migration_2-add_hyperparameter_settings.sql
Executing migration: Migration_3-add_model_name.sql
Executing SQL took 0.00867 secs
Saving migration: Migration_3-add_model_name.sql
Executing SQL took 0.0074 secs
Migration complete Migration_3-add_model_name.sql
Closing database connection
Updating version number
Connecting using SQLite driver
Executing SQL took 0.0107 secs
Connecting using SQLite driver
Adding new model settings
Inserting data took 0.0273 secs
Adding new model settings
Inserting data took 0.0202 secs
Adding new model settings
Inserting data took 0.0204 secs
Adding new model settings
Inserting data took 0.0239 secs
Adding new model settings
Inserting data took 0.0185 secs
Adding new model settings
Inserting data took 0.0181 secs
Adding new model settings
Inserting data took 0.0177 secs
Connecting using SQLite driver
Executing SQL took 0.00863 secs
Executing SQL took 0.00639 secs
Executing SQL took 0.00652 secs
Executing SQL took 0.00658 secs
Executing SQL took 0.00787 secs
Connecting using SQLite driver
All or some PLP result tables do not exist, tables being recreated
Deleting existing tables
Creating PLP results tables
Executing SQL took 0.0371 secs
PLP result migration being applied
Migrating data set
Migrator using SQL files in PatientLevelPrediction
Connecting using SQLite driver
Creating migrations table
Executing SQL took 0.016 secs
Migrations table created
Executing migration: Migration_1-store_version.sql
Executing SQL took 0.00773 secs
Saving migration: Migration_1-store_version.sql
Executing SQL took 0.00668 secs
Migration complete Migration_1-store_version.sql
Executing migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.0105 secs
Saving migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.00682 secs
Migration complete Migration_2-add_hyperparameter_settings.sql
Executing migration: Migration_3-add_model_name.sql
Executing SQL took 0.00813 secs
Saving migration: Migration_3-add_model_name.sql
Executing SQL took 0.0163 secs
Migration complete Migration_3-add_model_name.sql
Closing database connection
Updating version number
Connecting using SQLite driver
Executing SQL took 0.0169 secs
Connecting using SQLite driver
Adding new hyperparameter settings
Inserting data took 0.037 secs
Hyperparameter setting exists
Connecting using SQLite driver
All or some PLP result tables do not exist, tables being recreated
Deleting existing tables
Creating PLP results tables
Executing SQL took 0.0433 secs
PLP result migration being applied
Migrating data set
Migrator using SQL files in PatientLevelPrediction
Connecting using SQLite driver
Creating migrations table
Executing SQL took 0.00732 secs
Migrations table created
Executing migration: Migration_1-store_version.sql
Executing SQL took 0.00787 secs
Saving migration: Migration_1-store_version.sql
Executing SQL took 0.00684 secs
Migration complete Migration_1-store_version.sql
Executing migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.0109 secs
Saving migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.00614 secs
Migration complete Migration_2-add_hyperparameter_settings.sql
Executing migration: Migration_3-add_model_name.sql
Executing SQL took 0.0079 secs
Saving migration: Migration_3-add_model_name.sql
Executing SQL took 0.00775 secs
Migration complete Migration_3-add_model_name.sql
Closing database connection
Updating version number
Connecting using SQLite driver
Executing SQL took 0.00709 secs
Connecting using SQLite driver
Adding TAR
Executing SQL took 0.00678 secs
tarId: 1
Adding cohort target
Inserting data took 0.0519 secs
Adding cohort target
Inserting data took 0.0177 secs
tId: 1
Adding cohort outcome
Inserting data took 0.0396 secs
Adding cohort outcome
Inserting data took 0.0175 secs
oId: 2
Adding new population settings
Inserting data took 0.0172 secs
popSetId: 1
Adding new covariate settings
Inserting data took 0.0181 secs
covSetId: 1
Adding new model settings
Inserting data took 0.0184 secs
modSetId: 1
Adding new plp data settings
Inserting data took 0.0168 secs
plpDataSetId: 1
Adding new feature_engineering settings
Inserting data took 0.017 secs
FESetId: 1
Adding new sample settings
Inserting data took 0.0158 secs
sampleSetId: 1
Adding new tidy covariates settings
Inserting data took 0.0167 secs
tidySetId: 1
Adding new split settings
Inserting data took 0.0217 secs
splitId: 1
Adding new hyperparameter settings
Inserting data took 0.0164 secs
hyperparameterSetId: 2
Executing SQL took 0.0065 secs
modelDesignId: 1
TAR exists
tarId: 1
json in jsons:TRUE
Cohort target exists in cohort_definition with cohort id1
Cohort target exists in cohorts with cohort id1
tId: 1
json in jsons:TRUE
Cohort outcome exists in cohort_definition with cohort id2
Cohort outcome exists in cohorts with cohort id2
oId: 2
Population settings exists
popSetId: 1
Covariate setting exists
covSetId: 1
Model setting exists
modSetId: 1
Split setting exists
plpDataSetId: 1
feature engineering setting exists
FESetId: 1
sample setting exists
sampleSetId: 1
tidy covariates setting exists
tidySetId: 1
Adding new split settings
Inserting data took 0.0314 secs
splitId: 2
Adding new hyperparameter settings
Inserting data took 0.0153 secs
hyperparameterSetId: 3
Executing SQL took 0.00671 secs
modelDesignId: 2
TAR exists
tarId: 1
json in jsons:TRUE
Cohort target exists in cohort_definition with cohort id1
Cohort target exists in cohorts with cohort id1
tId: 1
json in jsons:TRUE
Cohort outcome exists in cohort_definition with cohort id2
Cohort outcome exists in cohorts with cohort id2
oId: 2
Population settings exists
popSetId: 1
Covariate setting exists
covSetId: 1
Model setting exists
modSetId: 1
Split setting exists
plpDataSetId: 1
feature engineering setting exists
FESetId: 1
sample setting exists
sampleSetId: 1
tidy covariates setting exists
tidySetId: 1
Split setting exists
splitId: 1
Hyperparameter setting exists
hyperparameterSetId: 2
modelDesignId: 1
Prediction done in: 0.00389 secs
Creating directory to save model
New best performance 0.5 with param: lambda=1
Fit best model on whole training set
Calculating covariate summary @ 2026-11-19 21:27:46.75092
This can take a while...
calculating subset of strata 1
Restricting to subgroup
Calculating summary for subgroup
Aggregating with no labels or strata
Finished covariate summary @ 2026-11-19 21:27:49.675093
Time to calculate covariate summary: 2.92 secs
Calculating covariate summary @ 2026-11-19 21:27:49.714626
This can take a while...
calculating subset of strata 1
Restricting to subgroup
Calculating summary for subgroup
Aggregating with no labels or strata
Finished covariate summary @ 2026-11-19 21:27:53.279135
Time to calculate covariate summary: 3.56 secs
Calculating covariate summary @ 2026-11-19 21:27:53.308852
This can take a while...
Creating binary labels
calculating subset of strata 1
calculating subset of strata 2
Restricting to subgroup
Calculating summary for subgroup WithOutcome
Restricting to subgroup
Calculating summary for subgroup WithNoOutcome
Aggregating with only labels or strata
Finished covariate summary @ 2026-11-19 21:27:59.512886
Time to calculate covariate summary: 6.2 secs
Restricting to subgroup
Calculating summary for subgroup
variance needs to be >= 0
variance should be of class:numericvariance should be of class:integer
seed should be of class:numericseed should be of class:NULLseed should be of class:integer
threads should be of class:numericthreads should be of class:integer
lowerLimit should be of class:numericlowerLimit should be of class:integer
upperLimit should be of class:numericupperLimit should be of class:integer
upperLimit needs to be >= 3
variance needs to be >= 0
variance should be of class:numericvariance should be of class:integer
seed should be of class:numericseed should be of class:NULLseed should be of class:integer
threads should be of class:numericthreads should be of class:integer
lowerLimit should be of class:numericlowerLimit should be of class:integer
upperLimit should be of class:numericupperLimit should be of class:integer
upperLimit needs to be >= 3
testFraction should be of class:numerictestFraction should be of class:integer
testFraction needs to be >= 0
-1 * testFraction needs to be > -1
trainFraction should be of class:numerictrainFraction should be of class:integer
-1 * trainFraction needs to be >= -1
trainFraction needs to be > 0
splitSeed should be of class:numericsplitSeed should be of class:integer
splitSeed should be of class:numericsplitSeed should be of class:integer
nfold should be of class:numericnfold should be of class:integer
nfold should be of class:numericnfold should be of class:integer
Invalid type setting.
Pick from: 'stratified','time','subject'
type should be of class:character
type should be of class:character
maxTrainingOutcomes needs to be > 0
maxTrainingOutcomes should be of class:numericmaxTrainingOutcomes should be of class:integer
maxTrainingOutcomes must be a single whole number
splitSeed should be of class:numericsplitSeed should be of class:integer
nfold needs to be > 1
nfold must be a single whole number
type should be stratified or subject
Data split into 240 test cases and 160 train cases with 40 training outcome rows (maxTrainingOutcomes = 40; folds: 40, 40, 40, 40)
Data split into 240 test cases and 160 train cases with 40 training outcome rows (maxTrainingOutcomes = 40; folds: 40, 40, 40, 40)
Data split into 180 test cases and 120 train cases with 20 training outcome rows (maxTrainingOutcomes = 20; folds: 30, 30, 30, 30)
Data split into 110 test cases and 220 train cases with 20 training outcome rows (maxTrainingOutcomes = 20; folds: 60, 60, 50, 50)
Warning: Outcome-limited split did not trigger because the population has 40 outcome rows, which is not greater than maxTrainingOutcomes = 100. Using the default 25% test and 75% train split.
seed: 1
Creating a 25% test and 75% train (into 4 folds) random stratified split by class
Data split into 50 test cases and 150 train cases (38, 38, 37, 37)
seed: 1
Creating a 25% test and 75% train (into 4 folds) random stratified split by class
Data split into 50 test cases and 150 train cases (38, 38, 37, 37)
seed: 40633
seed: 40633
Creating a 30% test and 70% train (into 4 folds) random stratified split by class
Data split into 59 test cases and 141 train cases (36, 36, 35, 34)
seed: 40633
Creating a 20% test and 80% train (into 4 folds) random stratified split by class
Data split into 99 test cases and 401 train cases (101, 100, 100, 100)
seed: 40633
Creating a 20% test and 40% train (into 4 folds) random stratified split by class
Data split into 99 test cases and 201 train cases (51, 51, 50, 49)
200 were not used for training or testing
seed: 40633
Creating 20% test and 80% train (into 4 folds) stratified split at 2011-02-05
Data split into 100 test cases and 400 train samples (101, 101, 101, 97)
seed: 40633
Creating 20% test and 40% train (into 4 folds) stratified split at 2011-02-05
Data split into 100 test cases and 204 train samples (51, 51, 51, 51)
196 were not used for training or testing
seed: 40633
seed: 40633
Creating a 20% test and 80% train (into 4 folds) stratified split by subject
Data split into 40 test cases and 160 train cases (41, 41, 39, 39)
seed: 40633
Creating a 25% test and 75% train (into 3 folds) stratified split by subject
Data split into 52 test cases and 148 train cases (52, 48, 48)
Calculating Performance for All
=============
AUC 51.00
95% lower AUC: 39.57
95% upper AUC: 62.43
AUPRC: 52.01
Brier: 0.33
Eavg: 0.24
Calibration in large- Mean predicted risk 0.5 : observed risk 0.5
Calibration in large- Intercept 0
Weak calibration intercept: 0 - gradient:0.0349
Hosmer-Lemeshow calibration gradient: 0.00 intercept: 0.50
Average Precision: 0.53
Time to calculate evaluation metrics: 0.415 secs
Evaluating survival model at time: 365 days
C-statistic: 0.38931 (0.269-0.51)
E-statistic: 0.0319220138645564
E-statistic 90%: 0.0649360660190881
getCalibrationSummary error
Error in attributes(.Data) <- c(attributes(.Data), attrib): adding class "factor" to an invalid object
Warning: PredictionDistribution not available for survival models
Time to calculate evaluation metrics: 1.01 secs
Setting levels: control = 0, case = 1
Warning: Cannot compute AUC: outcomeCount has only one class
Warning: Cannot compute AUC: outcomeCount has only one class
Warning: Cannot compute AUPRC: outcomeCount has only one class
Warning: Cannot compute Average Precision: outcomeCount has no positive class
Calculating Performance for Validation
=============
Warning: Cannot compute AUC: outcomeCount has only one class
AUC NA
95% lower AUC: NA
95% upper AUC: NA
Warning: Cannot compute AUPRC: outcomeCount has only one class
AUPRC: NA
Brier: 0.35
Eavg: 0.53
Calibration in large- Mean predicted risk 0.527 : observed risk 0
Calibration in large- Intercept -26.5661
Weak calibration intercept: -26.5661 - gradient:0
Hosmer-Lemeshow calibration gradient: 0.00 intercept: -0.00
Warning: Cannot compute Average Precision: outcomeCount has no positive class
Average Precision: NA
Warning: Number of positives is zero
Time to calculate evaluation metrics: 0.216 secs
Column names of coefficients are not correct
Column types of coefficients are not correct
Column types of coefficients are not correct
intercept should be of class:numeric
mapping should be of class:charactermapping should be of class:function
targetId should be of class:numerictargetId should be of class:NULL
outcomeId should be of class:numericoutcomeId should be of class:NULL
populationSettings should be of class:NULLpopulationSettings should be of class:populationSettings
restrictPlpDataSettings should be of class:NULLrestrictPlpDataSettings should be of class:restrictPlpDataSettings
covariateSettings should be of class:listcovariateSettings should be of class:NULLcovariateSettings should be of class:covariateSettings
predict risk probabilities using predictGlm
Prediction took 0.541 secs
Prediction done in: 0.622 secs
databaseDetails should be of class:databaseDetails
databaseDetails should be of class:databaseDetails
databaseDetails should be of class:databaseDetails
No cdm database name entered so using cdmDatabaseSchema
threshold needs to be >= 0
threshold should be of class:numeric
threshold needs to be < 1
starting to map the columns and rows
finished MapCovariates
starting to map the columns and rows
finished MapCovariates
plpData size estimated to use 0GBs of RAM
starting toSparseM
starting to map the columns and rows
finished MapCovariates
toSparseM non temporal used
plpData size estimated to use 0GBs of RAM
finishing toSparseM
toSparseM took 7.20750570297241 secs
starting to map the columns and rows
finished MapCovariates
Saving _problems/test-getCalibration-65.R
Saving _problems/test-helperfunctions-45.R
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
missingThreshold needs to be > 0
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
missingThreshold should be of class:numeric
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
missingThreshold needs to be < 1
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
method should be mean or median
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
addMissingIndicator should be of class:logical
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
method should be pmm
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
missingThreshold needs to be > 0
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
missingThreshold should be of class:numeric
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
missingThreshold needs to be < 1
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
addMissingIndicator should be of class:logical
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
Imputing missing values with simpleImputer using: mean and missing threshold: 0.3
Calculating missingness in data
Found features with missing values
Imputation done in time: 16.8 secs
Applying imputation to test data with simpleImputer
using method: mean and missing threshold: 0.3
Imputation done in time: 13.7 secs
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
Imputing missing values with simpleImputer using: mean and missing threshold: 0.95
Calculating missingness in data
Found features with missing values
Imputation done in time: 11.6 secs
Applying imputation to test data with simpleImputer
using method: mean and missing threshold: 0.95
Imputation done in time: 10.6 secs
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
Imputing missing values with simpleImputer using: mean and missing threshold: 0.3
Calculating missingness in data
Found features with missing values
Imputation done in time: 12.1 secs
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
Imputing missing values with iterativeImputer using: pmm, missing threshold: 0.3 and method settings: 111
Calculating missingness in data
Found features with missing values
Imputation iteration: 1
Imputing variable: fakeMissingVariable
Imputation done in time: 26.4 secs
Applying imputation to test data with iterativeImputer
using method: pmm and missing threshold: 0.3
Imputing variable: fakeMissingVariable
PMM predict for variable fakeMissingVariable (covariateId=666): missing=32, donors=64, k=1, time=0.7 secs
Imputation done in time: 21.2 secs
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
Applying imputation to test data with iterativeImputer
using method: pmm and missing threshold: 0.3
Imputation done in time: 11.3 secs
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
Applying imputation to test data with iterativeImputer
using method: pmm and missing threshold: 0.8
Imputing variable: bmi
PMM predict for variable bmi (covariateId=100): missing=2, donors=2, k=3, time=0.591 secs
Imputation done in time: 17.9 secs
Warning: Imputation model only has intercept. It does not fit the data well
Use timeStamp: TRUE
Creating save directory at: /home/hornik/tmp/scratch/Rtmpcl7TXb/saveLoc388d6143aa4b12/lcc/learningCurve_1
Patient-Level Prediction Package version 6.7.0
Study started at: 2026-11-19 21:32:01.986076
AnalysisID: learningCurve_1
AnalysisName: Study details
TargetID: 1
OutcomeID: 3
Cohort size: 1800
Covariates: 75
Creating population
Outcome is 0 or 1
Population created with: 1767 observations, 1767 unique subjects and 355 outcomes
Population created in 0.132 secs
seed: 70165
Creating a 20% test and 60% train (into 2 folds) random stratified split by class
Data split into 353 test cases and 1061 train cases (531, 530)
353 were not used for training or testing
Data split in 9.63 secs
Train Set:
Fold 1 531 patients with 107 outcomes - Fold 2 530 patients with 106 outcomes
75 covariates in train data
Test Set:
353 patients with 71 outcomes
Removing 2 redundant covariates
Removing 0 infrequent covariates
Normalizing covariates
Tidying covariates took 8.98 secs
Train Set:
Fold 1 531 patients with 107 outcomes - Fold 2 530 patients with 106 outcomes
73 covariates in train data
Test Set:
353 patients with 71 outcomes
Running Cyclops
Done.
GLM fit status: OK
Creating variable importance data frame
Prediction took 0.693 secs
Time to fit model: 3.24 secs
Removing infrequent and redundant covariates and normalizing
Removing infrequent and redundant covariates covariates and normalizing took 2.93 secs
Prediction took 0.963 secs
Prediction done in: 7.24 secs
Calculating Performance for Test
=============
AUC 70.65
95% lower AUC: 63.83
95% upper AUC: 77.47
AUPRC: 32.86
Brier: 0.14
Eavg: 0.01
Calibration in large- Mean predicted risk 0.1994 : observed risk 0.2011
Calibration in large- Intercept 0.1296
Weak calibration intercept: 0.1296 - gradient:1.0968
Hosmer-Lemeshow calibration gradient: 1.09 intercept: -0.02
Average Precision: 0.35
Calculating Performance for Train
=============
AUC 74.41
95% lower AUC: 70.82
95% upper AUC: 77.99
AUPRC: 40.35
Brier: 0.14
Eavg: 0.03
Calibration in large- Mean predicted risk 0.2008 : observed risk 0.2008
Calibration in large- Intercept 0.1786
Weak calibration intercept: 0.1786 - gradient:1.1495
Hosmer-Lemeshow calibration gradient: 1.25 intercept: -0.04
Average Precision: 0.41
Calculating Performance for CV
=============
AUC 69.66
95% lower AUC: 65.74
95% upper AUC: 73.58
AUPRC: 32.83
Brier: 0.14
Eavg: 0.03
Calibration in large- Mean predicted risk 0.2012 : observed risk 0.2008
Calibration in large- Intercept 0.2838
Weak calibration intercept: 0.2838 - gradient:1.2355
Hosmer-Lemeshow calibration gradient: 1.24 intercept: -0.05
Average Precision: 0.31
Time to calculate evaluation metrics: 0.852 secs
Run finished successfully.
Saving PlpResult
Creating directory to save model
plpResult saved to ..\/home/hornik/tmp/scratch/Rtmpcl7TXb/saveLoc388d6143aa4b12/lcc/learningCurve_1\plpResult
runPlp time taken: 31.8 secs
Creating save directory at: /home/hornik/tmp/scratch/Rtmpcl7TXb/saveLoc388d6143aa4b12/lcc/learningCurve_2
Patient-Level Prediction Package version 6.7.0
Study started at: 2026-11-19 21:32:33.82517
AnalysisID: learningCurve_2
AnalysisName: Study details
TargetID: 1
OutcomeID: 3
Cohort size: 1800
Covariates: 75
Creating population
Outcome is 0 or 1
Population created with: 1767 observations, 1767 unique subjects and 355 outcomes
Population created in 0.151 secs
seed: 70165
Creating a 20% test and 70% train (into 2 folds) random stratified split by class
Data split into 353 test cases and 1237 train cases (619, 618)
177 were not used for training or testing
Data split in 11.1 secs
Train Set:
Fold 1 619 patients with 124 outcomes - Fold 2 618 patients with 124 outcomes
75 covariates in train data
Test Set:
353 patients with 71 outcomes
Removing 2 redundant covariates
Removing 0 infrequent covariates
Normalizing covariates
Tidying covariates took 10.1 secs
Train Set:
Fold 1 619 patients with 124 outcomes - Fold 2 618 patients with 124 outcomes
73 covariates in train data
Test Set:
353 patients with 71 outcomes
Running Cyclops
Done.
GLM fit status: OK
Creating variable importance data frame
Prediction took 0.776 secs
Time to fit model: 4.29 secs
Removing infrequent and redundant covariates and normalizing
Removing infrequent and redundant covariates covariates and normalizing took 3.01 secs
Prediction took 1.32 secs
Prediction done in: 7.67 secs
Calculating Performance for Test
=============
AUC 69.51
95% lower AUC: 62.68
95% upper AUC: 76.34
AUPRC: 30.91
Brier: 0.14
Eavg: 0.01
Calibration in large- Mean predicted risk 0.1987 : observed risk 0.2011
Calibration in large- Intercept 0.0621
Weak calibration intercept: 0.0621 - gradient:1.0373
Hosmer-Lemeshow calibration gradient: 0.90 intercept: 0.02
Average Precision: 0.33
Calculating Performance for Train
=============
AUC 74.47
95% lower AUC: 71.12
95% upper AUC: 77.83
AUPRC: 40.88
Brier: 0.14
Eavg: 0.02
Calibration in large- Mean predicted risk 0.2005 : observed risk 0.2005
Calibration in large- Intercept 0.1642
Weak calibration intercept: 0.1642 - gradient:1.1391
Hosmer-Lemeshow calibration gradient: 1.26 intercept: -0.04
Average Precision: 0.41
Calculating Performance for CV
=============
AUC 70.17
95% lower AUC: 66.55
95% upper AUC: 73.79
AUPRC: 33.71
Brier: 0.14
Eavg: 0.02
Calibration in large- Mean predicted risk 0.2004 : observed risk 0.2005
Calibration in large- Intercept 0.2634
Weak calibration intercept: 0.2634 - gradient:1.2179
Hosmer-Lemeshow calibration gradient: 1.24 intercept: -0.05
Average Precision: 0.33
Time to calculate evaluation metrics: 0.784 secs
Run finished successfully.
Saving PlpResult
Creating directory to save model
plpResult saved to ..\/home/hornik/tmp/scratch/Rtmpcl7TXb/saveLoc388d6143aa4b12/lcc/learningCurve_2\plpResult
runPlp time taken: 37.7 secs
Finished in 70 secs.
type should be minmax or robust
type should be minmax or robust
type should be minmax or robust
type should be minmax or robust
settings$clip needs to be a boolean
Starting min-max normalization of continuous features
Finished min-max normalization of continuous features in 3.47 secs
Applying min-max normalization of continuous features to test data
Finished min-max normalization of continuous features in 2.54 secs
Starting robust normalization of continuous features
Finished robust normalization in 3.52 secs
Applying robust normalization of continuous features to test data
Finished robust normalization in 2.86 secs
Starting robust normalization of continuous features
Finished robust normalization in 3.89 secs
Applying robust normalization of continuous features to test data
Finished robust normalization in 2.67 secs
Starting min-max normalization of continuous features
Finished min-max normalization of continuous features in 4.58 secs
Applying min-max normalization of continuous features to test data
Finished min-max normalization of continuous features in 2.5 secs
Starting robust normalization of continuous features
Finished robust normalization in 3.6 secs
Applying robust normalization of continuous features to test data
Finished robust normalization in 2.97 secs
1 needs to be a boolean
"tertet" needs to be a boolean
1 needs to be >= 2
"tertet" needs to be >= 1
2 needs to be <= 1
"tertet" needs to be <= 1
1 needs to be > 2
0 needs to be > 0
"tertet" needs to be > 1
2 needs to be < 1
0 needs to be < 0
"tertet" needs to be < 1
NULL cannot be empty
"dsdsds" should be of class:double
1.5 must be a single whole number
c(1, 2) must be a single whole number
"1" must be a single whole number
NA_real_ must be a single whole number
Inf must be a single whole number
c(1, 2) must be a single finite numeric value
"1" must be a single finite numeric value
NA_real_ must be a single finite numeric value
Inf must be a single finite numeric value
"dsdsds" should be dsds or double
c("dsdsds", "double") should be dsdsds or double
NA_character_ should be dsdsds or double
Creating save directory at: /home/hornik/tmp/scratch/Rtmpcl7TXb/file388d61192271b9
file does not exist
Column names of data.frame(a = 1:2, b = 1:2) are not correct
Column types of data.frame(a = 1:2, b = 1:2) are not correct
includeAllOutcomes should be of class:logical
includeAllOutcomes should be of class:logical
firstExposureOnly should be of class:logical
firstExposureOnly should be of class:logical
washoutPeriod should be of class:numericwashoutPeriod should be of class:integer
washoutPeriod needs to be >= 0
removeSubjectsWithPriorOutcome should be of class:logical
removeSubjectsWithPriorOutcome should be of class:logical
priorOutcomeLookback should be of class:numericpriorOutcomeLookback should be of class:integer
priorOutcomeLookback needs to be >= 0
requireTimeAtRisk should be of class:logical
requireTimeAtRisk should be of class:logical
minTimeAtRisk should be of class:numericminTimeAtRisk should be of class:integer
minTimeAtRisk needs to be >= 0
riskWindowStart should be of class:numericriskWindowStart should be of class:integer
riskWindowEnd should be of class:numericriskWindowEnd should be of class:integer
startAnchor should be of class:character
endAnchor should be of class:character
minFraction needs to be >= 0
minFraction should be of class:numericminFraction should be of class:integer
removeRedundancy should be of class:logical
removeRedundancy should be of class:logical
normalize should be of class:logical
normalize should be of class:logical
Removing infrequent and redundant covariates and normalizing
Removing infrequent and redundant covariates covariates and normalizing took 2.6 secs
recal initial baseline hazard: 0.8
recal initial offset: 0.15
recal initial timepoint: 220
recal initial baseline hazard: 0.85
recal initial offset: -0.05
recal initial timepoint: 240
recal initial baseline hazard: 0.9
recal initial offset: 0
recal initial timepoint: 365
Connecting using SQLite driver
All or some PLP result tables do not exist, tables being recreated
Deleting existing tables
Creating PLP results tables
Executing SQL took 0.0807 secs
PLP result migration being applied
Migrating data set
Migrator using SQL files in PatientLevelPrediction
Connecting using SQLite driver
Creating migrations table
Executing SQL took 0.0155 secs
Migrations table created
Executing migration: Migration_1-store_version.sql
Executing SQL took 0.0073 secs
Saving migration: Migration_1-store_version.sql
Executing SQL took 0.0102 secs
Migration complete Migration_1-store_version.sql
Executing migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.00903 secs
Saving migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.0251 secs
Migration complete Migration_2-add_hyperparameter_settings.sql
Executing migration: Migration_3-add_model_name.sql
Executing SQL took 0.00753 secs
Saving migration: Migration_3-add_model_name.sql
Executing SQL took 0.00638 secs
Migration complete Migration_3-add_model_name.sql
Closing database connection
Updating version number
Connecting using SQLite driver
Executing SQL took 0.0067 secs
Connecting using SQLite driver
runSplitData should be of class:logical
runSampleData should be of class:logical
runFeatureEngineering should be of class:logical
runPreprocessData should be of class:logical
runModelDevelopment should be of class:logical
runCovariateSummary should be of class:logical
numberOutcomestoNonOutcomes should be of class:numericnumberOutcomestoNonOutcomes should be of class:integer
numberOutcomestoNonOutcomes needs to be > 0
sampleSeed should be of class:numericsampleSeed should be of class:integer
type should be of class:character
plpData object
At risk concept ID:
Outcome concept ID(s):
plpData object summary
At risk cohort concept ID:
Outcome concept ID(s):
People:
Outcome counts:
Creating directory to save model
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Generating covariates
Generating cohorts
Generating outcomes
Removing infrequent and redundant covariates and normalizing
Removing infrequent and redundant covariates covariates and normalizing took 3.17 secs
Prediction took 0.718 secs
Prediction done in: 6.72 secs
Removing infrequent and redundant covariates and normalizing
Removing infrequent and redundant covariates covariates and normalizing took 3.29 secs
predict risk probabilities using predictGlm
Prediction took 0.7 secs
Prediction done in: 7.05 secs
predict risk probabilities using predictGlm
Prediction took 0.682 secs
Prediction done in: 0.756 secs
Restricting tarEnd to end of target cohort
Outcome is 0 or 1
Population created with: 20 observations, 20 unique subjects and 20 outcomes
Population created in 0.13 secs
Warning: restrictPlpDataSettings are not the same in models and validationDesign,
using from design
restrictPlpDataSettings not set in design, using model's
targetId should be of class:numerictargetId should be of class:integer
outcomeId should be of class:numericoutcomeId should be of class:integer
targetId should be of class:numerictargetId should be of class:integer
outcomeId should be of class:numericoutcomeId should be of class:integer
populationSettings should be of class:populationSettings
recalibrate should be of class:characterrecalibrate should be of class:NULL
runCovariateSummary should be of class:logical
[ FAIL 2 | WARN 0 | SKIP 149 | PASS 1127 ]
══ Skipped tests (149) ═════════════════════════════════════════════════════════
• On CRAN (145): 'test-LightGBM.R:18:3', 'test-LightGBM.R:54:3',
'test-LightGBM.R:75:3', 'test-UploadToDatabase.R:210:3',
'test-UploadToDatabase.R:284:3', 'test-UploadToDatabase.R:456:3',
'test-UploadToDatabase.R:507:3', 'test-UploadToDatabase.R:571:3',
'test-UploadToDatabase.R:635:3', 'test-UploadToDatabase.R:743:3',
'test-andromedahelperfunctions.R:19:3', 'test-barFitState.R:42:3',
'test-barFitState.R:109:3', 'test-compatModelSettings.R:248:3',
'test-compatModelSettings.R:303:3', 'test-compatModelSettings.R:367:3',
'test-cyclopsModels.R:159:3', 'test-cyclopsModels.R:216:3',
'test-cyclopsModels.R:233:3', 'test-cyclopsModels.R:293:3',
'test-cyclopsModels.R:328:3', 'test-cyclopsModels.R:361:3',
'test-cyclopsModels.R:382:3', 'test-cyclopsModels.R:435:3',
'test-cyclopsModels.R:473:3', 'test-cyclopsModels.R:524:3',
'test-cyclopsModels.R:581:3', 'test-cyclopsModels.R:619:3',
'test-cyclopsModels.R:641:3', 'test-cyclopsModels.R:681:3',
'test-cyclopsModels.R:713:3', 'test-cyclopsModels.R:740:3',
'test-cyclopsModels.R:777:3', 'test-dataSplitting.R:154:3',
'test-dataSplitting.R:378:3', 'test-dataSplitting.R:401:3',
'test-dataSplitting.R:595:3', 'test-diagnostic.R:60:3',
'test-diagnostic.R:111:3', 'test-evaluation.R:17:3',
'test-evaluation.R:46:3', 'test-existingModel.R:18:3',
'test-existingModel.R:74:3', 'test-existingModel.R:124:3',
'test-extractData.R:17:3', 'test-extractData.R:43:3',
'test-extractData.R:62:3', 'test-extractData.R:93:3',
'test-extractData.R:112:3', 'test-extractData.R:156:3',
'test-extractData.R:185:3', 'test-extractData.R:204:3',
'test-extractData.R:231:3', 'test-featureEngineering.R:40:3',
'test-featureEngineering.R:56:3', 'test-featureEngineering.R:84:3',
'test-featureEngineering.R:130:3', 'test-featureEngineering.R:158:3',
'test-featureEngineering.R:203:3', 'test-featureEngineering.R:248:3',
'test-featureEngineering.R:316:3', 'test-featureEngineering.R:343:3',
'test-featureImportance.R:19:3', 'test-featureImportance.R:45:3',
'test-fitting.R:20:3', 'test-fitting.R:32:3', 'test-formatting.R:103:3',
'test-helperfunctions.R:39:3', 'test-imputation.R:261:3',
'test-imputation.R:282:3', 'test-imputation.R:333:3',
'test-learningCurves.R:35:3', 'test-learningCurves.R:50:3',
'test-learningCurves.R:74:3', 'test-multiplePlp.R:86:3',
'test-multiplePlp.R:132:3', 'test-plotting.R:21:3', 'test-plotting.R:60:3',
'test-plotting.R:74:3', 'test-plotting.R:91:3', 'test-plotting.R:126:3',
'test-plotting.R:148:3', 'test-plotting.R:155:3', 'test-plotting.R:165:3',
'test-plotting.R:174:3', 'test-plotting.R:182:3', 'test-plotting.R:199:3',
'test-plotting.R:208:3', 'test-plotting.R:218:3', 'test-plotting.R:225:3',
'test-population.R:263:3', 'test-population.R:471:3',
'test-prediction.R:18:3', 'test-prediction.R:39:3', 'test-prediction.R:78:3',
'test-preprocessingData.R:56:3', 'test-preprocessingData.R:118:3',
'test-rclassifier.R:18:3', 'test-rclassifier.R:51:3',
'test-rclassifier.R:71:3', 'test-rclassifier.R:134:3',
'test-recalibration.R:61:3', 'test-runPlpHelpers.R:18:3',
'test-runPlpHelpers.R:32:3', 'test-sampling.R:91:3', 'test-sampling.R:132:3',
'test-sampling.R:168:3', 'test-saveloadplp.R:31:3',
'test-saveloadplp.R:46:3', 'test-saveloadplp.R:87:3',
'test-saveloadplp.R:171:3', 'test-sklearnClassifier.R:19:3',
'test-sklearnClassifier.R:43:3', 'test-sklearnClassifier.R:54:3',
'test-sklearnClassifier.R:117:3', 'test-sklearnClassifier.R:132:3',
'test-sklearnClassifier.R:164:3', 'test-sklearnClassifier.R:201:3',
'test-sklearnClassifier.R:237:3', 'test-sklearnClassifier.R:281:3',
'test-sklearnClassifier.R:301:3', 'test-sklearnClassifierSettings.R:19:3',
'test-sklearnClassifierSettings.R:37:3',
'test-sklearnClassifierSettings.R:46:3',
'test-sklearnClassifierSettings.R:81:3',
'test-sklearnClassifierSettings.R:94:3',
'test-sklearnClassifierSettings.R:122:3', 'test-sklearnJson.R:39:3',
'test-sklearnJson.R:57:3', 'test-sklearnJson.R:78:3',
'test-sklearnJson.R:99:3', 'test-sklearnJson.R:117:3',
'test-sklearnJson.R:137:5', 'test-sklearnJson.R:137:5',
'test-sklearnJson.R:186:3', 'test-validation.R:44:3',
'test-validation.R:68:3', 'test-validation.R:201:3',
'test-validation.R:226:3', 'test-validation.R:241:3',
'test-validation.R:262:3', 'test-validation.R:287:3',
'test-validation.R:315:3', 'test-validation.R:345:3',
'test-validation.R:368:3'
• PostgreSQL test connection is not configured (4):
'test-UploadToDatabase.R:51:3', 'test-UploadToDatabase.R:103:3',
'test-UploadToDatabase.R:123:3', 'test-UploadToDatabase.R:172:3'
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test-getCalibration.R:59:3'): getCalibration survival ───────────────
Error in `attributes(.Data) <- c(attributes(.Data), attrib)`: adding class "factor" to an invalid object
Backtrace:
▆
1. └─PatientLevelPrediction:::getCalibrationSummary_survival(...) at test-getCalibration.R:59:3
2. └─PatientLevelPrediction:::cut2(predictionOfInterest$value, g = gval)
3. └─base::structure(y, class = "factor", levels = labs)
── Error ('test-helperfunctions.R:45:3'): Borrowed cut2 ────────────────────────
Error in `attributes(.Data) <- c(attributes(.Data), attrib)`: adding class "factor" to an invalid object
Backtrace:
▆
1. └─PatientLevelPrediction:::cut2(x, g = 3) at test-helperfunctions.R:45:3
2. └─base::structure(y, class = "factor", levels = labs)
[ FAIL 2 | WARN 0 | SKIP 149 | PASS 1127 ]
Error:
! Test failures.
Execution halted
Flavor: r-devel-linux-x86_64-debian-clang
Version: 6.7.0
Check: tests
Result: ERROR
Running ‘testthat.R’ [396s/532s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> library(testthat)
> library(PatientLevelPrediction)
> test_check("PatientLevelPrediction")
Internet: TRUE
attempting to download GiBleed
trying URL 'https://raw.githubusercontent.com/OHDSI/EunomiaDatasets/main/datasets/GiBleed/GiBleed_5.5.zip'
Content type 'application/zip' length 7414767 bytes (7.1 MB)
==================================================
downloaded 7.1 MB
attempting to extract and load: /home/hornik/tmp/scratch/RtmpIlwUx3/GiBleed_5.5.zip to: /home/hornik/tmp/scratch/RtmpIlwUx3/GiBleed_5.5.sqlite
Cohorts created in table main.cohort
No cdm database id entered so using cdmDatabaseSchema - if cdmDatabaseSchema is the same for multiple different databases, please use cdmDatabaseId to specify a unique identifier for the database and version
Connecting using SQLite driver
Constructing the at risk cohort
Executing SQL took 0.0316 secs
Fetching cohorts from server
Loading cohorts took 0.0537 secs
Constructing features on server
Executing SQL took 0.121 secs
Fetching data from server
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
Fetching data took 1.59 secs
Fetching outcomes from server
Loading outcomes took 0.0724 secs
Use timeStamp: TRUE
Creating save directory at: /home/hornik/tmp/scratch/RtmpIlwUx3/saveLoc11103736fe6942/Test
Currently in a tryCatch or withCallingHandlers block, so unable to add global calling handlers. ParallelLogger will not capture R messages, errors, and warnings, only explicit calls to ParallelLogger. (This message will not be shown again this R session)
Patient-Level Prediction Package version 6.7.0
Study started at: 2026-11-20 14:58:42.007969
AnalysisID: Test
AnalysisName: Testing analysis
TargetID: 1
OutcomeID: 3
Cohort size: 1800
Covariates: 75
Creating population
Outcome is 0 or 1
Population created with: 1800 observations, 1800 unique subjects and 355 outcomes
Population created in 0.133 secs
seed: 12
Creating a 25% test and 75% train (into 3 folds) random stratified split by class
Data split into 449 test cases and 1351 train cases (451, 450, 450)
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
Data split in 7.96 secs
Train Set:
Fold 1 451 patients with 89 outcomes - Fold 2 450 patients with 89 outcomes - Fold 3 450 patients with 89 outcomes
75 covariates in train data
Test Set:
449 patients with 88 outcomes
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
Removing 2 redundant covariates
Removing 0 infrequent covariates
Normalizing covariates
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
Tidying covariates took 8.92 secs
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
Train Set:
Fold 1 451 patients with 89 outcomes - Fold 2 450 patients with 89 outcomes - Fold 3 450 patients with 89 outcomes
73 covariates in train data
Test Set:
449 patients with 88 outcomes
Running Cyclops
Done.
GLM fit status: OK
Creating variable importance data frame
Prediction took 0.585 secs
Time to fit model: 3.48 secs
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
Removing infrequent and redundant covariates and normalizing
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
Removing infrequent and redundant covariates covariates and normalizing took 2.52 secs
Prediction took 0.481 secs
Prediction done in: 5.73 secs
Calculating Performance for Test
=============
AUC 71.48
95% lower AUC: 65.41
95% upper AUC: 77.54
AUPRC: 35.74
Brier: 0.14
Eavg: 0.02
Calibration in large- Mean predicted risk 0.1897 : observed risk 0.196
Calibration in large- Intercept 0.3216
Weak calibration intercept: 0.3216 - gradient:1.2182
Hosmer-Lemeshow calibration gradient: 1.23 intercept: -0.06
Average Precision: 0.37
Calculating Performance for Train
=============
AUC 72.43
95% lower AUC: 68.99
95% upper AUC: 75.86
AUPRC: 39.39
Brier: 0.14
Eavg: 0.02
Calibration in large- Mean predicted risk 0.1976 : observed risk 0.1976
Calibration in large- Intercept 0.174
Weak calibration intercept: 0.174 - gradient:1.1421
Hosmer-Lemeshow calibration gradient: 1.21 intercept: -0.04
Average Precision: 0.39
Calculating Performance for CV
=============
AUC 67.83
95% lower AUC: 64.15
95% upper AUC: 71.50
AUPRC: 31.83
Brier: 0.14
Eavg: 0.02
Calibration in large- Mean predicted risk 0.1979 : observed risk 0.1976
Calibration in large- Intercept 0.1826
Weak calibration intercept: 0.1826 - gradient:1.149
Hosmer-Lemeshow calibration gradient: 1.17 intercept: -0.03
Average Precision: 0.32
Time to calculate evaluation metrics: 0.619 secs
Calculating covariate summary @ 2026-11-20 14:59:10.458293
This can take a while...
Creating binary labels
Joining with strata
calculating subset of strata 1
calculating subset of strata 2
calculating subset of strata 3
calculating subset of strata 4
Restricting to subgroup
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
Calculating summary for subgroup TestWithNoOutcome
Restricting to subgroup
duckdb keeps downloaded extensions and secrets in a temporary directory:
ℹ /home/hornik/tmp/scratch/RtmpIlwUx3/duckdb
This is removed when the R session ends.
• Extensions are re-downloaded each session.
• Secrets are lost.
ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
ℹ See ?duckdb_storage for details and alternatives.
ℹ This message has been shown 60 times and will not be shown again this session.
Calculating summary for subgroup TrainWithNoOutcome
Restricting to subgroup
Calculating summary for subgroup TrainWithOutcome
Restricting to subgroup
Calculating summary for subgroup TestWithOutcome
Aggregating with labels and strata
Finished covariate summary @ 2026-11-20 14:59:21.178845
Time to calculate covariate summary: 10.7 secs
Run finished successfully.
Saving PlpResult
Creating directory to save model
plpResult saved to ..\/home/hornik/tmp/scratch/RtmpIlwUx3/saveLoc11103736fe6942/Test\plpResult
runPlp time taken: 39.3 secs
Use timeStamp: TRUE
Diagnosing impact of minTimeAtRisk in populationSettings
Outcome is 0 or 1
Population created with: 1800 observations, 1800 unique subjects and 355 outcomes
Population created in 0.124 secs
Outcome is 0 or 1
Population created with: 1800 observations, 1800 unique subjects and 355 outcomes
Population created in 0.1 secs
Outcome is 0 or 1
Population created with: 1800 observations, 1800 unique subjects and 355 outcomes
Population created in 0.123 secs
Outcome is 0 or 1
Population created with: 1800 observations, 1800 unique subjects and 355 outcomes
Population created in 0.221 secs
Outcome is 0 or 1
Population created with: 1800 observations, 1800 unique subjects and 355 outcomes
Population created in 0.0767 secs
Outcome is 0 or 1
Population created with: 1800 observations, 1800 unique subjects and 355 outcomes
Population created in 0.102 secs
Outcome is 0 or 1
Population created with: 1800 observations, 1800 unique subjects and 355 outcomes
Population created in 0.137 secs
Saving diagnosePlp to /home/hornik/tmp/scratch/RtmpIlwUx3/saveLoc11103736fe6942/Test/diagnosePlp.rds
Outcome is 0 or 1
Population created with: 1800 observations, 1800 unique subjects and 355 outcomes
Population created in 0.11 secs
seed: 12
Creating a 25% test and 75% train (into 3 folds) random stratified split by class
Data split into 449 test cases and 1351 train cases (451, 450, 450)
Data split in 8.7 secs
seed: 12
Creating a 25% test and 75% train (into 3 folds) random stratified split by class
Data split into 449 test cases and 1351 train cases (451, 450, 450)
Data split in 8.49 secs
Use timeStamp: TRUE
Creating save directory at: /home/hornik/tmp/scratch/RtmpIlwUx3/saveLoc11103736fe6942/tinyResults/tinyFit
Patient-Level Prediction Package version 6.7.0
Study started at: 2026-11-20 14:59:57.295435
AnalysisID: tinyFit
AnalysisName: Study details
TargetID: 1
OutcomeID: 3
Cohort size: 865
Covariates: 2
Creating population
Outcome is 0 or 1
Population created with: 865 observations, 865 unique subjects and 262 outcomes
Population created in 0.0999 secs
seed: 123
Creating a 25% test and 75% train (into 3 folds) random stratified split by class
Data split into 215 test cases and 650 train cases (217, 217, 216)
Data split in 8.29 secs
Train Set:
Fold 1 217 patients with 66 outcomes - Fold 2 217 patients with 66 outcomes - Fold 3 216 patients with 65 outcomes
2 covariates in train data
Test Set:
215 patients with 65 outcomes
Removing 0 redundant covariates
Removing 0 infrequent covariates
Normalizing covariates
Tidying covariates took 7.61 secs
Train Set:
Fold 1 217 patients with 66 outcomes - Fold 2 217 patients with 66 outcomes - Fold 3 216 patients with 65 outcomes
2 covariates in train data
Test Set:
215 patients with 65 outcomes
Running Cyclops
Done.
GLM fit status: OK
Creating variable importance data frame
Prediction took 0.392 secs
Time to fit model: 2.4 secs
Removing infrequent and redundant covariates and normalizing
Removing infrequent and redundant covariates covariates and normalizing took 2.92 secs
Prediction took 0.384 secs
Prediction done in: 5.87 secs
Calculating Performance for Test
=============
AUC 63.68
95% lower AUC: 57.26
95% upper AUC: 70.10
AUPRC: 40.36
Brier: 0.20
Eavg: 0.01
Calibration in large- Mean predicted risk 0.3018 : observed risk 0.3023
Calibration in large- Intercept -0.118
Weak calibration intercept: -0.118 - gradient:0.8335
Hosmer-Lemeshow calibration gradient: 0.90 intercept: 0.03
Average Precision: 0.41
Calculating Performance for Train
=============
AUC 64.99
95% lower AUC: 61.50
95% upper AUC: 68.48
AUPRC: 41.44
Brier: 0.19
Eavg: 0.00
Calibration in large- Mean predicted risk 0.3031 : observed risk 0.3031
Calibration in large- Intercept 0.0121
Weak calibration intercept: 0.0121 - gradient:1.0174
Hosmer-Lemeshow calibration gradient: 1.02 intercept: -0.00
Average Precision: 0.41
Calculating Performance for CV
=============
AUC 63.91
95% lower AUC: 59.57
95% upper AUC: 68.24
AUPRC: 39.42
Brier: 0.20
Eavg: 0.04
Calibration in large- Mean predicted risk 0.3031 : observed risk 0.3031
Calibration in large- Intercept -0.0778
Weak calibration intercept: -0.0778 - gradient:0.8913
Hosmer-Lemeshow calibration gradient: 1.02 intercept: 0.01
Average Precision: 0.39
Time to calculate evaluation metrics: 1.12 secs
Run finished successfully.
Saving PlpResult
Creating directory to save model
plpResult saved to ..\/home/hornik/tmp/scratch/RtmpIlwUx3/saveLoc11103736fe6942/tinyResults/tinyFit\plpResult
runPlp time taken: 26.7 secs
Warning: PredictionDistribution not available for survival models
No databaseRefId specified so using schema as unique database identifier
Warning: Unable to coerce hyperParamSearch to data.frame; storing empty table for compatibility.
Connecting using SQLite driver
All or some PLP result tables do not exist, tables being recreated
Deleting existing tables
Creating PLP results tables
Executing SQL took 0.0269 secs
PLP result migration being applied
Migrating data set
Migrator using SQL files in PatientLevelPrediction
Connecting using SQLite driver
Creating migrations table
Executing SQL took 0.0054 secs
Migrations table created
Executing migration: Migration_1-store_version.sql
Executing SQL took 0.00537 secs
Saving migration: Migration_1-store_version.sql
Executing SQL took 0.00507 secs
Migration complete Migration_1-store_version.sql
Executing migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.00909 secs
Saving migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.00487 secs
Migration complete Migration_2-add_hyperparameter_settings.sql
Executing migration: Migration_3-add_model_name.sql
Executing SQL took 0.00633 secs
Saving migration: Migration_3-add_model_name.sql
Executing SQL took 0.00793 secs
Migration complete Migration_3-add_model_name.sql
Closing database connection
Updating version number
Connecting using SQLite driver
Executing SQL took 0.00614 secs
Connecting using SQLite driver
Adding new model settings
Inserting data took 0.0234 secs
Connecting using SQLite driver
All or some PLP result tables do not exist, tables being recreated
Deleting existing tables
Creating PLP results tables
Executing SQL took 0.0256 secs
PLP result migration being applied
Migrating data set
Migrator using SQL files in PatientLevelPrediction
Connecting using SQLite driver
Creating migrations table
Executing SQL took 0.00419 secs
Migrations table created
Executing migration: Migration_1-store_version.sql
Executing SQL took 0.00476 secs
Saving migration: Migration_1-store_version.sql
Executing SQL took 0.00416 secs
Migration complete Migration_1-store_version.sql
Executing migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.00714 secs
Saving migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.00463 secs
Migration complete Migration_2-add_hyperparameter_settings.sql
Executing migration: Migration_3-add_model_name.sql
Executing SQL took 0.00541 secs
Saving migration: Migration_3-add_model_name.sql
Executing SQL took 0.00403 secs
Migration complete Migration_3-add_model_name.sql
Closing database connection
Updating version number
Connecting using SQLite driver
Executing SQL took 0.00497 secs
Connecting using SQLite driver
Adding new model settings
Inserting data took 0.0275 secs
Adding new model settings
Inserting data took 0.0109 secs
Adding new model settings
Inserting data took 0.014 secs
Adding new model settings
Inserting data took 0.0292 secs
Adding new model settings
Inserting data took 0.00876 secs
Adding new model settings
Inserting data took 0.0105 secs
Adding new model settings
Inserting data took 0.0143 secs
Connecting using SQLite driver
Executing SQL took 0.00994 secs
Executing SQL took 0.00723 secs
Executing SQL took 0.00727 secs
Executing SQL took 0.00697 secs
Executing SQL took 0.0108 secs
Connecting using SQLite driver
All or some PLP result tables do not exist, tables being recreated
Deleting existing tables
Creating PLP results tables
Executing SQL took 0.0295 secs
PLP result migration being applied
Migrating data set
Migrator using SQL files in PatientLevelPrediction
Connecting using SQLite driver
Creating migrations table
Executing SQL took 0.00449 secs
Migrations table created
Executing migration: Migration_1-store_version.sql
Executing SQL took 0.00436 secs
Saving migration: Migration_1-store_version.sql
Executing SQL took 0.00371 secs
Migration complete Migration_1-store_version.sql
Executing migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.00641 secs
Saving migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.00381 secs
Migration complete Migration_2-add_hyperparameter_settings.sql
Executing migration: Migration_3-add_model_name.sql
Executing SQL took 0.0051 secs
Saving migration: Migration_3-add_model_name.sql
Executing SQL took 0.00352 secs
Migration complete Migration_3-add_model_name.sql
Closing database connection
Updating version number
Connecting using SQLite driver
Executing SQL took 0.00718 secs
Connecting using SQLite driver
Adding new hyperparameter settings
Inserting data took 0.0196 secs
Hyperparameter setting exists
Connecting using SQLite driver
All or some PLP result tables do not exist, tables being recreated
Deleting existing tables
Creating PLP results tables
Executing SQL took 0.0257 secs
PLP result migration being applied
Migrating data set
Migrator using SQL files in PatientLevelPrediction
Connecting using SQLite driver
Creating migrations table
Executing SQL took 0.00426 secs
Migrations table created
Executing migration: Migration_1-store_version.sql
Executing SQL took 0.0046 secs
Saving migration: Migration_1-store_version.sql
Executing SQL took 0.007 secs
Migration complete Migration_1-store_version.sql
Executing migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.00739 secs
Saving migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.00412 secs
Migration complete Migration_2-add_hyperparameter_settings.sql
Executing migration: Migration_3-add_model_name.sql
Executing SQL took 0.00604 secs
Saving migration: Migration_3-add_model_name.sql
Executing SQL took 0.00384 secs
Migration complete Migration_3-add_model_name.sql
Closing database connection
Updating version number
Connecting using SQLite driver
Executing SQL took 0.00461 secs
Connecting using SQLite driver
Adding TAR
Executing SQL took 0.00381 secs
tarId: 1
Adding cohort target
Inserting data took 0.0562 secs
Adding cohort target
Inserting data took 0.0104 secs
tId: 1
Adding cohort outcome
Inserting data took 0.012 secs
Adding cohort outcome
Inserting data took 0.0115 secs
oId: 2
Adding new population settings
Inserting data took 0.0114 secs
popSetId: 1
Adding new covariate settings
Inserting data took 0.0286 secs
covSetId: 1
Adding new model settings
Inserting data took 0.024 secs
modSetId: 1
Adding new plp data settings
Inserting data took 0.0112 secs
plpDataSetId: 1
Adding new feature_engineering settings
Inserting data took 0.0107 secs
FESetId: 1
Adding new sample settings
Inserting data took 0.0121 secs
sampleSetId: 1
Adding new tidy covariates settings
Inserting data took 0.0123 secs
tidySetId: 1
Adding new split settings
Inserting data took 0.00957 secs
splitId: 1
Adding new hyperparameter settings
Inserting data took 0.0149 secs
hyperparameterSetId: 2
Executing SQL took 0.00463 secs
modelDesignId: 1
TAR exists
tarId: 1
json in jsons:TRUE
Cohort target exists in cohort_definition with cohort id1
Cohort target exists in cohorts with cohort id1
tId: 1
json in jsons:TRUE
Cohort outcome exists in cohort_definition with cohort id2
Cohort outcome exists in cohorts with cohort id2
oId: 2
Population settings exists
popSetId: 1
Covariate setting exists
covSetId: 1
Model setting exists
modSetId: 1
Split setting exists
plpDataSetId: 1
feature engineering setting exists
FESetId: 1
sample setting exists
sampleSetId: 1
tidy covariates setting exists
tidySetId: 1
Adding new split settings
Inserting data took 0.011 secs
splitId: 2
Adding new hyperparameter settings
Inserting data took 0.0105 secs
hyperparameterSetId: 3
Executing SQL took 0.00375 secs
modelDesignId: 2
TAR exists
tarId: 1
json in jsons:TRUE
Cohort target exists in cohort_definition with cohort id1
Cohort target exists in cohorts with cohort id1
tId: 1
json in jsons:TRUE
Cohort outcome exists in cohort_definition with cohort id2
Cohort outcome exists in cohorts with cohort id2
oId: 2
Population settings exists
popSetId: 1
Covariate setting exists
covSetId: 1
Model setting exists
modSetId: 1
Split setting exists
plpDataSetId: 1
feature engineering setting exists
FESetId: 1
sample setting exists
sampleSetId: 1
tidy covariates setting exists
tidySetId: 1
Split setting exists
splitId: 1
Hyperparameter setting exists
hyperparameterSetId: 2
modelDesignId: 1
Prediction done in: 0.000587 secs
Creating directory to save model
New best performance 0.5 with param: lambda=1
Fit best model on whole training set
Calculating covariate summary @ 2026-11-20 15:00:38.865205
This can take a while...
calculating subset of strata 1
Restricting to subgroup
Calculating summary for subgroup
Aggregating with no labels or strata
Finished covariate summary @ 2026-11-20 15:00:41.189847
Time to calculate covariate summary: 2.32 secs
Calculating covariate summary @ 2026-11-20 15:00:41.214363
This can take a while...
calculating subset of strata 1
Restricting to subgroup
Calculating summary for subgroup
Aggregating with no labels or strata
Finished covariate summary @ 2026-11-20 15:00:43.385168
Time to calculate covariate summary: 2.17 secs
Calculating covariate summary @ 2026-11-20 15:00:43.441158
This can take a while...
Creating binary labels
calculating subset of strata 1
calculating subset of strata 2
Restricting to subgroup
Calculating summary for subgroup WithOutcome
Restricting to subgroup
Calculating summary for subgroup WithNoOutcome
Aggregating with only labels or strata
Finished covariate summary @ 2026-11-20 15:00:48.740033
Time to calculate covariate summary: 5.3 secs
Restricting to subgroup
Calculating summary for subgroup
variance needs to be >= 0
variance should be of class:numericvariance should be of class:integer
seed should be of class:numericseed should be of class:NULLseed should be of class:integer
threads should be of class:numericthreads should be of class:integer
lowerLimit should be of class:numericlowerLimit should be of class:integer
upperLimit should be of class:numericupperLimit should be of class:integer
upperLimit needs to be >= 3
variance needs to be >= 0
variance should be of class:numericvariance should be of class:integer
seed should be of class:numericseed should be of class:NULLseed should be of class:integer
threads should be of class:numericthreads should be of class:integer
lowerLimit should be of class:numericlowerLimit should be of class:integer
upperLimit should be of class:numericupperLimit should be of class:integer
upperLimit needs to be >= 3
testFraction should be of class:numerictestFraction should be of class:integer
testFraction needs to be >= 0
-1 * testFraction needs to be > -1
trainFraction should be of class:numerictrainFraction should be of class:integer
-1 * trainFraction needs to be >= -1
trainFraction needs to be > 0
splitSeed should be of class:numericsplitSeed should be of class:integer
splitSeed should be of class:numericsplitSeed should be of class:integer
nfold should be of class:numericnfold should be of class:integer
nfold should be of class:numericnfold should be of class:integer
Invalid type setting.
Pick from: 'stratified','time','subject'
type should be of class:character
type should be of class:character
maxTrainingOutcomes needs to be > 0
maxTrainingOutcomes should be of class:numericmaxTrainingOutcomes should be of class:integer
maxTrainingOutcomes must be a single whole number
splitSeed should be of class:numericsplitSeed should be of class:integer
nfold needs to be > 1
nfold must be a single whole number
type should be stratified or subject
Data split into 240 test cases and 160 train cases with 40 training outcome rows (maxTrainingOutcomes = 40; folds: 40, 40, 40, 40)
Data split into 240 test cases and 160 train cases with 40 training outcome rows (maxTrainingOutcomes = 40; folds: 40, 40, 40, 40)
Data split into 180 test cases and 120 train cases with 20 training outcome rows (maxTrainingOutcomes = 20; folds: 30, 30, 30, 30)
Data split into 110 test cases and 220 train cases with 20 training outcome rows (maxTrainingOutcomes = 20; folds: 60, 60, 50, 50)
Warning: Outcome-limited split did not trigger because the population has 40 outcome rows, which is not greater than maxTrainingOutcomes = 100. Using the default 25% test and 75% train split.
seed: 1
Creating a 25% test and 75% train (into 4 folds) random stratified split by class
Data split into 50 test cases and 150 train cases (38, 38, 37, 37)
seed: 1
Creating a 25% test and 75% train (into 4 folds) random stratified split by class
Data split into 50 test cases and 150 train cases (38, 38, 37, 37)
seed: 40633
seed: 40633
Creating a 30% test and 70% train (into 4 folds) random stratified split by class
Data split into 59 test cases and 141 train cases (36, 36, 35, 34)
seed: 40633
Creating a 20% test and 80% train (into 4 folds) random stratified split by class
Data split into 99 test cases and 401 train cases (101, 100, 100, 100)
seed: 40633
Creating a 20% test and 40% train (into 4 folds) random stratified split by class
Data split into 99 test cases and 201 train cases (51, 51, 50, 49)
200 were not used for training or testing
seed: 40633
Creating 20% test and 80% train (into 4 folds) stratified split at 2011-02-05
Data split into 100 test cases and 400 train samples (101, 101, 101, 97)
seed: 40633
Creating 20% test and 40% train (into 4 folds) stratified split at 2011-02-05
Data split into 100 test cases and 204 train samples (51, 51, 51, 51)
196 were not used for training or testing
seed: 40633
seed: 40633
Creating a 20% test and 80% train (into 4 folds) stratified split by subject
Data split into 40 test cases and 160 train cases (41, 41, 39, 39)
seed: 40633
Creating a 25% test and 75% train (into 3 folds) stratified split by subject
Data split into 52 test cases and 148 train cases (52, 48, 48)
Calculating Performance for All
=============
AUC 51.00
95% lower AUC: 39.57
95% upper AUC: 62.43
AUPRC: 52.01
Brier: 0.33
Eavg: 0.24
Calibration in large- Mean predicted risk 0.5 : observed risk 0.5
Calibration in large- Intercept 0
Weak calibration intercept: 0 - gradient:0.0349
Hosmer-Lemeshow calibration gradient: 0.00 intercept: 0.50
Average Precision: 0.53
Time to calculate evaluation metrics: 0.236 secs
Evaluating survival model at time: 365 days
C-statistic: 0.38931 (0.269-0.51)
E-statistic: 0.0319220138645564
E-statistic 90%: 0.0649360660190881
getCalibrationSummary error
Error in attributes(.Data) <- c(attributes(.Data), attrib): adding class "factor" to an invalid object
Warning: PredictionDistribution not available for survival models
Time to calculate evaluation metrics: 1.19 secs
Setting levels: control = 0, case = 1
Warning: Cannot compute AUC: outcomeCount has only one class
Warning: Cannot compute AUC: outcomeCount has only one class
Warning: Cannot compute AUPRC: outcomeCount has only one class
Warning: Cannot compute Average Precision: outcomeCount has no positive class
Calculating Performance for Validation
=============
Warning: Cannot compute AUC: outcomeCount has only one class
AUC NA
95% lower AUC: NA
95% upper AUC: NA
Warning: Cannot compute AUPRC: outcomeCount has only one class
AUPRC: NA
Brier: 0.35
Eavg: 0.53
Calibration in large- Mean predicted risk 0.527 : observed risk 0
Calibration in large- Intercept -26.5661
Weak calibration intercept: -26.5661 - gradient:0
Hosmer-Lemeshow calibration gradient: 0.00 intercept: -0.00
Warning: Cannot compute Average Precision: outcomeCount has no positive class
Average Precision: NA
Warning: Number of positives is zero
Time to calculate evaluation metrics: 0.119 secs
Column names of coefficients are not correct
Column types of coefficients are not correct
Column types of coefficients are not correct
intercept should be of class:numeric
mapping should be of class:charactermapping should be of class:function
targetId should be of class:numerictargetId should be of class:NULL
outcomeId should be of class:numericoutcomeId should be of class:NULL
populationSettings should be of class:NULLpopulationSettings should be of class:populationSettings
restrictPlpDataSettings should be of class:NULLrestrictPlpDataSettings should be of class:restrictPlpDataSettings
covariateSettings should be of class:listcovariateSettings should be of class:NULLcovariateSettings should be of class:covariateSettings
predict risk probabilities using predictGlm
Prediction took 0.296 secs
Prediction done in: 0.329 secs
databaseDetails should be of class:databaseDetails
databaseDetails should be of class:databaseDetails
databaseDetails should be of class:databaseDetails
No cdm database name entered so using cdmDatabaseSchema
threshold needs to be >= 0
threshold should be of class:numeric
threshold needs to be < 1
starting to map the columns and rows
finished MapCovariates
starting to map the columns and rows
finished MapCovariates
plpData size estimated to use 0GBs of RAM
starting toSparseM
starting to map the columns and rows
finished MapCovariates
toSparseM non temporal used
plpData size estimated to use 0GBs of RAM
finishing toSparseM
toSparseM took 5.7076690196991 secs
starting to map the columns and rows
finished MapCovariates
Saving _problems/test-getCalibration-65.R
Saving _problems/test-helperfunctions-45.R
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
missingThreshold needs to be > 0
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
missingThreshold should be of class:numeric
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
missingThreshold needs to be < 1
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
method should be mean or median
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
addMissingIndicator should be of class:logical
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
method should be pmm
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
missingThreshold needs to be > 0
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
missingThreshold should be of class:numeric
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
missingThreshold needs to be < 1
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
addMissingIndicator should be of class:logical
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
Imputing missing values with simpleImputer using: mean and missing threshold: 0.3
Calculating missingness in data
Found features with missing values
Imputation done in time: 13.7 secs
Applying imputation to test data with simpleImputer
using method: mean and missing threshold: 0.3
Imputation done in time: 11.7 secs
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
Imputing missing values with simpleImputer using: mean and missing threshold: 0.95
Calculating missingness in data
Found features with missing values
Imputation done in time: 9.2 secs
Applying imputation to test data with simpleImputer
using method: mean and missing threshold: 0.95
Imputation done in time: 8.78 secs
Warning: Imputation is experimental and may have bugs,
please report any issues on the GitHub repository.
Imputing missing values with simpleImputer using: mean and missing threshold: 0.3
Calculating missingness in data
Found features with missing values
Imputation done in time: 10.7 secs
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
Imputing missing values with iterativeImputer using: pmm, missing threshold: 0.3 and method settings: 111
Calculating missingness in data
Found features with missing values
Imputation iteration: 1
Imputing variable: fakeMissingVariable
Imputation done in time: 21.1 secs
Applying imputation to test data with iterativeImputer
using method: pmm and missing threshold: 0.3
Imputing variable: fakeMissingVariable
PMM predict for variable fakeMissingVariable (covariateId=666): missing=32, donors=64, k=1, time=1.03 secs
Imputation done in time: 19.8 secs
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
Applying imputation to test data with iterativeImputer
using method: pmm and missing threshold: 0.3
Imputation done in time: 10.8 secs
Warning: Imputation is experimental and may have bugs.
Please report any issues on the GitHub repository.
Applying imputation to test data with iterativeImputer
using method: pmm and missing threshold: 0.8
Imputing variable: bmi
PMM predict for variable bmi (covariateId=100): missing=2, donors=2, k=3, time=0.637 secs
Imputation done in time: 17.3 secs
Warning: Imputation model only has intercept. It does not fit the data well
Use timeStamp: TRUE
Creating save directory at: /home/hornik/tmp/scratch/RtmpIlwUx3/saveLoc11103736fe6942/lcc/learningCurve_1
Patient-Level Prediction Package version 6.7.0
Study started at: 2026-11-20 15:04:10.353146
AnalysisID: learningCurve_1
AnalysisName: Study details
TargetID: 1
OutcomeID: 3
Cohort size: 1800
Covariates: 75
Creating population
Outcome is 0 or 1
Population created with: 1767 observations, 1767 unique subjects and 355 outcomes
Population created in 0.159 secs
seed: 70165
Creating a 20% test and 60% train (into 2 folds) random stratified split by class
Data split into 353 test cases and 1061 train cases (531, 530)
353 were not used for training or testing
Data split in 9.14 secs
Train Set:
Fold 1 531 patients with 107 outcomes - Fold 2 530 patients with 106 outcomes
75 covariates in train data
Test Set:
353 patients with 71 outcomes
Removing 2 redundant covariates
Removing 0 infrequent covariates
Normalizing covariates
Tidying covariates took 9.52 secs
Train Set:
Fold 1 531 patients with 107 outcomes - Fold 2 530 patients with 106 outcomes
73 covariates in train data
Test Set:
353 patients with 71 outcomes
Running Cyclops
Done.
GLM fit status: OK
Creating variable importance data frame
Prediction took 0.521 secs
Time to fit model: 2.53 secs
Removing infrequent and redundant covariates and normalizing
Removing infrequent and redundant covariates covariates and normalizing took 2.42 secs
Prediction took 0.726 secs
Prediction done in: 6.26 secs
Calculating Performance for Test
=============
AUC 70.65
95% lower AUC: 63.83
95% upper AUC: 77.47
AUPRC: 32.86
Brier: 0.14
Eavg: 0.01
Calibration in large- Mean predicted risk 0.1994 : observed risk 0.2011
Calibration in large- Intercept 0.1296
Weak calibration intercept: 0.1296 - gradient:1.0968
Hosmer-Lemeshow calibration gradient: 1.09 intercept: -0.02
Average Precision: 0.35
Calculating Performance for Train
=============
AUC 74.41
95% lower AUC: 70.82
95% upper AUC: 77.99
AUPRC: 40.35
Brier: 0.14
Eavg: 0.03
Calibration in large- Mean predicted risk 0.2008 : observed risk 0.2008
Calibration in large- Intercept 0.1786
Weak calibration intercept: 0.1786 - gradient:1.1495
Hosmer-Lemeshow calibration gradient: 1.25 intercept: -0.04
Average Precision: 0.41
Calculating Performance for CV
=============
AUC 69.66
95% lower AUC: 65.74
95% upper AUC: 73.58
AUPRC: 32.83
Brier: 0.14
Eavg: 0.03
Calibration in large- Mean predicted risk 0.2012 : observed risk 0.2008
Calibration in large- Intercept 0.2838
Weak calibration intercept: 0.2838 - gradient:1.2355
Hosmer-Lemeshow calibration gradient: 1.24 intercept: -0.05
Average Precision: 0.31
Time to calculate evaluation metrics: 0.538 secs
Run finished successfully.
Saving PlpResult
Creating directory to save model
plpResult saved to ..\/home/hornik/tmp/scratch/RtmpIlwUx3/saveLoc11103736fe6942/lcc/learningCurve_1\plpResult
runPlp time taken: 29.7 secs
Creating save directory at: /home/hornik/tmp/scratch/RtmpIlwUx3/saveLoc11103736fe6942/lcc/learningCurve_2
Patient-Level Prediction Package version 6.7.0
Study started at: 2026-11-20 15:04:40.100132
AnalysisID: learningCurve_2
AnalysisName: Study details
TargetID: 1
OutcomeID: 3
Cohort size: 1800
Covariates: 75
Creating population
Outcome is 0 or 1
Population created with: 1767 observations, 1767 unique subjects and 355 outcomes
Population created in 0.168 secs
seed: 70165
Creating a 20% test and 70% train (into 2 folds) random stratified split by class
Data split into 353 test cases and 1237 train cases (619, 618)
177 were not used for training or testing
Data split in 8.98 secs
Train Set:
Fold 1 619 patients with 124 outcomes - Fold 2 618 patients with 124 outcomes
75 covariates in train data
Test Set:
353 patients with 71 outcomes
Removing 2 redundant covariates
Removing 0 infrequent covariates
Normalizing covariates
Tidying covariates took 9.87 secs
Train Set:
Fold 1 619 patients with 124 outcomes - Fold 2 618 patients with 124 outcomes
73 covariates in train data
Test Set:
353 patients with 71 outcomes
Running Cyclops
Done.
GLM fit status: OK
Creating variable importance data frame
Prediction took 0.689 secs
Time to fit model: 3.11 secs
Removing infrequent and redundant covariates and normalizing
Removing infrequent and redundant covariates covariates and normalizing took 2.94 secs
Prediction took 0.439 secs
Prediction done in: 6.36 secs
Calculating Performance for Test
=============
AUC 69.51
95% lower AUC: 62.68
95% upper AUC: 76.34
AUPRC: 30.91
Brier: 0.14
Eavg: 0.01
Calibration in large- Mean predicted risk 0.1987 : observed risk 0.2011
Calibration in large- Intercept 0.0621
Weak calibration intercept: 0.0621 - gradient:1.0373
Hosmer-Lemeshow calibration gradient: 0.90 intercept: 0.02
Average Precision: 0.33
Calculating Performance for Train
=============
AUC 74.47
95% lower AUC: 71.12
95% upper AUC: 77.83
AUPRC: 40.88
Brier: 0.14
Eavg: 0.02
Calibration in large- Mean predicted risk 0.2005 : observed risk 0.2005
Calibration in large- Intercept 0.1642
Weak calibration intercept: 0.1642 - gradient:1.1391
Hosmer-Lemeshow calibration gradient: 1.26 intercept: -0.04
Average Precision: 0.41
Calculating Performance for CV
=============
AUC 70.17
95% lower AUC: 66.55
95% upper AUC: 73.79
AUPRC: 33.71
Brier: 0.14
Eavg: 0.02
Calibration in large- Mean predicted risk 0.2004 : observed risk 0.2005
Calibration in large- Intercept 0.2634
Weak calibration intercept: 0.2634 - gradient:1.2179
Hosmer-Lemeshow calibration gradient: 1.24 intercept: -0.05
Average Precision: 0.33
Time to calculate evaluation metrics: 0.479 secs
Run finished successfully.
Saving PlpResult
Creating directory to save model
plpResult saved to ..\/home/hornik/tmp/scratch/RtmpIlwUx3/saveLoc11103736fe6942/lcc/learningCurve_2\plpResult
runPlp time taken: 31.5 secs
Finished in 61 secs.
type should be minmax or robust
type should be minmax or robust
type should be minmax or robust
type should be minmax or robust
settings$clip needs to be a boolean
Starting min-max normalization of continuous features
Finished min-max normalization of continuous features in 2.45 secs
Applying min-max normalization of continuous features to test data
Finished min-max normalization of continuous features in 2.03 secs
Starting robust normalization of continuous features
Finished robust normalization in 2.4 secs
Applying robust normalization of continuous features to test data
Finished robust normalization in 2.5 secs
Starting robust normalization of continuous features
Finished robust normalization in 3.49 secs
Applying robust normalization of continuous features to test data
Finished robust normalization in 1.83 secs
Starting min-max normalization of continuous features
Finished min-max normalization of continuous features in 3.91 secs
Applying min-max normalization of continuous features to test data
Finished min-max normalization of continuous features in 2.49 secs
Starting robust normalization of continuous features
Finished robust normalization in 2.87 secs
Applying robust normalization of continuous features to test data
Finished robust normalization in 2.01 secs
1 needs to be a boolean
"tertet" needs to be a boolean
1 needs to be >= 2
"tertet" needs to be >= 1
2 needs to be <= 1
"tertet" needs to be <= 1
1 needs to be > 2
0 needs to be > 0
"tertet" needs to be > 1
2 needs to be < 1
0 needs to be < 0
"tertet" needs to be < 1
NULL cannot be empty
"dsdsds" should be of class:double
1.5 must be a single whole number
c(1, 2) must be a single whole number
"1" must be a single whole number
NA_real_ must be a single whole number
Inf must be a single whole number
c(1, 2) must be a single finite numeric value
"1" must be a single finite numeric value
NA_real_ must be a single finite numeric value
Inf must be a single finite numeric value
"dsdsds" should be dsds or double
c("dsdsds", "double") should be dsdsds or double
NA_character_ should be dsdsds or double
Creating save directory at: /home/hornik/tmp/scratch/RtmpIlwUx3/file11103712d0d55c
file does not exist
Column names of data.frame(a = 1:2, b = 1:2) are not correct
Column types of data.frame(a = 1:2, b = 1:2) are not correct
includeAllOutcomes should be of class:logical
includeAllOutcomes should be of class:logical
firstExposureOnly should be of class:logical
firstExposureOnly should be of class:logical
washoutPeriod should be of class:numericwashoutPeriod should be of class:integer
washoutPeriod needs to be >= 0
removeSubjectsWithPriorOutcome should be of class:logical
removeSubjectsWithPriorOutcome should be of class:logical
priorOutcomeLookback should be of class:numericpriorOutcomeLookback should be of class:integer
priorOutcomeLookback needs to be >= 0
requireTimeAtRisk should be of class:logical
requireTimeAtRisk should be of class:logical
minTimeAtRisk should be of class:numericminTimeAtRisk should be of class:integer
minTimeAtRisk needs to be >= 0
riskWindowStart should be of class:numericriskWindowStart should be of class:integer
riskWindowEnd should be of class:numericriskWindowEnd should be of class:integer
startAnchor should be of class:character
endAnchor should be of class:character
minFraction needs to be >= 0
minFraction should be of class:numericminFraction should be of class:integer
removeRedundancy should be of class:logical
removeRedundancy should be of class:logical
normalize should be of class:logical
normalize should be of class:logical
Removing infrequent and redundant covariates and normalizing
Removing infrequent and redundant covariates covariates and normalizing took 2.17 secs
recal initial baseline hazard: 0.8
recal initial offset: 0.15
recal initial timepoint: 220
recal initial baseline hazard: 0.85
recal initial offset: -0.05
recal initial timepoint: 240
recal initial baseline hazard: 0.9
recal initial offset: 0
recal initial timepoint: 365
Connecting using SQLite driver
All or some PLP result tables do not exist, tables being recreated
Deleting existing tables
Creating PLP results tables
Executing SQL took 0.0238 secs
PLP result migration being applied
Migrating data set
Migrator using SQL files in PatientLevelPrediction
Connecting using SQLite driver
Creating migrations table
Executing SQL took 0.00791 secs
Migrations table created
Executing migration: Migration_1-store_version.sql
Executing SQL took 0.0052 secs
Saving migration: Migration_1-store_version.sql
Executing SQL took 0.00425 secs
Migration complete Migration_1-store_version.sql
Executing migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.00753 secs
Saving migration: Migration_2-add_hyperparameter_settings.sql
Executing SQL took 0.00612 secs
Migration complete Migration_2-add_hyperparameter_settings.sql
Executing migration: Migration_3-add_model_name.sql
Executing SQL took 0.00528 secs
Saving migration: Migration_3-add_model_name.sql
Executing SQL took 0.00411 secs
Migration complete Migration_3-add_model_name.sql
Closing database connection
Updating version number
Connecting using SQLite driver
Executing SQL took 0.00615 secs
Connecting using SQLite driver
runSplitData should be of class:logical
runSampleData should be of class:logical
runFeatureEngineering should be of class:logical
runPreprocessData should be of class:logical
runModelDevelopment should be of class:logical
runCovariateSummary should be of class:logical
numberOutcomestoNonOutcomes should be of class:numericnumberOutcomestoNonOutcomes should be of class:integer
numberOutcomestoNonOutcomes needs to be > 0
sampleSeed should be of class:numericsampleSeed should be of class:integer
type should be of class:character
plpData object
At risk concept ID:
Outcome concept ID(s):
plpData object summary
At risk cohort concept ID:
Outcome concept ID(s):
People:
Outcome counts:
Creating directory to save model
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Computing covariate prevalence
Fitting outcome model(s)
Generating covariates
Generating cohorts
Generating outcomes
Removing infrequent and redundant covariates and normalizing
Removing infrequent and redundant covariates covariates and normalizing took 2.83 secs
Prediction took 0.738 secs
Prediction done in: 6.21 secs
Removing infrequent and redundant covariates and normalizing
Removing infrequent and redundant covariates covariates and normalizing took 3.28 secs
predict risk probabilities using predictGlm
Prediction took 0.78 secs
Prediction done in: 6.89 secs
predict risk probabilities using predictGlm
Prediction took 0.365 secs
Prediction done in: 0.413 secs
Restricting tarEnd to end of target cohort
Outcome is 0 or 1
Population created with: 20 observations, 20 unique subjects and 20 outcomes
Population created in 0.127 secs
Warning: restrictPlpDataSettings are not the same in models and validationDesign,
using from design
restrictPlpDataSettings not set in design, using model's
targetId should be of class:numerictargetId should be of class:integer
outcomeId should be of class:numericoutcomeId should be of class:integer
targetId should be of class:numerictargetId should be of class:integer
outcomeId should be of class:numericoutcomeId should be of class:integer
populationSettings should be of class:populationSettings
recalibrate should be of class:characterrecalibrate should be of class:NULL
runCovariateSummary should be of class:logical
[ FAIL 2 | WARN 0 | SKIP 149 | PASS 1127 ]
══ Skipped tests (149) ═════════════════════════════════════════════════════════
• On CRAN (145): 'test-LightGBM.R:18:3', 'test-LightGBM.R:54:3',
'test-LightGBM.R:75:3', 'test-UploadToDatabase.R:210:3',
'test-UploadToDatabase.R:284:3', 'test-UploadToDatabase.R:456:3',
'test-UploadToDatabase.R:507:3', 'test-UploadToDatabase.R:571:3',
'test-UploadToDatabase.R:635:3', 'test-UploadToDatabase.R:743:3',
'test-andromedahelperfunctions.R:19:3', 'test-barFitState.R:42:3',
'test-barFitState.R:109:3', 'test-compatModelSettings.R:248:3',
'test-compatModelSettings.R:303:3', 'test-compatModelSettings.R:367:3',
'test-cyclopsModels.R:159:3', 'test-cyclopsModels.R:216:3',
'test-cyclopsModels.R:233:3', 'test-cyclopsModels.R:293:3',
'test-cyclopsModels.R:328:3', 'test-cyclopsModels.R:361:3',
'test-cyclopsModels.R:382:3', 'test-cyclopsModels.R:435:3',
'test-cyclopsModels.R:473:3', 'test-cyclopsModels.R:524:3',
'test-cyclopsModels.R:581:3', 'test-cyclopsModels.R:619:3',
'test-cyclopsModels.R:641:3', 'test-cyclopsModels.R:681:3',
'test-cyclopsModels.R:713:3', 'test-cyclopsModels.R:740:3',
'test-cyclopsModels.R:777:3', 'test-dataSplitting.R:154:3',
'test-dataSplitting.R:378:3', 'test-dataSplitting.R:401:3',
'test-dataSplitting.R:595:3', 'test-diagnostic.R:60:3',
'test-diagnostic.R:111:3', 'test-evaluation.R:17:3',
'test-evaluation.R:46:3', 'test-existingModel.R:18:3',
'test-existingModel.R:74:3', 'test-existingModel.R:124:3',
'test-extractData.R:17:3', 'test-extractData.R:43:3',
'test-extractData.R:62:3', 'test-extractData.R:93:3',
'test-extractData.R:112:3', 'test-extractData.R:156:3',
'test-extractData.R:185:3', 'test-extractData.R:204:3',
'test-extractData.R:231:3', 'test-featureEngineering.R:40:3',
'test-featureEngineering.R:56:3', 'test-featureEngineering.R:84:3',
'test-featureEngineering.R:130:3', 'test-featureEngineering.R:158:3',
'test-featureEngineering.R:203:3', 'test-featureEngineering.R:248:3',
'test-featureEngineering.R:316:3', 'test-featureEngineering.R:343:3',
'test-featureImportance.R:19:3', 'test-featureImportance.R:45:3',
'test-fitting.R:20:3', 'test-fitting.R:32:3', 'test-formatting.R:103:3',
'test-helperfunctions.R:39:3', 'test-imputation.R:261:3',
'test-imputation.R:282:3', 'test-imputation.R:333:3',
'test-learningCurves.R:35:3', 'test-learningCurves.R:50:3',
'test-learningCurves.R:74:3', 'test-multiplePlp.R:86:3',
'test-multiplePlp.R:132:3', 'test-plotting.R:21:3', 'test-plotting.R:60:3',
'test-plotting.R:74:3', 'test-plotting.R:91:3', 'test-plotting.R:126:3',
'test-plotting.R:148:3', 'test-plotting.R:155:3', 'test-plotting.R:165:3',
'test-plotting.R:174:3', 'test-plotting.R:182:3', 'test-plotting.R:199:3',
'test-plotting.R:208:3', 'test-plotting.R:218:3', 'test-plotting.R:225:3',
'test-population.R:263:3', 'test-population.R:471:3',
'test-prediction.R:18:3', 'test-prediction.R:39:3', 'test-prediction.R:78:3',
'test-preprocessingData.R:56:3', 'test-preprocessingData.R:118:3',
'test-rclassifier.R:18:3', 'test-rclassifier.R:51:3',
'test-rclassifier.R:71:3', 'test-rclassifier.R:134:3',
'test-recalibration.R:61:3', 'test-runPlpHelpers.R:18:3',
'test-runPlpHelpers.R:32:3', 'test-sampling.R:91:3', 'test-sampling.R:132:3',
'test-sampling.R:168:3', 'test-saveloadplp.R:31:3',
'test-saveloadplp.R:46:3', 'test-saveloadplp.R:87:3',
'test-saveloadplp.R:171:3', 'test-sklearnClassifier.R:19:3',
'test-sklearnClassifier.R:43:3', 'test-sklearnClassifier.R:54:3',
'test-sklearnClassifier.R:117:3', 'test-sklearnClassifier.R:132:3',
'test-sklearnClassifier.R:164:3', 'test-sklearnClassifier.R:201:3',
'test-sklearnClassifier.R:237:3', 'test-sklearnClassifier.R:281:3',
'test-sklearnClassifier.R:301:3', 'test-sklearnClassifierSettings.R:19:3',
'test-sklearnClassifierSettings.R:37:3',
'test-sklearnClassifierSettings.R:46:3',
'test-sklearnClassifierSettings.R:81:3',
'test-sklearnClassifierSettings.R:94:3',
'test-sklearnClassifierSettings.R:122:3', 'test-sklearnJson.R:39:3',
'test-sklearnJson.R:57:3', 'test-sklearnJson.R:78:3',
'test-sklearnJson.R:99:3', 'test-sklearnJson.R:117:3',
'test-sklearnJson.R:137:5', 'test-sklearnJson.R:137:5',
'test-sklearnJson.R:186:3', 'test-validation.R:44:3',
'test-validation.R:68:3', 'test-validation.R:201:3',
'test-validation.R:226:3', 'test-validation.R:241:3',
'test-validation.R:262:3', 'test-validation.R:287:3',
'test-validation.R:315:3', 'test-validation.R:345:3',
'test-validation.R:368:3'
• PostgreSQL test connection is not configured (4):
'test-UploadToDatabase.R:51:3', 'test-UploadToDatabase.R:103:3',
'test-UploadToDatabase.R:123:3', 'test-UploadToDatabase.R:172:3'
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test-getCalibration.R:59:3'): getCalibration survival ───────────────
Error in `attributes(.Data) <- c(attributes(.Data), attrib)`: adding class "factor" to an invalid object
Backtrace:
▆
1. └─PatientLevelPrediction:::getCalibrationSummary_survival(...) at test-getCalibration.R:59:3
2. └─PatientLevelPrediction:::cut2(predictionOfInterest$value, g = gval)
3. └─base::structure(y, class = "factor", levels = labs)
── Error ('test-helperfunctions.R:45:3'): Borrowed cut2 ────────────────────────
Error in `attributes(.Data) <- c(attributes(.Data), attrib)`: adding class "factor" to an invalid object
Backtrace:
▆
1. └─PatientLevelPrediction:::cut2(x, g = 3) at test-helperfunctions.R:45:3
2. └─base::structure(y, class = "factor", levels = labs)
[ FAIL 2 | WARN 0 | SKIP 149 | PASS 1127 ]
Error:
! Test failures.
Execution halted
Flavor: r-devel-linux-x86_64-debian-gcc