Package {coreval}

coreval: Check Clinical Trial Data Against 'CDISC' Open Rules

Description

Finds conformance problems in clinical trial data without leaving R, using the openly published 'CORE' conformance rules: the 'CORE' 1.0 rules maintained by Verisian for 'SDTM', and the 'CDISC' Open Rules for 'SEND', 'TIG' and 'USDM'. Check a single dataset while you are still writing the code that builds it, or a whole study folder once it exists, and get the findings back as a tidy data frame pointing at the exact row and variable. Reads transport ('XPT'), 'SAS', comma-separated and 'Dataset-JSON' files, plus 'Define-XML' when present, and 'USDM' study-design documents. Covers rules for the 'SDTM', 'SEND', 'TIG' and 'USDM' standards. The rules are bundled inside the package, so nothing is downloaded and your data never leaves your machine: no internet, no API key, no account. When a rule cannot be checked - because it needs a dataset you did not supply, for instance - it is reported as skipped with the reason, never counted as a pass. Meant as a quick first pass before a qualified validation system, never as a replacement for one. An independent project: not affiliated with or endorsed by 'CDISC' or Verisian, and not a 'CORE'-certified conformance engine.

Details

coreval finds CDISC conformance problems in clinical trial data without leaving R. Run it early and often, while you are still writing the code that produces the data, so problems turn up while they are cheap to fix.

Two ways in, depending on what you have:

• check_dataset() - one dataset, either a data frame you already have open or a single file. Use this while writing code.

• check_study() - a whole study folder. Use this once the datasets exist, since the cross-dataset rules need everything present.

Both entry points give you two tables, and both matter. ⁠$findings⁠ is what is wrong. ⁠$skipped⁠ is what could not be checked, with a reason for each. An empty ⁠$findings⁠ can mean clean data or rules that never ran, and those look identical if you only read the first table.

write_findings() saves both to Excel or CSV. vignette("coreval") walks through all of it, and vignette("technical") explains how it works.

Status

coreval is a personal open-source project. It is not a CDISC product, is not affiliated with or endorsed by CDISC, and is not qualified or validated software. The rules are bundled inside the package, so nothing is downloaded and your data never leaves your machine: no internet, no API key, no account.

Treat every result as a hint, not a verdict. A qualified validation system and your own review are still what decide whether data is good to submit. coreval does not remove that step - it just leaves that step less to find.

Author(s)

Maintainer: Hrach Gevorgyan hrach.gevorgyan@yandex.com [copyright holder]

Authors:

• Hrach Gevorgyan hrach.gevorgyan@yandex.com [copyright holder]

See Also

Useful links:

• https://hrach-gevorgyan.github.io/coreval/

• https://github.com/hrach-gevorgyan/coreval

• Report bugs at https://github.com/hrach-gevorgyan/coreval/issues

Check one dataset, without needing a study folder

Description

For when you are writing the code that builds a domain and want to know what is wrong with it right now. Give it the data frame you already have open, or the path to a single file.

Usage

check_dataset(
  x,
  domain = NULL,
  standard = NULL,
  version = NULL,
  use_case = NULL,
  max_records = 1000,
  include_deprecated = FALSE,
  ct_package = NULL,
  substandard = NULL
)

Arguments

Value

An object of class coreval_result: a list of the three tables findings, skipped and truncated, the same shape check_study() returns, so write_findings() and filter_findings() work on it unchanged. Because it has a class, typing the result's name prints a readable report rather than dumping the list.

skipped carries a reason for every rule that did not run - a dataset you did not supply, a missing Define-XML, or, for the rules that ask whether a value is an allowed term, CDISC's controlled terminology if you did not say which package the study follows - see ct_package. Nothing skipped is ever counted as a pass.

Provenance rides along as attributes: checks_run (how many rules were evaluated), domains, and excluded_by_standard (how many rules the standard/version filter set aside). write_findings() writes these into the file it saves.

What it cannot check on its own

Plenty of CDISC rules compare one dataset against another - an adverse event date against the subject's reference dates in DM, a visit against the trial design. Hand over a single dataset and those questions cannot be answered.

coreval does not guess. Those rules are skipped, and ⁠$skipped⁠ names the dataset each one wanted. Running them anyway would compare your data against columns that are not there and report problems that do not exist.

Most rules still run on a single dataset. But the ones that cannot are the cross-dataset checks, which are often the ones that matter.

So a short ⁠$findings⁠ table here does not mean the data is clean. It is a quick first pass, not a verdict. Run check_study() on the whole folder before drawing conclusions.

See Also

check_study() to check a whole study folder.

Examples

ae <- data.frame(
  STUDYID = "S1", DOMAIN = "AE", USUBJID = c("01", "01"),
  AESEQ = c(1, 2), AETERM = c("Headache", "Rash"),
  AESTDTC = c("2024-01-10", "2024-02-30") # 30 February is not a date
)
result <- check_dataset(ae, standard = "SDTMIG")
result$findings[result$findings$Value == "2024-02-30", ]

# Always look at what could not run:
nrow(result$skipped)

Check a whole study against CDISC Open Rules

Description

Runs every rule that applies to every dataset in the study, including the ones that compare datasets against each other. Use this once the datasets exist as files; to check a single dataset while you are still writing the code that builds it, see check_dataset().

Usage

check_study(
  study,
  standard = NULL,
  version = NULL,
  use_case = NULL,
  max_records = 1000,
  include_deprecated = FALSE,
  ct_package = NULL,
  usdm = FALSE,
  substandard = NULL
)

Arguments

Details

Findings come back one row per (dataset, record, variable), pointing at the exact spot. Some rules ask about a dataset as a whole rather than a particular row - those leave Record blank. A few ask about the study as a whole, such as "is DM present at all?"; those are answered once and reported under Dataset = "STUDY" rather than repeated for every domain.

Rules comparing against a define.xml do run, as long as the study has one and the xml2 package is installed. Without both, they are skipped with a reason instead of being run against columns that are not there, which would report problems that do not exist. The same goes for any rule needing an operator or join coreval does not implement yet.

Value

An object of class coreval_result, holding three tables. Because it has a class, typing the result's name prints a readable report rather than dumping the list, and provenance rides along as attributes: checks_run, ran (each rule and dataset checked, one row per check), domains, excluded_by_standard, and define_only (TRUE when the folder held a define.xml and no datasets, so only the rules comparing the define with the study ran).

• findings - what is wrong. One row per affected record and variable, with Dataset, Record (its row in the file, or for a rule about the variables themselves, such as a type or label check, the variable's position), USUBJID and SEQ (its subject and --SEQ, blank where the dataset has none or the row is not a record), Variable, Value, the issue in words, its triage and rule_id. ⁠Not in dataset⁠ under Value means the rule wanted a variable you do not have, which is usually the finding itself.

• skipped - what could not be checked, with a reason for each. Read this one: an empty findings table can mean clean data or rules that never ran, and they look identical otherwise. Reasons include a dataset the study does not have, a missing Define-XML, and a controlled terminology version that neither the study nor ct_package names. Nothing skipped is ever counted as a pass.

• truncated - rules that flagged more records than max_records kept, with how many they really found.

Progress

A large study takes long enough that silence looks like a hang, so an interactive session shows a progress bar naming the domain being checked and how far through the study it is:

  AE        4/7  |=====================       |  75%

The percentage is weighted by how many records each domain holds, not by a plain count of rules, because a check against a 161,600-row AE costs hundreds of times one against a 200-row SJ. It tracks elapsed time closely but is still an estimate - rules differ in cost among themselves too - so treat it as "roughly how far through", not a clock.

It is off in scripts and non-interactive runs, where it would only clutter a log. Turn it on or off with options(coreval.progress = TRUE) or FALSE.

Examples

dir <- tempfile("coreval_study_")
dir.create(dir)
haven::write_xpt(data.frame(USUBJID = c("1", "2"), AGE = c(30, 65)), file.path(dir, "dm.xpt"))

result <- check_study(dir, standard = "SDTMIG")
result$findings
unlink(dir, recursive = TRUE)

Narrow a result to the findings you care about

Description

Saves writing subset() over ⁠$findings⁠ by hand, and - because it returns a result rather than a plain table - what comes back still prints as a readable report.

Usage

filter_findings(
  result,
  triage = NULL,
  dataset = NULL,
  rule = NULL,
  variable = NULL
)

Arguments

Value

A coreval_result holding the matching findings. ⁠$skipped⁠ is left alone: what could not be checked does not become less true because you narrowed what you are looking at.

Examples

ae <- data.frame(
  STUDYID = "S1", DOMAIN = "AE", USUBJID = c("01", "01"),
  AESEQ = c(1, 2), AETERM = c("Headache", "Rash"),
  AESTDTC = c("2024-01-10", "2024-02-30")
)
result <- check_dataset(ae, standard = "SDTMIG")

# Just the things that are definitely wrong:
filter_findings(result, triage = "wrong value")

Controlled Terminology packages coreval bundles

Description

Which version of CDISC's Controlled Terminology a study follows decides whether a value is a legal term, and the answer changes between releases - SEX gained INTERSEX and lost UNDIFFERENTIATED. So coreval never picks one for you: pass the one your study declares as check_study(ct_package =) and this is the list to pick from.

Usage

list_ct_packages(family = NULL)

Arguments

Details

Only what a conformance rule can ask about is bundled - each codelist's submission value and C-code, its terms' submission values and C-codes, and whether it is extensible. Definitions and synonyms are not, which is how 438 MB of CDISC's own caches becomes half a megabyte here.

Value

A character vector of package names, oldest first.

Examples

head(list_ct_packages("sdtm"))

Look up CORE rules

Description

One way in for every question about the rule set: what rules exist, what a particular one checks, and which of them apply to a domain.

Usage

list_rules(
  id = NULL,
  domain = NULL,
  standard = NULL,
  version = NULL,
  use_case = NULL,
  include_deprecated = TRUE,
  substandard = NULL
)

Arguments

Details

The columns are the same whatever you ask for, so the result is safe to filter, join and script against.

Value

A data.table::data.table(), one row per rule: id, the one-line issue it reports, its fuller description, the guidance sentence from the Implementation Guide it enforces, its legacy_ids (the ids the CDISC Conformance Rules and FDA Business Rules give it), standard and standard_version, authority, rule_type, sensitivity, executability, source and status.

The commit the bundled rules came from is on the result as attr(x, "rules_version"); write_findings() records it in every exported file.

What source tells you

Where a rule comes from, and what standing it has there:

• "core_contributor" - a CORE 1.0 SDTMIG or FDA Business Rule, from Verisian's core-contributor repository. These check an SDTM study.

• "published" - ⁠Published/⁠ in CDISC's cdisc-open-rules, fully tested.

• "sendig_draft", "sdtmig_draft", "fda_business_rules_draft" - drafts upstream that already ship test data.

• "deprecated_dir" - superseded by a published replacement. A check does not run it unless you ask, since running both reports the same defect twice.

Examples

# Everything
nrow(list_rules())

# What does the rule the report just named actually check?
list_rules(id = "CORE-000547")$issue

# What applies to AE under SDTMIG 3.4?
nrow(list_rules(domain = "AE", standard = "SDTMIG", version = "3.4"))

# Which snapshot of the rules is this?
attr(list_rules(), "rules_version")

Print a coreval check result as a readable report

Description

Describes each problem in the rule's own words, worst first, with the rows and values that caused it. A whole-study result is grouped by dataset, with a summary first, so you can see where the trouble is before reading detail.

Usage

## S3 method for class 'coreval_result'
print(x, n = 10, rows = 3, guidance = FALSE, ...)

Arguments

Value

x, invisibly.

Examples

ae <- data.frame(
  STUDYID = "S1", DOMAIN = "AE", USUBJID = c("01", "01"),
  AESEQ = c(1, 2), AETERM = c("Headache", "Rash"),
  AESTDTC = c("2024-01-10", "2024-02-30")
)
check_dataset(ae, standard = "SDTMIG")

Read a study into coreval's internal representation

Description

Reads a study folder: SAS transport (.xpt) and SAS (.sas7bdat) files, Dataset-JSON (.json, .ndjson), or a USDM study document, plus a Define-XML when one is there. A CORE test-case ⁠data/⁠ folder (⁠_variables.csv⁠ and one CSV per dataset) is read too. All of them come out in the same internal representation, so the evaluator never has to know which one it got.

Usage

read_study(path, usdm = FALSE)

Arguments

Details

Character columns use "" for blank/missing (never NA) to match how SAS XPT round-trips blanks; numeric columns use NA. Column types are taken from the source (XPT's own types, or ⁠_variables.csv⁠'s declared Char/Num) rather than guessed from the data, so numeric-looking identifiers (e.g. "007") are never silently coerced.

Value

A study object: ⁠list(datasets = <named list of domain -> list(data, meta, label, file_size)>, define, stf, ct = NULL, standard = list(product, version))⁠. define is the parsed Define-XML if one was found in path and the xml2 package is installed, and NULL otherwise; stf likewise holds the study tagging file (stf*.xml), one row per document file tag; ct is always NULL. Each data is a data.table::data.table(); each meta is a data.table with columns variable, label, type; label is the dataset's own label (e.g. "Adverse Events"), or NA if unavailable; file_size is the size in bytes of the file it was read from, which CORE 1.0's dataset size rules compare. standard is the study's declared standard/version (e.g. list(product = "SDTMIG", version = "3-4")), read from a CORE test case's .env file or a USDM document, and both NA for a folder of datasets, which does not declare one.

Examples

dir <- tempfile("coreval_study_")
dir.create(dir)
haven::write_xpt(data.frame(USUBJID = c("1", "2"), AGE = c(30, 65)), file.path(dir, "dm.xpt"))
study <- read_study(dir)
study$datasets$DM$data
unlink(dir, recursive = TRUE)

Summarize a check result in a few lines

Description

The short form of print.coreval_result(), for when you have run a check inside a script, or just want to know whether the last fix helped.

Usage

## S3 method for class 'coreval_result'
summary(object, ...)

Arguments

Value

A one-row data.table::data.table(), invisibly, with the counts it printed - so it can be logged or compared.

Examples

ae <- data.frame(
  STUDYID = "S1", DOMAIN = "AE", USUBJID = c("01", "01"),
  AESEQ = c(1, 2), AETERM = c("Headache", "Rash"),
  AESTDTC = c("2024-01-10", "2024-02-30")
)
summary(check_dataset(ae, standard = "SDTMIG"))

Write conformance findings to a file

Description

Saves the result of check_study() to CSV or Excel, so findings can be shared with people who don't use R, tracked in a spreadsheet, or attached to a data-review document.

Usage

write_findings(result, path, tracking = TRUE)

Arguments

Details

Both tables are always written, never just the findings. A short findings table can mean clean data, or it can mean many rules were skipped, and those two situations look identical if the skipped table is dropped:

• Excel (.xlsx) - four sheets laid out for a person to read:

  • Problems: one row per problem, definite errors first, then the ones touching most records. What is wrong and the rule's number, the variables it names (or the missing ones), and how many records it affects, or ⁠whole dataset⁠.

  • Records: one row per affected record and problem, in the order of Problems, with USUBJID, SEQ and the values, such as AESTDTC = "2024-02-30". A dataset with no subject gives the row instead.

  • ⁠Not checked⁠: the checks that could not run, grouped by reason, with what to do about each, then when and how the check was run.

  • Rules: every rule, and whether it found problems, found none or could not run. A rule that found nothing appears nowhere else.

• CSV (.csv) - the tables as they are in R, for reading back into a program. Findings go to path; the others go to sibling files with a suffix before the extension, so issues.csv gives issues_skipped.csv, issues_about.csv and issues_summary.csv (one row per problem).

An existing file at path, or at a sibling path, is replaced.

In CSV files, a text value that begins with =, +, - or @ and is not a plain number is written with a leading ⁠'⁠. Findings quote the submission data, and a spreadsheet program opening the CSV would otherwise run such a value as a formula. Excel files store every value as text and need no such change. CSV files are written as UTF-8 with a byte order mark, which Excel needs to show non-English text correctly.

Excel output needs the writexl package. It is a Suggests, so if it isn't installed you get a clear message telling you to install it or use .csv instead, rather than a failure part-way through writing.

Value

The paths actually written, invisibly. One element for Excel (a single workbook). For CSV, one path per file written: the findings, plus siblings for skipped, about and summary, plus one for truncated when any rule flagged more records than were kept.

Columns for tracking

Empty columns are added for you to fill in by hand once the file is open: Status and Notes on Problems and Notes on Records in Excel, Status, Owner and Notes in CSV. They exist so a finding you have looked at and decided not to act on ("expected, see protocol deviation log") can be recorded next to the finding itself, rather than in a separate document nobody reads.

Examples

dir <- tempfile("coreval_study_")
dir.create(dir)
haven::write_xpt(data.frame(USUBJID = c("1", "2"), AGE = c(30, 65)), file.path(dir, "dm.xpt"))
study <- read_study(dir)
result <- check_study(study, standard = "SDTMIG")

out <- file.path(dir, "findings.csv")
written <- write_findings(result, out)
basename(written)

unlink(dir, recursive = TRUE)