Glossary

To improve the readability of the code, we provide the glossary to serve as an educational document to grow people’s understanding of the graphical approach to multiple comparison procedures. Most terms are inputs or outputs of exported functions and some are used only in the internal code. If any definition could be clarified or improved, please submit an issue to the GitHub repository.

Entity	Definition	Aliases	Variable(s)
Graph	A set of nodes and directed edges representing a graphical multiple comparison procedure. Each node correspond to hypothesis and each edge corresponds to transition.		`graph`
Delete a hypothesis	Remove a hypothesis from a graph, and update the graph according to algorithm 1 of Bretz et al. (2011). This is an operation on a graph.		`updated_graph`
Reject a hypothesis	Under a given graph, testing strategy, and `alpha`, a hypothesis is rejected if its p-value is sufficiently small, which is determined by the graphical multiple comparison procedure.
Hypothesis	A node with weight in a graph. Each node represents a null hypothesis, associated with a hypothesis weights. The corresponding significance level is the weight times `alpha`.	weight, hypothesis weight	`hypotheses`
	Terms associated with hypothesis get their variable names: hypothesis name, and number of hypotheses.		`hyp_names`, `num_hyps`
Transition	A directed edge with weight in a graph. Each edge defines the proportion of the hypothesis weight to be propagated from the origin hypothesis to the end hypothesis, when the origin is rejected.	edge, transition weight	`transitions`
Intersection hypothesis	An intersection hypothesis is an intersection of multiple null hypotheses, which means that all associated null hypotheses are true. Plural often implies all intersections of all subsets of hypotheses.	intersection, sub-graph(s), closure	`intersections`
Weighting strategy	The set of all intersections and their hypothesis weights according to Algorithm 1 in Bretz et al. (2011).	intersection weights, closure weights	`weighting_strategy`
Adjusted weight	The hypothesis weight, adjusted according to a multiple comparison procedure: Bonferroni: No change from the hypothesis weight from a graph Simes: Sum of hypothesis weights for hypotheses with smaller p-values in an intersection hypothesis Parametric: Multiply Bonferroni weight by c-value, which is calculated based on the joint distribution of test statistics of the associated intersection hypothesis A hypothesis may be rejected if its p-value is less than or equal to its adjusted weight times `alpha`.		`adjusted_weights`
P-value	A p-value before multiplicity adjustment. Also could be called as an unadjusted p-value or a raw p-value.		`p`
Ordered p-value	A p-value sorted from the smallest to the largest. They are mainly used to perform Simes tests.		`ordered_p`
Adjusted p-value	A p-value that has been adjusted according to a multiple comparison procedure. A hypothesis may be rejected if its adjusted p-value is less than or equal to `alpha`.		`adjusted_p`
Significance level	A threshold chosen to test a null hypothesis, which may be rejected if its p-value is less than or equal to its significance level. The overall significance level to test all hypotheses is `alpha`.		`alpha`
Test type	A specification of which testing type to use for an intersection hypothesis - Bonferroni, Simes, and parametric are currently supported.	tests	`test_types`
Test group	A partition of all null hypotheses in a graph specifying which hypotheses should be tested together using a test type.	groups	`groups`, `test_groups`
Testing strategy	Test types and test groups combined with a graph.	multiple comparison procedure
Marginal power	The power to reject a null hypothesis at the significance level `alpha` (without multiplicity adjustment).		`marginal_power`
Correlation matrix	Specification of correlations between test statistics for hypotheses. The correlation for testing `test_corr` is used to perform parametric tests. The correlation for simulation `test_sim` is used to simulate p-values from the alternative hypotheses for to assess power, under assumptions.		`corr`, `test_corr`, `sim_corr`
Power	With a given graph, testing strategy, `alpha`, and the underlying distribution of test statistics under the alternative hypotheses, the estimated likelihood that a particular success criterion is met.	probability of achieving success criterion	`power_*`
Success	Specification of the success criterion, which could be a combination of null hypotheses. Being success means that the combination of null hypotheses has been rejected.		`sim_success`

Graph

A set of nodes and directed edges representing a graphical multiple comparison procedure. Each node correspond to hypothesis and each edge corresponds to transition.

graph

Delete a hypothesis

Remove a hypothesis from a graph, and update the graph according to algorithm 1 of Bretz et al. (2011). This is an operation on a graph.

updated_graph

Reject a hypothesis

Under a given graph, testing strategy, and alpha, a hypothesis is rejected if its p-value is sufficiently small, which is determined by the graphical multiple comparison procedure.

Hypothesis

A node with weight in a graph. Each node represents a null hypothesis, associated with a hypothesis weights. The corresponding significance level is the weight times alpha.

weight, hypothesis weight

hypotheses

Terms associated with hypothesis get their variable names: hypothesis name, and number of hypotheses.

hyp_names, num_hyps

Transition

A directed edge with weight in a graph. Each edge defines the proportion of the hypothesis weight to be propagated from the origin hypothesis to the end hypothesis, when the origin is rejected.

edge, transition weight

transitions

Intersection hypothesis

An intersection hypothesis is an intersection of multiple null hypotheses, which means that all associated null hypotheses are true. Plural often implies all intersections of all subsets of hypotheses.

intersection, sub-graph(s), closure

intersections

Weighting strategy

The set of all intersections and their hypothesis weights according to Algorithm 1 in Bretz et al. (2011).

intersection weights, closure weights

weighting_strategy

Adjusted weight

The hypothesis weight, adjusted according to a multiple comparison procedure:

Bonferroni: No change from the hypothesis weight from a graph
Simes: Sum of hypothesis weights for hypotheses with smaller p-values in an intersection hypothesis
Parametric: Multiply Bonferroni weight by c-value, which is calculated based on the joint distribution of test statistics of the associated intersection hypothesis A hypothesis may be rejected if its p-value is less than or equal to its adjusted weight times alpha.

adjusted_weights

P-value

A p-value before multiplicity adjustment. Also could be called as an unadjusted p-value or a raw p-value.

p

Ordered p-value

A p-value sorted from the smallest to the largest. They are mainly used to perform Simes tests.

ordered_p

Adjusted p-value

A p-value that has been adjusted according to a multiple comparison procedure. A hypothesis may be rejected if its adjusted p-value is less than or equal to alpha.

adjusted_p

Significance level

A threshold chosen to test a null hypothesis, which may be rejected if its p-value is less than or equal to its significance level. The overall significance level to test all hypotheses is alpha.

alpha

Test type

A specification of which testing type to use for an intersection hypothesis - Bonferroni, Simes, and parametric are currently supported.

tests

test_types

Test group

A partition of all null hypotheses in a graph specifying which hypotheses should be tested together using a test type.

groups

groups, test_groups

Testing strategy

Test types and test groups combined with a graph.

multiple comparison procedure

Marginal power

The power to reject a null hypothesis at the significance level alpha (without multiplicity adjustment).

marginal_power

Correlation matrix

Specification of correlations between test statistics for hypotheses. The correlation for testing test_corr is used to perform parametric tests. The correlation for simulation test_sim is used to simulate p-values from the alternative hypotheses for to assess power, under assumptions.

corr, test_corr, sim_corr

Power

With a given graph, testing strategy, alpha, and the underlying distribution of test statistics under the alternative hypotheses, the estimated likelihood that a particular success criterion is met.

probability of achieving success criterion

power_*

Success

Specification of the success criterion, which could be a combination of null hypotheses. Being success means that the combination of null hypotheses has been rejected.

sim_success

Bretz, Frank, Martin Posch, Ekkehard Glimm, Florian Klinglmueller, Willi Maurer, and Kornelius Rohmeyer. 2011. “Graphical Approaches for Multiple Comparison Procedures Using Weighted Bonferroni, Simes, or Parametric Tests.” Biometrical Journal 53 (6): 894–913. https://onlinelibrary.wiley.com/doi/10.1002/bimj.201000239.