decisionfacets: Decision Accuracy and Consistency for Rater-Mediated Exams
Answers "would this candidate have passed with a different set
of raters?" for rater-mediated exams such as oral examinations,
objective structured clinical examinations and essay scoring. Builds on
the many-facet extension of the rating scale model (Andrich, 1978,
<doi:10.1007/BF02293814>) to compute counterfactual pass probabilities
under the observed, an average-severity and a random rater panel, under
an explicit decision rule (raw total, fair average or measure), and
splits expected misclassification into measurement error and rater
assignment, extending item response theory classification accuracy
(Lee, 2010, <doi:10.1111/j.1745-3984.2009.00096.x>) to rater effects.
Supports panels in which every rater scores every task and designs in
which each case is scored by a different examiner, with rating
categories or continuous scores such as 0-100 (through a linear
many-facet model). Models can be fitted with 'TAM' or built-in
estimators.
Documentation:
Downloads:
Linking:
Please use the canonical form
https://CRAN.R-project.org/package=decisionfacets
to link to this page.
[aut,
cre, cph]