In applied statistics and machine learning, the "gold standards" used for
training are often biased and almost always noisy. Dawid and Skene's
justifiably popular crowdsourcing model adjusts for rater (coder, annotator)
sensitivity and specificity, but fails to capture distributional properties of
rating data gathered for training, which in turn biases training. In this
study, we introduce a general purpose measurement-error model with which we can
infer consensus categories by adding item-level effects for difficulty,
discriminativeness, and guessability. We further show how to constrain the
bimodal posterior of these models to avoid (or if necessary, allow) adversarial
raters. We validate our model's goodness of fit with posterior predictive
checks, the Bayesian analogue of $\chi^2$ tests. Dawid and Skene's model is
rejected by goodness of fit tests, whereas our new model, which adjusts for
item heterogeneity, is not rejected. We illustrate our new model with two
well-studied data sets, binary rating data for caries in dental X-rays and
implication in natural language.

在应用统计学和机器学习领域，我们介绍了一种通用的测量误差模型，通过对困难度、区分度和猜测度添加项目级效应，用于推断共识类别，并通过后验预测检验验证了模型的拟合优度，该模型调整了项目异质性。