Analytical method for detecting outlier evaluators

by   Yujie Wu, et al.

Epidemiologic and medical studies often rely on evaluators to obtain measurements of exposures or outcomes for study participants, and valid estimates of associations depends on the quality of data. Even though statistical methods have been proposed to adjust for measurement errors, they often rely on unverifiable assumptions and could lead to biased estimates if those assumptions are violated. Therefore, methods for detecting potential `outlier' evaluators are needed to improve data quality during data collection stage. In this paper, we propose a two-stage algorithm to detect `outlier' evaluators whose evaluation results tend to be higher or lower than their counterparts. In the first stage, evaluators' effects are obtained by fitting a regression model. In the second stage, hypothesis tests are performed to detect `outlier' evaluators, where we consider both the power of each hypothesis test and the false discovery rate (FDR) among all tests. We conduct an extensive simulation study to evaluate the proposed method, and illustrate the method by detecting potential `outlier' audiologists in the data collection stage for the Audiology Assessment Arm of the Conservation of Hearing Study, an epidemiologic study for examining risk factors of hearing loss in the Nurses' Health Study II. Our simulation study shows that our method not only can detect true `outlier' evaluators, but also is less likely to falsely reject true `normal' evaluators. Our two-stage `outlier' detection algorithm is a flexible approach that can effectively detect `outlier' evaluators, and thus data quality can be improved during data collection stage.


page 1

page 2

page 3

page 4


Hypothesis Testing for Detecting Outlier Evaluators

In epidemiological studies, very often, evaluators obtain measurements o...

Outlier Detection for Improved Data Quality and Diversity in Dialog Systems

In a corpus of data, outliers are either errors: mistakes in the data th...

RODD: Robust Outlier Detection in Data Cubes

Data cubes are multidimensional databases, often built from several sepa...

Multiple outlier detection tests for parametric models

We propose a simple multiple outlier identification method for parametri...

Outcome measurement error correction for survival analyses with multiple failure types: application to hearing loss studies

In epidemiological studies, participants' disease status is often collec...

Valid Inference Corrected for Outlier Removal

Ordinary least square (OLS) estimation of a linear regression model is w...

Detecting outlying demand in multi-leg bookings for transportation networks

Network effects complicate demand forecasting in general, and outlier de...

Please sign up or login with your details

Forgot password? Click here to reset