Issues with criteria to create blacklists: An epidemiological approach

Screening criteria are a vital part of society, medicine and publishing. In this paper, a new framework, based on epidemiological principles, is developed to assess the effectiveness of criteria that are used to detect predatory behavior, or to assess whether a journal or publisher is predatory, and create blacklists. Applying epidemiological measures such as specificity, sensitivity, prevalence rates, the likelihood ratio, posterior and prior probabilities and odds, as well as Bayesian analysis, we elaborate on the false discovery rate and work towards assessing the likelihood that an entity is in fact predatory when screening criteria are used. We applied the framework to three different prevalence cases: a low prevalence rate where all journals are screened for predatory behavior assuming Jeffrey Beall's criteria are used; a higher rate when only open access journals are assessed; the highest rate where only Walt Crawford grayOA journals were screened for deceptive publishing practices. In all cases, we found a very high false discovery rate even when using reasonable values for the sensitivity and specificity rate for Beall's screening criteria.