This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Vecchio, R. P. Search for Related Content Social Bookmarking                   What's this?

Journal Reviewer Ratings: Issues of Particularistic Bias, Agreement, and Predictive Validity Within the Manuscript Review Process

University of Notre Dame

Reviewer evaluations and recommendations for 853 manuscript submissions, over a span of 4 years, are analyzed for evidence of particularistic bias, reviewer agreement, and predictive validity for forecasting a published manuscript's citation impact. Attributes of the submitters, their affiliated institutions, and the reviewers have little consistent association with reviewers' recommendations or editorial decision outcomes. Furthermore, reviewers' recommendations demonstrate a reasonable degree of agreement. However, neither reviewers' evaluative ratings across five dimensions nor publication recommendations can predict the number of citations that a published article subsequently receives. Strengths and limitations of various features of the manuscript review process, as well as the importance of monitoring the process for particularistic biases and evidence of predictive validity, are discussed.

Key Words: citation analysis • journal reviewers • research impact

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?