Should SETAC Journals Move to a Double-blind Peer Review Process?
Chris Mebane, Globe Guest Contributor
Just how blind should peer reviewers be? Should information on who did the studies, where it was conducted, and who paid for it be hidden from peer reviewers? Peer review is one of the most important factors distinguishing published journal articles from gray literature. The peer review imprimatur signals to the readers that the material received impartial, expert reviews prior to being published and is likely to be credible. Likewise, earning publication in a credible journal is important for building their reputation, obtaining support, career development and influence within one’s field for many if not most SETAC scientists. But, what if peer reviews were perceived as not being impartial, and instead, the playing field was tipped in favor of some authors and against others?
The peer review process for the SETAC journals has been a subject of lively debate at Publications Advisory Committee (PAC) meetings and editorial board meetings of the society’s two journals, Environmental Toxicology and Chemistry (ET&C) and Integrated Environmental Assessment and Management (IEAM). Like the vast majority of scientific journals in the environmental sciences, ET&C and IEAM use a single-blind peer review process. The process is “single-blind” because the authors don’t know who the reviewers are, but the reviewers know who the authors are, and the identity of the handling editor is known to all. The single-blind review process is intended to allow frank assessments of manuscripts without the reviewers having to worry about retribution when roles flip. Peer reviewers and editors are practicing scientists themselves, and today’s critical reviewer will be tomorrow’s author hoping to get their manuscript blessed by the reviewers.
But, how is an author to be assured that their submission will be evaluated solely on the quality of the science and the presentation rather than preconceptions that reviewers might hold about who did the science? Do papers with “big name” scientists as senior authors get an easier go than unfamiliar authors? Do studies conducted at prestigious research centers get treated with less skepticism than those from say, a backwater teaching university with more limited resources? Might details like nationality, whether they are native English speakers or gender at least unconsciously affect reviewers’ views toward the work? For instance, overall acceptance rates of manuscripts submitted to ET&C are higher for submissions from North America and Europe than from Latin America, Asia-Pacific or African authors. Does this solely reflect the quality of the science or might implicit bias by reviewers play some role? Why not just obscure the authors’ identity from reviewers to be sure it doesn’t?
These concerns of implicit bias have led to many calls across scientific communities to move to a double-blind peer review of journals. For the double-blind review process, the identities of the authors, their organization and other identifying details, such as where the study was conducted are masked from the reviewers. Usually, this is done by requiring authors to submit two versions of their manuscripts, a regular version seen only by the editors, and a scrubbed version for reviewers that redacts all potentially identifying information.
This double-blind proposal has led to much debate. The counter-arguments against going to double-blind peer review processes are practical: It won’t work as intended, and the hassle will turn off authors. Attempts to mask identities often don’t work. Authors usually cite their previous related work, which makes them no longer anonymous; certain topics only have a handful of laboratories or teams actively working on them, and trying to obscure the laboratory or study site is either ineffective or obscures important details from reviewers. The increased work for the editorial office to check for compliance could be costly and slow down the publishing process. Authors may balk at having to go to the trouble of preparing and reconciling redacted and full versions and may submit elsewhere. SETAC scientists have a variety of outlets available to publish their works. If publishing in SETAC journals becomes a greater hassle than publishing in non-society, for-profit journals, is it worth the trouble? Shouldn’t we be striving for greater transparency in science and publishing, not less? Finally, if there is an actual problem rather than just a hypothesized problem, look first to the editors. Reviewers advise, but editors decide. It’s not practical to make the editor blind, and a biased editor could direct an article to allies or assassins to review. For such an editor, a double-blind review policy would only provide a falsely reassuring veneer. Thus, to skeptics, double-blind reviews sound like a disruptive and dubious solution for a problem that isn’t even certain to exist.
From this raucous-caucus of argument and counter-argument, a compromise suggestion arose: Moving to a “soft mandatory double-blind” review policy. As explained by the editors of a journal giving it a try (Global Ecology and Biogeography[GEB]), this is intended to pursue the perceived benefits of double-blind reviews, while trying not to overburden authors and the editorial office. According to the GEB editors,“…‘soft’ is recognizing that it is nearly impossible to create 100% effective blinding, and we are not even going to try. Policing the degree of blinding would involve somebody reading the entire paper to remove any hints in citation patterns, etc. We feel that our approach is capturing 90% of the benefit without burdening authors or the peer-review system with unwarranted (and probably inefficient) effort. Authors will now submit two title pages, one with author names and institutions and one without (i.e., only title and keywords).”
A theme from the discussions that this correspondent attended was although the PAC and the journal editors-in-chiefs are giving serious consideration to the journals’ peer review processes, they are not eager to get ahead of the wishes of Society authors and readers of the SETAC journals. The journals belong to SETAC, and SETAC belongs to its members, so major shifts in publication policy should engage the SETAC community. As part of that discussion, watch for an IEAM Blog post with more of an in-depth treatment of the issues, and more importantly, an opportunity for authors and readers to make their views known.
Author’s contact information: globe@setac.org
Return to the Globe
|
