{"id":3471,"date":"2018-03-05T23:04:36","date_gmt":"2018-03-06T04:04:36","guid":{"rendered":"http:\/\/blogs.ams.org\/blogonmathblogs\/?p=3471"},"modified":"2018-03-05T23:04:36","modified_gmt":"2018-03-06T04:04:36","slug":"blind-review-review","status":"publish","type":"post","link":"https:\/\/blogs.ams.org\/blogonmathblogs\/2018\/03\/05\/blind-review-review\/","title":{"rendered":"Blind Review Review"},"content":{"rendered":"

Theoretical computer scientists have been talking about double blind peer review, and it\u2019s an interesting discussion. The current incarnation of this discussion started when Rasmus Pagh and Suresh Venkatasubramanian used a double blind refereeing process for submissions to the ALENEX18 conference they co-chaired. Venkatasubramanian posted about their motivations and how they pulled it off in two posts on his blog, The Geomblog<\/span><\/a> (post 1<\/span><\/a>, post 2<\/span><\/a>, post 3<\/span><\/a>).<\/span><\/p>\n

\"\"<\/a>

Artist’s rendition of blinded peer review. (Justice, from the Cardinal Virtues by Nicolaes de Bruyn) Credit: Public domain, via the Metropolitan Museum of Art<\/p><\/div>\n

Why double-blind? First, it\u2019s the standard for computer science conferences outside of the theory subdiscipline. More importantly, many people worry that single-blind peer review, where the reviewer knows the identity of the author, leads to some objectionable outcomes based on implicit and explicit biases. More famous authors or authors from more prominent institutions may have their work reviewed more favorably, and more broadly, the bias in favor of these authors combined with other biases reviewers have can continue systemic bias against women and other groups that are underrepresented in the field.<\/span><\/p>\n

Obviously, a major change in the paper submission system is not without controversy. The discussion has continued in posts by Boaz Barak<\/span><\/a>, Michael Mitzenmacher<\/span><\/a>, Omer Reingold<\/span><\/a>, and Lance Fortnow<\/span><\/a>. In general, the conversation I have seen has been civil and thoughtful. In one post<\/a>, Venkatasubramanian writes,<\/span><\/p>\n

\n

First up, I think it’s gratifying to see that the the basic premise: \u2018single blind review has the potential for bias, especially with respect to institutional status, gender and other signifiers of in\/out groups\u2019 is granted at this point. There was a time in the not-so-distant past that I wouldn’t be able to even establish this baseline in conversations that I’d have.<\/span><\/p>\n

\u201cThe argument therefore has moved to one of tradeoffs: does the installation of DB review introduce other kinds of harm while mitigating harms due to bias?<\/span><\/p>\n<\/blockquote>\n

A few math journals\u2014mostly in math education and undergraduate research, as far as I can tell\u2014do use double-blind peer review. But it is not standard. One of the biggest barriers to double blind reviewing in computer science, physics, or math is the fact that so many preprints are posted on arxiv or authors\u2019 websites before they are submitted, making it that much more difficult for a reviewer to avoid knowing who wrote the paper. (Venkatasubramanian writes about how they dealt with that problem in his posts; one point he makes is that double-blinding the process won\u2019t necessarily prevent reviewers from being able to determine authors eventually, but it could prevent some knee-jerk reactions. He also points to a post by Regina Barzilay<\/span><\/a> that delves into the issue in more depth) In some fairly narrow subdisciplines, there are few enough researchers that even without seeing the paper online, others in the field will be able to tell who wrote it anyway.<\/span><\/p>\n

While societies and individual humans in them have biases, there will be no way to completely eliminate these biases when people (or algorithms<\/span><\/a>) make decisions about paper and conference submissions. It is important for academics to look at the advantages and disadvantages of different strategies to mitigate the effects of bias. I am looking forward to seeing how this conversation evolves.<\/span><\/p>\n

<\/div>","protected":false},"excerpt":{"rendered":"

Theoretical computer scientists have been talking about double blind peer review, and it\u2019s an interesting discussion. The current incarnation of this discussion started when Rasmus Pagh and Suresh Venkatasubramanian used a double blind refereeing process for submissions to the ALENEX18 … Continue reading →<\/span><\/a><\/p>\n

<\/div>\n","protected":false},"author":61,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[18],"tags":[779],"class_list":["post-3471","post","type-post","status-publish","format-standard","hentry","category-publishing-in-math","tag-peer-review"],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p3tW3N-TZ","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/blogs.ams.org\/blogonmathblogs\/wp-json\/wp\/v2\/posts\/3471","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.ams.org\/blogonmathblogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.ams.org\/blogonmathblogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.ams.org\/blogonmathblogs\/wp-json\/wp\/v2\/users\/61"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.ams.org\/blogonmathblogs\/wp-json\/wp\/v2\/comments?post=3471"}],"version-history":[{"count":2,"href":"https:\/\/blogs.ams.org\/blogonmathblogs\/wp-json\/wp\/v2\/posts\/3471\/revisions"}],"predecessor-version":[{"id":3474,"href":"https:\/\/blogs.ams.org\/blogonmathblogs\/wp-json\/wp\/v2\/posts\/3471\/revisions\/3474"}],"wp:attachment":[{"href":"https:\/\/blogs.ams.org\/blogonmathblogs\/wp-json\/wp\/v2\/media?parent=3471"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.ams.org\/blogonmathblogs\/wp-json\/wp\/v2\/categories?post=3471"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.ams.org\/blogonmathblogs\/wp-json\/wp\/v2\/tags?post=3471"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}