In the kingdom of the blind, could double-blind be king?

During the Joint Math Meetings, I found myself discussing with a large and diverse group of people whether double-blind refereeing would be a good idea for math publications. This is not standard practice, although the referee usually does have anonymity in the process. This conversation began because a recent study (to be published soon in MAA FOCUS in an article written by Francis Su and Betty Mayfield) showed that women in mathematics are underrepresented when it comes to publishing. By this I mean that given how many women are mathematicians (and active members of professional societies like AMS or MAA), only a small percentage of those (I forget the number) are publishing in the big journals. As you can tell I have very few details (this is a study that hasn’t been published yet, after all), but what I was interested in doing in this post was discussing whether double-blind refereeing could solve the problem of representation of women and minorities in journals, and whether this under-representation is really caused by the refereeing process.  What I’m most interested in, actually, is opening this topic for discussion.

The first thing I think about is whether this problem (i.e. that the ratio of women publishing to the women who are mathematicians is small) just has to do with general issues of women in mathematics. By this I mean that there is definitely something stopping women from pursuing mathematics as a career, whether it’s societal pressures, preference, stereotype threat, or being too pretty. Maybe some of these underlying issues also affect women who are well into mathematician-hood? It wouldn’t be hard to imagine (from my own impostor syndrome) that women mathematicians are just not as likely to submit papers to journals as their male counterparts. Of course, I am speculating again, because I have no data. But if this is the case (women just don’t submit as much) then using double-blind refereeing is not really addressing the problem. Although if it’s stereotype threat (women or minorities feel they are representing a whole group of people), then maybe knowing that their gender is not going to factor into the reviewing process might make women more likely to submit papers.

The problem is to figure out if there is some hidden gender bias that is affecting how papers are being evaluated. Luckily for me and many of my contemporaries, I have not seen much open sexism in mathematics. But biases might be involuntary. In this case, it would definitely be beneficial to have a double-blind system, and thus eliminate any danger of biases that people are not even aware of.

I can think of other biases, too. Biases depending on institution or even your Ph.D. advisor (this is easy to look up given the author’s name, thanks to the math genealogy project). I’m sure that people’s opinion of a paper would change if it was Jane Doe from Big Famous School or Jane Doe from Small Liberal Arts College or even Fulana De Tal from Some South American University.

In fact, I can see very few drawbacks to a double-blind system. So I went online and found an editorial on double-blind refereeing by Gillian Raab and Uma Moorthy in the Journal of the Royal Statistical Society. In the “advantages” section they list many of the things I have mentioned. In the “disadvantages” they mention things I hadn’t thought about, like “many referees feel that they provide extra advice to new researchers” and the fact that “it may be advantageous, particularly for new researchers, to publicize their pre-publication work through the refereeing process.” They also discuss an experiment they conducted, and their conclusions were that “acceptance rates are lower and referees more critical under a double-blind system and the authors from top rated and low rated institutions are little affected by double-blind refereeing, but that double-blind refereeing disadvantages authors from middle ranked institutions as well as foreign authors and those from non-academic institutions.” As this was an editorial it didn’t have all the details of the study, but it was definitely interesting to see results that went against my initial intuition (their experiment was conducted through an Economics journal, I don’t know if that matters).

On a final note, I have mentioned this conundrum to friends from other disciplines, and they were quite appalled that double-blind refereeing wasn’t the norm. Some even said that double-blind journal publications count more heavily for tenure, given that it is (as stated above) more difficult to get a publication in such a journal.

What are your thoughts, dear readers? Do you think switching math journals to double-blind refereeing could be a good idea? How about just moving the name and institution to the end of the paper? Do you think this would add unreasonable burdens on editors? Have you refereed a paper using a double-blind system? I look forward to reading your comments!

This entry was posted in Uncategorized. Bookmark the permalink.

yyy8 Responses to In the kingdom of the blind, could double-blind be king?

  1. Ursula says:

    My impression is that the types of institutions where women work also come into play: people who aren’t at R1 institutions feel less pressure to publish in the “big” journals.

    Also, wouldn’t double-blind refereeing be hard to enforce when so many math papers are posted as preprints on the arxiv?

  2. Maria says:

    I do like the idea of double-blind reviews, and not only for papers but also for research proposals and grants. A way to minimize unintended biases in evaluation is exactly a double-blind review: that can minimize biases in (1) women vs men (2) top-ranked institution vs law-ranked institution (3) senior scientists vs junior scientists (4) well-konwn scientists vs unknown scientists. The fact that “acceptance rates are lower and referees more critical under a double-blind system” is actually only a plus for younger scientists. In cases of grants and research proposals it is true that ““many referees feel that they provide extra advice to new researchers”, but I found this argument less compelling when applied to papers.
    Blind auditions are a common procedure in music; and looking at the statistics, it is a very successful method. Why not also in math ?

  3. Michelle says:

    I am fairly new at this refereeing game, but I would say I could have correctly identified the author (or at least one author on multi-author papers) on all but one or two papers I’ve been asked to referee, even if it was double-blind. It’s possible that I’m being asked to referee quite narrowly right now, since I don’t have a long track record in very many areas.

    I also have pretty good educated guesses about my referees sometimes, though less often. I’ve never followed up to see if I’m correct.

    I guess what I’m saying is: When you work in a small-ish subspecialty, I wonder how “real” double blind refereeing would be? And this is (it seems to me) nearly impossible for grant proposals, since part of what you do is try to convince the panel that you’re the right person for the job. How do you do that and still remain anonymous? You have to describe your previous work!

    In other news: Luckily for me and many of my contemporaries, I have not seen much open sexism in mathematics. But biases might be involuntary.

    Maybe not involuntary, but subconscious?

    Anyway, luckily for you indeed. I’ll tell you some stories over drinks when you’re in Providence!

  4. Ray says:

    If this is true, women with non-identifiably female names should not suffer much underrepresentation. That would be an interesting check on the existing data.

  5. James F. Epperson says:

    I think a double blind process has merit, and not just for gender bias issues, and it should be used for grant proposals as well.

  6. Chris Sinclair says:

    I like the idea of double-blind refereeing in principle. However, I always post my papers on the arXiv a couple of weeks before submitting (this sometimes produces interesting feedback or comments that can be included in the submitted version), and I look daily at all the newly posted titles in the mathematics and mathematical physics sections of arXiv. Thus, anybody can discover easily the authorship of my submissions, and I theirs, assuming they post to the arXiv before they submit.

    Moreover, for most of the referee requests made of me, I have known at least one of the authors, and would have been able to guess their identity even if it was not presented to me. There have been a minority of requests for authors unknown to me, and for these a double-blind system may work.

    Finally, while it can be irksome that senior scientists often get their papers or grants rubber-stamped, it is also often the case that these individuals have made profound contributions to the field and their opinion of what is interesting or important may deserve a certain reverence. I’m not sure I necessarily buy this argument, being non-senior myself, but the case could be made …

    As for the paucity of women in mathematics departments, I don’t know the underlying cause, nor the solution to this problem. But I would very much like to know a solution. I feel like our students (grad students in particular) are wanting for role-models. -=C

  7. Emily Sprague says:

    Before I ventured into mathematics I was a symphony orchestra player. The American Federation of Musicians decreed half a century or more ago, that most auditions be conducted behind a screen to ensure that racial or gender bias did not enter into the selection process. Union stewards to this day police this process. Now an experienced musician may be able to guess whose student is playing behind that screen, perhaps something about the instrument being used, and almost surely which commercial recording was used to study the required excerpts, and even sometimes who the actual player might be. Nevertheless much is gained from the anonymity, not the least of which is the reminder to the reviewer that focus should be directed to the worthiness of the material at hand rather than to the pedigree of the author.

    The best way to tell whether blind decisions improve the number of women who get published is simply to institute a practice of conducting blind reviewing.

  8. Scott Sallberg says:

    I have a few points that are more or less relevant to your discussion:
    - several books by Thomas Sowell deal with the issues that it is rare for subgroups of people to be equally represented in most any activity; wish I could give you a title off the top of my head, but this comes up all the time in many of his books on race and culture
    - many if not most people that are really good at what they’re doing whether it is music, dance, math, or martial arts can identify people and their teachers/school without too much trouble, so this business of double blind refereeing is simply, in my opinion, to passifiy people who feel that they or their subgroup might have been wronged or passed over some how
    - but maybe it’s real, it’s happened to me, I know cuz they told me, strange, I know, but going around looking for possible wrongs based on assumed statistical models isn’t useful for much anything but catching fraudulent charges on credit cards

Leave a Reply

Your email address will not be published. Required fields are marked *

HTML tags are not allowed.