{"id":1825,"date":"2019-05-31T18:20:43","date_gmt":"2019-05-31T22:20:43","guid":{"rendered":"http:\/\/blogs.ams.org\/inclusionexclusion\/?p=1825"},"modified":"2019-05-31T18:20:43","modified_gmt":"2019-05-31T22:20:43","slug":"set-theory-on-reading-student-evaluations-of-teaching","status":"publish","type":"post","link":"https:\/\/blogs.ams.org\/inclusionexclusion\/2019\/05\/31\/set-theory-on-reading-student-evaluations-of-teaching\/","title":{"rendered":"SET Theory: On reading student evaluations of teaching"},"content":{"rendered":"

\"\"The school year is over, commencement has come and gone, grades are in, and the summer lies ahead of us, with all of its promise of research or rest or travel, and only one potential obstacle looms in the horizon \u2013 the dreaded teaching evaluations. We have all been traumatized and scarred by teaching evals at some point in our lives. If you\u2019re in a privileged position like mine, with tenure, chair of your department, and no promotion coming any time soon (I am only eligible to go up for promotion in like three years), you can avoid the trauma using one simple trick: just don\u2019t look, until you have to. \u00a0You know no one else will, either. But this is not the case for early career tenure-track faculty, postdocs and other visiting faculty, or for \u201ctenure exempt\u201d faculty (Linse, 2017). The fact is that so-called Student Evaluations of Teaching (SET) are still heavily used for reappointment and promotion, and sometimes requested by hiring committees. But another fact is that this data could potentially be useful even to senior faculty \u2013 for our own teaching but also as colleagues and mentors to these more vulnerable faculty. Last week, I attended a workshop at my institution run by my colleague in Chemistry Dr. Lynn Mandletort<\/a>, designed to help us make the most of these teaching evaluations. In this post, I summarize some of my main takeaways from this workshop, and some suggestions and resources for further reading.<\/p>\n

<\/p>\n

First, a disclaimer:<\/strong> This is NOT a post about how teaching evaluations are biased and awful \u2013 in my personal opinion they are, and there is a lot of writing on this very issue you can find elsewhere. For example, see this great post by Anna Haensch on the AMS Blog on Math Blogs<\/a> for a few resources. Also, after doing some of the reading recommended by Lynn, it is possible that evals are not as biased and awful as I thought, but more on that later. This is a post about what we do with the measures we already have.<\/strong><\/p>\n

But why is this on the i\/e blog? <\/strong>Good question, dear reader. Like I said above, there is a lot of research showing that SETs are biased, especially against women professors and professors of color. This is so well known by now, that there are even comedic pieces about it, see this McSweeney\u2019s piece for a \u201cfun\u201d drinking game on this topic<\/a>\u2026 Linse, in her article \u201cInterpreting and using student ratings data\u201d<\/a>, claims that much of this research is overblown and maybe gives too much weight to what she calls \u201crare\u201d data or outliers. I am not a sociologist nor am I an expert in these matters. But the fact is that even if not making a statistically significant difference to our ratings, the impact on women faculty, faculty of color, and other marginalized groups seen as \u201coutsiders\u201d to academia, is not just about statistical significance. This is emotionally taxing, on top of other emotionally taxing things that these faculty already have to do. I also wonder about how a person’s teaching\u00a0 is affected when they are constantly exposed to micro and macro aggressions from other faculty and students. At any rate, if you are faculty of color or woman faculty, \u201cwell, your low ratings might not ONLY be due to your gender\/ethnicity\/race\u201d is not really all that helpful.<\/p>\n

SO, that said, after being through that workshop last week, I realized that there were ways to digest the data that were more useful and less traumatic to me, and that I should share this with the i\/e community. The bonus is that this should be helpful to everyone, not just mathematics instructors from minoritized groups.<\/p>\n

Finally, the workshop!<\/strong><\/p>\n

Lynn started by emphasizing a new term\/language for what we have so far called SETs \u2013 instead, we should call them Student Ratings of Instruction (SRIs). She gave us three guiding principles for what the data are (and are not), similar to those in (Linse, 2017):<\/p>\n