Evaluating Evaluations

Last month a friend in the history department passed along a notice from the American Historical Association entitled “AHA Signs onto ASA Statement on Teaching Evaluations.” This ASA is the American Sociological Association, and their statement is a devastating takedown of using Student Evaluations of Teaching (SETs) as a tool for evaluating faculty performance. Just to give a taste,

Despite the ubiquity of SETs, a growing body of evidence suggests that their use in personnel decisions is problematic. SETs are weakly related to other measures of teaching effectiveness and student learning (Boring, Ottoboni, and Stark 2016; Uttl, White, and Gonzalez 2017); they are used in statistically problematic ways (e.g., categorical measures are treated as interval, response rates are ignored, small differences are given undue weight, and distributions are not reported) (Boysen 2015; Stark and Freishtat 2014); and they can be influenced by course characteristics like time of day, subject, class size, and whether the course is required, all of which are unrelated to teaching effectiveness.

While some schools are doing away with SETs voluntarily, many faculty still depend on their numbers at the end of the semester to keep their job. The last time I was on the job market, some schools required evaluations as a part of their application process. I will certainly be submitting mine next fall when I compile my tenure dossier. But we all jump through these hoops despite the well-documented fact that these evaluations are strongly influenced by all the factors above, but even worse, by how white and/or male the professor happens to be.

Those looking for more specifics on the unreliability of student evaluations are welcome to look at the extensive reference list of the ASA statement, or this lovely post by Anna Haensch on the AMS Blog on Math Blogs, or the other posts linked below. What I’d like to talk about is What do we, the untenured, do about this?!

The ASA statement gives lists of suggestions, though most of them are at the institution level. They recommend everything from renaming the evaluations themselves, to tweaking the language of the questions, to implementing a much broader and more holistic process of evaluating teaching effectiveness. They even go so far as to recommend that individuals not be compared to campus or even departmental averages.

My school has not (yet) done any of that. I could technically refuse to provide my evaluations in my tenure dossier. Emphasis on technically. The faculty promotion committee would definitely view it as possibly (probably?) troublesome without a boatload of documentation on my reasons for omitting them, and I’d certainly need to provide a lot of other evidence of teaching effectiveness. Who knows what the Provost and Trustees would think. I don’t think anybody’s ever risked it. I certainly won’t.

What I have done is cited relevant research on the unreliability of SETs during workshops with the faculty on the promotion committee, and at least publicly they’ve all agreed that student evaluations are at best problematic and at worst useless. But, of course, they still want to see them. There’s probably room at my institution to begin a more formal transition away from SETs, but I don’t think anybody in a position to make change has the energy for that particular fight right now. Certainly official statements from professional societies like the ASA’s will provide some useful ammunition. Maybe after I get tenure…

What I should do in the meantime is follow Jaqueline Dewer’s advice from the On Teaching and Learning Mathematics blog on interpreting your evaluations, which she calls ratings (“I refer to them as student ratings, not evaluations, because “evaluation” indicates that a judgment of value or worth…while “ratings” denote data that need interpretation.”) She recommends possible tweaks to the language of the questions your school uses, and also gives good targets for response rates (75-80%), and advice for how to compare your course to others. The “When Good Teaching is the Average” section really jumped out at me. I’ve had friends (not at my school) who’ve scored 4.0 out of 5 when the department average is a 4.3, and been made to feel like they wouldn’t pass their mid-tenure review because of it.

I can also follow Adriana Salerno’s advice from a workshop she attended on interpreting evaluations without losing your mind. That is, if I can get up the guts to look at them before it’s time to put them in my dossier.



Posted in Uncategorized | Tagged | Leave a comment

It’s Not a Competition…But We’re Still Ahead

Life is full of ironies and contradictions. Case in point: I write for two competitions (Who Wants to Be a Mathematician, and MATHCOUNTS), but I never have participated in a math competition myself. I hadn’t even heard of the Putnam until graduate school. It gets better: while I now write for and have helped students prep for competitions (not the ones I just mentioned, don’t worry. I know what a conflict of interest is.), “the jury is still out” as to whether or not I think they’re good ideas.

Continue reading

Posted in math circles, networking, outreach, parenting | Tagged , , , , | 1 Comment

To New Faculty (and those who hope to join us): what life is like at my small private liberal arts college

Private colleges were never really on my radar. Everyone in my family attended a state school. I taught in a public school, as did about a half dozen family members. A hefty dose of public school pride, mixed with the slobs vs. snobs zeitgeist of the movies of my childhood, had convinced me that the private college life wasn’t for me, and the feeling was probably mutual.

Students making ziplines on the first day of calculus at Hood College, because this is how fun my job is.

And yet I’m beginning my seventh year of teaching at small, private, not particularly selective liberal arts colleges, and I’ve genuinely loved it. So I wanted to write a little post for two audiences of people who also may not have any experience in this world: those of you starting a job at a school like this for the first time, and those of you on the market this year and wondering if you should apply at a place like Hood.

  • The students may not be what you expect. I figured the students were all entitled, high-maintenance, helicopter-parented rich kids. I’d spent a decade teaching students from disadvantaged groups, and I was reluctant to leave behind students who I felt really needed me. It turns out there are a ton of students from all kinds of backgrounds at private colleges. Many of my students are working multiple jobs outside of class, caring for family members or children, even dealing with food insecurity and lack of housing. They value their education, and they value the people working hard to help them learn.

When I was applying for jobs, I specifically sought out schools that valued diversity and providing inclusive environments for students, and who had well-developed programs for supporting students of color and first generation college students. If that’s something that’s important to you, mention it in your application materials, and not just in a diversity statement.

  • The faculty may not be either. You will see people with staggeringly impressive pedigrees who turn out to be humble and gracious and generous with their time, and people who look like nobodies on paper and act like tin pot dictators (and, of course, vice versa). You may find yourself on a committee with people who have been sparring in meetings for twenty years and have no idea whose side you’re on. Or a small dispute with the administration may come up in a faculty meeting, and suddenly encoded grievances from a decade ago bubble to the surface. You get to ignore all this as a graduate student or a postdoc, but as faculty at a small school you need to keep an ear to the ground.

You’re entering a very, very small collective of highly intelligent – and possibly maladapted – colleagues, and you will probably need a little guidance about which ones to pay close attention to and which ones to put on the back burner. Your fellow new faculty will be very helpful in this, and any effort you put in to establish a strong bond among your cohort will repay itself many times over. Get email addresses and phone numbers as soon as you can and invite people out to dinner or happy hour or bowling, and then keep inviting them to things until you have a strong network established.

  • Your priorities may have to change. You will be teaching more, and probably teaching more preps. You may not get to teach four sections of the same course, because there may not be four sections of the same course. Your courses won’t be giant lectures, but you will probably be expected to be considerably more hands-on with your students than if you had big classes. For instance, I would not be able to get away with only using online homework for my students. The classes are small, so I should be able to provide them better feedback on their work. Delivering pure lecture classes is strongly discouraged in my department, if not straight up forbidden. Active learning, or at least a willingness to begin to move in that direction, is becoming more and more the standard. If you’re applying to a place like this, you need to at least know the buzzwords and have some thoughts on how you would incorporate non-lecture techniques into your teaching, even if you haven’t had the opportunity to try yet.

Keeping up with all this work will require a reshuffling of your priorities, and probably a long, hard look at your time management. Between teaching, service, and – oh yeah, that – research, you will have more work than you can comfortably do in a week. You can fix that by spending all your nights and weekends grading and prepping and trying to write, or you can figure out how to fit your obligations into a standard work week without getting fired. There are people on your campus who are doing just that, even untenured ones. Find out who they are and ask them how they’re doing it. The academy is a tough job and we’re all lucky to be working. But this is still just a job. It doesn’t have to be your entire life unless you want it to be.

  • You won’t ever feel completely comfortable. This is a scary time to be in higher education period, but small colleges like mine feel especially vulnerable. Sweet Briar College managed to stay open, but it seems pretty clear that this is just an unsustainable business model (for lack of a better phrase) for a lot of colleges, and nobody knows who will be next to close. Or your school may decide to restructure and suddenly axe the math major. Or your accreditation may fail to be renewed. Or your state (or country?) may make state schools tuition-free. Or your state university may decide to open a branch in your community. As in the above bullet point, you will suddenly find yourself paying close attention to your enrollment reports, the state of your school’s endowment, and how to increase your retention rate.

Also, you will constantly see your friends and colleagues moving on to new things, and making way more money than you. Many people hustle for years for a job as a professor and find out that it just wasn’t what they thought it would be. Or they burn out. Or want a job where they can comfortably spring for guac on their burrito without thinking about when payday is. So even if you think you’ve landed your dream job, it might not hurt to keep your website updated, and maybe start up a linkedin. You never know when you might need it.

This post got a little longer than I intended, but I hope somebody out there finds it useful. Every semester when classes start up again I feel excited and renewed, because the energy I get from (most of) my students is infectious. My colleagues are amazing and I rely on their wisdom and support every day. This life isn’t for everyone, but if you think it’s for you I hope you’ll give it a shot and make it your own.

Posted in Uncategorized | Tagged , | Leave a comment