by Katz
If I had to condense my teaching philosophy into one phrase it would be something like the following:
The practice that students do and the evaluation of their progress should match
the context of the goal skill/attitude/behavior as closely as possible.
The challenge then becomes determining the goals and their contexts. Almost every reason I can think of for an undergraduate to take a math class ends with them taking time to produce nice written and oral versions of their work to communicate with others. I believe that I create appropriate contexts in the classroom, but I have not been happy with my exams for several years. Recently I have experimented with removing in-class written exams entirely in favor of a combination of a take-home exam and an oral exam.
I see huge advantages including allowing more useful and immediate feedback for the students and a more pleasant experience grading. I also see some obstacles including initial fear from the students, complex administrative requirements, and a slightly different focus while evaluating. In what follows, I will share my thoughts on the benefits and costs of this different exam structure.
Question: In which classes do I use take-home and oral exams?
I have paired take-home and oral exams in Discrete Mathematics (our sophomore introduction-to-proof course), Real Analysis, and Modern Geometry. I have given take-home exams instead of 1-hour written exams in my Calculus courses as well, and I would like to try an oral component.
In-class written exams may make pedagogical sense in certain contexts. I’d imagine that students who go on to take a Calculus-based Physics course will be expected to compute derivatives quickly and accurately in the middle of other problems; this skill does resemble the ones practiced in a traditional Calculus exam. But this is honestly the only example I can think of right now. There is no Calculus on the MCAT. Working Engineers will have plenty of people to check their work with multiple people and resources. I suppose that speedy facility with computations is important for portions of the Mathematics subject GRE, but I think the best way to prepare for that is to be a grader, not a student. Mathematicians take weeks to prepare their papers and conference presentations.
I allow my students to use all resources that have been available to them during their regular work when working on their take-home exams. This usually means a textbook, if we’ve been using one, and any resources that we have built together like a WikiTextbook. The internet is explicitly disallowed, which may seem counter to my goal to match future contexts. I believe that my students would spend all of their time looking for solutions instead of thinking about solutions if they could use the internet. I also think that slight variations in Google-skill could lead to massively different exam experiences for similar students, and I am not teaching these search skills, so it seems appropriate to remove this tool. But most importantly, I am trying to teach the skill of creating new arguments, not understanding old arguments. It seems virtually impossible to create new problems of appropriate difficulty for each student each year, so there needs to be something artificial. In addition, the rest of the course is highly collaborative, so the solo work can be a bit of a shock for the students. I am not thrilled with either of these last two aspects, but I can see no other way to guarantee that each student must meet a base-line skill level. Suggestions are encouraged.
Question: How are the exams structured in my courses?
The students get the take-home exam questions on a Friday or Wednesday. A week later, the students submit written solutions to these questions. The students are instructed to work alone, and we have an Honor Code signed by all students. After they submit their work, I give extensive feedback, and they are encouraged to re-submit for partial credit. The grade is heavily weighted towards both the rigor and completeness of the argument as well as the quality of the writing.
Sometime during the week that they are working on the take-home, the students schedule an oral exam. The oral exams are just presentations of some portion of the work for the take-home exam. I have tried (a) letting the student choose which portion s/he wishes to present and (b) having the student select a card for a random selection of a portion. I have tried holding these oral exams with individual students and with groups of students each presenting different questions. I generally let the students dictate whether they would like to present and then get feedback or use the time as more of a dialogue. I have not yet given an opportunity for corrections on an oral exam, but I would like to find a way. The oral exams are scheduled before the take-home is due in order to allow time for the feedback from the presentation to be applied to the writing. The grade on the oral is weighted more heavily towards communication and less heavily on rigor. They have a choice: present the parts they know well to get a high grade on the oral but get no useful feedback for the take-home or present the weaker parts and receive a lower grade on the oral but get more help on the take-home. I want the grading structure to make the latter choice less painful.
Question: How much time and energy does this structure take?
Simply setting up the oral exam times could be onerous for some people. I pick the length of the sessions and then pick a couple of windows of time adding to about the total needed for all students. I create a page in the class Wiki with the times, and the students take care of signing up and organizing themselves. This is exactly the same as passing around a sign-up sheet except that (1) it requires no in-class time, (2) it saves each version, so I can tell if someone tampered with another’s time, (3) it’s visible to all of them at any time, and (4) they can handle any swaps amongst themselves without involving me.
With my largest class (64 people in Discrete), I believe that I managed 10 6-minute sessions per hour. This is also the first course with proofs, so the proofs are regularly shorter than in other courses. With my smaller classes, I have scheduled 15 or 20 minutes for each student. I bet I could do 10 6-minute orals for a Calculus course if the class were huge. A senior colleague suggested using 3 students in a 30-minute time slot instead of 3 10-minute time slots, even if the students are not presenting in the same room. This guarantees that there are enough people around when you need them, and the students seem to like the option of going before or after their slot-mates.
The first round of orals for a term is going to be hectic, but the students learn how the schedule works, and I got better at warning them that we were running out of time. Also, allowing them to select a portion of the work to present beforehand allows you to expect that they have timed their presentation.
The previous paragraphs may seem like enough to make this idea unfeasible; however, there are hidden time-benefits. First, I can grade and give feedback on the oral exams during the time scheduled for the oral. In part this is due to a rubric; for communication quality, I like the AAC&U rubric for Oral Communication. Second, the oral exam dramatically reduces the amount of time I need to spend reading and grading the take-home exams because they are generally higher quality, because I can often reference a comment from the oral for clarification instead of writing a paragraph and because I basically know their thinking before I start reading. Third, cheating is even more transparent in oral form, so I can save some energy thinking about it later. Fourth, I find it very hard to write the same paragraph repeatedly on a paper exam, but it’s easy to give the same feedback at the end of the orals. I also find the oral exam environment to be much more dynamic and engaging than grading written work, so I don’t dread it at all. In the end, I think it depends on your personal grading style and quality of life preferences whether this is easier or harder than a traditional round of exams on the teacher, though I think the benefits for the students could balance quite a lot.
Question: What is the student response?
Some students were terrified of the idea of an oral exam, but other students spontaneously expressed anxiety about traditional timed exams. During an oral exam, if a student gets stuck, I can ask questions. I can seek for the student’s level of mastery instead of hoping that a static set of questions will make it clear to me. Explaining that I’m looking for what they know, not trying to trick them into saying dumb things, helped some of the nervous students. Only about 2-4 students per term were still significantly more nervous about the second oral exam than they would have been about a written exam. However, my students expect to present at least once a week in my classes, so an oral presentation is more in line with my daily activities than those in more traditional math classes.
Students also expressed some fear about a test structure that only asked them to do new things without any portion that they knew they could just prepare for by memorizing. In class exams tend to ask students to exhibit skills that they’ve had to practice and to recall facts that they know; a take-home exam allows time for the students to think about an entirely new object. I think a big transition during college is learning to deal with this fear, so I don’t worry about its presence here. However, the students did seem significantly more nervous about having only a take-home when compared to having a take-home and oral exam. In any format, I am going to give the students a new definition and ask them to prove some theorems about it; once they accept this premise, they generally see the oral/take-home structure as the least of all evils. Frankly, I’m a pretty demanding professor, and their response doesn’t seem out of line with the rest of my expectations.
I believe that it would be best for the students to present at the oral exams without using notes. For the first few rounds of orals, I offered a tiny penalty for using notes, and they could pick them up in the middle if they felt they needed them. The students who were going to do a passable job rarely looked at their notes, regardless of whether or not they were holding them, and it seems silly to me now to expect them to memorize a technical choice of epsilon in an analysis proof. I’ve let this go altogether, and it seems to make them feel more comfortable.
Question: Is cheating a problem?
I now ask the students to hand-write a version of the Honor Code with each take-home exam. In addition, I have added a little direct instruction on the Code, which seems to be sufficient. However, you should run any honor statement past your chair/supervisor, and you should consider the culture on your campus about take-homes and cheating. After a big cheating problem earlier in my career, a senior student told me that “cheating is a learned behavior” on my campus. Simply put, there is more time to possibly cheat on a take-home, but that doesn’t mean it’s not 100% transparent. Also, as mentioned above, my take-homes usually contain definitions that are new to the students. I like to make up intuitive words for the objects or properties that are different from any existing terms (if they exist), which seems enough to block most simple attempts at cheating with Google.
Question: Are there other major differences between an in-class and take-home exam?
Absolutely, yes. The major difference for me is substituting a very small number of meaty questions for a larger number of more routine exercises. At the undergraduate level, it is really easy to leap from a straight-forward problem to an extremely challenging question, and the difference is completely hidden from the novice student. As a result, you needed to have a more detailed understanding of the process that will take students through the work. I think it’s critical to find problems where there aren’t “choking points”, steps that require leaps that only make sense in retrospect. These leaps can be extremely frustrating for the students, and I think they point to very subtle skills that we have built over years. I think the oral exam component compensates for this, especially for the weaker students; a few days before the written version is due, they have a scheduled session in which they have to at least articulate the steps they want to take.
The in-class exams I used to give were related to the content already covered. As a result, corrections on the in-class exam were painfully boring for everyone involved. Some students would try to submit old work as corrections to proofs from the exam. The take-home, being about a new term, doesn’t have this problem.
The in-class exams I used to give also asked students to summarize in 3-4 sentences a proof we had completed together. The way this was graded made sure it was a high-level activity to pick the truly clever steps in a proof. This doesn’t fit very naturally in the take-home or oral. I have tried requiring a summary on the take-home for any theorem quoted from previous work; I have also tried using any extra time during the oral to ask for summaries. Neither has been great, so I am working on ways to incorporate this activity into other parts of the course more explicitly. I also like that the students must review and synthesize the material while studying for an in-class exam, and I have been trying to build this into the course in other ways.
Other Thoughts:
*** During a week-long take-home exam, class would regularly meet 3 times. I usually cancel one of those days in order to schedule at least some of the orals during our regular class hour. But we need to keep working on other things in class during the exam. We usually move on to a new topic, so at least one day of group exploration of the new ideas is appropriate. The way I run my classes usually requires preparation from the students, so the second day of class can be quite a challenge. It’s a great time to spend a day in the computer lab learning about LaTex or Mathematica.
*** There seems to be a big difference between a Friday-Friday exam and a Wednesday-Wednesday exam schedule. The best choice depends on when the students are going to need access to your office hours as well as things like the schedule of their other course work. I see no reason not to let them decide collectively on the schedule. I would avoid Monday-Monday because you will not be around on campus for the last two days of work for questions.
*** The same senior colleague from above suggested that I videotape the oral exams, at least until I get tenure. He seemed worried about some form of push-back or claims of unfairness from students, and he thought the videos would provide sufficient evidence to support me in the case of any trouble. He is a Biology professor, and his department does feel far more grade pressure than mine. I have recorded all of these oral exams, but I have not found a useful way to share those videos with students. I did, however, use some of them as “evidence of student learning” at a recent pre-tenure review, so it was nice to have.
*** In general, I have been disappointed in the response to feedback given at the oral exams. Recently, I have started audio-recording the oral exams as well with a LiveScribe pen. The pen makes a synched file of my writing and the audio from the student. These are easy to share with students, and the few students who have used them have told me that the ability to connect my feedback to exactly what they said was incredible. At my last round of orals (with a tiny class), I left the recorder on while I talked through the notes so that the students also had an explanation of my notes. This format let me make connections between their errors in a way that is very hard to manage in other formats. In addition, I tried using the pen to give feedback while reading their take-home exams. I essentially dictated their work while interspersing my comments. This took me less time than writing feedback and it seemed way more useful to the students.
*** At its core, this exam structure was inspired by the Physics department at my undergraduate institution. Their students receive the problems with only about 24 hours of lead-time for the oral, and there is no written version after the oral, but it still feels similar to me. There are lots of ways to do this, clearly.