This New York Times opinion piece proposes a reform to the tenure system in which there are three tracks: the “teaching track,” the “research track,” and the “teaching and research track.” Worth a read. But I’m not here to talk about that, I’m here to question one of the findings used to support this argument, that “students rarely favor teachers who grade leniently — and give higher ratings to teachers who assign heavier workloads.”

This finding screams “selection bias.” Let me explain how.

Let’s say there are two types of students: gluttons for punishment and easy riders. Gluttons for punishment prefer the challenge of a hard class, and in my experience, you can actually get them more excited for your class by telling them how hard it is and how much work you will assign. I was a glutton for punishment, and so were some of my friends—I remember my sophomore roommate and I comparing our battle scars from Engin Akarli‘s insane Middle Eastern History course. (“Some people say, no dates. I say, “know dates!”)

Easy riders prefer an easier course, with lower workloads and easier grading. They like courses like my cousin Hal’s criminology courses.

The problem is, so long as students have a choice in what classes they take, we should expect them to self-select into easy courses if they are easy riders, and into hard courses if they are gluttons for punishment. The results: hard courses full of students who like hard courses, and who evaluate them favorably, and easy courses full of students who don’t like hard courses, and who evaluate them accordingly. If it is the case that even gluttons for punishment take easy courses sometimes (ahem), but easy riders never take hard courses[1], then in aggregate, evaluations will suggest that students prefer hard courses because only the students who prefer hard courses will be evaluating them.

What I want, of course, is for someone to exploit a natural experiment in which the same course is taught with different workloads to students who don’t have a choice to take the hard version or the easy version. But even that won’t be a relevant design if the rest of us have to anticipate the effects of workload on enrollment.[2]

***

[1] Actually, what we really need to assume is simply that easy riders are less likely to take hard courses than gluttons for punishment are to take easy courses.

[2] As Director of Undergraduate Studies in my department, I think about this.