Writing and studying for an exam is a game played between Professor and student. In this game the Professor has to pick which questions to ask and the student has to pick which topics to study. The game has the flavor of rock-scissors-paper in that the Professor would like to be unpredictable. That way the students will have to devote studying time to all topics rather than focus on just one that they know the Professor will ask about.
But the Professor might not want the students to spend too much time memorizing concepts from the book. Instead he may want them to spend their time thinking about how to apply those concepts to new problems. How can the Professor be unpredictable and still deter the students from trying to memorize the book? The solution is to use an open book exam. This way the Professor is committing not to ask rote questions which would turn the exam into nothing more than a contest to see which students are the fastest to search through the book and find the topic.
With an open-book exam, the students can predict that the Professor will not ask such questions and they will not bother studying for them. They bring their books to the exam and never have to open them. And they still cannot predict which questions (apart from the mundane ones) the Professor will ask.
Keep in mind that this means students should dislike open-book exams. Many students don’t understand this and are always asking for open-book.
Also, the worst possible format is the common practice of allowing students to write out crib notes on one sheet of paper. This turns studying into pure rent-seeking. All students will predict which are the essential concepts from the book and will write them down and, predicting this, the Professor will not ask questions about those topics. In the end the outcome is just like open-book except the poor students lose valuable studying time while they squeeze the book onto a sheet of paper.
(To my PhD students and undergrads taking exams today: aren’t you glad the exams are closed book? Oh and good luck!)
13 comments
Comments feed for this article
April 29, 2009 at 7:29 am
Mikko
Your conclusion about what students should like and dislike may be wrong.
Perhaps the kind of students who request for open book examinations are the ones who would much rather spend their time trying to understand the topic and how it is relevant to new things than trying to utilize rote memory? After all, those two things require different skills (and different people are natural in one or the other) and thus may tilt the playing field toward students like me: bad memory for details, but relatively good head for understanding and applying systems.
So why should I dislike open book exams? (Apart from disliking all examinations).
Lucky Mikko who has passed all examination but one. The oral one.
May 20, 2009 at 11:29 pm
Phillip
Closed book exams are generally for lazy professors. Open book exams require a professor to craft a test that requires understanding about the fundamental concepts in question. This is difficult, it requires much more thought than singling out a few facts that they feel are important. Rarely in the real life are you strapped without access to resources to help you solve a problem. With omnipresent WiFi, wikipedia, and the beauty of instant searchable access to almost any fact the memorization of facts becomes ridiculous. The real world needs us to be able to solve problems not to be fact storage machines. My phone has instant access to that is better at it than I am. We need to train our primary, undergraduate, and graduate students to be good at quickly finding and applying facts to solve real, and complicated problems. Way too much time is spent in primary school doing long division when calculators which everywhere can do it more accurately. As a previous teacher I know our children are very deficient in solving word-problems. The only real ones that matter. They need to be able to take a problem, distill it down into meaningful parts and know how set the math up for the machine to do. They have to be able to do it by hand but who cares if they are good at it? When was the last time you did 10,938 X 3864.14 by hand? Who cares if an engineer can do math problems if you hand them to her if she cannot figure out what math to use to solve the problem at hand?
If you don’t understand the material you will never be able to solve well crafted problems when given them on a test, you need to not only understand the problem, but know really well where to get it quickly. Real life is an open book test and industry has been complaining for years that our students are not graduation from their schools prepared to handle the working world. It is about time our schools start preparing our students for life and successful careers rather than just to pass tests. You cannot cram for a well written open book test. As a cognitive neuroscientist I know that gist memory lasts far longer than verbatim memory. Why should we require a less durable memory trace when it is not necessary or useful in the really real world.
April 29, 2009 at 7:36 am
hbi
Not so obvious that the one page crib sheet is the worst possible outcome (or for whom) if you believe that by thinking about which are the essential concepts, and writing them out, the students end up learning some of them.
The question is what the purpose of the exam is:
+ to “test” the student and give potential employers, supervisors etc information about the ability/knowledge of the students (then probably we should be thinking about what it is that they want to know)
+ to give students incentives to learn (then what happens in the exam only matters to the extent that it affects how they spend the time outside of the exam (… and thinking through what is “essential” does not seem insane)
+ or (which I consider actually quite important) to give the students some feedback on the extent to which they do or don’t understand what’s happening in the class.
Frankly, it’s hard for me to believe that one strategy for format/questions/grading policies etc optimizes on all these dimensions, though there are clearly some interdependencies among these objectives. Also hard for me to figure out what you think the professor’s objective is in your description.
April 29, 2009 at 9:39 am
j
I always liked open book tests. The student is motivated to spend time understanding concepts instead of memorizing. Why wouldn’t anybody like that?
April 29, 2009 at 12:50 pm
Anshu
I hated all exams types, but actually preferred open book or cheat sheet exams for engineering classes. That way, for any questions I didn’t understand, I’d simply look on my cheat sheet and utilize a solution methodology for a question I did understand, thereby securing part marks despite not having the faintest clue.
April 29, 2009 at 3:23 pm
econ phd
Clearly in an exam like today’s 410-3 mid-term, there is no real scope for a cheat sheet. If someone cannot remember the definition of weak dominance on their own, there are bigger problems. However, for subjects like econometrics, where there are an avalanche of formulas (particularly things like CLTs, LLNs, and the like), it seems very burdensome to have to remember these. I think there is a 4th option which you did not mention: a pre-determined, available-in-advance professor-written cheat sheet. That way, you (or the professor…) have control over the content on them, students know what is up to them to remember (and you can accordingly use the cheat sheet as a signal for this), and there is no time wasted a) on memorizing formulas that are just going to be forgotten the next day anyway, and b) on writing out the cheat sheet.
The only time I ever took exams were like this were in my high school math exams — the final ones administered centrally (the way it goes in my country of origin). I thought it worked well…
April 29, 2009 at 11:39 pm
jeff
this is a great idea.
April 29, 2009 at 5:21 pm
Anush
I agree that open-book tests are best for the professor, but I think they are great for the student. An open-book test usually allows for a bit more predictability than a closed-book test. The student no longer has to memorize much data. The student can also expect to do more higher-level work on the test. He will probably have to solve problems or analyze a scenario rather than regurgitate information. Mikko pointed out that some people might prefer to do this or are better suited to this. I also think it gives the student information about the kind of question that might be asked. I also find them preferable because studying how to solve a problem is more interesting than memorizing, and seems like it might actually be useful later on.
The mobile web has made nearly everything open-book, hasn’t it?
April 29, 2009 at 9:42 pm
The Wife
Books? I thought you never assign text books to your class?! It is a waste of money for your students.
April 29, 2009 at 10:57 pm
jeff
I agree that there is good reason for students to like open book exams. But it takes the rare student who can see those good reasons. Not surprisingly, our commenters meet that description. I was instead writing about students who think they want open-book exams for the superficial reason that the open book will help them.
April 29, 2009 at 11:20 pm
w
The only downside to having open-book exams is actually having to buy the book… and having no book is one aspect I really enjoy about our class.
Having said that, today’s exam was quite conceptual.
April 30, 2009 at 2:32 am
Quel est le bon format pour un examen ? « Rationalité Limitée
[...] format idéal d’un examen est celui où les étudiants ont le droit de consulter les ouvrages qui ont servi de support au cours. L’idée est la suivante : si vous organisez un tel examen, les étudiants anticiperont que [...]
April 30, 2009 at 2:18 pm
mike
When I took econometrics from Art Goldberger in the early 80s, all of his exams were open book. I liked it. I worried less about memorizing formulas, and worried more about understanding the concepts he was trying to teach. As I remember, I actually consulted my books/notes very little in those exams.