orgtheory.net

multiple choice tests are: (a) totally useless, (b) ideal for all classrooms, (c) useful in some situations, or (d) all of the above

The answer is (c), if you read Tereance Tao’s post. He makes the case that there’s a good place for multiple choice questions in learning, just not for final exams and such. He thinks carefully crafted multiple choice can help student self-assess what they know. Here’s the punchline:

In summary, I believe that there are a number of interesting ways – many of which appear to be underexplored at present – in which some well-designed and self-administered online multiple choice questions can efficiently assess one’s strengths and weaknesses in a given mathematical subject.  Of course, having one-on-one interaction with a lecturer or teaching assistant would be a greatly preferable way to achieve this sort of instant feedback, but this is impractical for larger classes.  It is also true that a certain level of maturity and discipline is needed on the student’s part in order to actually benefit from these sort of self-assessments, especially since they are not directly contributing to the student’s grade in the class, but my philosophy here is to give the students the benefit of the doubt in this regard; I feel that being able to explore beyond the bare minimum of what is needed to obtain a passing grade is part of what an upper-division course should be about.

I personally use a bit of MC. One use is policing. It’s a quick and easy way to assess who actually did the readings, and creates an incentive to at least skim the books. I also use MC because you can easily vary the difficulty and I can see who can add concepts together beyond what was listed on the review sheet. However, I always have short answers and essays. They don’t tell me much that the MC sections don’t because MC and written essay scores are very highly correlated (about .7 on my tests). But they do send a message – I want you to express yourself in complete sentences and see the big picture.

Written by fabiorojas

December 16, 2008 at 4:48 am

Posted in education, fabio

5 Responses

Subscribe to comments with RSS.

  1. I used mc questions just in the way that Tao suggests in a social problems course last year. I had a test bank in our course management software and each day before class students had to answer 3 or 4 mc questions on the reading. The students hated it and did terribly even when they had done the reading, as almost inevitably the questions were detail-oriented and nitpicky. I was happier with the questions I wrote myself (in contrast to the ones that came with the testbank), but I found it rather difficult to come up with three plausible wrong answers. The only mc questions I wrote that I was really happy with were of the format:

    Which of the following ….
    A)
    B)
    C) A and B
    D) Neither A nor B

    In any case, for groups of 50 or less I find that short answer is really almost as fast to grade and requires much more thinking on their part. Plus it avoids the problem I always had in high school–bubbling the wrong answer choice simply because of problems managing the bubble sheet.

    Like

    Mikaila

    December 16, 2008 at 2:00 pm

  2. Jacques Barzun (1945:300–302) wrote the following regarding the use of multiple choice tests: “They should go. They are an insult to Intelligence, except when played with as parlor games. And something else must go at the same time; I mean the form of such tests. Every man of education ought to take a solemn vow that he will never ‘check’ anything on a printed list. Students should not be asked to pass so-called objective examinations, which are the kind composed of mimeographed questions to be marked Yes or No, or to be solved by matching the right name with a definition. I have kept track for some ten years of the effect of such tests upon the upper half of each class. The best men go down one grade and the next best go up. It is not hard to see why. The second-rate do well in school and in life because of their ability to grasp what is accepted and conventional, the ‘ropes’ of the subject. They become pillars of society and I have no quarrel with them. But first-rate men are rarer and equally indispensable. They see into situations quickly, and with the fresh, clear eye of Intelligence, and they must be encouraged to continue. To them, a ready-made question is an obstacle. It paralyzes thought by cutting off all connections but one. Or else it sets them thinking and doubting whether in that form any possible answers really fits. Their minds have finer adjustments, more imagination, which the test deliberately penalizes as encumbrances. This basic difficulty occurs no matter how carefully the questions are drafted and how extensive their coverage. I sat and worked on a committee that prepared objective questions in history for the so-called Graduate Record Examination, which is now widely used to test college seniors’ readiness for graduate work. In committee, it was revealing to see how a question that seemed ‘foolproof’ and ;obvious’ to two or three men, thoroughly trained in their field, struck others of the same caliber as ‘ambiguous’ or misleading.’ Add modifiers and you can make the question so unwieldy that it can hardly be grasped at one reading; simplify and you reduce it to bare common fact. Neither extreme, moreover, brings anything out of the student’s mind; yet the power to relate, to think up, to see into, is what distinguishes the first rank from the second in all walks of life. The results of the Graduate examination no doubt correlate very satisfactorily with other indices, but they scarcely give data for the most needful kind of diagnosis. Nor have they ever been tried on the masters of the profession, which would be the test of tests, provided running comments were allowed. When one courageous man proposed just this at an institution that thrives on endless testing, the idea was dismissed as a joke in poor taste.”

    Barzun wrote more about multiple choice tests in Begin Here: The Fogotten Condition of Learning (1991). See in particular chapter 3.

    Like

    Isaac

    December 16, 2008 at 6:10 pm

  3. So, Barzun would say (a)?

    Like

    fabiorojas

    December 16, 2008 at 7:36 pm

  4. I am not sure whether he would *take* a multiple choice test, but my guess is that he might even use a capital letter in this case: (A)! As an aside it is unfortunate that social scientists do not pay more attention to what Barzun has to say.

    Like

    Isaac

    December 16, 2008 at 10:57 pm

  5. Maybe Tao would like to grade my 480 students’ essay questions?

    Like

    violet

    December 17, 2008 at 4:58 am


Comments are closed.