Multiple choice exams are a reliable means of measuring knowledge and understanding at various levels of learning, as well as enabling large amounts of material to be covered efficiently. Below we examine 5 best practices to get the most out of all your multiple choice exams.
1. Invest in a software package
Investing in a multiple choice software package can enable one person to produce and administer an entire exam in just a matter of hours. The primary advantages are of course speed and accuracy when used in conjunction with an OMR scanner, as well as the immediate access to in-depth statistical analysis on the performance of not just the candidates but the exam as well.
2. Question Banking
Using a question or item banking system allows you to build a library of exam questions with associated statistical analysis on the performance of each and every question. The advantage of using question banking in conjunction with exam software for analysis is that it contains historical data, enabling comparisons to be made to previous exams and any statistics are directly and easily accessible to the exam authors.
3. Review the stats
Multi-choice exams can provide you with a wide range of statistical data allowing you to make changes and produce better performing questions. The following commonly used stats are often used to give an indication on the quality of an exam question:
Mean – The mean score is one of the most basic stat tools. A high mean score suggests the question was relatively easy and makes distinguishing between A-level and B-level students difficult.
Facility – Facility, as the name implies, will give you an indication of how easy or difficult a question was. The higher the facility, the easier the question.
Discrimination – Discrimination is a measure of how well a question discriminates between good and poor candidates. This can be calculated in various ways, but typically it compares the top third of students with the bottom third. The higher the value, the better the correlation between the question and the exam.
Point Biserial – Point biserial coefficient measures the correlation between a candidate’s performance on a question and the exam as a whole. Results range from –1 to 1. A higher value indicates that the students who had high test scores got the question correct whereas students who had low test scores got the question wrong. Questions with discrimination values near or less than zero should be removed from the exam.
Pentile Histogram – This shows the percentage of students in each 20% score band who have answered the question correctly. An indication of a good performing question has a pentile showing a downward trend.
For reliability, the Kuder Richardson Formula 20 (KR20) value can be used to give an overall indication of test reliability.
4. Evaluate your discriminators
Other possible areas to explore when evaluating exam questions, are how well your distractors are working. The quality of distractors can influence student performance on an exam question. A strong distractor will test the top scoring students. For distractors to work they must be plausible; could possibly be argued as correct; and have a reasonable chance of being selected by someone who has not learnt the material. Distractors that are not chosen by any students should be replaced in the exam.
5. Make improvements
Analysing how well your exam questions are performing and their overall quality is useless unless the data is used to make improvements to any future exams. Therefore any findings should be fedback to the exam authors and changes made.
For more information about our software and how they can help you optimise your multiple choice exams, please contact our team here at Speedwell via this website, via email at firstname.lastname@example.org or by telephone on +44 (0)1223 851703.
Published 19th January, 2015