How can you get the most out of your question bank system? Here are our top five tips.
Let’s face facts – not all questions perform as well as you like during examinations. It takes significant amounts of time to prepare, collate and review examination questions. And finding out after your students have sat the exam that your questions have not performed as expected is disappointing – and can call the validity of your exam into question.
There can be many reasons why a question doesn’t perform as you would like. The beauty of a sophisticated question bank system is that you can use your past experience to continuously improve – easily. It is similar to the sentiment of Lord Kelvin’s comment – “if you cannot measure it, you cannot improve it.”
This kind of approach was probably ok twenty years ago, but ultimately now it is likely that you have a stack of data that is unsearchable and un-index-able. It’s likely that this is holding you back from improving your questions from exam to exam because you can’t access powerful insights at your fingertips.
It might seem obvious but really take notice of what the statistics in your question bank tell you. A sophisticated question bank system will analyse several parameters for each question or group of questions allowing you to gain an in-depth understanding of question effectiveness as well as making comparisons with its performance in previous exams.
Yes, we have already mentioned this, but it is worthwhile considering the statistics behind OSCE (practical) exams as well as those for multiple choice exams. The set up for OSCE exams can be extremely complicated – and with multiple assessors making it even more challenging. However there are real benefits in understanding the differences between assessors and their performance during the day – especially when it comes to any subjective marking of a candidates performance.
Automatic controls ensure that the exam questions remain in sequence following the addition, withdrawal or movement of questions within an exam. Our recommendation is to always manage all your exam editing and re-ordering within the question bank. Occasionally we see clients creating their exam in question bank software, and then exporting the questions to another format (most likely word). It is at this point that exam questions are edited and then re-imported into the question bank system. This process usually overrides the automatic controls that help to minimise human error – and of course introduces a numbering error.
However, the best solution for removing human error in exam preparation and management are next-generation exam management tools, such as the eSystem. eSystem manages the entire preparation, administration and analysis of exams managed within one solution. There is no need to export exams and circulate them to stakeholders as each author can review specific questions or the entire exam in a controlled and audited manner.
This is known as blueprinting (view this short video). When a question bank includes information on how many times a question has been used, when it was last used, how it performed and so on, you can use it as a starting point for designing new exams. When you are close to finishing the exam design phase, use your question bank to cross check that you have covered all your learning outcomes.
The power of a robust question bank system isn’t in doubt. It saves time, energy, resources – and at times stress. As well as ensuring that you set the exam that best evaluates candidates’ knowledge. Our specialists here at Speedwell are happy to give you a host of other tips which will ensure you get the most out of your question bank system. Please get in touch and ask!