Transforming Exam Management to Support Equitable Healthcare Education in Africa

Photo Credit: University of Global Health Equity (UGHE)

Streamlining an African Medical School’s Assessments with eSystem Exam Software.

Recognising the need for a modern, reliable system to manage examinations, the University of Global Health Equity (UGHE) sought a solution capable of supporting both written and practical (OSCE) assessments securely and efficiently, ultimately choosing Speedwell’s eSystem as the trusted solution.

Founded in 2014 by Partners in Health, the University of Global Health Equity (UGHE) set out to transform healthcare education and delivery across Africa and beyond. Based in Rwanda, the University is dedicated to advancing health equity by preparing the next generation of health professionals to not only be highly skilled clinicians but also leaders with the insight and social awareness needed to build high-quality, equitable health systems in underserved communities worldwide.

Dr Ibrahim Olayinka, Assistant Professor at the School of Medicine, details that they run an integrated programme, the MBBS-MGHD. This is a combined degree and masters spanning six and a half years. The course begins with a foundation in social medicine, before progressing through four and a half years of intensive study in health sciences and clinical practice. It concludes with a year-long internship, which students must undertake before completing their masters in global health delivery. At the end of the programme, students take a final clinical exam, the exit exam, which is a combination of MCQ and practical OSCE assessments. It was for the management and delivery of this critical examination that UGHE chose Speedwell’s eSystem.

Identifying Exam Management Challenges

Dr Olayinka mentions that as a relatively new institution, the University is only in its second cohort. The exam was set up and delivered using a mix of Word or Excel files for question creation and storage and then Google Forms or the University’s Learning Management System (LMS) for delivery.  There wasn’t a platform in place that could manage all aspects of the exam.

It soon became clear that this approach was insufficient for the University’s needs. The LMS’ lack of a question banking facility was a problem. With no way to store questions centrally or track previous exams, visibility into exam performance was difficult, and there was a limited opportunity to gain insights for improving assessments. Furthermore, there were concerns over exam security; the LMS lacked the user permissions to restrict access. This led to concerns that there was the potential for published exams to be viewed by unauthorised individuals.

It was apparent that the University required a solution capable of tackling these challenges whilst also allowing them to improve overall exam management.

Choosing a trusted Exam Management Solution

It was through UGHE’s collaborative links with Aga Khan University (East Africa) that they first learned about Speedwell. Aga Khan University has been using Speedwell’s eSystem since 2020, initially adopting it to run their admissions exams during the unprecedented challenges of the Covid-19 pandemic. Kennedy Ngenga, Manager for Academic Programmes at Aga Khan’s Medical College, was keen to share their experience and recommended the eSystem to UGHE.

Natalie McCall, MD, Chair of the Division of Clinical Medicine, explains that Aga Khan not only endorsed the software but also provided UGHE’s key decision-makers with a demonstration of the software in action.

Impressed by how effectively it was supporting another leading medical institution, UGHE were confident the eSystem could meet their needs and proceeded to secure a licence.

“I personally was happy to approve the use of the eSystem. I saw how intuitive and easy to use it was, and was confident it could meet the need we had to improve the quality of our exams.” – Natalie McCall, MD, Chair of Division of Clinical Medicine, UGHE.

Delivering Exams with Confidence

Dr Olayinka led the team from a paediatric perspective and was one of the first people to interact with the eSystem platform practically, taking the lead in using it for the first exit exams.

He underscores that the MBBS-MGHD exit exam is a high-profile, high-stakes assessment, so it was critical that everything ran smoothly:

“Speedwell’s support was always available to make sure the MBBS-MGHD exit exam ran perfectly the first time, which was vital for such an important exam.” – Dr Olayinka, Assistant Professor, UGHE.

In addition to support from Speedwell, the team held weekly faculty meetings with Aga Khan University to observe how the software was used in practice and to receive guidance on setting up similar exams effectively.

Before implementing the eSystem for the main exit exam, Dr Olayinka says they initially ran a pilot for the mock MCQ exams. The purpose of this was to ensure that students gained experience and became familiar with the software before using it in the actual exam. It also allowed the team to iron out any teething issues that arose, such as login issues or the students not having the latest version of Safe ExamBrowser.

Dr Olayinka notes that he was particularly impressed by the full audit that the eSystem holds. It meant that when there was a slight issue with a student not having the correct version of Safe ExamBrowser, he was able to log in and see everything that was happening – in real-time. Having this visibility meant that he could quickly troubleshoot and resolve the problem with minimal impact on the exam, even though he was away from the university in Nigeria at the time. This was hugely beneficial and would not have been possible under the old system.

When it came to the practical component of the exit exam – a 15-station OSCE, Dr Olayinka recalls there was some nervousness about internet connectivity on campus. Understanding this common concern, the eSystem offers flexibility to operate in both online and offline modes, so institutions can choose the approach that best suits the circumstances.

To ensure that they chose the right solution for their needs, UGHE met with the Speedwell Support team to explore in detail how both the online and offline modes function in practice.

The team ultimately decided to run the exam online. They were reassured by the built-in protections of the eSystem’s online mode. These protections handle brief connectivity drops, storing all activity locally and automatically syncing with the server once the connection is restored.

Given the high-stakes nature of the exam, as a precaution, tablets were prepared with an offline version of the OSCE to ensure the assessment could continue in the event of significant connectivity issues. This approach gave the team confidence that the assessment could proceed seamlessly while being fully prepared for all eventualities.

In practice, the offline backup was not needed. The exam largely ran smoothly; however, there was one instance where an examiner experienced a temporary connectivity loss and, out of caution, chose to pause the exam. Once reconnected, the eSystem performed exactly as intended, automatically syncing all activity and data without loss, and the exam continued as planned.

This experience reinforced the eSystem’s reliability and strengthened the team’s trust that the platform could deliver a smooth and dependable exam experience, even in challenging conditions.

“The good thing with the eSystem is that even if the internet is weak or drops briefly, it automatically logs and syncs when connectivity is restored so that the exam can proceed without interruption.”

Photo Credit: University of Global Health Equity (UGHE)

As with the MCQ exam, a pilot was conducted the day before the actual exam for examiners to familiarise themselves with the software and address any initial questions. This preparation, combined with the system’s flexible approach, ensured that on the day of the exam, everything ran smoothly and examiners were confident using the platform.

Dr Olayinka explains that this pilot helped address some hesitancy surrounding the software from examiners. Their main concern was about making mistakes because they were using a new and unfamiliar process.

However, as soon as they used the eSystem, any reservations quickly disappeared due to its intuitive nature he continues:

“Everybody likes the eSystem because it’s so simple to use, everybody was very complimentary of the platform and sees it as a huge benefit for the university”  

After the success of the exit exam, the eSystem is now used for the vast majority of exams in the programme, both summative and formative.

The Speedwell Difference

UGHE are now enjoying the benefits of a true question bank for their questions. Aside from the fact that it is quick and effortless for question authors to add questions from anywhere, there is now a full history and audit trail. Questions are stored in one place, it’s easy to see any changes, how many times a question has been used, and problems such as missing passwords for password-protected Word question files are a thing of the past.

Dr Olayinka emphasises that having full visibility of any login, edit, or action at any time, from anywhere, is hugely valuable, as are the eSystem’s exam monitoring capabilities. He recalls a situation when he was away from the university during an exam. A couple of students were experiencing difficulties accessing their exam, which was understandably causing some concern.

Using the audit, he quickly established that the students had logged on early and attempted to take the exam. The eSystem’s safeguards had prevented them from doing so, but it registered their attempt. Since the exam was set as a single-attempt, high-stakes assessment, the students were unable to log in at the correct start time. With the exam already underway for other students, it could not be edited to allow a re-take.

Having all of this information immediately to hand, Dr Olayinka was able to make quick decisions and promptly create a duplicate exam for the students, allowing them to proceed without further disruption. He explains that this combination of visibility and flexibility is a major advantage, and something they could not have achieved with their previous process.

He continues that the analysis and feedback features of the eSystem are also proving very beneficial for the university, with the data being used for both post-exam analysis and academic research; ‘We’re now able to get data very easily, I have just completed a manuscript and several others are being written using the data from the eSystem’, he explains.

For post-exam analysis, before implementing the eSystem, only limited psychometric data was available for the MCQ exam; however, it took significant time and effort to collate. They did not have any data for their OSCE exams.

Now, for the first time, detailed analytics for both the MCQ and OSCE exams are available instantly, with the click of a button. The deeper understanding gained from the data is driving meaningful improvements and has delivered tangible value for the university.

“We’ve been quite amazed by the data and what it revealed, it’s allowed us to have greater insight into what is working and identify areas where action is needed – it’s very revealing”

The team are extending the use of the eSystem to formative exams, and they can see great potential for using the candidate feedback reports. This will enable students to receive more detailed, personalised feedback and identify any knowledge gaps to improve future performance.

The Future: Shaping Fairer, More Effective Assessments

UGHE is now exploring the adoption of the eSystem across the entire School of Medicine and the wider University, so other departments can experience the benefits too.

Summing up his experience, Dr Olayinka highlights that the eSystem’s greatest impact has been on improving overall exam management. It’s transformed the way they manage their exams, and he happily recommends it to others.

With questions, exams, and data all in one place, it enables quick access to information and informed decision-making, something that was previously very time-consuming to do in detail.

The team’s confidence in their exam processes is strengthened, and with access to valuable insights, they are shaping higher-quality assessments for the future.