In Focus: LKCMedicine takes its Multiple Mini Interviews online

By Anne Loh, Assistant Director, Communications and Outreach

On Friday, April 3, most of Singapore was glued to the TV, local news sites and the Prime Minister's Facebook page. At 4pm, Prime Minister Lee Hsien Loong announced to the nation that the country would enter a circuit-breaker mode on April 7. Schools would switch to home-based learning, university campuses, offices and shops would shut. While most learning had already been switched to online at LKCMedicine, the School was due to start its Multiple Mini Interviews (MMIs), an integral part of the admission process, on April 4.

The admissions team had originally planned for a semi-remote MMI format, where the 403 shortlisted candidates would Zoom in from home while interviewers and support staff would operate out of the Novena campus. With the circuit breaker in play, the MMIs had to be postponed, while the School set up an entirely remote interviewing process.

To be viable as an assessment tool, Assistant Dean for Admissions Associate Professor Kwek Tong Kiat said the eMMI had to be as close to the structure of original MMIs as possible. "We had to ensure the security of the sessions, that the candidates' identity was verified and that they were not getting help from others in the room or from the internet," he said.

The Digital Learning team also had some concerns about taking the MMIs wholly online. While Singapore may have one of the highest internet penetration rates in the world, that didn't mean interviewers and candidates had equal means to participate in a fully online MMI.

"The other area we were concerned about was whether the MMI workflow would easily fit into an online delivery modality, and we really had to work closely with the MMI team on best practices for online engagement," said the Digital Learning Department's Senior Assistant Director of Content and Instructional Design Mr Ng Aik Song.

But some stations just couldn't be translated into a fully online format. Stations that included role-play and the assessment of the candidates' ability to tackle real-life scenarios were dropped because interviewers would find it difficult to observe their body language and non-verbal cues. MMI stations were therefore reduced from eight to five. The rubrics of the five stations remained the same as those of the physical MMIs.  

With fewer stations, the admissions team also fielded fewer interviewers. However, the eMMIs still required the participation of 30 faculty, clinicians, educators, staff, and allied health professionals, who had to be trained in conducting a virtual interview. Each station also had a support staff in the session so that interviewers could just concentrate on interviewing and grading the candidates.

A pictorial step-by-step guide and FAQs were created by the Digital Learning team and provided to the interviewers. The briefing sessions were not only to ensure the interviewers had the means to join the fully online MMI sessions but that they were familiar with the functions that would be used.

It was also important that the candidates were able to perform at the eMMIs on an equal technological footing with their peers, and that they would not be penalised for something out of their control. Each candidate was provided with a detailed guide to the equipment and version of Zoom needed. Almost all the candidates tested their internet connectivity with practice runs organised by the MMI team. One candidate, who was unable to travel to Singapore due to the restrictions on movements in place in Malaysia, completed his connectivity check and MMI from Malaysia. In normal times, LKCMedicine would expect all candidates to be physically present for the MMIs.

With fewer interviewers, fewer stations and, therefore, a shorter eMMI, more sessions could be conducted in one day, up to a total of eight sessions of five candidates per day. Developing the traditional MMI into first a semi-remote and then a fully remote activity took 2.5 weeks from planning to implementation, with all candidates interviewed over 11 days. 

The MMI team acknowledged that there were some plusses in organising eMMIs as many of the usual logistical demands associated with a physical MMI session, such as signages and hospitality needs, were no longer concerns. "Of course, we do lose out on the possibility of assessing the candidates' communication skills and use of body language," said Prof Kwek.

Going up to the full MMI with eight stations virtually wouldn't be just a simple matter of scaling up from this first eMMI. "We would need to go back to the drawing board and reassess logistics, manpower and process flow again," said Mr Ng.

Prof Kwek said, "The main takeaway from the first online MMI is that the concept works and that we have the confidence to attempt the full eight stations should there be a need to in the future. It would require adjustments and testing to be sure that the assessment of the desired skills is still viable."

Since completing the eMMI successfully, the School's medical education faculty including Professor of Medical Education Research Jennifer Cleland, Prof Kwek and Vice-Dean for Education Professor Naomi Low-Beer together with the admissions team published their experience in the journal Medical Teacher, a leading international journal on medical education.