Imagine an anatomy class where students observe the workings of the heart in real action, or how a set of cancerous lungs look like right in front of them. This can be made possible with technologies such as augmented and virtual reality (AR & VR), and it is exactly what a team of LKCMedicine faculty and students is developing to make the study of anatomy both accessible and interactive for students.
With a curriculum that encourages independent learning, students at LKCMedicine spend most of their time studying outside of the classroom. And with anatomy being a subject that heavily relies on actual specimens may be an issue not having ready access to the laboratories that house these specimens. That is where the MagicBook comes in, an application made for smart phones and tablets created to bring anatomy to life through augmented and virtual reality.
“There is now a gap in the system where students have access to these materials in the lab, but the access is limited,” said Head of Anatomy Assistant Professor Sreenivasulu Reddy Mogali, who together with Associate Professor of Human and Microbial Genetics Eric Yap, are in the midst of developing the MagicBook. “The idea of the app is to promote self-directed and mobile learning, and to have authentic materials for the students to prepare themselves outside of their classes.”
How the MagicBook works is simple. Students coming into class can fire up the app and scan markers in their anatomy textbooks or images in worksheets. The app will pick up the image of the anatomy specimen, like a heart, and show an augmented reality of that heart with accompanied annotations, labels and interactions that are specifically curated by Asst Prof Mogali for the class. This allows students to interact with the specimen, clicking on touch points that explain the functions of the specimen, even showing how it looks like in different medical scenarios.
To better understand the students’ studying habits and needs in an anatomy class, Asst Prof Mogali and A/Prof Yap employed the help of three Year 5 LKCMedicine students for the project: Sophia Wong, Cecilia Chen and Brian Ho. The team has been working together for over a year, developing the prototype and creating a database of anatomy materials.
“All of us brainstorm together, with the professors refining our thought processes and research focus,” said Sophia. “Administrative work is also split up among us and this allows us to gain more experience in the field of research and collaboration, without being too bogged down by excessive paperwork.”
The materials and preparation needed to create a database of all of the anatomy parts available in LKCMedicine for the app is long and tedious. The AR models are created through photogrammetry where specimens are digitised in a small, make-shift studio at the anatomy resource centre. Different angles of the specimen are photographed by an automatic turn-table that was cleverly designed and developed by A/Prof Yap to automate the process. After being photographed, the images are stitched together and embedded into the app, and Asst Prof Mogali then starts working on annotating labels and writing up notes for the specimen. Currently, the team is almost finished with the head, neck and limbs, and are now aiming to complete the bigger specimens like the lower limbs and torso by using hand-held scanners instead. The team plans to introduce VR capability into the system as well, which brings anatomy even closer to the user.
At the same time, the team is also working on a second project to help students identify MRI scans easily with the help of 3D anatomy models through integration of 3D printing with virtual and mixed reality technologies. The team has begun working on the thoracic region, and allows students to better visualise and match the MRI scans to the anatomy specimens they are used to.
“What we need to learn and the way we learn, are being transformed by disruptive social and technological forces like social media, pervasive internet and deep learning,” said A/Prof Yap. “So, if our schools don’t continually transform the way they teach, they risk going the way of Xerox and Kodak in ineffectiveness or worse still, become irrelevant. We’re a young school, but extinction knows no age.”
As these projects gain traction, Asst Prof Mogali and A/Prof Yap recently received the Nanyang Technological University, Singapore (NTU) EdeX Grant, which offers faculty the opportunity to develop new strategies to improve student learning. This $75,000 grant will enable the team to further improve the two projects, and expedite their implementation. Asst Prof Mogali and A/Prof Yap also presented their work to the Prime Minister of India, His Excellency Narendra Modi during his visit of NTU on 1 June.
“The EdeX grant is for developing and evaluating new teaching materials and methods for anatomy education. We are creating photorealistic 3D models of our specimens and exploring Mixed Reality visualisation for practicals and online pre-TBL preparation,” added A/Prof Yap.
Hopes are high for these two projects, where these tools could be implemented in the anatomy classes here in LKCMedicine in the near future.
Student Cecilia said, “I do hope that the students will find learning anatomy fun through AR and the MagicBook. I also hope that the project can potentially be incorporated into the school curriculum if it is deemed to be beneficial for the learning of the student.”
“Perhaps one day, VR and AR could replace cadavers that date back to medieval times, and plastinated specimens which can be costly!” added A/Prof Yap.