Pixel power

Imagine opening a door and walking into a strange universe where Newton’s laws do not apply and imagination rules the day. This can be achieved thanks to digital tools such as virtual reality (VR) and augmented reality (AR), which allow us to visualise and interact with digital information as if they were real.

At the Multi-plAtform Game Innovation Centre (MAGIC) at NTU’s School of Computer Science and Engineering (SCSE), we conduct research in VR such as 3D modelling, simulation and real-time graphics, as well as advance possibilities in AR, from computer vision to image processing. Beyond VR and AR, our research includes artificial intelligence and cloud computing.

Simulating deformable models
Avatars that resemble people in appearance, character and behaviour in crowds are important elements in any virtual environment, as well as in applications such as architectural validation and emergency evacuation. To develop more realistic avatar animations, the MAGIC team is working with other researchers from SCSE to simulate real-world stiffness and predict physical or mechanical behaviours under deformation.

Introducing the concept of fibre into deformable models, we use fibre materials to animate a skeletal character that has its skin modelled by a surface mesh. Unlike geometrically based skinning techniques that suffer from obvious volume distortion artefacts, our technique allows us to achieve real-time performance with large deformations (Figure 1).

Figure 1: Skeletal animation of a 3D model. (a) A surface mesh with underlying skeletal structure. (b) Proper skinning without volume distortion artefacts. Credit: Cai Jianping.

Relieving persistent neck pain
In a collaboration with Singapore’s Sengkang General Hospital, we are studying the neck mobility of patients who suffer from neck pain. By wearing a VR headset while following the path of an object, patients generate data that enable doctors to analyse their cervical joint position sense as well as the accuracy and time taken to complete the task (Figure 2).

Figure 2: Real-time head motion measurement with a mobile VR headset. Credit: MAGIC.

Our VR approach makes it possible for patients to complete the assessment remotely at home. Fun and interactive tasks can also be incorporated to increase compliance with exercises. On a population scale, we aim to create a registry and analyse the collected data.

A blast to the past
VR and AR are also excellent tools to make history accessible to the general public. In collaboration with the National Archives of Singapore and NTU’s Wee Kim Wee School of Communication and Information, our team developed a mobile AR-based game that features scenes from Bukit Merah, a township in the heart of Singapore that includes the country’s first housing estate (Figure 3).