Other Projects 

Abstract:

Chatbots, also known as conversational agents, are increasingly popular in education and are one of the most useful Artificial Intelligence (AI) technologies that can be integrated into various aspects of education. We propose a project to develop a chatbot to support Singapore MOE K-12 students in learning Computing and evaluate its effectiveness on students' learning. The project will develop new insights, adds to the current theoretical understandings, and enriches international literature in the research field of education (technology) and artificial intelligence. The project will also demonstrate rigorous innovation by developing a chatbot that serves as a learning support tool in a form of agent-augmented intervention to help students to have a better understanding of computing concepts. The chatbot will serve a learning role in supporting students by providing instant feedback to students' queries related to the course topics. The dataset for the chatbot will be derived from a list of frequently asked questions about the topic that is developed by the teachers and discussed with the researchers. To investigate the students' perception of the use of chatbot for learning and the effectiveness of using agent-based learning, we will conduct a quasi-experimental study. Pre- and post-survey instruments will include the use of the Technology Acceptance Model on the perception of the chatbot and students' content knowledge of the topic along with open-ended questions. Analysis of quantitative data will use descriptive and inferential statistics while qualitative data will thematically analysed through coding. Participants in the study will include 70 upper secondary and Junior college students taking O and A level Computing subject. The project is helmed by an inter-disciplinary research team made of members from National Institute of Education, Computer Science and Engineering and MOE bringing in their expertise in education research, AI, software engineering, and classroom intervention design. 

Funding body: MOE

Lead PI: Dr Khor Ean Teng Karen

Co-PIs: Dr Vidya Sudarson, Dr Shen Zhiqi

Collaborators: Dr Seow Sen Kee, Peter, Dr Koh Ruilin, Elizabeth, Mr Gi Soong Chee

Abstract:

We propose SingaKids Pic2Speak - a picture-prompted multilingual virtual tutor for Singaporean students in primary 1 and primary 2 to practice their oral skills in MTL (see Figure 1). In question generation mode, SingaKids Pic2Speak can generate question answer pairs from a picture so teachers can select suitable ones for students to practice. In dialogue mode, these teacher selected question answer pairs can also be used in to prompt students with both the picture and question to elicit spoken answers. SingaKids Pic2Speak will evaluate the student's response in terms of content accuracy, grammatical correctness, and oral delivery (e.g. pronunciation, fluency) and give corresponding feedback. In addition, the engagement level of the student will also be assessed through the speech signal. By considering both engagement and capability, we can preposition recommendations and intervention strategies to increase student learning outcome and interest. The technical challenges of this grand challenge stem from data sparsity for technology deployment needs, including under-resourced languages, child usage in the education context, and processing of pictorial drawings. We will empirically fill in these gaps in academic research, which focuses largely on adults speaking American English, applications used by adults, and processing photographic images in computer vision. As far as we know, few scientists have investigated all these gaps empirically. We use hybrid AI as an overarching technical framework to tackle these real-world challenges. We will exploit domain knowledge from education experts to sharpen the solution space to reduce entropy of unseen scenarios (e.g. leveraging on expected error patterns in Singapore's English dominant multilingual landscape), integrate classic parse trees in linguistics and neural representations to address multilingual challenges, combine high level visual understanding (e.g. scene graphs) with low level semantics (e.g. image recognition) when adapting pre-trained models to work well on pictorial drawings with domain data, and take advantage of adversarial adaptation and acousticphonetics insights to address sparsity challenges of child data. We will also develop SingaKids-VQAP, a multilingual dataset for visual question, answer, and paraphrasing to better anchor these targeted technical endeavours. We will investigate these scientific endeavours by building upon our rich suite of background IPs and data assets in multilingual speech and language technology, which have led to recent commercial spin-offs (e.g. nomopai, KiteSense) and a multitude of government deployments (e.g. Whole of Government deployment for SG Translate, Malay and Tamil speech synthesis and evaluation at MOE SLS and iMTL, English speech evaluation deployment at SEAB, amongst numerous confidential deployments). Our team is skilled in efficient data modelling for translation and deployment needs. In particular, we exploit deep learning integrated with domain knowledge modelling to achieve high performance with minimal data usage. Such efforts have won us numerous local and international awards. We also have expertise in constructing and executing protocols for data collection, curation, annotation and analysis for real-world deployment to service partners and clients from education, healthcare, and defence sectors. In addition, we have strong backing in the local ecosystem from both commercial collaborators (e.g. Ednovation, NovoLearning) and public partners (e.g. SEAB) to jointly construct multimodal data assets, adopt our technology to enhance services, and spin-off companies. The total value-add from this project to these collaborators is estimated to be S$5.25 M/yr.

Funding body: AI Singapore - AI in Education Grand Challenge

Lead PI: Dr Nancy F. Chen

Co-PIs: Ms Suryani Binte Atan, Assoc Prof Seetha Lakshmi, Dr Tan Hui Li, Dr Dong Minghui, Asst Prof Zhang Hanwang, Assoc Prof Kan Min Yen, Assoc Prof Chng Eng Siong, Dr Wong Lung Hsiang, Dr Sun He, Dr Goh Hock Huan, Dr Khor Ean Teng Karen

Collaborators: Mr Richard Yen, Mr Martijn Enter, Dr Kang Mei-Ling

Abstract: 

With four official languages, the linguistic environment of Singapore is globally unique, with rich, inter-generational bilingualism in diverse languages, with diverse writing systems. This landscape provides unique advantages for Singapore’s progress in an increasingly globalised world. It also presents unique challenges in education, as teaching practices established in monolingual communities (like the US, the UK and Australia) may be poorly suited to the needs of Singapore’s bilingual learners. For this reason, it is critical to conduct investigations into how language and literacy develop within Singapore’s own bilingual communities, in order to support the educational needs of future generations. The lack of global research into early bilingualism means that Singaporeans do not have good guidance about how best to support children’s language needs in early care contexts and the pre-school years. Similarly, a lack of research into bilingual reading development means that dyslexia diagnosis and support is not tailored to the needs of Singaporeans whose linguistic experience may generate different patterns of strengths and weaknesses in the early reading years.

This research will evaluate how different patterns of exposure influence bilingual and biliterate language development in three research streams. (Birth to 2 years). We will record language spoken around infants and toddlers in the home, and evaluate how it contributes to the sensitivity, flexibility, and speed of toddlers’ aural-language skills. (pre-K to Grade 1). We will investigate the link between early language skills and learning to read, in a group of children evaluated for their early aural and pre-reading skills pre-kindergarten, and their reading and brain network connectivity after school entry.

Lead PI: Asst Prof Suzy Styles
Co-PI: Dr Beth O'Brien (study lead), Prof Annabel Chen, A/P Justin Dauwels (TU Delft)