-
Do Students Write Better Post-AI Support? Effects of Generative AI Literacy and Chatbot Interaction Strategies on Multimodal Academic Writing
Authors:
Yueqiao Jin,
Kaixun Yang,
Roberto Martinez-Maldonado,
Dragan Gašević,
Lixiang Yan
Abstract:
Academic writing increasingly involves multimodal tasks requiring students to integrate visual information and textual arguments. While generative AI (GenAI) tools, like ChatGPT, offer new pathways for supporting academic writing, little is known about how students' GenAI literacy influences their independent multimodal writing skills or how chatbot interaction strategies (passive reactive vs. pro…
▽ More
Academic writing increasingly involves multimodal tasks requiring students to integrate visual information and textual arguments. While generative AI (GenAI) tools, like ChatGPT, offer new pathways for supporting academic writing, little is known about how students' GenAI literacy influences their independent multimodal writing skills or how chatbot interaction strategies (passive reactive vs. proactive scaffolding) impact learning. This study examined 79 higher education students' multimodal academic writing performance using a comparative research design. Students completed writing tasks integrating visual data under two chatbot-assisted conditions (passive vs. proactive) and subsequently without AI assistance. Their writing performance was rigorously evaluated across five dimensions, including insightfulness, visual data integration, organisation, linguistic quality, and critical thinking. Ordinal logistic regression and correlation analyses revealed that higher levels of GenAI literacy significantly predicted stronger independent multimodal writing performance immediately after AI assistance removal, particularly for students using passive chatbots requiring active prompting. These results highlight the critical role of GenAI literacy and specific chatbot interaction strategies in shaping students' capacities for independent multimodal academic writing. Our findings emphasise the need for purposeful integration of GenAI literacy training into curricula and balancing external scaffolding support with autonomous learning opportunities. This research offers valuable recommendations for educators leveraging AI-enhanced pedagogies to optimise student writing outcomes and technological engagement strategies.
△ Less
Submitted 6 July, 2025;
originally announced July 2025.
-
Novobo: Supporting Teachers' Peer Learning of Instructional Gestures by Teaching a Mentee AI-Agent Together
Authors:
Jiaqi Jiang,
Kexin Huang,
Roberto Martinez-Maldonado,
Huan Zeng,
Duo Gong,
Pengcheng An
Abstract:
Instructional gestures are essential for teaching, as they enhance communication and support student comprehension. However, existing training methods for developing these embodied skills can be time-consuming, isolating, or overly prescriptive. Research suggests that developing these tacit, experiential skills requires teachers' peer learning, where they learn from each other and build shared kno…
▽ More
Instructional gestures are essential for teaching, as they enhance communication and support student comprehension. However, existing training methods for developing these embodied skills can be time-consuming, isolating, or overly prescriptive. Research suggests that developing these tacit, experiential skills requires teachers' peer learning, where they learn from each other and build shared knowledge. This paper introduces Novobo, an apprentice AI-agent stimulating teachers' peer learning of instructional gestures through verbal and bodily inputs. Positioning the AI as a mentee employs the learning-by-teaching paradigm, aiming to promote deliberate reflection and active learning. Novobo encourages teachers to evaluate its generated gestures and invite them to provide demonstrations. An evaluation with 30 teachers in 10 collaborative sessions showed Novobo prompted teachers to share tacit knowledge through conversation and movement. This process helped teachers externalize, exchange, and internalize their embodied knowledge, promoting collaborative learning and building a shared understanding of instructional gestures within the local teaching community. This work advances understanding of how teachable AI agents can enhance collaborative learning in teacher professional development, offering valuable design insights for leveraging AI to promote the sharing and construction of embodied and practical knowledge.
△ Less
Submitted 25 May, 2025; v1 submitted 23 May, 2025;
originally announced May 2025.
-
TeamVision: An AI-powered Learning Analytics System for Supporting Reflection in Team-based Healthcare Simulation
Authors:
Vanessa Echeverria,
Linxuan Zhao,
Riordan Alfredo,
Mikaela Milesi,
Yuequiao Jin,
Sophie Abel,
Jie Fan,
Lixiang Yan,
Xinyu Li,
Samantha Dix,
Rosie Wotherspoon,
Hollie Jaggard,
Abra Osborne,
Simon Buckingham Shum,
Dragan Gasevic,
Roberto Martinez-Maldonado
Abstract:
Healthcare simulations help learners develop teamwork and clinical skills in a risk-free setting, promoting reflection on real-world practices through structured debriefs. However, despite video's potential, it is hard to use, leaving a gap in providing concise, data-driven summaries for supporting effective debriefing. Addressing this, we present TeamVision, an AI-powered multimodal learning anal…
▽ More
Healthcare simulations help learners develop teamwork and clinical skills in a risk-free setting, promoting reflection on real-world practices through structured debriefs. However, despite video's potential, it is hard to use, leaving a gap in providing concise, data-driven summaries for supporting effective debriefing. Addressing this, we present TeamVision, an AI-powered multimodal learning analytics (MMLA) system that captures voice presence, automated transcriptions, body rotation, and positioning data, offering educators a dashboard to guide debriefs immediately after simulations. We conducted an in-the-wild study with 56 teams (221 students) and recorded debriefs led by six teachers using TeamVision. Follow-up interviews with 15 students and five teachers explored perceptions of its usefulness, accuracy, and trustworthiness. This paper examines: i) how TeamVision was used in debriefing, ii) what educators found valuable and challenging, and iii) perceptions of its effectiveness. Results suggest TeamVision enables flexible debriefing and highlights the challenges and implications of using AI-powered systems in healthcare simulation.
△ Less
Submitted 4 February, 2025; v1 submitted 16 January, 2025;
originally announced January 2025.
-
Chatting with a Learning Analytics Dashboard: The Role of Generative AI Literacy on Learner Interaction with Conventional and Scaffolding Chatbots
Authors:
Yueqiao Jin,
Kaixun Yang,
Lixiang Yan,
Vanessa Echeverria,
Linxuan Zhao,
Riordan Alfredo,
Mikaela Milesi,
Jie Fan,
Xinyu Li,
Dragan Gašević,
Roberto Martinez-Maldonado
Abstract:
Learning analytics dashboards (LADs) simplify complex learner data into accessible visualisations, providing actionable insights for educators and students. However, their educational effectiveness has not always matched the sophistication of the technology behind them. Explanatory and interactive LADs, enhanced by generative AI (GenAI) chatbots, hold promise by enabling dynamic, dialogue-based in…
▽ More
Learning analytics dashboards (LADs) simplify complex learner data into accessible visualisations, providing actionable insights for educators and students. However, their educational effectiveness has not always matched the sophistication of the technology behind them. Explanatory and interactive LADs, enhanced by generative AI (GenAI) chatbots, hold promise by enabling dynamic, dialogue-based interactions with data visualisations and offering personalised feedback through text. Yet, the effectiveness of these tools may be limited by learners' varying levels of GenAI literacy, a factor that remains underexplored in current research. This study investigates the role of GenAI literacy in learner interactions with conventional (reactive) versus scaffolding (proactive) chatbot-assisted LADs. Through a comparative analysis of 81 participants, we examine how GenAI literacy is associated with learners' ability to interpret complex visualisations and their cognitive processes during interactions with chatbot-assisted LADs. Results show that while both chatbots significantly improved learner comprehension, those with higher GenAI literacy benefited the most, particularly with conventional chatbots, demonstrating diverse prompting strategies. Findings highlight the importance of considering learners' GenAI literacy when integrating GenAI chatbots in LADs and educational technologies. Incorporating scaffolding techniques within GenAI chatbots can be an effective strategy, offering a more guided experience that reduces reliance on learners' GenAI literacy.
△ Less
Submitted 23 November, 2024;
originally announced November 2024.
-
From Complexity to Parsimony: Integrating Latent Class Analysis to Uncover Multimodal Learning Patterns in Collaborative Learning
Authors:
Lixiang Yan,
Dragan Gašević,
Linxuan Zhao,
Vanessa Echeverria,
Yueqiao Jin,
Roberto Martinez-Maldonado
Abstract:
Multimodal Learning Analytics (MMLA) leverages advanced sensing technologies and artificial intelligence to capture complex learning processes, but integrating diverse data sources into cohesive insights remains challenging. This study introduces a novel methodology for integrating latent class analysis (LCA) within MMLA to map monomodal behavioural indicators into parsimonious multimodal ones. Us…
▽ More
Multimodal Learning Analytics (MMLA) leverages advanced sensing technologies and artificial intelligence to capture complex learning processes, but integrating diverse data sources into cohesive insights remains challenging. This study introduces a novel methodology for integrating latent class analysis (LCA) within MMLA to map monomodal behavioural indicators into parsimonious multimodal ones. Using a high-fidelity healthcare simulation context, we collected positional, audio, and physiological data, deriving 17 monomodal indicators. LCA identified four distinct latent classes: Collaborative Communication, Embodied Collaboration, Distant Interaction, and Solitary Engagement, each capturing unique monomodal patterns. Epistemic network analysis compared these multimodal indicators with the original monomodal indicators and found that the multimodal approach was more parsimonious while offering higher explanatory power regarding students' task and collaboration performances. The findings highlight the potential of LCA in simplifying the analysis of complex multimodal data while capturing nuanced, cross-modality behaviours, offering actionable insights for educators and enhancing the design of collaborative learning interventions. This study proposes a pathway for advancing MMLA, making it more parsimonious and manageable, and aligning with the principles of learner-centred education.
△ Less
Submitted 23 November, 2024;
originally announced November 2024.
-
GLAT: The Generative AI Literacy Assessment Test
Authors:
Yueqiao Jin,
Roberto Martinez-Maldonado,
Dragan Gašević,
Lixiang Yan
Abstract:
The rapid integration of generative artificial intelligence (GenAI) technology into education necessitates precise measurement of GenAI literacy to ensure that learners and educators possess the skills to engage with and critically evaluate this transformative technology effectively. Existing instruments often rely on self-reports, which may be biased. In this study, we present the GenAI Literacy…
▽ More
The rapid integration of generative artificial intelligence (GenAI) technology into education necessitates precise measurement of GenAI literacy to ensure that learners and educators possess the skills to engage with and critically evaluate this transformative technology effectively. Existing instruments often rely on self-reports, which may be biased. In this study, we present the GenAI Literacy Assessment Test (GLAT), a 20-item multiple-choice instrument developed following established procedures in psychological and educational measurement. Structural validity and reliability were confirmed with responses from 355 higher education students using classical test theory and item response theory, resulting in a reliable 2-parameter logistic (2PL) model (Cronbach's alpha = 0.80; omega total = 0.81) with a robust factor structure (RMSEA = 0.03; CFI = 0.97). Critically, GLAT scores were found to be significant predictors of learners' performance in GenAI-supported tasks, outperforming self-reported measures such as perceived ChatGPT proficiency and demonstrating external validity. These results suggest that GLAT offers a reliable and valid method for assessing GenAI literacy, with the potential to inform educational practices and policy decisions that aim to enhance learners' and educators' GenAI literacy, ultimately equipping them to navigate an AI-enhanced future.
△ Less
Submitted 19 November, 2024; v1 submitted 31 October, 2024;
originally announced November 2024.
-
The Effects of Generative AI Agents and Scaffolding on Enhancing Students' Comprehension of Visual Learning Analytics
Authors:
Lixiang Yan,
Roberto Martinez-Maldonado,
Yueqiao Jin,
Vanessa Echeverria,
Mikaela Milesi,
Jie Fan,
Linxuan Zhao,
Riordan Alfredo,
Xinyu Li,
Dragan Gašević
Abstract:
Visual learning analytics (VLA) is becoming increasingly adopted in educational technologies and learning analytics dashboards to convey critical insights to students and educators. Yet many students experienced difficulties in comprehending complex VLA due to their limited data visualisation literacy. While conventional scaffolding approaches like data storytelling have shown effectiveness in enh…
▽ More
Visual learning analytics (VLA) is becoming increasingly adopted in educational technologies and learning analytics dashboards to convey critical insights to students and educators. Yet many students experienced difficulties in comprehending complex VLA due to their limited data visualisation literacy. While conventional scaffolding approaches like data storytelling have shown effectiveness in enhancing students' comprehension of VLA, these approaches remain difficult to scale and adapt to individual learning needs. Generative AI (GenAI) technologies, especially conversational agents, offer potential solutions by providing personalised and dynamic support to enhance students' comprehension of VLA. This study investigates the effectiveness of GenAI agents, particularly when integrated with scaffolding techniques, in improving students' comprehension of VLA. A randomised controlled trial was conducted with 117 higher education students to compare the effects of two types of GenAI agents: passive agents, which respond to student queries, and proactive agents, which utilise scaffolding questions, against standalone scaffolding in a VLA comprehension task. The results show that passive agents yield comparable improvements to standalone scaffolding both during and after the intervention. Notably, proactive GenAI agents significantly enhance students' VLA comprehension compared to both passive agents and standalone scaffolding, with these benefits persisting beyond the intervention. These findings suggest that integrating GenAI agents with scaffolding can have lasting positive effects on students' comprehension skills and support genuine learning.
△ Less
Submitted 19 November, 2024; v1 submitted 17 September, 2024;
originally announced September 2024.
-
Generative AI in Higher Education: A Global Perspective of Institutional Adoption Policies and Guidelines
Authors:
Yueqiao Jin,
Lixiang Yan,
Vanessa Echeverria,
Dragan Gašević,
Roberto Martinez-Maldonado
Abstract:
Integrating generative AI (GAI) into higher education is crucial for preparing a future generation of GAI-literate students. Yet a thorough understanding of the global institutional adoption policy remains absent, with most of the prior studies focused on the Global North and the promises and challenges of GAI, lacking a theoretical lens. This study utilizes the Diffusion of Innovations Theory to…
▽ More
Integrating generative AI (GAI) into higher education is crucial for preparing a future generation of GAI-literate students. Yet a thorough understanding of the global institutional adoption policy remains absent, with most of the prior studies focused on the Global North and the promises and challenges of GAI, lacking a theoretical lens. This study utilizes the Diffusion of Innovations Theory to examine GAI adoption strategies in higher education across 40 universities from six global regions. It explores the characteristics of GAI innovation, including compatibility, trialability, and observability, and analyses the communication channels and roles and responsibilities outlined in university policies and guidelines. The findings reveal a proactive approach by universities towards GAI integration, emphasizing academic integrity, teaching and learning enhancement, and equity. Despite a cautious yet optimistic stance, a comprehensive policy framework is needed to evaluate the impacts of GAI integration and establish effective communication strategies that foster broader stakeholder engagement. The study highlights the importance of clear roles and responsibilities among faculty, students, and administrators for successful GAI integration, supporting a collaborative model for navigating the complexities of GAI in education. This study contributes insights for policymakers in crafting detailed strategies for its integration.
△ Less
Submitted 20 May, 2024;
originally announced May 2024.
-
FATE in MMLA: A Student-Centred Exploration of Fairness, Accountability, Transparency, and Ethics in Multimodal Learning Analytics
Authors:
Yueqiao Jin,
Vanessa Echeverria,
Lixiang Yan,
Linxuan Zhao,
Riordan Alfredo,
Yi-Shan Tsai,
Dragan Gašević,
Roberto Martinez-Maldonado
Abstract:
Multimodal Learning Analytics (MMLA) integrates novel sensing technologies and artificial intelligence algorithms, providing opportunities to enhance student reflection during complex, collaborative learning experiences. Although recent advancements in MMLA have shown its capability to generate insights into diverse learning behaviours across various learning settings, little research has been con…
▽ More
Multimodal Learning Analytics (MMLA) integrates novel sensing technologies and artificial intelligence algorithms, providing opportunities to enhance student reflection during complex, collaborative learning experiences. Although recent advancements in MMLA have shown its capability to generate insights into diverse learning behaviours across various learning settings, little research has been conducted to evaluate these systems in authentic learning contexts, particularly regarding students' perceived fairness, accountability, transparency, and ethics (FATE). Understanding these perceptions is essential to using MMLA effectively without introducing ethical complications or negatively affecting how students learn. This study aimed to address this gap by assessing the FATE of MMLA in an authentic, collaborative learning context. We conducted semi-structured interviews with 14 undergraduate students who used MMLA visualisations for post-activity reflection. The findings highlighted the significance of accurate and comprehensive data representation to ensure visualisation fairness, the need for different levels of data access to foster accountability, the imperative of measuring and cultivating transparency with students, and the necessity of transforming informed consent from dichotomous to continuous and measurable scales. While students value the benefits of MMLA, they also emphasise the importance of ethical considerations, highlighting a pressing need for the LA and MMLA community to investigate and address FATE issues actively.
△ Less
Submitted 29 February, 2024;
originally announced February 2024.
-
Data Storytelling in Data Visualisation: Does it Enhance the Efficiency and Effectiveness of Information Retrieval and Insights Comprehension?
Authors:
Honbo Shao,
Roberto Martinez-Maldonado,
Vanessa Echeverria,
Lixiang Yan,
Dragan Gasevic
Abstract:
Data storytelling (DS) is rapidly gaining attention as an approach that integrates data, visuals, and narratives to create data stories that can help a particular audience to comprehend the key messages underscored by the data with enhanced efficiency and effectiveness. It has been posited that DS can be especially advantageous for audiences with limited visualisation literacy, by presenting the d…
▽ More
Data storytelling (DS) is rapidly gaining attention as an approach that integrates data, visuals, and narratives to create data stories that can help a particular audience to comprehend the key messages underscored by the data with enhanced efficiency and effectiveness. It has been posited that DS can be especially advantageous for audiences with limited visualisation literacy, by presenting the data clearly and concisely. However, empirical studies confirming whether data stories indeed provide these benefits over conventional data visualisations are scarce. To bridge this gap, we conducted a study with 103 participants to determine whether DS indeed improve both efficiency and effectiveness in tasks related to information retrieval and insights comprehension. Our findings suggest that data stories do improve the efficiency of comprehension tasks, as well as the effectiveness of comprehension tasks that involve a single insight compared with conventional visualisations. Interestingly, these benefits were not associated with participants' visualisation literacy.
△ Less
Submitted 20 May, 2024; v1 submitted 19 February, 2024;
originally announced February 2024.
-
Human-Centred Learning Analytics and AI in Education: a Systematic Literature Review
Authors:
Riordan Alfredo,
Vanessa Echeverria,
Yueqiao Jin,
Lixiang Yan,
Zachari Swiecki,
Dragan Gašević,
Roberto Martinez-Maldonado
Abstract:
The rapid expansion of Learning Analytics (LA) and Artificial Intelligence in Education (AIED) offers new scalable, data-intensive systems but also raises concerns about data privacy and agency. Excluding stakeholders -- like students and teachers -- from the design process can potentially lead to mistrust and inadequately aligned tools. Despite a shift towards human-centred design in recent LA an…
▽ More
The rapid expansion of Learning Analytics (LA) and Artificial Intelligence in Education (AIED) offers new scalable, data-intensive systems but also raises concerns about data privacy and agency. Excluding stakeholders -- like students and teachers -- from the design process can potentially lead to mistrust and inadequately aligned tools. Despite a shift towards human-centred design in recent LA and AIED research, there remain gaps in our understanding of the importance of human control, safety, reliability, and trustworthiness in the design and implementation of these systems. We conducted a systematic literature review to explore these concerns and gaps. We analysed 108 papers to provide insights about i) the current state of human-centred LA/AIED research; ii) the extent to which educational stakeholders have contributed to the design process of human-centred LA/AIED systems; iii) the current balance between human control and computer automation of such systems; and iv) the extent to which safety, reliability and trustworthiness have been considered in the literature. Results indicate some consideration of human control in LA/AIED system design, but limited end-user involvement in actual design. Based on these findings, we recommend: 1) carefully balancing stakeholders' involvement in designing and deploying LA/AIED systems throughout all design phases, 2) actively involving target end-users, especially students, to delineate the balance between human control and automation, and 3) exploring safety, reliability, and trustworthiness as principles in future human-centred LA/AIED systems.
△ Less
Submitted 19 December, 2023;
originally announced December 2023.
-
Generative Artificial Intelligence in Learning Analytics: Contextualising Opportunities and Challenges through the Learning Analytics Cycle
Authors:
Lixiang Yan,
Roberto Martinez-Maldonado,
Dragan Gašević
Abstract:
Generative artificial intelligence (GenAI), exemplified by ChatGPT, Midjourney, and other state-of-the-art large language models and diffusion models, holds significant potential for transforming education and enhancing human productivity. While the prevalence of GenAI in education has motivated numerous research initiatives, integrating these technologies within the learning analytics (LA) cycle…
▽ More
Generative artificial intelligence (GenAI), exemplified by ChatGPT, Midjourney, and other state-of-the-art large language models and diffusion models, holds significant potential for transforming education and enhancing human productivity. While the prevalence of GenAI in education has motivated numerous research initiatives, integrating these technologies within the learning analytics (LA) cycle and their implications for practical interventions remain underexplored. This paper delves into the prospective opportunities and challenges GenAI poses for advancing LA. We present a concise overview of the current GenAI landscape and contextualise its potential roles within Clow's generic framework of the LA cycle. We posit that GenAI can play pivotal roles in analysing unstructured data, generating synthetic learner data, enriching multimodal learner interactions, advancing interactive and explanatory analytics, and facilitating personalisation and adaptive interventions. As the lines blur between learners and GenAI tools, a renewed understanding of learners is needed. Future research can delve deep into frameworks and methodologies that advocate for human-AI collaboration. The LA community can play a pivotal role in capturing data about human and AI contributions and exploring how they can collaborate most effectively. As LA advances, it is essential to consider the pedagogical implications and broader socioeconomic impact of GenAI for ensuring an inclusive future.
△ Less
Submitted 30 November, 2023;
originally announced December 2023.
-
Human-AI Collaboration in Thematic Analysis using ChatGPT: A User Study and Design Recommendations
Authors:
Lixiang Yan,
Vanessa Echeverria,
Gloria Fernandez Nieto,
Yueqiao Jin,
Zachari Swiecki,
Linxuan Zhao,
Dragan Gašević,
Roberto Martinez-Maldonado
Abstract:
Generative artificial intelligence (GenAI) offers promising potential for advancing human-AI collaboration in qualitative research. However, existing works focused on conventional machine-learning and pattern-based AI systems, and little is known about how researchers interact with GenAI in qualitative research. This work delves into researchers' perceptions of their collaboration with GenAI, spec…
▽ More
Generative artificial intelligence (GenAI) offers promising potential for advancing human-AI collaboration in qualitative research. However, existing works focused on conventional machine-learning and pattern-based AI systems, and little is known about how researchers interact with GenAI in qualitative research. This work delves into researchers' perceptions of their collaboration with GenAI, specifically ChatGPT. Through a user study involving ten qualitative researchers, we found ChatGPT to be a valuable collaborator for thematic analysis, enhancing coding efficiency, aiding initial data exploration, offering granular quantitative insights, and assisting comprehension for non-native speakers and non-experts. Yet, concerns about its trustworthiness and accuracy, reliability and consistency, limited contextual understanding, and broader acceptance within the research community persist. We contribute five actionable design recommendations to foster effective human-AI collaboration. These include incorporating transparent explanatory mechanisms, enhancing interface and integration capabilities, prioritising contextual understanding and customisation, embedding human-AI feedback loops and iterative functionality, and strengthening trust through validation mechanisms.
△ Less
Submitted 7 November, 2023;
originally announced November 2023.
-
Practical and Ethical Challenges of Large Language Models in Education: A Systematic Scoping Review
Authors:
Lixiang Yan,
Lele Sha,
Linxuan Zhao,
Yuheng Li,
Roberto Martinez-Maldonado,
Guanliang Chen,
Xinyu Li,
Yueqiao Jin,
Dragan Gašević
Abstract:
Educational technology innovations leveraging large language models (LLMs) have shown the potential to automate the laborious process of generating and analysing textual content. While various innovations have been developed to automate a range of educational tasks (e.g., question generation, feedback provision, and essay grading), there are concerns regarding the practicality and ethicality of th…
▽ More
Educational technology innovations leveraging large language models (LLMs) have shown the potential to automate the laborious process of generating and analysing textual content. While various innovations have been developed to automate a range of educational tasks (e.g., question generation, feedback provision, and essay grading), there are concerns regarding the practicality and ethicality of these innovations. Such concerns may hinder future research and the adoption of LLMs-based innovations in authentic educational contexts. To address this, we conducted a systematic scoping review of 118 peer-reviewed papers published since 2017 to pinpoint the current state of research on using LLMs to automate and support educational tasks. The findings revealed 53 use cases for LLMs in automating education tasks, categorised into nine main categories: profiling/labelling, detection, grading, teaching support, prediction, knowledge representation, feedback, content generation, and recommendation. Additionally, we also identified several practical and ethical challenges, including low technological readiness, lack of replicability and transparency, and insufficient privacy and beneficence considerations. The findings were summarised into three recommendations for future studies, including updating existing innovations with state-of-the-art models (e.g., GPT-3/4), embracing the initiative of open-sourcing models/systems, and adopting a human-centred approach throughout the developmental process. As the intersection of AI and education is continuously evolving, the findings of this study can serve as an essential reference point for researchers, allowing them to leverage the strengths, learn from the limitations, and uncover potential research opportunities enabled by ChatGPT and other generative AI models.
△ Less
Submitted 22 July, 2023; v1 submitted 17 March, 2023;
originally announced March 2023.
-
Lessons Learnt from a Multimodal Learning Analytics Deployment In-the-wild
Authors:
Roberto Martinez-Maldonado,
Vanessa Echeverria,
Gloria Fernandez-Nieto,
Lixiang Yan,
Linxuan Zhao,
Riordan Alfredo,
Xinyu Li,
Samantha Dix,
Hollie Jaggard,
Rosie Wotherspoon,
Abra Osborne,
Dragan Gašević,
Simon Buckingham Shum
Abstract:
Multimodal Learning Analytics (MMLA) innovations make use of rapidly evolving sensing and artificial intelligence algorithms to collect rich data about learning activities that unfold in physical learning spaces. The analysis of these data is opening exciting new avenues for both studying and supporting learning. Yet, practical and logistical challenges commonly appear while deploying MMLA innovat…
▽ More
Multimodal Learning Analytics (MMLA) innovations make use of rapidly evolving sensing and artificial intelligence algorithms to collect rich data about learning activities that unfold in physical learning spaces. The analysis of these data is opening exciting new avenues for both studying and supporting learning. Yet, practical and logistical challenges commonly appear while deploying MMLA innovations "in-the-wild". These can span from technical issues related to enhancing the learning space with sensing capabilities, to the increased complexity of teachers' tasks and informed consent. These practicalities have been rarely discussed. This paper addresses this gap by presenting a set of lessons learnt from a 2-year human-centred MMLA in-the-wild study conducted with 399 students and 17 educators. The lessons learnt were synthesised into topics related to i) technological/physical aspects of the deployment; ii) multimodal data and interfaces; iii) the design process; iv) participation, ethics and privacy; and v) the sustainability of the deployment.
△ Less
Submitted 16 March, 2023;
originally announced March 2023.
