-
LLM-Assisted Knowledge Graph Completion for Curriculum and Domain Modelling in Personalized Higher Education Recommendations
Authors:
Hasan Abu-Rasheed,
Constance Jumbo,
Rashed Al Amin,
Christian Weber,
Veit Wiese,
Roman Obermaisser,
Madjid Fathi
Abstract:
While learning personalization offers great potential for learners, modern practices in higher education require a deeper consideration of domain models and learning contexts, to develop effective personalization algorithms. This paper introduces an innovative approach to higher education curriculum modelling that utilizes large language models (LLMs) for knowledge graph (KG) completion, with the…
▽ More
While learning personalization offers great potential for learners, modern practices in higher education require a deeper consideration of domain models and learning contexts, to develop effective personalization algorithms. This paper introduces an innovative approach to higher education curriculum modelling that utilizes large language models (LLMs) for knowledge graph (KG) completion, with the goal of creating personalized learning-path recommendations. Our research focuses on modelling university subjects and linking their topics to corresponding domain models, enabling the integration of learning modules from different faculties and institutions in the student's learning path. Central to our approach is a collaborative process, where LLMs assist human experts in extracting high-quality, fine-grained topics from lecture materials. We develop a domain, curriculum, and user models for university modules and stakeholders. We implement this model to create the KG from two study modules: Embedded Systems and Development of Embedded Systems Using FPGA. The resulting KG structures the curriculum and links it to the domain models. We evaluate our approach through qualitative expert feedback and quantitative graph quality metrics. Domain experts validated the relevance and accuracy of the model, while the graph quality metrics measured the structural properties of our KG. Our results show that the LLM-assisted graph completion approach enhances the ability to connect related courses across disciplines to personalize the learning experience. Expert feedback also showed high acceptance of the proposed collaborative approach for concept extraction and classification.
△ Less
Submitted 21 January, 2025;
originally announced January 2025.
-
Knowledge Graphs as Context Sources for LLM-Based Explanations of Learning Recommendations
Authors:
Hasan Abu-Rasheed,
Christian Weber,
Madjid Fathi
Abstract:
In the era of personalized education, the provision of comprehensible explanations for learning recommendations is of a great value to enhance the learner's understanding and engagement with the recommended learning content. Large language models (LLMs) and generative AI in general have recently opened new doors for generating human-like explanations, for and along learning recommendations. Howeve…
▽ More
In the era of personalized education, the provision of comprehensible explanations for learning recommendations is of a great value to enhance the learner's understanding and engagement with the recommended learning content. Large language models (LLMs) and generative AI in general have recently opened new doors for generating human-like explanations, for and along learning recommendations. However, their precision is still far away from acceptable in a sensitive field like education. To harness the abilities of LLMs, while still ensuring a high level of precision towards the intent of the learners, this paper proposes an approach to utilize knowledge graphs (KG) as a source of factual context, for LLM prompts, reducing the risk of model hallucinations, and safeguarding against wrong or imprecise information, while maintaining an application-intended learning context. We utilize the semantic relations in the knowledge graph to offer curated knowledge about learning recommendations. With domain-experts in the loop, we design the explanation as a textual template, which is filled and completed by the LLM. Domain experts were integrated in the prompt engineering phase as part of a study, to ensure that explanations include information that is relevant to the learner. We evaluate our approach quantitatively using Rouge-N and Rouge-L measures, as well as qualitatively with experts and learners. Our results show an enhanced recall and precision of the generated explanations compared to those generated solely by the GPT model, with a greatly reduced risk of generating imprecise information in the final learning explanation.
△ Less
Submitted 5 March, 2024;
originally announced March 2024.
-
Experimental Interface for Multimodal and Large Language Model Based Explanations of Educational Recommender Systems
Authors:
Hasan Abu-Rasheed,
Christian Weber,
Madjid Fathi
Abstract:
In the age of artificial intelligence (AI), providing learners with suitable and sufficient explanations of AI-based recommendation algorithm's output becomes essential to enable them to make an informed decision about it. However, the rapid development of AI approaches for educational recommendations and their explainability is not accompanied by an equal level of evidence-based experimentation t…
▽ More
In the age of artificial intelligence (AI), providing learners with suitable and sufficient explanations of AI-based recommendation algorithm's output becomes essential to enable them to make an informed decision about it. However, the rapid development of AI approaches for educational recommendations and their explainability is not accompanied by an equal level of evidence-based experimentation to evaluate the learning effect of those explanations. To address this issue, we propose an experimental web-based tool for evaluating multimodal and large language model (LLM) based explainability approaches. Our tool provides a comprehensive set of modular, interactive, and customizable explainability elements, which researchers and educators can utilize to study the role of individual and hybrid explainability methods. We design a two-stage evaluation of the proposed tool, with learners and with educators. Our preliminary results from the first stage show high acceptance of the tool's components, user-friendliness, and an induced motivation to use the explanations for exploring more information about the recommendation.
△ Less
Submitted 29 January, 2024;
originally announced February 2024.
-
Building Contextual Knowledge Graphs for Personalized Learning Recommendations using Text Mining and Semantic Graph Completion
Authors:
Hasan Abu-Rasheed,
Mareike Dornhöfer,
Christian Weber,
Gábor Kismihók,
Ulrike Buchmann,
Madjid Fathi
Abstract:
Modelling learning objects (LO) within their context enables the learner to advance from a basic, remembering-level, learning objective to a higher-order one, i.e., a level with an application- and analysis objective. While hierarchical data models are commonly used in digital learning platforms, using graph-based models enables representing the context of LOs in those platforms. This leads to a f…
▽ More
Modelling learning objects (LO) within their context enables the learner to advance from a basic, remembering-level, learning objective to a higher-order one, i.e., a level with an application- and analysis objective. While hierarchical data models are commonly used in digital learning platforms, using graph-based models enables representing the context of LOs in those platforms. This leads to a foundation for personalized recommendations of learning paths. In this paper, the transformation of hierarchical data models into knowledge graph (KG) models of LOs using text mining is introduced and evaluated. We utilize custom text mining pipelines to mine semantic relations between elements of an expert-curated hierarchical model. We evaluate the KG structure and relation extraction using graph quality-control metrics and the comparison of algorithmic semantic-similarities to expert-defined ones. The results show that the relations in the KG are semantically comparable to those defined by domain experts, and that the proposed KG improves representing and linking the contexts of LOs through increasing graph communities and betweenness centrality.
△ Less
Submitted 24 January, 2024;
originally announced January 2024.
-
Supporting Student Decisions on Learning Recommendations: An LLM-Based Chatbot with Knowledge Graph Contextualization for Conversational Explainability and Mentoring
Authors:
Hasan Abu-Rasheed,
Mohamad Hussam Abdulsalam,
Christian Weber,
Madjid Fathi
Abstract:
Student commitment towards a learning recommendation is not separable from their understanding of the reasons it was recommended to them; and their ability to modify it based on that understanding. Among explainability approaches, chatbots offer the potential to engage the student in a conversation, similar to a discussion with a peer or a mentor. The capabilities of chatbots, however, are still n…
▽ More
Student commitment towards a learning recommendation is not separable from their understanding of the reasons it was recommended to them; and their ability to modify it based on that understanding. Among explainability approaches, chatbots offer the potential to engage the student in a conversation, similar to a discussion with a peer or a mentor. The capabilities of chatbots, however, are still not sufficient to replace a human mentor, despite the advancements of generative AI (GenAI) and large language models (LLM). Therefore, we propose an approach to utilize chatbots as mediators of the conversation and sources of limited and controlled generation of explanations, to harvest the potential of LLMs while reducing their potential risks at the same time. The proposed LLM-based chatbot supports students in understanding learning-paths recommendations. We use a knowledge graph (KG) as a human-curated source of information, to regulate the LLM's output through defining its prompt's context. A group chat approach is developed to connect students with human mentors, either on demand or in cases that exceed the chatbot's pre-defined tasks. We evaluate the chatbot with a user study, to provide a proof-of-concept and highlight the potential requirements and limitations of utilizing chatbots in conversational explainability.
△ Less
Submitted 24 January, 2024; v1 submitted 16 January, 2024;
originally announced January 2024.
-
Context based learning: a survey of contextual indicators for personalized and adaptive learning recommendations. A pedagogical and technical perspective
Authors:
Hasan Abu-Rasheed,
Christian Weber,
Madjid Fathi
Abstract:
Learning personalization has proven its effectiveness in enhancing learner performance. Therefore, modern digital learning platforms have been increasingly depending on recommendation systems to offer learners personalized suggestions of learning materials. Learners can utilize those recommendations to acquire certain skills for the labor market or for their formal education. Personalization can b…
▽ More
Learning personalization has proven its effectiveness in enhancing learner performance. Therefore, modern digital learning platforms have been increasingly depending on recommendation systems to offer learners personalized suggestions of learning materials. Learners can utilize those recommendations to acquire certain skills for the labor market or for their formal education. Personalization can be based on several factors, such as personal preference, social connections or learning context. In an educational environment, the learning context plays an important role in generating sound recommendations, which not only fulfill the preferences of the learner, but also correspond to the pedagogical goals of the learning process. This is because a learning context describes the actual situation of the learner at the moment of requesting a learning recommendation. It provides information about the learner current state of knowledge, goal orientation, motivation, needs, available time, and other factors that reflect their status and may influence how learning recommendations are perceived and utilized. Context aware recommender systems have the potential to reflect the logic that a learning expert may follow in recommending materials to students with respect to their status and needs. In this paper, we review the state-of-the-art approaches for defining a user learning-context. We provide an overview of the definitions available, as well as the different factors that are considered when defining a context. Moreover, we further investigate the links between those factors and their pedagogical foundations in learning theories. We aim to provide a comprehensive understanding of contextualized learning from both pedagogical and technical points of view. By combining those two viewpoints, we aim to bridge a gap between both domains, in terms of contextualizing learning recommendations.
△ Less
Submitted 31 August, 2023;
originally announced August 2023.
-
EduCOR: An Educational and Career-Oriented Recommendation Ontology
Authors:
Eleni Ilkou,
Hasan Abu-Rasheed,
Mohammadreza Tavakoli,
Sherzod Hakimov,
Gábor Kismihók,
Sören Auer,
Wolfgang Nejdl
Abstract:
With the increased dependence on online learning platforms and educational resource repositories, a unified representation of digital learning resources becomes essential to support a dynamic and multi-source learning experience. We introduce the EduCOR ontology, an educational, career-oriented ontology that provides a foundation for representing online learning resources for personalised learning…
▽ More
With the increased dependence on online learning platforms and educational resource repositories, a unified representation of digital learning resources becomes essential to support a dynamic and multi-source learning experience. We introduce the EduCOR ontology, an educational, career-oriented ontology that provides a foundation for representing online learning resources for personalised learning systems. The ontology is designed to enable learning material repositories to offer learning path recommendations, which correspond to the user's learning goals, academic and psychological parameters, and the labour-market skills. We present the multiple patterns that compose the EduCOR ontology, highlighting its cross-domain applicability and integrability with other ontologies. A demonstration of the proposed ontology on the real-life learning platform eDoer is discussed as a use-case. We evaluate the EduCOR ontology using both gold standard and task-based approaches. The comparison of EduCOR to three gold schemata, and its application in two use-cases, shows its coverage and adaptability to multiple OER repositories, which allows generating user-centric and labour-market oriented recommendations.
△ Less
Submitted 13 July, 2021; v1 submitted 12 July, 2021;
originally announced July 2021.
-
Explainable Graph-based Search for Lessons-Learned Documents in the Semiconductor Industry
Authors:
Hasan Abu-Rasheed,
Christian Weber,
Johannes Zenkert,
Roland Krumm,
Madjid Fathi
Abstract:
Industrial processes produce a considerable volume of data and thus information. Whether it is structured sensory data or semi- to unstructured textual data, the knowledge that can be derived from it is critical to the sustainable development of the industrial process. A key challenge of this sustainability is the intelligent management of the generated data, as well as the knowledge extracted fro…
▽ More
Industrial processes produce a considerable volume of data and thus information. Whether it is structured sensory data or semi- to unstructured textual data, the knowledge that can be derived from it is critical to the sustainable development of the industrial process. A key challenge of this sustainability is the intelligent management of the generated data, as well as the knowledge extracted from it, in order to utilize this knowledge for improving future procedures. This challenge is a result of the tailored documentation methods and domain-specific requirements, which include the need for quick visibility of the documented knowledge. In this paper, we utilize the expert knowledge documented in chip-design failure reports in supporting user access to information that is relevant to a current chip design. Unstructured, free, textual data in previous failure documentations provides a valuable source of lessons-learned, which expert design-engineers have experienced, solved and documented. To achieve a sustainable utilization of knowledge within the company, not only the inherent knowledge has to be mined from unstructured textual data, but also the relations between the lessons-learned, uncovering potentially unknown links. In this research, a knowledge graph is constructed, in order to represent and use the interconnections between reported design failures. A search engine is developed and applied onto the graph to answer queries. In contrast to mere keyword-based searching, the searchability of the knowledge graph offers enhanced search results beyond direct matches and acts as a mean for generating explainable results and result recommendations. Results are provided to the design engineer through an interactive search interface, in which, the feedback from the user is used to further optimize relations for future iterations of the knowledge graph.
△ Less
Submitted 20 June, 2021; v1 submitted 18 May, 2021;
originally announced May 2021.
-
A Text Extraction-Based Smart Knowledge Graph Composition for Integrating Lessons Learned during the Microchip Design
Authors:
H. Abu-Rasheed,
C. Weber,
J. Zenkert,
P. Czerner,
R. Krumm,
M. Fathi
Abstract:
The production of microchips is a complex and thus well documented process. Therefore, available textual data about the production can be overwhelming in terms of quantity. This affects the visibility and retrieval of a certain piece of information when it is most needed. In this paper, we propose a dynamic approach to interlink the information extracted from multisource production-relevant docume…
▽ More
The production of microchips is a complex and thus well documented process. Therefore, available textual data about the production can be overwhelming in terms of quantity. This affects the visibility and retrieval of a certain piece of information when it is most needed. In this paper, we propose a dynamic approach to interlink the information extracted from multisource production-relevant documents through the creation of a knowledge graph. This graph is constructed in order to support searchability and enhance user's access to large-scale production information. Text mining methods are firstly utilized to extract data from multiple documentation sources. Document relations are then mined and extracted for the composition of the knowledge graph. Graph search functionality is then supported with a recommendation use-case to enhance users' access to information that is related to the initial documents. The proposed approach is tailored to and tested on microchip design-relevant documents. It enhances the visibility and findability of previous design-failure-cases during the process of a new chip design.
△ Less
Submitted 11 May, 2021;
originally announced May 2021.
