-
Evaluating AI-Powered Learning Assistants in Engineering Higher Education: Student Engagement, Ethical Challenges, and Policy Implications
Authors:
Ramteja Sajja,
Yusuf Sermet,
Brian Fodale,
Ibrahim Demir
Abstract:
As generative AI tools become increasingly integrated into higher education, understanding how students interact with and perceive these technologies is essential for responsible and effective adoption. This study evaluates the use of the Educational AI Hub, an AI-powered learning framework, in undergraduate civil and environmental engineering courses at a large R1 public university. Using a mixed…
▽ More
As generative AI tools become increasingly integrated into higher education, understanding how students interact with and perceive these technologies is essential for responsible and effective adoption. This study evaluates the use of the Educational AI Hub, an AI-powered learning framework, in undergraduate civil and environmental engineering courses at a large R1 public university. Using a mixed-methods approach that combines pre- and post-surveys, system usage logs, and qualitative analysis of the open-ended prompts and questions students posed to the AI chatbot, the research explores students' perceptions of trust, ethical concerns, usability, and learning outcomes. Findings reveal that students appreciated the AI assistant for its convenience and comfort, with nearly half reporting greater ease in using the AI tool compared to seeking help from instructors or teaching assistants. The tool was seen as most helpful for completing homework and understanding course concepts, though perceptions of its instructional quality were mixed. Ethical concerns emerged as a key barrier to full engagement: while most students viewed AI use as ethically acceptable, many expressed uncertainties about institutional policies and apprehension about potential academic misconduct. This study contributes to the growing body of research on AI in education by highlighting the importance of usability, policy clarity, and faculty guidance in fostering meaningful AI engagement. The findings suggest that while students are ready to embrace AI as a supplement to human instruction, thoughtful integration and transparent institutional frameworks are critical for ensuring student confidence, trust, and learning effectiveness.
△ Less
Submitted 5 June, 2025;
originally announced June 2025.
-
An Open-Source Dual-Loss Embedding Model for Semantic Retrieval in Higher Education
Authors:
Ramteja Sajja,
Yusuf Sermet,
Ibrahim Demir
Abstract:
Recent advances in AI have catalyzed the adoption of intelligent educational tools, yet many semantic retrieval systems remain ill-suited to the unique linguistic and structural characteristics of academic content. This study presents two open-source embedding models fine-tuned for educational question answering, particularly in the context of course syllabi. A synthetic dataset of 3,197 sentence…
▽ More
Recent advances in AI have catalyzed the adoption of intelligent educational tools, yet many semantic retrieval systems remain ill-suited to the unique linguistic and structural characteristics of academic content. This study presents two open-source embedding models fine-tuned for educational question answering, particularly in the context of course syllabi. A synthetic dataset of 3,197 sentence pairs, spanning synonymous terminology, paraphrased questions, and implicit-explicit mappings, was constructed through a combination of manual curation and large language model (LLM)-assisted generation. Two training strategies were evaluated: (1) a baseline model fine-tuned using MultipleNegativesRankingLoss (MNRL), and (2) a dual-loss model that combines MNRL with CosineSimilarityLoss to improve both semantic ranking and similarity calibration. Evaluations were conducted on 28 university course syllabi using a fixed set of natural language questions categorized into course, faculty, and teaching assistant information. Results demonstrate that both fine-tuned models outperform strong open-source baselines, including all-MiniLM-L6-v2 and multi-qa-MiniLM-L6-cos-v1, and that the dual-loss model narrows the performance gap with high-performing proprietary embeddings such as OpenAI's text-embedding-3 series. This work contributes reusable, domain-aligned embedding models and provides a replicable framework for educational semantic retrieval, supporting downstream applications such as academic chatbots, retrieval-augmented generation (RAG) systems, and learning management system (LMS) integrations.
△ Less
Submitted 7 May, 2025;
originally announced May 2025.
-
Enhancing Collective Intelligence in Large Language Models Through Emotional Integration
Authors:
Likith Kadiyala,
Ramteja Sajja,
Yusuf Sermet,
Ibrahim Demir
Abstract:
This research investigates the integration of emotional diversity into Large Language Models (LLMs) to enhance collective intelligence. Inspired by the human wisdom of crowds phenomenon, where group decisions often outperform individual judgments, we fine-tuned the DarkIdol-Llama-3.1-8B model using Google's GoEmotions dataset and Low-Rank Adaptation (LoRA) to simulate emotionally diverse responses…
▽ More
This research investigates the integration of emotional diversity into Large Language Models (LLMs) to enhance collective intelligence. Inspired by the human wisdom of crowds phenomenon, where group decisions often outperform individual judgments, we fine-tuned the DarkIdol-Llama-3.1-8B model using Google's GoEmotions dataset and Low-Rank Adaptation (LoRA) to simulate emotionally diverse responses. Evaluating the model on a distance estimation task between Fargo, ND, and Seattle, WA, across 15,064 unique persona configurations, we analyzed how emotional states and social attributes influence decision-making. Our findings demonstrate that emotional integration shapes response patterns while maintaining acceptable prediction accuracy, revealing its potential to enhance artificial collective intelligence. This study provides valuable insights into the interplay of emotional diversity and decision-making in LLMs, suggesting pathways for creating emotionally aware AI systems that balance emotional depth with analytical precision.
△ Less
Submitted 5 March, 2025;
originally announced March 2025.
-
Integrating Generative AI in Hackathons: Opportunities, Challenges, and Educational Implications
Authors:
Ramteja Sajja,
Carlos Erazo Ramirez,
Zhouyayan Li,
Bekir Z. Demiray,
Yusuf Sermet,
Ibrahim Demir
Abstract:
Hackathons have emerged as pivotal platforms in the software industry, driving both innovation and skill development for organizations and students alike. These events enable companies to quickly prototype new ideas while offering students practical, hands-on learning experiences. Over time, hackathons have transitioned from purely competitive events to valuable educational tools, integrating theo…
▽ More
Hackathons have emerged as pivotal platforms in the software industry, driving both innovation and skill development for organizations and students alike. These events enable companies to quickly prototype new ideas while offering students practical, hands-on learning experiences. Over time, hackathons have transitioned from purely competitive events to valuable educational tools, integrating theory with real-world problem-solving through collaboration between academia and industry. The infusion of artificial intelligence (AI) and machine learning is now reshaping hackathons, providing enhanced learning opportunities while also introducing ethical challenges. This study explores the influence of generative AI on students' technological choices, focusing on a case study from the 2023 University of Iowa Hackathon. The findings offer insights into AI's role in these events, its educational impact, and propose strategies for integrating such technologies in future hackathons, ensuring a balance between innovation, ethics, and educational value.
△ Less
Submitted 18 September, 2024; v1 submitted 30 January, 2024;
originally announced January 2024.
-
Integrating AI and Learning Analytics for Data-Driven Pedagogical Decisions and Personalized Interventions in Education
Authors:
Ramteja Sajja,
Yusuf Sermet,
David Cwiertny,
Ibrahim Demir
Abstract:
This research study explores the conceptualization, development, and deployment of an innovative learning analytics tool, leveraging OpenAI's GPT-4 model to quantify student engagement, map learning progression, and evaluate diverse instructional strategies within an educational context. By analyzing critical data points such as students' stress levels, curiosity, confusion, agitation, topic prefe…
▽ More
This research study explores the conceptualization, development, and deployment of an innovative learning analytics tool, leveraging OpenAI's GPT-4 model to quantify student engagement, map learning progression, and evaluate diverse instructional strategies within an educational context. By analyzing critical data points such as students' stress levels, curiosity, confusion, agitation, topic preferences, and study methods, the tool provides a comprehensive view of the learning environment. It also employs Bloom's taxonomy to assess cognitive development based on student inquiries. In addition to technical evaluation through synthetic data, feedback from a survey of teaching faculty at the University of Iowa was collected to gauge perceived benefits and challenges. Faculty recognized the tool's potential to enhance instructional decision-making through real-time insights but expressed concerns about data security and the accuracy of AI-generated insights. The study outlines the design, implementation, and evaluation of the tool, highlighting its contributions to educational outcomes, practical integration within learning management systems, and future refinements needed to address privacy and accuracy concerns. This research underscores AI's role in shaping personalized, data-driven education.
△ Less
Submitted 18 September, 2024; v1 submitted 15 December, 2023;
originally announced December 2023.
-
Artificial Intelligence-Enabled Intelligent Assistant for Personalized and Adaptive Learning in Higher Education
Authors:
Ramteja Sajja,
Yusuf Sermet,
Muhammed Cikmaz,
David Cwiertny,
Ibrahim Demir
Abstract:
This paper presents a novel framework, Artificial Intelligence-Enabled Intelligent Assistant (AIIA), for personalized and adaptive learning in higher education. The AIIA system leverages advanced AI and Natural Language Processing (NLP) techniques to create an interactive and engaging learning platform. This platform is engineered to reduce cognitive load on learners by providing easy access to in…
▽ More
This paper presents a novel framework, Artificial Intelligence-Enabled Intelligent Assistant (AIIA), for personalized and adaptive learning in higher education. The AIIA system leverages advanced AI and Natural Language Processing (NLP) techniques to create an interactive and engaging learning platform. This platform is engineered to reduce cognitive load on learners by providing easy access to information, facilitating knowledge assessment, and delivering personalized learning support tailored to individual needs and learning styles. The AIIA's capabilities include understanding and responding to student inquiries, generating quizzes and flashcards, and offering personalized learning pathways. The research findings have the potential to significantly impact the design, implementation, and evaluation of AI-enabled Virtual Teaching Assistants (VTAs) in higher education, informing the development of innovative educational tools that can enhance student learning outcomes, engagement, and satisfaction. The paper presents the methodology, system architecture, intelligent services, and integration with Learning Management Systems (LMSs) while discussing the challenges, limitations, and future directions for the development of AI-enabled intelligent assistants in education.
△ Less
Submitted 19 September, 2023;
originally announced September 2023.
-
Platform-Independent and Curriculum-Oriented Intelligent Assistant for Higher Education
Authors:
Ramteja Sajja,
Yusuf Sermet,
David Cwiertny,
Ibrahim Demir
Abstract:
Miscommunication and communication challenges between instructors and students represents one of the primary barriers to post-secondary learning. Students often avoid or miss opportunities to ask questions during office hours due to insecurities or scheduling conflicts. Moreover, students need to work at their own pace to have the freedom and time for the self-contemplation needed to build concept…
▽ More
Miscommunication and communication challenges between instructors and students represents one of the primary barriers to post-secondary learning. Students often avoid or miss opportunities to ask questions during office hours due to insecurities or scheduling conflicts. Moreover, students need to work at their own pace to have the freedom and time for the self-contemplation needed to build conceptual understanding and develop creative thinking skills. To eliminate barriers to student engagement, academic institutions need to redefine their fundamental approach to education by proposing flexible educational pathways that recognize continuous learning. To this end, we developed an AI-augmented intelligent educational assistance framework based on a power language model (i.e., GPT-3) that automatically generates course-specific intelligent assistants regardless of discipline or academic level. The virtual intelligent teaching assistant (TA) system will serve as a voice-enabled helper capable of answering course-specific questions concerning curriculum, logistics and course policies. It is envisioned to improve access to course-related information for the students and reduce logistical workload for the instructors and TAs. Its GPT-3-based knowledge discovery component as well as the generalized system architecture is presented accompanied by a methodical evaluation of the system accuracy and performance.
△ Less
Submitted 15 February, 2023;
originally announced February 2023.
