An AI Educational Agent for Medical Machine Learning Courses

Официальное название

Application and Effectiveness of a Large Language Model-Based Educational Agent in Medical Education: A Study on the Machine Learning and Data Mining Course

Обзор

The goal of this interventional study is to evaluate the effectiveness of a Large Language Model (LLM)-based educational AI Agent in graduate students (Masters and PhD) specializing in medicine or nursing who are enrolled in the "Machine Learning and Data Mining" course. The main questions it aims to answer are: Does the use of an educational AI Agent improve students' academic performance and practical skills in machine learning compared to traditional methods? Does the AI intervention enhance students' learning confidence, satisfaction, and cognitive engagement? Researchers will compare students currently using the AI Agent (experimental group) to a historical control group (students from the previous cohort who did not use the AI tool) to see if the AI-assisted learning model leads to significantly higher learning achievements and better educational experiences. Participants will: Utilize the Teaching Agent for real-time answers to theoretical questions, personalized study planning, and knowledge reinforcement. Engage with the Research Agent to assist with literature reviews, research design optimization, and academic writing structure. Use the Practice Innovation Agent for guidance on coding, algorithm debugging, and applying machine learning models to medical data analysis projects.

Подробное описание

Background : Artificial Intelligence (AI) and data mining are becoming essential skills in modern medical and nursing research. However, traditional teaching methods for the graduate-level course "Machine Learning and Data Mining" often struggle to meet the personalized learning needs of students with varying technical backgrounds (e.g., programming, mathematics). To address this, this study introduces a custom-developed AI Educational Agent based on Large Language Models (LLMs) to serve as an intelligent teaching assistant.

Objectives: The primary objective is to evaluate the effectiveness of the AI Agent in improving learning outcomes, practical coding skills, and academic self-efficacy among medical and nursing graduate students. The study also aims to assess the feasibility and student satisfaction of integrating AI agents into the medical curriculum.

Study Design: This is a non-randomized interventional study utilizing a historical control design.

Study Design: This is a non-randomized interventional study utilizing a historical control design.

Experimental Group (Intervention): Students in the 2025-2026 academic year who will receive access to the AI Agent system.

Control Group (Historical): Students from the previous academic cohort (2024-2025) who completed the same curriculum using standard instruction methods without AI support.

Intervention Details: The intervention involves the deployment of an AI Agent system powered by LLMs and Knowledge Graph-based Retrieval-Augmented Generation (KGRAG). The KGRAG framework restricts the AI's responses to a verified knowledge base (course textbooks, lecture slides, and curated code repositories) to minimize "hallucinations" and ensure medical/scientific accuracy. The system includes three specialized functional modules:

Teaching Agent: Functions as a 24/7 tutor, providing concept explanations, summarizing key knowledge points, and offering personalized study plans based on student progress.

Research Agent: Supports research training by assisting with literature review, refining research questions, and optimizing academic writing structures.

Practice Innovation Agent: Facilitates practical skill acquisition by guiding students through code generation, debugging algorithms, and applying machine learning models to real-world medical datasets. The agent employs a Socratic tutoring method to guide problem-solving rather than providing direct answers.

Вмешательства

• Другое KGRAG-based AI Educational Agent System
  The intervention involves a custom-developed AI educational system powered by Large Language Models (LLMs) and Knowledge Graph-based Retrieval-Augmented Generation (KGRAG) technology. The system comprises three specialized agents to support self-directed learning: 1. Teaching Agent: Provides real-time concept explanations, personalized study plans, and knowledge reinforcement based on the course curriculum. 2. Research Agent: Assists with literature review, research question refinement, and acad

Первичные конечные точки

• Composite Academic Performance Score [Срок оценки: After the intervention (at the end of the course, approximately week 3)]

Вторичные конечные точки (5)

• Objective Knowledge Acquisition Rate [Срок оценки: After the intervention (at the end of the course, approximately week 3)]
• Perceived Usefulness and Technology Acceptance [Срок оценки: After the intervention (at the end of the course, approximately week 3)]
• AI Agent Engagement: Interaction Frequency [Срок оценки: At the end of the course (approximately Week 3)]
• AI Agent Engagement: Temporal Patterns [Срок оценки: At the end of the course (approximately Week 3)]
• AI Agent Engagement: Query Themes [Срок оценки: At the end of the course (approximately Week 3)]

Критерии участия

Критерии включения

• Medical graduate students from universities in the Guangdong-Hong Kong-Macao Greater Bay Area;
• Graduate students who have taken the "Machine Learning and Data Mining" course;
• Have completed the required prerequisite courses: "Medical Statistics" and "Nursing Research";
• Capable of operating the AI Educational Agent system normally and willing to undergo relevant teaching interventions and assessments during the study period.

Критерии исключения

• Unwilling to use the AI education agent system, or refusing to allow the research team to collect their relevant data;
• Students who cannot commit to the full duration of the course or have known scheduling conflicts that would prevent regular attendance;
• Students who have previously enrolled in or audited this course in prior academic years to avoid learning effect bias

Критерии приведены из реестра в оригинале (на английском). Окончательную оценку соответствия проводит исследовательский центр.

Здоровые добровольцы: Да

Дизайн исследования

Распределение

Не применимо

Модель

Одна группа

Маскирование

Открытое

Основная цель

Другое

Центры проведения

Китай · 1 центр

• North Campus of Sun Yat-sen University — Гуанчжоу

Идентификаторы

NCT: NCT07449182 · L2025SYSU-HL-032 · 25XJ0215

Первоисточники (государственные реестры)

• ClinicalTrials.gov (NIH, США)

Открыть это исследование на ClinicalTrials.gov ↗