Research and Application of an AI Agent-Based MBBS Course Assistant

Обзор

Prior to the quasi-experiment, this study performed semi-structured interviews among Bachelor of Medicine and Bachelor of Surgery(MBBS) undergraduate students from the Belt and Road International Medical College, Zhejiang University. Based on learning science, cognitive load theory and causal inference frameworks, the interview outline focused on system usability, feedback quality, learning reasoning processes, cognitive burden and instructional optimization advice. Each 20-30 minute individual interview was audio-recorded with participants' informed consent and verbatim transcribed for standardized qualitative analysis. The quasi-experiment recruited no fewer than 60 MBBS students. The sample size was determined by intergroup statistical power analysis to guarantee around 30 participants per group for valid intergroup comparison. All eligible students were randomly assigned into two groups with balanced demographic and academic baseline characteristics. The experimental group (n=30) received structured epidemiological learning assisted by the (Socratic Agent for Guided Epidemiology) SAGE (Artificial Intelligence)AI agent with professional Socratic cognitive guidance. The control group (n=30) adopted general large language model (LLM)-based learning without systematic thinking intervention. Participants with clinically diagnosed severe mental disorders, cognitive dysfunction or inability to finish the complete research process were excluded. A baseline epidemiological knowledge pre-test confirmed no significant academic differences between the two groups via independent samples t-test, and all participants had no prior experience of AI-assisted medical learning. The semi-structured interviews were conducted to collect students' authentic learning experiences, interactive perceptions and cognitive characteristics during the use of SAGE agent and general LLMs, providing qualitative evidence for the iterative optimization of AI teaching tools. All interviewees completed baseline assessments and preliminary AI learning trials, ensuring qualified professional foundation and genuine interactive experience. Conducted by trained researchers, the standardized interviews centered on three core themes: system usability and feedback clarity; AI-induced changes in information extraction, hypothesis formulation and causal inference; and common learning barriers including interactive obstacles, comprehension difficulties, cognitive overload and potential AI over-reliance. Transcribed interview data were analyzed through thematic analysis to summarize typical user experience patterns. Qualitative outcomes were triangulated with quantitative experimental results to revise the SAGE teaching protocol, optimize agent prompt chains and improve the interpretation of experimental findings. The quasi-experiment consisted of three standardized stages. In the pre-test stage, all participants signed informed consent, completed a 25-item clinical epidemiology knowledge scale, an 11-item reasoning ability test and a demographic questionnaire to establish consistent baseline levels. In the intervention stage, the experimental group received standardized training in confounder identification and causal inference construction in strict accordance with the SAGE teaching protocol. The SAGE agent improved students' advanced epidemiological reasoning ability through continuous multi-round Socratic questioning and targeted cognitive guidance. The control group received equal-duration learning in the same experimental environment, only using conventional search engines and unguided LLMs for basic information retrieval without any cognitive and thinking intervention. All participants submitted screenshots to record their accurate AI tool usage duration after completing learning tasks. In the post-test stage, all participants finished parallel-version epidemiological knowledge assessments and unified reasoning ability tests. Validated scales were adopted to evaluate students' cognitive load, system usability, learning satisfaction and academic self-confidence. Students' final scores of the Epidemiology course were collected as supplementary indicators of long-term learning effectiveness. Upon the completion of data collection, backend AI interaction logs were summarized and strictly screened. Invalid samples with insufficient interaction rounds or incomplete responses were excluded to ensure high data quality and reliable experimental conclusions.

Вмешательства

• Поведенческое Socratic Agent for Guided Epidemiology
  A total of 30 participants are assigned to the experimental group receiving the SAGE teaching model, namely the Socratic Agent for Guided Epidemiology.
• Поведенческое General LLM
  30 participants are allocated to the control group with teaching assistance from a general large language model (General LLM).

Первичные конечные точки

• Clinical Epidemiology Knowledge Assessment Scale (Utrecht questionnaire on knowledge on clinical epidemiology for evidence-based practice) [Срок оценки: Before the intervention (One to seven days before starting the epidemiology course) and After the intervention(Within half a month after completing the epidemiology course)]
• Epidemiological Reasoning Test [Срок оценки: Before the intervention (One to seven days before starting the epidemiology course ) and After the intervention(Within half a month after completing the epidemiology course)]

Вторичные конечные точки (6)

• System Usability Scale(SUS) [Срок оценки: Immediately after intervention(One to seven days after completing the epidemiology course)]
• Needs and perceptions questionnaire (AI-powered simulation-based teaching agent) [Срок оценки: Qualitative research stage (one to five months before starting the epidemiology course]
• Student's final exam score in the Epidemiology course [Срок оценки: Immediately after intervention(One to seven days after completing the epidemiology course)]
• Demographic information questionnaire [Срок оценки: Before the intervention (One to seven days before starting the epidemiology course)]
• Learning Satisfaction and Self-confidence Scale [Срок оценки: Immediately after intervention(One to seven days after completing the epidemiology course)]
• Cognitive Load Scale [Срок оценки: After the intervention(Within half a month after completing the epidemiology course)]

Критерии участия

Критерии включения

• 1: The undergraduate students of the "Belt and Road Initiative" International Medical School of Zhejiang University who have officially registered for the MBBS program
• 2: Voluntary signing of the informed consent form

Критерии исключения

• 1: Having severe mental illnesses (such as severe depression, bipolar disorder, acute phase of schizophrenia), serious physical diseases or cognitive impairments
• 2: Is currently participating in other studies that may affect the outcome indicators of this research
• 3: Have systematically studied or participated in research that is highly similar to this study
• 4: No experience of AI agent learning

Критерии приведены из реестра в оригинале (на английском). Окончательную оценку соответствия проводит исследовательский центр.

Здоровые добровольцы: Да

Дизайн исследования

Распределение

Рандомизированное

Модель

Параллельные группы

Маскирование

Простое слепое

Основная цель

Фундаментальное исследование

Центры проведения

Китай · 1 центр

• Zhejiang University International School of Medicine — Yiwu

Идентификаторы

NCT: NCT07752160 · KY-2025-374

Первоисточники (государственные реестры)

• ClinicalTrials.gov (NIH, США)

Открыть это исследование на ClinicalTrials.gov ↗