Menu
Recruiting NCT07651280

Improving the Reliability of LLMs as Medical Assistants for the General Public

No phase Interventional Relevant Conditions Identification

For patients and families

In plain language

An automatic summary of structured registry data. It is an orientation aid, not a substitute for the official protocol or a physician assessment.

What is being studied
The protocol lists: three minutes six dimensions education, ChatGPT, Gemini.
Who it may be relevant to
Registry conditions: Relevant Conditions Identification. Basic parameters: from 18 years · All.
What needs checking
Age, condition and sex are only basic indicators. Prior treatment, laboratory values and other mandatory requirements appear in the eligibility criteria below.
Where it takes place
China
Next step
Save the trial, show it to the treating physician, and confirm current recruitment with the study center. Costs, documents and travel →

Unsure about the terms? Read our patient guide →

Official title

Improving the Reliability of LLMs as Medical Assistants for the General Public: a Proof of Concept Simulation Trial

Overview

This study will evaluate whether three-minute six-dimensions education(3M-6D education) can improve the reliability of large language models as medical assistants for the general public. Participants will be randomly assigned to receive or not receive 3M-6D education and then use ChatGPT, Gemini, or non-AI information resources. The study will assess relevant condition identification, disposition concordance, red-flag identification, and NASA-TLX score.

Detailed description

This randomized, controlled, proof-of-concept simulation trial will evaluate whether three-minute six-dimensions education (3M-6D education) can improve the reliability of large language models as medical assistants for the general public.

Eligible participants will be randomly assigned in a 1:1:1:1:1 ratio to one of five study groups: the 3M-6D education GPT group, the GPT group, the 3M-6D education Gemini group, the Gemini group, or the control group. Participants in the 3M-6D education GPT and 3M-6D education Gemini groups will receive approximately three minutes of education before using ChatGPT or Gemini.Each participant will be randomly assigned one of 10 standardized clinical scenarios and complete a simulated counseling task in unrestricted natural language within approximately 10 minutes. The study will assess relevant condition identification, disposition concordance, red-flag identification, and NASA-TLX score.

Interventions

  • Behavioral three minutes six dimensions education
    3M-6D education is designed based on Cognitive Load Theory to reduce the cognitive burden on patients during medical interactions with AI and to improve the clarity and completeness of symptom reporting. Guided by cognitive load theory and the natural process physicians use to take medical histories, the investigators identified candidate information dimensions and developed a structured expression framework with six dimensions for public health queries through a Delphi expert consensus process
  • Other ChatGPT
    Participants use ChatGPT to complete a standardized simulated clinical scenarios in unrestricted natural language.
  • Other Gemini
    Participants use Gemini to complete a standardized simulated clinical scenarios in unrestricted natural language.

Primary outcome measures

  • Relevant conditions identification of the 3M-6D education GPT group compared with the GPT group [Time frame: 1 hour.]
  • Disposition concordance of the 3M-6D education GPT group compared with the GPT group [Time frame: 1 hour.]
  • Relevant conditions identification of the 3M-6D education Gemini group compared with the Gemini group [Time frame: 1 hour.]
  • Disposition concordance of the 3M-6D education Gemini group compared with the Gemini group [Time frame: 1 hour.]
Secondary outcome measures (12)
  • Relevant conditions identification of the 3M-6D education GPT group compared with the control group [Time frame: 1 hour.]
  • Relevant conditions identification of the 3M-6D education Gemini group compared with the control group [Time frame: 1 hour.]
  • Disposition concordance of the 3M-6D education GPT group compared with the control group [Time frame: 1 hour.]
  • Disposition concordance of the 3M-6D education Gemini group compared with the control group [Time frame: 1 hour.]
  • Red-flag identification in the 3M-6D education GPT group compared with the GPT group [Time frame: 1 hour.]
  • Red-flag identification in the 3M-6D education GPT group compared with the control group [Time frame: 1 hour.]
  • Red-flag identification in the 3M-6D education Gemini group compared with the Gemini group [Time frame: 1 hour.]
  • Red-flag identification in the 3M-6D education Gemini group compared with the control group [Time frame: 1 hour.]
  • NASA Task Load Index score of the 3M-6D education GPT group compared with the GPT group [Time frame: 1 hour.]
  • NASA Task Load Index score of the 3M-6D education GPT group compared with the control group [Time frame: 1 hour.]
  • NASA Task Load Index score of the 3M-6D education Gemini group compared with the Gemini group [Time frame: 1 hour.]
  • NASA Task Load Index score of the 3M-6D education Gemini group compared with the control group [Time frame: 1 hour.]

Eligibility criteria

Inclusion criteria

  • Age 18 years or greater, male or female;
  • Completed primary school or higher education;
  • Able to use a smartphone or computer to complete online interaction;
  • No history of acute ischemic stroke, systemic lupus erythematosus, gastric ulcer, pneumonia, acute cardiac infarction, urinary tract infection, uterine fibroids, diabetes, osteoarthritis, or migraine.
  • Able to understand and comply with study procedures and to provide written informed consent.

Exclusion criteria

  • Currently or previously employed as a healthcare worker;
  • Previously received systematic medical training;
  • Currently involved in concurrent research that may interfere with the results of the present trial;
  • The investigator considered that the participant had other conditions that might affect compliance or preclude participation.

Criteria are shown verbatim from the registry (in English). Final eligibility is always assessed by the study center.

Healthy volunteers: Yes

Study design

Allocation
Randomized
Model
Parallel assignment
Masking
Single blind
Primary purpose
Health services research

Study locations

China · 1 center
  • Beijing Ctiy — Beijing

Identifiers

NCT: NCT07651280 · LAMP-1

Primary sources (government registries)

View this study on ClinicalTrials.gov ↗