Evaluation of AI Large Models for Diagnosis and Treatment in Real-World Cases: Multicenter Retrospective Study
For patients and families
In plain language
An automatic summary of structured registry data. It is an orientation aid, not a substitute for the official protocol or a physician assessment.
- What is being studied
- The protocol lists: Large Language Model Assessment (ChatGPT, Gemini, DeepSeek).
- Who it may be relevant to
- Registry conditions: Urologic Diseases. Basic parameters: from 18 years · All.
- What needs checking
- Age, condition and sex are only basic indicators. Prior treatment, laboratory values and other mandatory requirements appear in the eligibility criteria below.
- Where it takes place
- China
- Next step
- Save the trial, show it to the treating physician, and confirm current recruitment with the study center. Costs, documents and travel →
Unsure about the terms? Read our patient guide →
Overview
This multicenter retrospective study aims to evaluate the diagnostic and therapeutic performance of three large language models-ChatGPT, Gemini and Deepseek-using 800 archived inpatient medical records from urology departments across four tertiary hospitals. The study will focus on the accuracy and applicability of these models in disease recognition, preliminary diagnosis and treatment recommendation generation, in order to explore their potential value and limitations in supporting clinical decision-making in real-world settings.
Interventions
- Other Large Language Model Assessment (ChatGPT, Gemini, DeepSeek)
De-identified inpatient medical records were retrospectively collected from the urology departments of four tertiary hospitals (200 cases per site, 800 in total). Each case included standardized clinical information such as demographics, chief complaint, history of present illness, past medical history, physical examination, laboratory and imaging findings, discharge diagnosis and treatment plan. To simulate the role of an AI system in a "first-visit physician" scenario, all diagnostic conclusi
Primary outcome measures
- Diagnostic Accuracy: Assessed by Top-1 accuracy [Time frame: Through study completion, an average of 3 months]
- Diagnostic Accuracy: Assessed by Top-3 accuracy [Time frame: Through study completion, an average of 3 months]
- Diagnostic Completeness [Time frame: Through study completion, an average of 3 months]
- Differential Diagnosis Quality [Time frame: Through study completion, an average of 3 months]
- Treatment Plan Quality [Time frame: Through study completion, an average of 3 months]
- Analysis Time [Time frame: Through study completion, an average of 3 months]
Eligibility criteria
Inclusion criteria
- The case data is sourced from the four hospitals involved in the study, with complete and authentic diagnosis and treatment records.
- Patients must be 18 years or older, with no gender restrictions.
- Complete medical records, including the following core information: patient' s basic information, present illness history, past medical history, physical examination, and auxiliary examinations (including laboratory and imaging tests).
- A clear discharge diagnosis and treatment plan (including therapeutic measures and follow-up arrangements).
- Medical records have been archived, with objective and accurate information that has not been altered.
- The patient or their legal representative has provided informed consent, agreeing to the use of their anonymized medical data for research analysis.
Exclusion criteria
- Medical records with significant missing information, such as key clinical details (present illness history, diagnostic or treatment records, etc.).
- Cases where the diagnosis or treatment plan is unclear, or where treatment has not been fully completed for an initial diagnosis.
- Cases where the primary diagnosis is not urological.
- Cases with major errors or inconsistencies in the records that could affect further assessment.
- Medical records in special formats or images that are not readable (e.g., handwritten notes, non-standard documentation).
- Patients who have not signed the informed consent form or who refuse to allow their medical data to be used for research.
Criteria are shown verbatim from the registry (in English). Final eligibility is always assessed by the study center.
Healthy volunteers: No
Study design
- Observational model
- Cohort
Study locations
China · 1 center
- The First Affiliated Hospital of Fujian Medical University — Fuzhou
Identifiers
NCT: NCT07378358 · MRCTA,ECFAH of FMU[2025]902