Multicenter Prospective Validation of AI Models for Malignancy Risk Prediction in Pulmonary Nodules

Lung cancer is the leading cause of cancer-related morbidity and mortality worldwide. Low-dose CT (LDCT) screening improves early detection but generates a high prevalence of indeterminate pulmonary nodules and substantial false positives, leading to unnecessary invasive procedures and anxiety while still risking missed early cancers. Traditional radiologic assessment of pulmonary nodules relies on visual features and clinical risk factors, with limited performance and substantial reader variability, especially for small or subsolid nodules.

Recent advances in deep learning enable AI models to extract high-dimensional imaging features from CT scans and to predict nodule malignancy and risk stratification. Several commercial AI systems for pulmonary nodule assessment have been approved and deployed in clinical practice, and academic groups have proposed novel algorithms such as the multi-view coupled self-attention (MVCS) model. However, most prior studies have been single-center, retrospective, and used reference standards such as imaging follow-up or expert reading rather than biopsy pathology. Head-to-head comparisons of different AI models in prospective, multicenter real-world populations with pathological gold standard are lacking.

This study is a multicenter, prospective, diagnostic accuracy comparison of three purely imaging-based AI models-MVCS, LungDoc (Shukun Technology), and a United Imaging AI model-for predicting the malignancy of pulmonary nodules. Eligible patients are adults (≥18 years) with at least one pulmonary nodule ≤3 cm on CT, who undergo surgical or biopsy pathology with a definitive benign or malignant diagnosis, and with a CT-pathology interval ≤6 months. CT images in DICOM format will be collected using standardized acquisition parameters across centers and processed by the three AI models, which output continuous malignancy probabilities or suspicion scores. Investigators will be blinded to AI outputs.

The primary endpoint is the AUC for malignancy prediction for each model, and pairwise AUC comparisons using DeLong's test. Secondary endpoints include binary performance metrics (sensitivity, specificity, PPV, NPV, accuracy, F1 score) at model-native thresholds and optimal Youden index thresholds, as well as calibration (calibration curves, intercept, slope). Prespecified subgroup analyses will examine performance by age, sex, smoking status, nodule size, morphology, and study center, and random-effects methods will be used to assess center effects.

An exploratory aim will validate the MVCS model for predicting pathological invasion degree by classifying nodules into pre-invasive lesions (atypical adenomatous hyperplasia \[AAH\] / adenocarcinoma in situ \[AIS\]), minimally invasive adenocarcinoma (MIA), and invasive adenocarcinoma (ADC), using metrics such as multi-class accuracy, weighted F1 score, confusion matrix, Matthews correlation coefficient, and AUC for predefined binary sub-tasks. Another exploratory analysis will evaluate the MVCSN model, which incorporates CT images plus clinical and radiologic features, in a subset with complete data.

The study plans to enroll 3,000 pathologically confirmed pulmonary nodules across five centers in China over approximately 30 months. No experimental treatment is administered; all clinical management, imaging, and pathology follow standard of care. Risks are limited to those associated with clinically indicated pathology procedures (e.g., surgery or biopsy).

• Sensitivity and Specificity for Malignancy Prediction [Time frame: At the time of availability of pathology results, up to 6 months after index chest CT]
• Positive Predictive Value (PPV) and Negative Predictive Value (NPV) [Time frame: At the time of availability of pathology results, up to 6 months after index chest CT]
• Overall Diagnostic Accuracy and F1 Score [Time frame: At the time of availability of pathology results, up to 6 months after index chest CT]
• Calibration Metrics [Time frame: At the time of availability of pathology results, up to 6 months after index chest CT]