Large language models for orthodontic classification using intraoral digital models: a diagnostic accuracy and reproducibility study
BMC ORAL HEALTH, cilt.0, 2026 (SCI-Expanded, Scopus)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 0
- Basım Tarihi: 2026
- Doi Numarası: 10.1186/s12903-026-10465-5
- Dergi Adı: BMC ORAL HEALTH
- Derginin Tarandığı İndeksler: Natural Science Collection (ProQuest), Biological Science Database (ProQuest), Biomedical Reference Collection: Corporate Edition (EBSCO), Health Research Premium Collection (ProQuest), Scopus, Science Citation Index Expanded (SCI-EXPANDED), CINAHL, EMBASE, MEDLINE, Directory of Open Access Journals
- İstanbul Kent Üniversitesi Adresli: Evet
Özet
Background Multimodal artificial intelligence (AI) systems built on large language models are consulted by
patients and clinicians, but their performance on visual orthodontic classification is not well characterized. This
study evaluated three such systems for agreement with an expert orthodontist reference standard, category-spe-
cific diagnostic performance and four-week reproducibility, using intraoral digital scan exports.
Methods In this retrospective diagnostic accuracy study, 90 patients were selected from archived records with
equal numbers in each Angle class (n =30). Five standardized two-dimensional views exported from each iTero
scan were submitted to Claude Opus 4.7, GPT-5.5 and Gemini 3 Flash with one zero-shot prompt. Two orthodon-
tists independently classified each case from the complete scan record with metric measurements; their agreed
categories formed the reference standard. Sagittal classification, vertical relationship, crowding severity and arch
form were assessed. Twenty patients were reassessed after four weeks, and the orthodontists also classified all
cases from the same five images. Accuracy, class-specific sensitivity and specificity, macro-average recall and
Cohen’s kappa were calculated with 95% confidence intervals (CIs).
Results Overall accuracy ranged from 29.2% (Gemini 3 Flash, arch form) to 64.0% (GPT-5.5, arch form), macro-
average recall from 38.7% (Gemini 3 Flash, arch form) to 60.9% (GPT-5.5, vertical relationship), and unweighted
kappa from 0.044 (Gemini 3 Flash, arch form) to 0.299 (Gemini 3 Flash, crowding severity). Class III sensitivity
was 10.0–13.3%, compared with 60.0–80.0% for Class I. Classifying the same five images, the orthodontists
agreed with the reference standard in 96.6–100% of cases. Test–retest kappa ranged from − 0.238 (Gemini 3
Flash, sagittal classification) to 0.433 (GPT-5.5, vertical relationship), with most CIs including zero; in the retest
subsample, sagittal accuracy fell from 55.0 to 60.0% to 5.0–35.0%. Each orthodontist reproduced at least 19 of
20 classifications for each parameter.
Conclusions Under the conditions tested, the three systems showed low agreement with the expert reference
standard, very low sensitivity for Class III malocclusion and limited four-week reproducibility. Overall accuracy
did not reflect category-specific performance, and reproducibility needs to be evaluated separately from accuracy.
The systems tested are not suitable for standalone orthodontic diagnosis under these conditions.