Large language models for orthodontic classification using intraoral digital models: a diagnostic accuracy and reproducibility study


Altundal G., Kürkçü İ., Altın M., Biren S.

BMC ORAL HEALTH, cilt.0, 2026 (SCI-Expanded, Scopus)

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 0
  • Basım Tarihi: 2026
  • Doi Numarası: 10.1186/s12903-026-10465-5
  • Dergi Adı: BMC ORAL HEALTH
  • Derginin Tarandığı İndeksler: Natural Science Collection (ProQuest), Biological Science Database (ProQuest), Biomedical Reference Collection: Corporate Edition (EBSCO), Health Research Premium Collection (ProQuest), Scopus, Science Citation Index Expanded (SCI-EXPANDED), CINAHL, EMBASE, MEDLINE, Directory of Open Access Journals
  • İstanbul Kent Üniversitesi Adresli: Evet

Özet

Background Multimodal artificial intelligence (AI) systems built on large language models are consulted by

patients and clinicians, but their performance on visual orthodontic classification is not well characterized. This

study evaluated three such systems for agreement with an expert orthodontist reference standard, category-spe-

cific diagnostic performance and four-week reproducibility, using intraoral digital scan exports.

Methods In this retrospective diagnostic accuracy study, 90 patients were selected from archived records with

equal numbers in each Angle class (n =30). Five standardized two-dimensional views exported from each iTero

scan were submitted to Claude Opus 4.7, GPT-5.5 and Gemini 3 Flash with one zero-shot prompt. Two orthodon-

tists independently classified each case from the complete scan record with metric measurements; their agreed

categories formed the reference standard. Sagittal classification, vertical relationship, crowding severity and arch

form were assessed. Twenty patients were reassessed after four weeks, and the orthodontists also classified all

cases from the same five images. Accuracy, class-specific sensitivity and specificity, macro-average recall and

Cohen’s kappa were calculated with 95% confidence intervals (CIs).

Results Overall accuracy ranged from 29.2% (Gemini 3 Flash, arch form) to 64.0% (GPT-5.5, arch form), macro-

average recall from 38.7% (Gemini 3 Flash, arch form) to 60.9% (GPT-5.5, vertical relationship), and unweighted

kappa from 0.044 (Gemini 3 Flash, arch form) to 0.299 (Gemini 3 Flash, crowding severity). Class III sensitivity

was 10.0–13.3%, compared with 60.0–80.0% for Class I. Classifying the same five images, the orthodontists

agreed with the reference standard in 96.6–100% of cases. Test–retest kappa ranged from − 0.238 (Gemini 3

Flash, sagittal classification) to 0.433 (GPT-5.5, vertical relationship), with most CIs including zero; in the retest

subsample, sagittal accuracy fell from 55.0 to 60.0% to 5.0–35.0%. Each orthodontist reproduced at least 19 of

20 classifications for each parameter.

Conclusions Under the conditions tested, the three systems showed low agreement with the expert reference

standard, very low sensitivity for Class III malocclusion and limited four-week reproducibility. Overall accuracy

did not reflect category-specific performance, and reproducibility needs to be evaluated separately from accuracy.

The systems tested are not suitable for standalone orthodontic diagnosis under these conditions.