Large language models in OMFS triage

Introduction:
Accurate triage of OMFS referrals is essential for appropriate care. However, increasing volume and often poor referral quality place strain on consultant time. Large language models (LLMs), such as GPT-3.5 and GPT-4, are emerging as potential decision-support tools. This project combined a scoping review and pilot study to evaluate the effectiveness of LLMs in OMFS referral triage compared to clinicians.

Method:
A scoping review was conducted using the PRISMA-ScR framework to identify literature on LLMs in triage, diagnosis, and referral optimisation. Findings informed a pilot study. Fifty synthetic referrals were constructed using anonymised real-patient case summaries. Original referrals lacked sufficient detail, so synthetic versions were designed to reflect plausible clinical scenarios with adequate information for assessment. Each referral was input into GPT-3.5 and GPT-4 using a standardised prompt requesting: (1) suggested investigations, (2) likely diagnosis, and (3) initial management. Outputs were compared to actual clinical decisions for those cases.

Results:
The review found limited but growing evidence for LLMs in referral triage, with sparse coverage in surgical specialties. In the pilot, GPT-4 showed better clinical alignment than GPT-3.5 across all domains. Both models produced more appropriate suggestions with structured information, but struggled with vague inputs, mirroring challenges in human triage.

Conclusion:
LLMs, particularly GPT-4, show promise as triage adjuncts in OMFS. While not a replacement for clinician judgement, they may help streamline decision-making when referral detail is sufficient. Further validation is needed before clinical implementation.