Çмú´ëȸ ¹ßÇ¥ ¿¬Á¦ ÃÊ·Ï

¹ßÇ¥Çü½Ä : Á¢¼ö¹øÈ£ - 990163    OTOP 8-3 
FEASIBILITY OF LLM-BASED ANALYSIS OF DIZZINESS PATIENT ELECTRONIC MEDICAL RECORDS
DEPARTMENT OF OTORHINOLARYNGOLOGY-HEAD AND NECK SURGERY, YONSEI UNIVERSITY WONJU COLLEGE OF MEDICINE©ö, DEPARTMENT OF INFORMATION STATISTICS, YONSEI UNIVERSITY©÷
DONG HYUN KIM, DONG HYUN KIM©ö©÷, JAE HA KIM©ö, HYUN SU LEE©ö, TAE HOON KONG©ö
¸ñÀû: This study explores the feasibility of using Large Language Models (LLMs) to analyze electronic medical records (EMR) of dizziness patients. Specifically, it evaluates whether LLMs can extract meaningful key terms from EMR and transform them into patient-friendly language, potentially improving medical information accessibility. ¹æ¹ý:This study analyzed the initial outpatient clinical records of patients suffered with dizziness in Wonju Severance Christian Hospital from 2011 to 2023. Using GPT models with prompt engineering, 131 key terms were extracted. The study compared GPT-3.5 Turbo and GPT-4o under two conditions: with and without additional context about dizziness patient data. To assess accuracy, semantic similarity was measured using BERTScore, SBERT, and KoSBERT. Two otologists provided reference key terms, and the LLM-extracted terms were compared against these expert annotations. °á°ú:Inter-expert agreement was BERTScore (0.817), SBERT (0.743), and KoSBERT (0.637). Compared to Expert A, LLM results ranged from 0.797 to 0.820 (BERTScore), 0.634 to 0.667 (SBERT), and 0.535 to 0.601 (KoSBERT). Compared to Expert B, scores ranged from 0.718 to 0.746, 0.594 to 0.623, and 0.516 to 0.558, respectively. GPT-4o with context showed the highest alignment with expert annotations. Considering the agreement rates among the two otologists using BERTScore, SBERT, and KoSBERT, the agreement between GPT models and a single otologist ranged from 80.0% to 100%, indicating that LLMs can achieve near-expert-level accuracy in key term extraction from dizziness patient EMRs. °á·Ð:GPT-based key term extraction and automatic transformation in dizziness patient EMR demonstrate the potential for LLM-driven medical record analysis. This approach could enhance medical information accessibility and facilitate communication between patients and healthcare providers. Further research is needed to optimize and validate this methodology in clinical practice.


[´Ý±â]