Validity of Migraine Diagnoses in Korean National Health Insurance Claims Data
Article information
Abstract
Purpose
Accurate case identification using administrative datasets relies on diagnostic coding, yet these codes’ accuracy for migraine remains uncertain. This study aimed to validate the diagnostic accuracy of International Statistical Classification of Diseases and Related Health Problems 10th Revision (International Classification of Diseases, ICD-10) codes for migraine, migraine without aura (MOA), and migraine with aura (MA) in the Korean National Health Insurance Service database.
Methods
We retrospectively reviewed the electronic medical records of 500 patients (migraine [G43.X], 200; MOA [G43.0], 200; MA [G43.1], 100) from secondary and tertiary hospitals between January 2019 and December 2024. Diagnoses confirmed by headache specialists using the International Classification of Headache Disorders, third edition served as the gold standard. Validation metrics included the positive predictive value (PPV), negative predictive value, sensitivity, specificity, and the kappa statistic. Diagnostic accuracy was assessed based on ICD-10 claim frequency and improved by combining diagnostic codes with prescriptions for migraine medications.
Results
A single ICD-10 claim had a PPV of 74.00%. Accuracy improved significantly with increased claim frequency (≥3 claims: PPV, 81.14%; sensitivity, 98.61%; specificity, 28.26%), particularly when combined with medication prescriptions (≥3 claims with medication: PPV, 94.96%; sensitivity, 91.87%; specificity, 85.37%). MOA (≥3 claims with medication: PPV, 95.20%) and MA (≥3 claims with medication: PPV, 93.65%) showed similar trends. Excellent inter-rater reliability was observed (kappa, 0.806–0.951), with no significant accuracy differences between hospitals.
Conclusion
Employing multiple claims and prescriptions improved the accuracy of migraine diagnoses using ICD-10 codes, supporting the use of this method in epidemiological studies and health policy decisions.
INTRODUCTION
Migraine is one of the most prevalent neurological disorders and a leading cause of disability worldwide.1-3 Advances in treatment modalities and preventive strategies have significantly improved the quality of life for patients suffering from migraines.4-6 Despite these advancements, the disease continues to impose a substantial burden due to its chronic nature and its association with comorbidities such as depression, anxiety, and cardiovascular conditions.7-9 These factors underscore the importance of accurately understanding and addressing the prognosis and management of migraine as a repetitive and chronic serious neurological disorder.
Recent nationwide population-based studies have provided valuable insights into this issue. Population-based research covers diverse populations and is especially beneficial in cases where conducting randomized controlled trials is challenging. Such studies help uncover new risk factors, comorbidities, and outcomes associated with specific diseases. Additionally, population-based research reflects real-world evidence and practices, offering practical implications for healthcare systems.10
In Korea, the National Health Insurance Service (NHIS) provides universal healthcare coverage, encompassing demographic information, diagnostic codes, treatment records, procedures, and prescription data for the entire population.11,12 This dataset is a robust resource for conducting large-scale epidemiological studies.13,14 In particular, the International Classification of Diseases (ICD) codes are widely used to define diseases and outcomes in research.15 However, the ICD codes often include diagnoses made on suspicion of disease, making it challenging to determine the precise presence of a condition or the occurrence of specific outcomes. This limitation underscores the importance of validating diagnostic algorithms based on ICD codes to enhance the accuracy of research findings and disease burden estimates. Despite the utility of the NHIS dataset, no research to date has validated diagnostic algorithms for migraine within the Korean NHIS database. Given the significant burden of migraine and its potential underestimation due to diagnostic challenges, there is a critical need to develop and validate algorithms for identifying migraine cases and related outcomes accurately.
In this study, we aim to validate diagnostic algorithms for migraine using the NHIS dataset. This validation process will focus on ensuring the accuracy of identifying migraine diagnoses and associated outcomes, thereby contributing to more reliable research and better-informed healthcare policies.
MATERIALS AND METHODS
1. Ethics approval and consent to participate
The study received approval from the Institutional Review Board of Ewha Womans University Seoul Hospital (IRB 2025-01-005).
2. Subjects
The study utilized data from discharged patients and outpatient visits at two hospitals in Korea: Ewha Womans University Seoul Hospital (a secondary medical center) and Ewha Womans University Mokdong Hospital (a tertiary medical center). A total of 5,327 patients aged 20 years or older, diagnosed with migraines at least once, were identified based on the Korean Standard Classification of Diseases, which aligns with codes from the 10th revision of the ICD-10, during the period between January 2019 and December 2024. For abortive or preventive medications prescribed as non-reimbursable under NHIS, their claim status could not be verified in the NHIS data; therefore, such cases (n=497) were excluded from the evaluation. Then, from this cohort, a random sample of migraine (n=200), migraine without aura (n=200), and migraine with aura (n=100) cases were selected for analysis evenly split between 100, 100, and 50 patients from each hospital (Figure 1). Cases where migraine, migraine without aura, or migraine with aura were claimed as the main diagnosis in outpatient or inpatient settings were included. There were no overlaps in patients’ data between the two hospitals.
Flow chart.
3. Diagnosis for migraine, migraine without aura, and migraine with aura
The ICD-10 codes for migraine, migraine without aura, and migraine with aura are G43.X, G43.0, and G43.1, respectively. Clinical information for the selected patients was stored in the electronic medical records (EMR) system of Ewha Womans University Hospital and retrospectively retrieved from hospital case notes. These case notes included details on migraine diagnoses based on Headache Classification Committee of the International Headache Society the International Classification of Headache Disorders, 3rd edition.16 Moreover, information for medical history, symptom descriptions, neurological examination findings, brain imaging results (if applicable), and prescription records for both acute and preventive migraine therapies were also investigated. Acute medications for migraine included acetaminophen, non-steroidal anti-inflammatory drugs, ergot derivatives, and triptans. Antiemetics were also considered as acute medications, but only when used in combination with pain control medications. For preventive medications, cases were defined as taking migraine preventive medications when prescribed as such by the rater, including beta-blockers, calcium channel blockers (flunarizine), anticonvulsants, antidepressants, antihypertensive agents, or calcitonin gene-related peptide monoclonal antibodies based on previous study (Supplementary Table 1, available online).17,18
The processes of data extraction and disease classification were carried out by different researchers to ensure accuracy and reduce potential bias. Specifically, Y.C. handled the classification of migraine cases, S.C. focused on migraine without aura, and B.S.K. was responsible for migraine with aura. To further minimize the risk of misclassification, an audit of the classifications was conducted by different authors who independently reviewed and reassessed each diagnosis. Any ambiguous or uncertain cases were thoroughly evaluated through discussions between Y.C. and T.J.S. to reach a consensus and ensure consistency in the final classifications.
4. Statistical analysis
To assess the percentage of accurate diagnostic codes for migraine, they were compared with the gold standard confirmed by headache specialists based on the International Headache Classification, 3rd edition.16 The performance parameters for ICD-10 code validation for diagnostic accuracy included the positive predictive value (PPV), negative predictive value (NPV), error rate, sensitivity, and specificity. The PPV were used to quantify diagnostic accuracy (number of correctly classified cases). PPV referred to the proportion of gold-standard diagnoses for migraine, migraine without aura, and migraine with aura cases relative to all of those identified with the diagnostic codes in the Korean NHIS data. The NPV was defined as the ratio of patients truly diagnosed as negative relative to all those who had negative test results (Supplementary Methods, available online).19 The error rate was defined as the proportion of patients with false results relative to all patients. Sensitivity and specificity analyses assessed the effect of reclassifying diagnoses using a combination of diagnostic codes and cofactors (specific tests and prescriptions for each diagnosis). Simple random sampling was used to select and allocate for each disease from each hospital. Estimates for each validation were assessed based on the frequency of claims and the use of medications for migraine, migraine without aura, and migraine with aura, respectively. The kappa coefficient was calculated after being independently investigated and compared by each of the two researchers (Y.C., S.C.). Each diagnostic value of migraine, migraine without aura, and migraine with aura had an overall kappa value higher than 0.8, indicating an excellent degree of agreement between the researchers. For sensitivity analysis, we estimated validation parameters by stratifying cases into two groups: those managed in the neurology department and those managed in non-neurology departments. All statistical analyses were performed using the open-source statistical package R (version 3.6.3; R Project for Statistical Computing).
RESULTS
1. Migraine
A total of 200 cases were reviewed for validation using ICD-10 code G43.X for migraine. When using a single ICD-10 claim, the PPV was 74.00% (148 true positives out of 200). Diagnostic accuracy improved with increased frequency of claims: two or more claims yielded a PPV of 75.66% and sensitivity of 98.62%, whereas three or more claims demonstrated an even higher PPV of 81.14%, specificity of 28.26%, and sensitivity of 98.61%. Incorporating medication prescriptions along with diagnostic codes further improved accuracy, with two or more claims plus medication showing a PPV of 79.88% and three or more claims with medication achieving the highest PPV of 94.96%, specificity of 85.37%, and sensitivity of 91.87%. The kappa values indicated excellent inter-rater reliability, ranging from 0.806 to 0.896 (Table 1). The analysis showed no significant differences in diagnostic accuracy between secondary and tertiary hospitals. In sensitivity analysis, among the 200 individuals who had at least one claim with a migraine diagnosis, the following distributions were observed: neurology (n=155, 77.5%), neurosurgery (n=19, 9.5%), emergency medicine (n=16, 8.0%), and family medicine (n=10, 5.0%). From an arithmetic perspective, the accuracy estimates for migraine diagnosis were generally higher in the neurology department compared to non-neurology departments. Notably, in cases with ≥3 migraine claims accompanied by acute or chronic medication use, the error rate remained below 10% in both neurology and non-neurology departments (Supplementary Table 2, available online).
Validation of diagnostic codes for migraine
2. Migraine without aura
Validation of migraine without aura using ICD-10 code G43.0 involved analysis of 200 cases. A single ICD-10 claim resulted in a PPV of 74.00%. Accuracy increased with multiple claims: two or more claims resulted in a PPV of 75.92%, specificity of 11.54%, and sensitivity of 97.97%; three or more claims increased PPV to 82.02%, specificity to 38.46%, and sensitivity to 98.65%. When medication prescriptions were included, two or more claims combined with medication prescriptions reached a PPV of 81.66% and specificity of 36.73%, and three or more claims with medication prescriptions yielded the highest PPV of 95.20%, specificity of 87.76%, and sensitivity of 90.15%. Kappa statistics confirmed high consistency among raters, from 0.878 to 0.951 (Table 2). No substantial difference in diagnostic accuracy was observed between secondary and tertiary hospitals.
Validation of diagnostic codes for migraine without aura
3. Migraine with aura
For migraine with aura validated through ICD-10 code G43.1, 100 cases were analyzed. Using one ICD-10 claim alone provided a PPV of 76.00%. Diagnostic accuracy increased significantly with frequency: two or more claims achieved a PPV of 78.72%, specificity of 13.04%, and sensitivity of 100%; three or more claims improved PPV to 82.95%, specificity to 31.82%, and maintained sensitivity at 98.65%. Including medication prescriptions further enhanced accuracy, achieving PPVs of 81.93% (≥2 claims) and 93.65% (≥3 claims), with specificity values improving to 31.82% and 78.95%, respectively. Inter-rater agreement was excellent, with kappa values consistently above 0.9 (Table 3). Similarly, no marked difference was noted in diagnostic performance between secondary and tertiary hospitals.
Validation of diagnostic codes for migraine with aura
DISCUSSION
Our study investigated the applicability of ICD-10 codes in identifying migraine, migraine without aura, and migraine with aura within the Korean NHIS data. Our analysis revealed that incorporating factors such as medication details and the frequency (2 or more times) of claims with ICD-10 codes improves the accuracy of diagnosing migraine, migraine without aura, and migraine with aura. The findings from our validation study highlight important considerations in accurately identifying migraine cases within the Korean NHIS dataset. Notably, we confirmed that reliance on a single ICD-10 code claim yields suboptimal diagnostic accuracy, reflecting challenges inherent in distinguishing definitive diagnoses from suspected cases solely through administrative data. By contrast, we demonstrated substantial improvements in accuracy when claims frequency increased to three or more and when combined with prescription data for acute or preventive medications.
In our study, we found that using a single claim based on the ICD-10 code for diagnosing migraine did not achieve a reliable level of diagnostic accuracy. Furthermore, we observed that the diagnostic accuracy improved as the number of claims for migraines increased, reflecting a similar principle to diagnostic criteria for migraines, which require experiencing at least five episodes of headache consistent with migraines. Additionally, when medications related to migraines—such as those specific to migraine, migraine without aura, and migraine with aura—were taken into account alongside multiple claims, the diagnostic accuracy was significantly higher compared to relying solely on a single claim. Our study underscores the importance of incorporating multifactorial diagnostic criteria, specifically multiple claims and prescription data, into epidemiological research and clinical management practices.
Determining a definitive diagnosis based solely on ICD-10 codes can be challenging, as healthcare providers may use these codes for insurance purposes even when a disease is only suspected. For instance, a prior investigation into inflammatory bowel disease (IBD) demonstrated that incorporating multiple factors—such as ICD-10 codes, repeated healthcare visits, and prescriptions for IBD-specific medications—into an identification algorithm significantly enhanced its accuracy in detecting true IBD cases.20 Similarly, research on stroke diagnosis algorithms revealed that combining brain imaging results with prescription data yields more reliable diagnostic outcomes than relying solely on primary ICD-10 codes.21 In contrast, for other conditions, the overall sensitivity of cancer diagnosis based on only one-time claim for ICD-10 codes was reported to be 92.8%.22 The reason for the lower diagnostic accuracy of migraine when relying solely on a single claim with ICD-10 codes, compared to using multiple claims or prescriptions related to migraine, remains unclear. This discrepancy could potentially be attributed to variations in study methodologies, differences in the characteristics of the study populations, or other contextual factors influencing the findings. Additionally, for conditions such as cancer or rare diseases, the existence of a copayment reduction system may contribute to a relatively higher diagnostic accuracy compared to migraines.
This study had several limitations. First, because it focused on patients who had already been diagnosed, we were unable to compare the prevalence of migraines with findings from previous studies. Second, potential bias could have arisen since, although we validated patients from secondary and tertiary hospitals, the proximity of these two hospitals may have minimized differences between the patient groups. Third, the retrospective nature of the EMR review may have led to missing records, potentially impacting the study results. Missing data related to migraine symptoms or diagnostic criteria could also have influenced diagnostic accuracy in this study. Fourth, although we extracted years of EMR records to construct the dataset, conditions such as chronic migraine or complicated migraine had very few cases recorded with diagnostic codes, making it impossible to conduct research on this aspect. Fifth, because this validation study was limited to a secondary and a tertiary hospital setting, its findings may not be generalizable to primary care providers, where diagnostic accuracy and coding practices may differ substantially. Finally, although the NHIS cannot verify claims for non-reimbursable medications, and such cases were excluded from validation in this study, this could introduce bias in the validation of migraine diagnosis.
In conclusion, diagnosing migraine, migraine without aura, and migraine with aura using only ICD-10 codes may lack accuracy. However, when ICD-10 codes are supplemented with information on migraine medications and multiple claims for migraine, the diagnostic accuracy for these conditions may be more reliable. Our study showed that multiple claims and prescriptions for migraine should be combined with ICD-10 codes to increase the accuracy of migraine, migraine without aura, and migraine with aura diagnoses among the Korean NHIS data set.
Notes
AVAILABILITY OF DATA AND MATERIAL
Upon request, the dataset for this study can be provided after consultation with the corresponding author and approval from the IRB of Ewha Womans University.
AUTHOR CONTRIBUTIONS
Conceptualization: YC, SC, BSK, TJS; Data curation: YC, SC, BSK, TJS; Formal analysis: YC, SC, BSK, TJS; Funding acquisition: TJS; Investigation: YC, SC, BSK; Methodology: YC, SC; Project administration: YC, SC, TJS; Resources: YC, SC, TJS; Software: YC, SC, TJS; Supervision: TJS; Validation: TJS; Visualization: YC, SC; Writing–original draft: YC, SC; Writing–review & editing: YC, SC, BSK, TJS.
CONFLICT OF INTEREST
No potential conflict of interest relevant to this article was reported.
FUNDING STATEMENT
This study was supported by a 2025 Ewha Fellow research grant from Ewha Womans University.
ACKNOWLEDGMENTS
Not applicable.
SUPPLEMENTARY MATERIAL
Supplementary materials are available from https://doi.org/10.62087/hpr.2025.0004.
List of ingredients for migraine treatment regimens
Validation of diagnostic codes for migraine according to clinical specialty