| WISA-2026-001 | ||||
|---|---|---|---|---|
| Title |
AI-Powered Classification of Training-Related Aviation Accidents: Performance Evaluation of a Gemini-Based TSB Report Filtering System |
|||
| Authors | Leyao Cui, Jacob Nowak, Shi Cao | |||
| Abstract |
Aviation accident reports are an important source of information for identifying safety issues and preventing future occurrences. However, finding reports related to contributing factors can be time-consuming, since investigation databases do not always include detailed labels for every safety issue discussed in the full report. This study evaluates an artificial intelligence (AI) based classification system designed to identify General Aviation (GA) accident reports that contain potential training-related systemic deficiencies. The system analyzed 1,312 aviation accident investigation reports published by the Transportation Safety Board of Canada (TSB) between 1991 and November 29, 2025, using Google’s Gemini Flash API as part of a two-stage filtering process. Stage 1 filtered for GA fixed-wing aircraft accidents (including Air Taxi operations), reducing the dataset from 1,312 to 595 reports (45.4%). Stage 2 applied a rigorous 4-filter classification protocol to identify training-related systemic gaps, ultimately identifying 97 preventable training-related accidents (7.4% of original dataset). Manual validation on a stratified sample of 178 reports (30% of Stage 2 outputs) demonstrated strong performance: 88.8% overall accuracy, 75.0% precision, and 96.4% recall. The high recall rate ensures comprehensive capture of genuine training gaps with minimal false negatives, supporting the research objective of identifying preventable accidents through targeted training interventions. |
|||
| Report | ||||