Innovation in ECG Interpretation:
Using Eye Movements to Predict
Accuracy
Raj Ratwani, PhD
Scientific Director, National Center for Human Factors in Healthcare
MedStar Institute for Innovation, MedStar Health
Assistant Professor of Emergency Medicine
Georgetown University School of Medicine
Inferring Cognition from Eye Movements
• Use eye movements as an “online” indicator of
cognitive process
– Procedural errors (Ratwani & Trafton, 2011,2012)
– Fatigue (Ji & Zhu, 2004)
– Workload (Ahlstrom & Friedman-Berg, 2006)
• Outputs: x,y coordinate indicating the location of the
eyes
Sampling rate: 60Hz
Fixations: defined areas of focus (min 100ms)
Emergency Medicine and ECGs
• ~ 130 million annual ED visits, 22m ECGs (CDC,2010)
• Error rate of 5-45% depending on the
abnormality (Davidenko et al, 2014)
• Goal: Improve ECG interpretation accuracy by
understanding cognitive processing of ECGs
– Training systems for residents and medical students
– Develop real-time predictive systems to reduce errors
(Ratwani & Trafton, 2010, 2011)
Method
• Controlled study
–
–
–
–
Eye movement and RT data
30 ECGs (easy, medium, hard)
No prior patient information
Interruptions (30 sec) consisting of dosage calculations
• 30 trials: ECG inspection -> Interpretation
– 6 interrupted trials (2 easy, 2 medium, 2 hard)
• Participants:
– 11 Residents
– 6 Attending Physicians
Control Trials
8 Easy
8 Medium
8 Hard
+
ECG
Response
Interrupt Trials
2 Easy
2 Medium
2 Hard
+
ECG
Interrupt
ECG
Response
Variability in Visual Processes
Avg. Fixation Count
Resident
Attending
Avg. Fixation Duration
Time Spent Focusing on Specific Areas-- Avg Fixation Durations
Attendings
Residents
Dispersion of Attention- Avg Count of Fixations
Attendings
Residents
Disruptive Interruptions
• Longer interpretation times and increased
fixation counts during interrupted trials
Control
Interrupt
Improving ECG Interpretation
• Clear differences:
– Residents search more exhaustively (more areas
examined, more fixations, longer durations)
– Attendings know where to look and are processing
more quickly
– The impact of interruptions is more pronounced with
residents
• Eye movement based logistic regression models
to predict accurate performance (AUC - .88)
Predicted Error
Predicted Accurate
Actual Error
49.6%(62) TP
50.4%(63) FN
Actual Accurate
6% (22) FP
94%(360) TN
Thank You
• [email protected]
• Collaborators:
Zach Hettinger, MD
Allan Fong, MS
• This work was supported by a grant from the Charles
and Mary Latham Foundation