JMIR Publications

ABSTRACT

Background:

FDA-cleared artificial intelligence (AI) triage tools for intracranial hemorrhage (ICH) are increasingly deployed in clinical radiology. In real-world practice, perceived utility may depend not only on diagnostic performance but also on workflow friction, false-alarm burden, and calibrated trust when AI outputs conflict with radiologist interpretation.

Objective:

To characterize radiologists’ perceptions, trust calibration, and self-reported vigilance behaviors when using an FDA-cleared ICH AI triage tool in a national teleradiology network and to evaluate differences by neuroradiology subspecialty training.

Methods:

We conducted an anonymous cross-sectional survey of radiologists in a national teleradiology practice who had access to an FDA-cleared ICH detection AI overlay during routine noncontrast head CT interpretation. Survey domains included perceived reliability and usefulness, false-alarm burden, workflow integration, medicolegal concerns, and items designed to probe self-reported vigilance behaviors consistent with automation complacency. Responses used a 5-point Likert scale (Strongly agree, Agree, Neutral, Disagree, Strongly disagree). Results are summarized as agreement proportions (“agree”/“strongly agree”). We evaluated subgroup differences between neuroradiologists and non-neuroradiologists using Fisher exact tests. To reduce risk of spurious findings from multiple comparisons, we prespecified a primary endpoint and treated other items as exploratory with false discovery rate (FDR) control using the Benjamini–Hochberg procedure. Optional free-text responses were analyzed qualitatively to identify recurring themes.

Results:

Sixty-five radiologists responded (23 neuroradiologists; 42 non-neuroradiologists). Only 18.5% (12/65) agreed that false-positive alerts were infrequent enough to be acceptable. Trust was highly conditional: 50.8% (33/65) trusted the AI when it agreed with their interpretation, whereas only 3.1% (2/65) trusted it when it conflicted. The primary endpoint—agreement that false-positive workload outweighed benefits—was endorsed by 33.9% (22/65) overall and was more common among neuroradiologists than non-neuroradiologists (52.2% vs 23.8%; unadjusted P=.029). However, after FDR correction across exploratory items, no subgroup differences remained statistically significant. Self-reported vigilance reduction on AI-negative outputs was uncommon (6.2% overall; 0% neuroradiologists; 9.5% non-neuroradiologists). Free-text feedback emphasized artifact-driven false positives, delayed or inconsistent AI availability, consult burden, and medicolegal concerns.

Conclusions:

In a national teleradiology environment, radiologists reported substantial false-alarm burden and highly conditional trust when using an FDA-cleared ICH AI triage tool. Self-reported vigilance reduction was uncommon but present in a minority of users. Human factors–oriented optimization—including specificity improvements, earlier availability, better localization, and workflow-aware triage routing—may improve acceptance and perceived utility.