Artificial Intelligence Healthcare Resource Hub
Insights Advancing the Safe Use of AI in Healthcare
As more healthcare organizations strive to use AI to improve the efficiency and quality of medical care, ECRI continues to share support services and insights on the safe and innovative use of AI.
Report AI Problems
AI-enabled devices and applications can sometimes lead to adverse events in patient care. When you report a problem, ECRI investigates. Report adverse events involving any type of device—and those potentially involving AI—through ECRI's Problem Reporting Network.
Key Takeaways
AI-enabled tools have benefits spanning the healthcare field, from identifying at-risk patients and enabling earlier interventions; to assisting clinical decision-making by surfacing relevant insights that might otherwise go unnoticed; to automating administrative tasks, leaving more time for direct patient care.
However, AI also poses significant risks to patient safety if it is not properly assessed and managed. These systems depend on high-quality data, robust clinical validation, and a clear understanding of their intended use. Inadequate training data, poor integration, or lack of transparency can lead to inappropriate outputs and degraded care.
AI Resources
- Quick Guide for Reporting Problems with AI-Enabled Devices - This guide addresses common questions on reporting AI problems - including where to submit reports, how to know if a device uses AI, and types of events.
- Navigating the AI Diagnostic Dilemma was named ECRI's #1 patient safety concern for 2026. Download the Top 10 Patient Safety Concerns report for strategies to mitigate risk in diagnostic AI applications.
- Misuse of AI Chatbots in Healthcare was ranked the #1 health technology hazard for 2026 in ECRI's annual research report. Download the Top 10 Health Technology Hazards for guidance on using these chatbots safely.
- ECRI’s AI Position Paper – This includes a seven-point plan to use AI safely and mitigate risk, plus answers to questions on regulatory clearance assurances and how to assess functionality.
- AI is the #1 hazard in ECRI’s Top 10 Health Technology Hazards for 2025 Report – Download the report for recommendations to implement AI safely through a system-design, human-centered safety infrastructure.
- Webcast recording: Managing Risks in AI-Enabled Health Tech – Seven percent of webinar attendees reported they were aware of instances at their facility in which AI could have contributed to an incident that did (or had potential to) cause harm.
The resources listed below are excerpts from ECRI’s member-only website that have been made accessible to non-members. To learn more about accessing other member-only resources from ECRI, contact us.
- Evidence-Based Risk Assessment Models for Preventing Sepsis in Critically Ill Adults: Several research items in this curated literature search address how AI can be used to prevent sepsis.
- Using AI for Aiding Venous Thromboembolism Risk Assessment: This curated literature search addresses how AI applications can potentially extract relevant risk factors from electronic health record and create preliminary risk stratifications.
- Managing the Risks of AI in Healthcare – This article addresses AI-enabled medical devices that use machine learning, including those that are "configuration locked" and those that are subject to ongoing updating.
- Ethical Use of AI in Healthcare – This article provides guidelines for data management and privacy; bias and fairness; governance; monitoring and evaluation; and provider and patient education.
- Understanding FDA’s Predetermined Change Control Plans (PCCPs) for AI-Enabled Medical Devices – Even with a PCCP, AI-enabled device performance may be unexpectedly altered, presenting new risks that require mitigation strategies.
- Application of AI in Histopathology and Cytology Practice – This evidence analysis indicates AI-enabled diagnostic reviews of high-resolution whole-slide images demonstrate specificity and diagnostic accuracy relative to manual review across various malignancies including gastric, breast, and prostate cancer.
- AI Software for Improving Outpatient Scheduling and Patient Chart Management – This evidence analysis focuses on the risks and benefits of using AI for administrative functions. AI applications may appear to predict patient behaviors and length of appointments and create efficiencies in appointment scheduling.
AI Technology Evaluations
ECRI has evaluated the effectiveness and clinical evidence surrounding numerous AI tools in healthcare, including:
Learn more about device evaluation and clinical evidence assessments.
Experts: AI in Health Tech
These healthcare safety and technology experts from ECRI are often called on to share insights on the safe and strategic use of AI in care delivery and coordination.
Scott Lucas, PhD, PE
Francisco Rodriguez-Campos, MSc, PhD, MRSO (MRSC™)
Jim Martucci, MSBME, MEM
Christie Bergerson, PhD
Scott Lucas, PhD, PE
Vice President, Device Safety
Scott has over twenty years of experience in engineering safety in the healthcare, automotive, maritime, and defense industries. He is head of medical device safety activities, including evaluation and testing of AI software in or as a medical device.
Francisco Rodriguez-Campos, MSc, PhD, MRSO (MRSC™)
Principal Project Officer, Device Safety
Francisco has more than 15 years of experience helping to deploy new technologies in research and healthcare, including evaluating AI software used in different modalities of medical imaging.
Jim Martucci, MSBME, MEM
Strategy Consultant
Jim has 40 years of experience in heavily regulated pharmaceutical and medical device environments leading complex software product development programs, verification and validation processes, risk management, and safety assurance.
Christie Bergerson, PhD
Consultant
Christie brings extensive expertise in AI-enabled medical devices across industry and consulting. She has led market entry for multiple technologies, served on AI ISO committees, and is a driving force behind many of ECRI’s AI initiatives.
To interview or request an AI expert from ECRI as a speaker, contact Yvonne Rhodes at YRhodes@ECRI.org.
Award-Winning AI Innovation
ECRI has honored several healthcare organizations for the innovative use of AI with the Health Technology Excellence Award, including these examples featured in TechNation:
This rural integrated health system used AI-enhanced video to create a tele-sitting program that recognizes when patients prone to falls are getting out of bed. Their novel use of AI led to immediate improvements in safety and helped address a staffing crisis exacerbated by the pandemic.
The Institute developed a safety event reporting software application that uses natural-language processing to sort and categorize incident reports. The application, called “Hero,” led to measurable improvements in organizational reporting and learning culture.
Insights for Government
ECRI has provided briefings and recommendations to government and policy leaders on the safe use of AI, including these publicly available examples:
- Accelerating Adoption and Use of AI Clinical Care: ECRI submitted input on accelerating AI in clinical care in response to a U.S. Department of Health and Human Services (HHS) request for information. (Feb. 2026)
- Response to HTI-5 Proposed Rule, Implications for Patient Safety,Clinicians, and AI Adoptions: The rule proposed removing more than 50% of certification criteria, including requirements on clinical decision support. (Feb. 2026)
- Measuring and Evaluating AI-Enabled Medical Device Performance in the Real World: ECRI provided insights in response to FDA's request on measuring and evaluating AI-enabled device performance. (Dec. 2025)
- Responsibly Developing and Sharing Generative AI Tools: ECRI submitted comments to National Institutes of Health (NIH) on Generative AI Tools that use NIH-Controlled Data (Jul. 2025)
- White House AI Action Plan: ECRI submitted practical recommendations to the Office of Science and Technology Policy to support the development of their AI Action Plan. (Mar. 2025)
Timeline of Red Flags
ECRI has sounded the alarm about potential risks in the use and misuse of AI. AI-related topics were addressed in ECRI’s Top Ten Health Technology Hazards report four out of the last five years, and they have also been covered in ECRI’s Top 10 Patient Safety Concerns report.
2021
#8 Health Technology Hazard
ECRI identified "Artificial Intelligence (AI) Applications for Diagnostic Imaging May Misrepresent Certain Patient Populations" as the number 8 concern in the Top 10 Health Technology Hazards for 2021 report. This ranking highlighted growing concerns within the medical community about potential biases in AI-driven diagnostic tools, particularly how these systems may produce inaccurate or misleading results for patients from underrepresented or diverse demographic groups. The report underscored the urgent need for more equitable data collection and algorithm training to ensure that AI applications in healthcare serve all populations effectively and fairly.
2022
#7 Health Technology Hazard
For the Top 10 Health Technology Hazards for 2022, ECRI once again emphasized the potential risks associated with artificial intelligence in healthcare by naming “AI-Based Reconstruction Can Distort Images, Threatening Diagnostic Outcomes” as one of the top hazards of the year. This designation brought attention to a critical issue: the use of AI algorithms in reconstructing medical images—such as those from CT scans or MRIs—can sometimes lead to image artifacts or distortions that may not be immediately apparent to clinicians. These distortions have the potential to obscure or mimic clinical findings, ultimately jeopardizing diagnostic accuracy and patient safety.
2024
#5 Health Technology Hazard
In the Top 10 Health Technology Hazards for 2024 report, ECRI pinpointed insufficient governance of AI in medical technologies as a potential source of danger and offered practical recommendations for reducing risks. The report emphasized that the rapid integration of AI into clinical environments—ranging from diagnostic tools to decision-support systems—has outpaced the development of appropriate oversight structures, regulatory frameworks, and institutional policies. Without robust governance mechanisms in place, healthcare organizations face increased risks related to data bias, lack of transparency, algorithmic errors, and the misuse or misunderstanding of AI outputs.
2024
#4 Patient Safety Concern
In the Top 10 Patient Safety Concerns for 2024 report, ECRI identified "Unintended Consequences of Technology Adoption including AI" as one of the most pressing challenges facing healthcare organizations. As hospitals and health systems increasingly turn to advanced technologies—such as AI-powered tools, electronic health records, and remote monitoring devices—to improve care delivery, they also face new and often unforeseen risks. ECRI warned that without thoughtful planning, training, and oversight, these technologies can inadvertently introduce safety hazards, such as clinician overreliance on AI recommendations, workflow disruptions, alert fatigue, or missed diagnoses due to algorithmic errors.
2025
#1 Health Technology Hazard
ECRI ranked “Risks with AI-Enabled Health Technologies” as the number one health technology hazard in the Top 10 Health Technology Hazards for 2025 report. As AI becomes increasingly embedded in diagnostic tools, decision-support systems, and patient monitoring platforms, ECRI warned that the potential for harm has escalated—particularly when these technologies are deployed without sufficient oversight, validation, or understanding of their limitations.
2025
#2 Patient Safety Concern
In the Top 10 Patient Safety Concerns 2025 report, ECRI highlighted “Insufficient Governance of AI in Healthcare” as the
second most critical concern impacting patient safety across care settings. ECRI emphasized that while AI offers transformative potential—enhancing diagnostic accuracy, streamlining workflows, and supporting clinical decision-making—the absence of robust governance structures can lead to significant risks.
2026
#1 Patient Safety Concern
Navigating the AI Diagnostic Dilemma was named ECRI’s #1 patient safety concern for 2026. AI has the potential to improve diagnostic accuracy by automating data retrieval, decreasing cognitive load, reducing cognitive biases, and providing clinicians with information to help guide their decisions. But users should not treat AI as a replacement for clinical expertise. Placing too much trust in an AI model to diagnose patients without factoring in clinician expertise can lead to misdiagnosis—the very problem AI was intended to solve.
2026
#1 Health Technology Hazard
The Misuse of AI Chatbots in Healthcare earned the #1 spot in ECRI’s annual health tech hazards report. AI chatbots and other large language models (LLMs) are not designed or regulated for healthcare purposes. Nevertheless, many people turn to LLMs for advice about medical conditions and treatments. This can seem innocuous but can have critical implications for safety. LLMs respond to queries with seemingly authoritative answers, but these responses can be incorrect. Users must recognize the limitations of these models and carefully scrutinize responses whenever using an LLM for an application that could influence patient care.
Get in Touch
Learn more about our impactful work can benefit you and your organization.