What does the EU AI Act mean for healthcare organisations?

The EU AI Act (Regulation (EU) 2024/1689) is the European Union's first comprehensive legal framework for artificial intelligence. For healthcare organisations, it adds a new layer of legally binding obligations on top of existing rules, and it applies to hospitals, GP practices, and health technology vendors operating in or supplying to EU markets.

The European Parliament adopted the Act in March 2024. It sets a risk-based framework, scaling obligations to the potential harm an AI system could cause, and establishes the European AI Office within the European Commission to oversee general-purpose AI (GPAI) models, meaning large AI models trained on broad datasets that can perform a wide range of tasks.

When do the Act's obligations apply?

The Act entered into force on 1 August 2024, but its obligations phase in over several years.

• February 2025: Prohibitions on unacceptable-risk AI systems take effect, alongside an AI literacy obligation requiring staff who work with AI systems to understand them sufficiently.

• August 2025: Rules governing general-purpose AI models, including the foundation models behind many clinical AI tools, become applicable.

• August 2026: Core obligations for high-risk AI systems apply in full, including conformity assessments, technical documentation, and human oversight requirements. This is the critical deadline for most healthcare AI.

• August 2027: AI systems already regulated as medical devices, and which require Notified Body assessment, get an extra year to meet the Act's Article 6(1) requirements.

The European Commission has proposed adjustments to the high-risk timeline through its Digital Omnibus initiative, partly in response to delays in developing harmonised standards. Healthcare organisations should monitor this, since it may shift when certain documentation requirements become enforceable.

Why healthcare is a priority sector under the Act

Healthcare is explicitly identified as a high-risk domain in the AI Act because AI systems used in clinical settings can directly affect patient safety, access to care, and fundamental rights. An AI system that influences a diagnostic decision, recommends a treatment pathway, or triages patients based on predicted urgency carries material potential for harm if it is inaccurate, biased, or used without adequate human oversight.

The European Commission's public health guidance on AI in healthcare notes that high-risk AI systems intended for medical purposes must satisfy requirements around risk mitigation, data quality, transparency, and human oversight. This reflects a broader recognition that the stakes in healthcare differ fundamentally from those in sectors such as marketing or logistics.

A comparative analysis of AI governance frameworks across five jurisdictions found that risk classification schemes for healthcare AI are converging internationally, with the EU approach serving as an influential model, particularly in how it distinguishes between AI that supports decision-making and AI that could autonomously influence clinical outcomes.

Why does healthcare count as high risk?

The Act treats healthcare as a high-risk domain because AI systems used in clinical settings can directly affect patient safety, access to care, and fundamental rights. A system that influences a diagnosis, recommends a treatment pathway, or triages patients by predicted urgency carries real potential for harm if it's inaccurate, biased, or used without adequate oversight.

The European Commission's own guidance on AI in health notes that high-risk AI systems for medical purposes must satisfy requirements around risk mitigation, data quality, transparency, and human oversight. A comparative governance study across five jurisdictions, published in Social Science & Medicine, found healthcare AI risk classification schemes converging internationally around transparency and accountability, with the EU's approach acting as an influential model, even as enforcement maturity varies widely between countries.

How does the Act classify AI systems?

The Act sorts AI systems into four risk tiers, each carrying different obligations. Unacceptable risk systems, such as social scoring by public authorities, are prohibited outright, a rule that applied from February 2025. High risk systems pose significant risks to health, safety, or fundamental rights, but their benefits may justify use under strict conditions. Healthcare AI falls predominantly here. Limited risk systems carry specific transparency obligations, such as a chatbot disclosing that it's AI, and many patient-facing tools sit in this tier. Minimal risk systems, such as spam filters, carry no specific obligations.

A separate category covers general-purpose AI models. These foundation models underpin many clinical language tools and carry their own obligations regardless of the risk tier of the application built on top of them.

Which healthcare AI systems count as high risk?

Annex III of the Act sets out the high-risk categories. For healthcare, the most relevant is point 5(a): AI systems used as safety components of medical devices, or standalone AI systems that are themselves medical devices. This covers diagnosis and clinical decision support, treatment recommendations, patient triage, and continuous patient monitoring that flags deterioration.

A peer-reviewed analysis in npj Digital Medicine found that around 75 per cent of commercial AI-enabled medical devices sit in radiology, and nearly all are classified as Class IIa or above under the Medical Device Regulation, meaning most deployed clinical AI counts as high risk under the AI Act.

Virtual health assistants and clinical chatbots sit in a more nuanced position. Tools that provide clinical information influencing a patient's decisions may attract high-risk classification, while those functioning mainly as communication interfaces may only carry limited-risk transparency rules.

How does the EU AI Act interact with medical device regulation?

AI systems already regulated as medical devices under the Medical Device Regulation or the In Vitro Diagnostic Regulation must meet both frameworks at once, and the obligations don't simply merge. Reed Smith's legal analysis describes this as a dual compliance framework: medical device AI systems classified as Medical Device Regulation Class IIa, IIb, or III will generally also qualify as high risk under the Act, which then adds requirements around data quality, record keeping, transparency, and human oversight beyond what the Medical Device Regulation covers.

Organisations can fold the Act's requirements into an existing quality management system, and a single conformity assessment is possible where the Notified Body is accredited under both frameworks. Even so, the Hunton Andrews Kurth briefing is clear that Medical Device Regulation compliance alone isn't sufficient. The Act adds obligations with no direct equivalent, including a requirement to report serious incidents within 15 days. White & Case's analysis describes the Act and the revised Product Liability Directive, taken together with the MDR, as creating a tight liability framework for manufacturers of AI-powered medical devices, especially now that the standalone AI Liability Directive has been withdrawn.

Key compliance obligations for healthcare providers deploying high-risk AI

Organisations deploying or supplying high-risk AI in healthcare need to meet several core obligations:

• Conformity assessment before going to market, which can fold into existing Notified Body review for AI medical devices.

• Technical documentation, kept up to date, covering a system's design, development, training data, performance, and known limitations.

• Human oversight, so users can monitor, understand, and override or stop outputs where necessary.

• Transparency, so users know they're interacting with or relying on an AI system, including clear disclosure when it generates a diagnostic suggestion or clinical documentation.

• Data governance, ensuring training and testing data is relevant, representative, and free from errors likely to cause unsafe or discriminatory outputs.

• Post-market monitoring, collecting performance data after deployment and reporting serious incidents to the relevant national authority.

• AI literacy, required since February 2025, so staff understand a system's capabilities and limitations. HIMSS notes this applies across all risk tiers, not just high-risk systems.

A scoping review in npj Digital Medicine identified human oversight, transparency, and post-deployment monitoring as the most consistently cited components of effective AI governance in healthcare.

Who is responsible: AI providers vs deployers

The Act draws a clear line between two obligated parties. Providers develop an AI system, place it on the market, or put it into service under their own name, and this includes AI vendors and health technology companies. Deployers use a high-risk AI system under their own authority in a professional context, which in healthcare means hospitals, GP practices, and any organisation using a third-party AI tool in clinical or administrative work.

Both carry distinct, non-transferable obligations. Providers must ensure their systems meet technical and documentation requirements before deployment. Deployers must ensure appropriate use, adequate oversight, and trained staff. The Diagnostic and Interventional Radiology journal analysis is explicit that deployers can't simply rely on vendor compliance and must actively verify that the tools they use meet the Act's requirements.

Healthcare organisations that build AI tools in-house for their own use may qualify for a limited exemption, but only under specific conditions, and data quality and transparency standards still apply.

What does human oversight mean in a clinical setting?

The Act's human oversight requirement isn't simply a legal formality. It reflects the principle that consequential decisions affecting patients shouldn't be delegated entirely to automated systems: a clinician reviewing AI-generated clinical documentation before it's saved to the record, a radiologist independently judging an AI-flagged finding, a triage nurse treating an AI priority score as input rather than instruction.

A commentary in the BMJ argues that this "clinician in the loop" model can function less as a genuine safety mechanism and more as a way of shifting accountability for AI failures onto individual clinicians, particularly when workflow pressure leaves little real space to question or override an output. The authors argue oversight needs organisational structures, not just a named individual expected to catch errors. Healthcare organisations evaluating AI tools should check whether a product genuinely supports clinician review, or whether time pressure makes rubber-stamping the easier path.

How does the EU AI Act interact with GDPR?

The AI Act's data governance rules sit alongside, not instead of, the General Data Protection Regulation (GDPR), and GDPR compliance doesn't automatically satisfy the Act's data requirements. Where GDPR governs the lawful processing of personal data, the Act focuses on the quality of data used to train, validate, and test AI systems, requiring it to be relevant, representative, and free from errors or biases that could cause unsafe or discriminatory outputs.

The npj Digital Medicine commentary notes that AI systems trained on datasets that under-represent groups by age, ethnicity, sex, or comorbidity profile can produce systematically worse outputs for those groups, with direct patient safety implications.

Data residency is a further consideration. Organisations buying AI tools from non-EU vendors should also confirm where patient data is processed and stored, both for GDPR and for the Act's transparency requirements. The European Health Data Space, in force since 2025, adds a further layer of data governance specific to health data.

What should healthcare organisations do now?

• Audit current AI tools, including systems in medical record systems, diagnostic software, and patient-facing apps.

• Assess risk classifications for each system against Annex III.

• Review vendor contracts to see how compliance obligations are allocated high risk.

• Appoint an AI governance lead, whether internally or through external support in smaller practices.

• Prepare documentation, including evidence of conformity assessments and oversight mechanisms.

• Deliver AI literacy training, an obligation that has applied since February 2025.

• Set up incident reporting processes aligned with the 15-day requirement for serious incidents.

A comparative governance study across five jurisdictions found that organisations which built AI governance structures ahead of regulatory deadlines were better positioned to meet compliance requirements than those that waited for enforcement to start.

What to ask when evaluating an AI vendor for EU AI Act compliance

Procurement and clinical informatics teams should treat AI Act compliance as standard due diligence, alongside clinical evidence and data security. Useful questions include:

• How does the vendor classify this system under the AI Act, and what evidence supports that?

• Has it undergone a conformity assessment, and by which Notified Body?

• Can the vendor provide full technical documentation, including known limitations?

• For AI medical devices, is CE marking in place under the MDR or IVDR, and has it been updated to reflect AI Act requirements?

• How does the product's design support clinician review and override in a live workflow?

• Where is patient data processed and stored, and does this meet GDPR and EHDS requirements?

• If the product is built on a foundation model, what obligations apply to that model under the GPAI provisions, and how does the vendor manage them?

The European Commission's AI Act Single Information Platform, including its compliance checker and AI Act Explorer, gives procurement teams a structured, official starting point, though the Commission itself flags the tool as indicative rather than a substitute for formal advice.

What are the penalties for non-compliance?

Penalties scale with the severity of the breach: up to €35 million or 7 per cent of global turnover for prohibited AI systems, up to €15 million or 3 per cent for other breaches including high-risk requirements, and up to €7.5 million or 1.5 per cent for misleading authorities.

Member states enforce the Act through national authorities, and penalties can apply to deployers as well as providers where a deployer fails to meet its own obligations. White & Case's liability analysis notes that the withdrawal of the standalone AI Liability Directive in February 2025 means civil liability claims now run through the revised Product Liability Directive instead.

Glossary: key EU AI Act terms for healthcare professionals

Key terms

• Provider: develops an AI system, places it on the EU market, or puts it into service under its own name.

• Deployer: uses a high-risk AI system in a professional context under its own authority.

• High-risk AI system: a system listed in Annex III, or used as a safety component in a product regulated under law such as the Medical Device Regulation.

• Conformity assessment: the formal process by which a provider demonstrates that a high-risk system meets the Act's requirements.

• Post-market monitoring: collecting real-world performance data to catch risks not visible before deployment.

Frequently asked questions

▶ What is the EU AI Act and does it apply to healthcare organisations?

Regulation (EU) 2024/1689 is the EU's first comprehensive AI law. It applies to hospitals, GP practices, and health technology vendors placing AI on the EU market, and its extraterritorial reach means non-EU suppliers are covered too if their systems affect EU patients.

▶ When do healthcare organisations need to comply with the EU AI Act?

In phases: prohibitions and AI literacy from February 2025, general-purpose AI model rules from August 2025, and core high-risk obligations, including conformity assessments and human oversight, from August 2026. AI systems already regulated as medical devices get until August 2027.

▶ Which healthcare AI systems are classified as high risk under the Act?

Diagnosis, clinical decision support, treatment recommendations, triage, and patient monitoring. A peer-reviewed analysis in npj Digital Medicine found around 75% of commercial AI medical devices sit in radiology, nearly all classified Class IIa or above under the MDR, meaning most deployed clinical AI counts as high risk.

▶ What's the difference between a provider and a deployer under the EU AI Act?

Providers develop and place an AI system on the market. Deployers use it professionally, which in healthcare means hospitals and GP practices using third-party tools. Obligations can't be transferred between the two: deployers can't simply rely on vendor compliance and must actively verify systems meet the Act's requirements.

▶ Does MDR compliance satisfy the EU AI Act requirements for AI medical devices?

No. AI medical devices must meet both frameworks at once. Reed Smith's analysis notes the Act adds data governance, transparency, and human oversight requirements beyond the MDR, though a single conformity assessment is possible where the Notified Body is accredited under both.

▶ What does the human oversight requirement mean in practice for clinical staff?

Consequential decisions shouldn't be delegated entirely to automated systems: a clinician reviews an AI note before it's saved, a radiologist independently judges an AI-flagged finding, a triage nurse treats an AI score as input, not instruction. A BMJ commentary warns this only works if staff have real organisational permission to override outputs, not just the theoretical ability to.

▶ How does the EU AI Act interact with GDPR for healthcare organisations?

They sit alongside each other. GDPR governs lawful processing of personal data; the Act governs the quality of data used to train and test AI systems, requiring it to be relevant, representative, and checked for bias. Meeting one doesn't automatically satisfy the other.

▶ What are the penalties for non-compliance with the EU AI Act?

Tiered by severity: up to €35 million or 7% of global turnover for prohibited systems, up to €15 million or 3% for high-risk breaches, and up to €7.5 million or 1.5% for misleading authorities. These sit in the Regulation itself, and can apply to deployers as well as providers.

▶ What should healthcare organisations be doing now to prepare for the EU AI Act?

Audit current AI tools, assess each against Annex III, review vendor contracts, deliver AI literacy training (an obligation since February 2025), and set up incident reporting aligned with the 15-day requirement.

▶ What questions should procurement teams ask AI vendors about EU AI Act compliance?

How the vendor classifies the system and why, whether a conformity assessment is complete and by which Notified Body, whether full technical documentation is available, how the product supports clinician review and override, and where patient data is processed and stored. The EU's own compliance checker is a useful starting point, though the Commission flags it as indicative rather than a substitute for advice.