- Health data is special category data under the UK GDPR: each processing operation needs an Article 6 lawful basis and an Article 9 condition, documented
- Clinical AI is not automatically high-risk under the EU AI Act: classification depends on intended purpose and the Article 6 route, not on use in a clinical setting
- Providers and deployers carry different duties, and a healthcare organisation that builds or substantially modifies a system can become a provider
- A DPIA is mandatory where processing is likely to result in high risk; if high residual risk remains, you must consult the ICO before processing starts
- Depending on its intended purpose, AI software can be a medical device regulated by the MHRA, on top of your data protection duties
The actual risk: health data and AI need a specific assessment
Health data is special category data under Article 9 of the UK GDPR. Processing it requires a lawful basis under Article 6 and a separate condition under Article 9(2). For care providers the usual Article 9 condition is 9(2)(h), the provision of health or social care, read with the conditions in Schedule 1 of the Data Protection Act 2018. Explicit consent, vital interests and public health are other possible conditions. Legitimate interests can be an Article 6 basis, but there is no equivalent Article 9 condition.
Work this out per processing operation, not per tool. Many AI tools process data in ways the interface does not show. A clinical summarisation tool receives consultation notes. A scheduling AI may use diagnosis codes to prioritise appointments. An administrative chatbot may store conversations that mention symptoms or medication. Each of these is processing of health data and needs to be mapped before the tool goes live.
The vendor's role. A SaaS AI vendor is normally your processor when it handles data on your documented instructions. For some operations it may act as an independent controller, for example when it uses data to improve its own models or for its own commercial purposes. Before you sign, identify each party's role, the sub-processors, the permitted purposes, and the retention and deletion terms.
Clinical AI and the EU AI Act: when it is high-risk
The EU AI Act (Regulation (EU) 2024/1689) does not apply in the UK as such, but it reaches UK organisations that place AI systems on the EU market, use them in the EU, or use their outputs in the EU. Many UK suppliers will also follow it because they sell into Europe.
AI used in a clinical workflow is not automatically high-risk. Article 6 sets out two routes:
- Article 6(1) and Annex I: the system is a medical device, or a safety component of one, under the MDR (Regulation (EU) 2017/745) or IVDR (Regulation (EU) 2017/746), and must undergo conformity assessment by a notified body. Much diagnostic and treatment-support software follows this route, but it depends on the device class.
- Article 6(2) and Annex III: the system falls within a listed use case, such as assessing eligibility for essential public services including healthcare, or emergency patient triage. An Annex III system may fall outside high-risk if it does not materially influence the decision (Article 6(3)), unless it profiles individuals.
Scheduling, billing and administrative documentation tools usually follow neither route. What decides the classification is the intended purpose stated by the provider, not the setting in which the tool is used or how it is marketed.
The Digital Omnibus on AI (Regulation (EU) 2026/1744) moved the high-risk dates: obligations for systems classified under Annex III apply from 2 December 2027, and for systems under Annex I, such as AI in medical devices, from 2 August 2028. The deadline that applies to you depends on the classification route. See our EU AI Act guide and the overview of prohibited and high-risk AI.
The marketing description is not the legal test. For each tool, keep a documented assessment of its regulatory classification: intended purpose, medical device status, Article 6 route and your organisation's role. UK GDPR obligations apply whatever the AI Act classification.
Provider, deployer and healthcare professional
Where a system is high-risk, the compliance duties are split. The EU AI Act separates the provider, which develops the system and places it on the market, from the deployer, which uses it under its own authority, such as a hospital or clinic. In the NHS, the clinical safety standards DCB0129 (manufacturers) and DCB0160 (deploying organisations) follow a similar split.
| Role | Main responsibilities |
|---|---|
| Provider | Risk management, training data quality, technical documentation, conformity assessment, registration in the EU database, instructions for use and post-market monitoring (Articles 16 and 49) |
| Deployer (hospital, clinic) | Use the system according to the instructions, assign human oversight to competent people, ensure input data is relevant, monitor operation, keep the logs under its control for at least six months, and report risks and serious incidents (Article 26). Public bodies and private entities providing public services may need a fundamental rights impact assessment for Annex III systems (Article 27) |
| Healthcare professional | Apply their own clinical judgement, understand the system's limitations, be able to challenge or disregard the output, and report wrong outputs through the agreed route |
A healthcare organisation becomes a provider if it develops a high-risk system in-house, puts its name or trademark on one, makes a substantial modification, or changes the intended purpose so that the system becomes high-risk (Article 25).
Clinical AI in the UK: when MHRA rules apply
In the UK, software that meets the definition of a medical device is regulated by the MHRA under the UK Medical Devices Regulations 2002, and the MHRA publishes specific guidance on software and AI as a medical device. Whether a tool qualifies depends on the manufacturer's intended purpose: software for diagnosis, monitoring or treatment decisions can be a medical device, while purely administrative tools generally are not.
Documentation and ambient scribe tools need case-by-case assessment. Ask the supplier for its regulatory position and, where relevant, its UKCA or CE marking and MHRA registration. Clinical safety and performance assessment sits alongside data protection, and one does not replace the other.
Five questions to ask before deploying any AI tool
Run these questions across every AI tool under evaluation, including AI features added to existing systems such as electronic patient records, scheduling and patient communication.
What to verify before you contract
The five questions decide whether a tool can go further. The table below sets out the full assessment.
| Area | What to verify |
|---|---|
| Intended use | The supplier's stated purpose, intended users, clinical or administrative context and any uses the supplier excludes |
| Lawful basis | Article 6 basis and Article 9 condition for each operation, documented |
| Regulatory classification | Medical device status with the MHRA; EU AI Act route where applicable; the duties that come with your role |
| Clinical performance | Validation population, accuracy, bias, false positives and false negatives, known limitations and fit with your patient population |
| Human oversight | Who reviews outputs, with what information, and how a recommendation is challenged or overridden |
| Security | Access control, encryption, activity logging, security testing and vulnerability management |
| Data governance | Training on patient data, inference, logging of prompts and outputs, secondary use by the vendor, retention and deletion |
| Vendor governance | Sub-processors, notice of model updates, incident notification, audit evidence and exit plan |
| Operational continuity | Fallback procedure if the tool is unavailable and dependence on connectivity |
| Monitoring | Performance drift indicators, patient safety incident reporting and triggers for reassessment |
"We don't train on your data" does not cover every risk. The statement is about training. It says nothing about inference (sending data to produce an output), logging of prompts and responses by the vendor, support staff access, or secondary use for analytics and product improvement. Ask for written answers on each point.
Before go-live, define how a clinician reports a wrong output, who reviews it and when the tool is suspended. Do the same for performance drift and for changes in behaviour after vendor updates, which may require revalidation.
When a DPIA is mandatory
A Data Protection Impact Assessment is mandatory under Article 35 of the UK GDPR before processing that is likely to result in a high risk to individuals. Article 35(3) names large-scale processing of special category data, and the ICO's list of processing that requires a DPIA includes innovative technology such as AI combined with other risk factors. A healthcare AI tool deployed across a clinical service will almost always meet that threshold.
For a healthcare AI tool, the DPIA should cover the purpose and necessity of the processing, data flows and retention, risks to patients (including biased outputs), security measures, the vendor's role, and how solely automated decisions are avoided or safeguarded. A proper DPIA takes time, so start it during procurement, not after contract signature.
If your DPIA shows a high residual risk that you cannot mitigate, Article 36 requires prior consultation with the ICO before processing begins. The ICO has up to eight weeks to respond, extendable by six weeks for complex cases. If the EU GDPR also applies, consult the competent EU supervisory authority as well.
What safe deployment looks like in practice
Data minimisation. Audit what data the tool actually needs. Many AI tools are configured to receive more data than necessary. A summarisation tool that asks for the full patient record to summarise one consultation should be challenged: can it work from the consultation content alone, without other identifiers?
Deployment model. Local deployment is not automatically the right default. Compare on-premises, private cloud and third-party AI services on data flows, legal requirements, security controls, clinical validation and your team's operational capacity. Processing on your own infrastructure reduces some third-party access and international transfer risks. In exchange, your organisation takes on patching, infrastructure security, access control, monitoring, model validation, availability and retention.
Meaningful human oversight. For high-risk systems, the EU AI Act requires oversight that lets people understand the system's limitations, interpret the output, disregard it and stop the system (Article 14). Even outside that regime, design the workflow so review is real: a summary checked before it enters the record is different from a system that updates records without review.
Organisations that document the regulatory classification, the lawful basis and the oversight arrangements are best placed to demonstrate due diligence and compliance. For staff using unapproved tools, see shadow AI and GDPR exposure and our GDPR compliance guide.
Primary sources: EU AI Act (Regulation (EU) 2024/1689), Regulation (EU) 2026/1744 and the ICO guidance on DPIAs.