
Healthcare AI developers risk exposing sensitive patient data and deploying tools that go beyond their intended purpose without appropriate regulatory oversight, legal experts have warned.
Speaking at Health Tech World Live in August, two panellists from law firm Osborne Clarke highlighted gaps in the UK’s approach to AI accountability, the importance of securing patient consent and the challenges of regulating rapidly evolving technologies.
Marcus Vass, a legal expert specialising in technology and healthcare, warned that companies often seek legal advice too late in the development process, after large volumes of patient data have already been used without appropriate consent.
“Very often, what we find is that the health AI, especially a medical device, is already using lakes of data for which there is no consent that’s been obtained, and of course that is hugely problematic,” he said.
Vass argued that data protection requirements must be built into AI development from the outset, rather than treated as a problem to resolve once a product is ready for market.
He warned that the misuse of patient information to train large language models could trigger a major healthcare data scandal, undermining public trust in AI.
Vass added that companies should involve legal and regulatory specialists early to ensure their products comply with data protection law, including the UK General Data Protection Regulation (UK GDPR).
Who is accountable when healthcare AI goes wrong?
Michelle Brandt-Sarif, a lawyer and clinician, highlighted the lack of a dedicated UK framework for determining accountability when AI causes harm in healthcare.
Instead, she said, responsibility is assessed through a patchwork of existing legal frameworks, including negligence, the Consumer Protection Act and contract law, none of which was originally designed with AI in mind.
“There is actually no framework in the UK that tells you how to deal with these situations when things go wrong,” she said.
Brandt-Sarif warned that AI tools can perform well during testing but encounter unforeseen problems once deployed, including when they lack data needed to produce accurate results.
There is a risk that healthcare professionals or patients may rely on outputs without adequate checks.
She also highlighted the growing use of general-purpose AI for health advice, raising questions about who should be held responsible when people act on information that causes harm.
In clinical settings, she said, professionals must continue to exercise their own judgement rather than assume responsibility ends when an AI tool is involved.
Regulatory challenges as AI capabilities expand
The discussion also explored how AI tools can develop beyond their original intended use, creating new regulatory challenges.
Vass pointed to ambient voice technology, which is increasingly used to transcribe clinical conversations.
While a tool may initially function as a medical scribe, additional capabilities can allow it to recommend prescriptions, suggest treatments or generate clinical insights.
Such developments can take a product beyond its original regulatory classification, potentially introducing functions that have not undergone appropriate certification or validation.
“The problems we’re seeing are the problems of the capabilities and the functionality of AI are so much greater than those which they’re initially adopted for,” Vass said.
Vass praised the MHRA for its work with early-stage developers through AI sandboxes, which can help companies understand regulatory requirements while informing the regulator’s guidance.
He also highlighted the importance of post-market surveillance, particularly when AI tools acquire new capabilities after deployment.
Health Tech World Live in 2027
Health Tech World is set to deliver three in-person events in autumn 2027, with the first panellists to be confirmed by the end of this year.
Would you like a seat at the table?
Email Health Tech World Live co-ordinator Gordon to find out how you can get involved.










