Hospitals Are Scaling AI, The Workflow Needs Its Own Safety Test – OpEd
A year-long NHS evaluation of AI across 104 operating theatres and related suites at eight hospitals prioritises assessing real-world bottlenecks and capacity use before scaling, underscoring that clinical AI must be judged inside local workflows rather than by model benchmarks alone.
Studies and guidelines such as DECIDE-AI and FUTURE-AI show that structured implementation—training, governance, feedback and standardisation—can sharply raise clinician engagement with AI outputs, yet regulatory clearance does not guarantee equal benefit, safety or workflow fit in every hospital.
Local evaluation should therefore define the intended care change, distinguish model effects from organisational ones, monitor human factors and subgroups, test fallbacks for outages, and set clear expansion gates so that institutions can confirm the whole care system improves, not merely that an algorithm performs well.
A model can perform well and still fail in practice if alerts arrive at the wrong time, staff cannot act on them, or the fallback collapses. The next phase of medical AI should evaluate the care system around the model as rigorously as the model itself.
On September 9, eight NHS hospital sites launched a year-long evaluation of artificial intelligence across 104 operating theatres, endoscopy suites and catheterisation laboratories. The programme is operational rather than diagnostic: it is intended to identify bottlenecks, reduce delays and make better use of theatre capacity across more than 20,000 procedures a year. What makes the project interesting beyond Britain is not the promise that AI will improve efficiency. It is the decision to evaluate the technology across multiple hospitals under a common framework before treating early results as a reason to scale further.
That is the right instinct for clinical AI as well. Hospitals are increasingly asked to judge systems whose technical performance may already look impressive. The harder question begins after the model enters a live service. Who responds to its output? What other work is delayed? What happens when the input is missing, the alert arrives during a busy shift, or the system is unavailable? A benchmark can describe a model. It cannot describe the........
