Healthcare is on an accelerated path to digitisation, creating real opportunity to reduce administrative burden, improve risk detection, support prioritisation, and help people access the right care sooner. But without deliberate design, these same systems risk reinforcing the very disparities we’re trying to solve. In turn, this can reshape decisions in ways that lack transparency and unintentionally disadvantage those already facing barriers to care.
The scrutiny we’re seeing today around algorithmic decision-making is both justified and necessary. When AI influences high-stakes outcomes, accountability must remain clear, and risks must be actively managed. It should always be evident who is responsible for every decision, especially when technology is involved.
This next phase of healthcare transformation presents a significant opportunity, not just to digitise what exists, but to redesign how care is delivered in a way that is fairer. For me, this means embedding inclusion before systems are built, intentionally designing without barriers from the start and preventing bias from scaling.
Inclusion is practical risk management
We tend to look at AI bias as a data problem, underestimating the critical role of leadership in shaping these tools. Systems built on historical data, combined with decisions made by a narrow set of leadership voices, will inevitably produce solutions that only work well for the majority. In a healthcare setting, this looks like the “average” patient: someone with typical presentation, stable circumstances and straightforward access to care.
But healthcare is complex. No two patients are the same. Access, experience, and outcomes vary across communities, geographies, and lived experience. If these differences are not actively included in how systems are designed, implemented and governed, they remain invisible and are less likely to be noticed until they show up in care delivery.
Inclusion is what brings those differences into the room early, before they scale.
From experience, being “the only one” in a leadership setting changes how you listen and what you question. It sharpens awareness of what’s missing, what’s assumed and what isn’t being challenged. That matters, because early questions lead to early fixes, leaving far less damage to undo down the line.
But inclusion is not achieved by simply having diverse representation at the table. It’s about ensuring every voice shapes decisions, welcoming challenges and including diverse lived experiences in design, governance and outcomes.
For inclusion to be meaningful in healthcare, it must be embedded systemically through hiring, leadership development and success metrics. That means actively addressing where bias emerges, investing in sponsorship and pathways, and creating environments where differing perspectives are not only present, but expected.
When AI scales, humans and governance must scale with it
The next risk emerges once systems are deployed.
In healthcare, decisions are often made under pressure, with fragmented information and constrained resources. In these conditions, AI can quickly shift from being advisory to becoming the default, simply because it’s faster, clearer, or easier to follow.
This is where trust is either built or lost.
At scale, safe AI adoption depends on three things working together.
First, psychological safety. Clinicians and frontline staff must feel safe to question outputs and escalate concerns. Without that, errors and bias remain hidden. People will either defer to the system or work around it quietly, both of which introduce risk. Trust in AI relies on a culture where challenge is part of the system, not a failure of it.
Second, a true human-in-the-loop model. AI should augment judgment, not replace it. That means clear points where human expertise intervenes, validates, and ultimately owns the decision. Not as a formality, but as an active control, especially where decisions impact access, priority, or outcomes of care.
Third, governance that is designed for scale. This goes beyond policy into operational reality. Clear accountability frameworks, auditability of decisions, transparent model behaviour, and mechanisms for continuous feedback and improvement. Systems must be monitored not just for performance, but for unintended consequences. And importantly, there must be pathways for patients and clinicians to question and challenge outcomes.
Without these elements, even well-designed tools can drift, embedding risk silently into care delivery.
The choice in front of us
As AI becomes more embedded in healthcare, we have a clear choice.
We can treat it as a layer of efficiency, automating what exists today, including its disparities.
Or we can use this moment to fundamentally rethink how decisions are made. Ensuring they are transparent, accountable and inclusive by design is harder, but it is the only way true trust is built.
It leads to fairer access, decisions that can be understood and challenged, and systems that improve over time rather than entrenching past patterns.
That’s the real leadership test in this next phase of healthcare transformation. Not how quickly we deploy AI, but whether we build the cultural, human and structural conditions around it, to ensure it delivers better outcomes for everyone, not just the historical majority.
Author Bio
