Stewarding Healthcare in the Age of Artificial Intelligence From Trustworthy Systems to Shared Accountability

CO AUTHORS

Dr Alwin Tan, GAICD, MBBS, FRACS, EMBA (Melbourne Business School)

Senior Surgeon | Governance Leader | HealthTech Co-founder | Founder of Institute for Systems Integrity (ISI) |Harvard Medical School — AI in Healthcare| University of Oxford — Sustainable Enterprise | Bastas Academy for Healthcare Leadership - Triple Scholar

Vidoula Uckiah LL.M, GAICD, Accredited Mediator, AFCHSM, CHM

Healthcare Strategy & Governance Advisor | Multinational, Government and Institutional Reform Specialist | Workforce & Workplace Transformation and Mediator | Founder of Vision Consulting & Mediation | Graduated over 20 years ago as a Lawyer from the University of Amsterdam and admitted to the Supreme Court of the Northern Territory.

The Institute for Systems Integrity (ISI)

A Discussion Paper

Executive Summary

Healthcare stands at a defining moment. Advances in artificial intelligence (AI) are transforming the way health services diagnose disease, support clinical decisions, manage operations and plan for future demand. Around the world, governments, regulators, health services and technology companies are investing heavily in AI with the expectation that it will improve efficiency, strengthen decision-making and contribute to more sustainable models of care.

Much of the current discourse has focused on the governance of AI itself. Important questions are being asked about algorithmic bias, explainability, transparency, privacy, cybersecurity and regulation. While these issues are critical, they share a common assumption—that the primary challenge lies in governing the technology.

This discussion paper proposes a different perspective.

Artificial intelligence does not exist independently of healthcare. It learns from healthcare, is implemented within healthcare and ultimately influences decisions made by the people who deliver healthcare. Consequently, the trustworthiness of AI cannot be separated from the trustworthiness of the systems from which it learns. AI does not create organisational capability; it inherits it. It does not overcome poor governance, fragmented communication or weak organisational culture; it reflects and, in some circumstances, amplifies them.

Healthcare has always been, and will remain, a fundamentally human enterprise. Every patient encounter is shaped by professional judgement, relationships, ethical reasoning, communication and trust. These characteristics cannot be automated, nor should they be. As AI becomes increasingly embedded within healthcare, the value of these human capabilities becomes even more significant. Technology should strengthen professional judgement, not replace it; enhance relationships, not diminish them; and support better decisions, not transfer responsibility away from those who shape the system.

This paper argues that the future of healthcare depends less on building increasingly intelligent technologies than on stewarding healthcare systems capable of using those technologies wisely. It introduces The Integrity Chain™, a conceptual model demonstrating that trustworthy AI begins long before information reaches an algorithm. Patient reality is progressively transformed through observation, professional interpretation, communication, documentation and organisational learning before becoming the data upon which AI depends. Every stage in this journey either preserves or diminishes the integrity of what AI ultimately learns.

Building on this foundation, the paper proposes that the governance of AI must evolve from a model centred on individual professional accountability towards one of shared accountability. Safe AI-enabled healthcare cannot be achieved by clinicians alone. Governments, regulators, boards, executives, technology developers, researchers, professional bodies and clinicians each make decisions that shape how AI performs within healthcare. Responsibility should therefore be proportionate to influence, recognising that trustworthy AI is a collective achievement rather than an individual obligation.

Finally, the paper introduces the concept of intelligent stewardship as a leadership approach for the AI era. Intelligent stewardship extends beyond governing technology to strengthening the organisational conditions within which technology is used. It recognises that sustainable improvements in healthcare will arise not from AI alone, but from capable leaders, strong governance, healthy organisational cultures and a shared commitment to preserving the integrity of patient care.

This discussion paper is offered as a contribution to an emerging conversation. Rather than presenting AI as the solution to healthcare's challenges, it invites healthcare leaders, policymakers, regulators, clinicians, researchers and technology partners to consider a broader question:

How should healthcare systems be stewarded so that artificial intelligence strengthens, rather than diminishes, the human values upon which safe, equitable and compassionate care has always depended?

Background

Healthcare has always evolved in response to scientific discovery, technological advancement and changing societal needs. From the introduction of antisepsis and medical imaging to genomics and digital health, innovation has continually reshaped how care is delivered and improved the health outcomes of communities. Yet despite these advances, the fundamental purpose of healthcare has remained unchanged: to improve health, relieve suffering and care for people when they are at their most vulnerable.

Today's health systems face challenges of unprecedented scale and complexity. Ageing populations, increasing prevalence of chronic disease, rising consumer expectations, workforce shortages and escalating costs are placing considerable pressure on healthcare organisations worldwide. At the same time, the pace of scientific discovery continues to accelerate, generating ever-growing volumes of clinical knowledge that healthcare professionals are expected to interpret and apply. These pressures have intensified the search for new approaches capable of improving quality, safety, efficiency and sustainability without compromising person-centred care.

Artificial intelligence has emerged as one of the most significant developments in this search. Advances in machine learning, natural language processing, computer vision and predictive analytics have created opportunities to support clinical diagnosis, personalise treatment, improve operational efficiency, optimise resource allocation and strengthen population health management. Governments are investing in national AI strategies, health services are embedding AI into clinical workflows, and technology companies are rapidly expanding the development of healthcare applications. Increasingly, AI is being viewed not simply as another digital tool, but as a transformational capability with the potential to reshape the future of healthcare.

Much of the contemporary discussion has therefore focused on the governance of AI technologies. Considerable attention has been directed towards algorithmic bias, transparency, explainability, privacy, cybersecurity, regulatory oversight and the validation of AI systems before their introduction into clinical practice. These discussions have made an important contribution to ensuring that AI is developed responsibly and safely, and they remain essential as the technology continues to evolve.

However, the emphasis on governing AI can inadvertently obscure a more fundamental question.

Healthcare is not simply a collection of technologies. It is a complex adaptive system comprising people, organisations, processes, relationships and cultures that continually interact to deliver care. Clinical decisions emerge not only from evidence, but also from professional judgement, communication, experience, ethics, organisational priorities and the unique circumstances of individual patients. Technology participates within this system; it does not exist outside it.

Artificial intelligence is no exception.

AI does not encounter patients directly. It encounters representations of patients that have been created through the work of clinicians and healthcare organisations. Before information becomes data, it must first be observed, interpreted, communicated and documented. Before algorithms generate recommendations, organisations must establish governance arrangements, design workflows, train clinicians, procure technologies and determine how AI will be integrated into practice. Every one of these activities influences the quality, reliability and trustworthiness of the outputs that AI ultimately produces.

This distinction is significant because it shifts the focus of governance upstream. Rather than viewing trustworthy AI primarily as a technical achievement, it invites us to consider trustworthy AI as the product of trustworthy healthcare systems. If observations are incomplete, communication is fragmented, documentation is inconsistent or organisational cultures discourage learning, AI will inherit those weaknesses because they have become embedded within the information upon which it depends. Technology may process information at unprecedented speed, but it cannot restore integrity that has already been lost.

This perspective also highlights an emerging imbalance in contemporary approaches to accountability. As AI becomes more influential in supporting clinical and organisational decisions, responsibility continues to rest predominantly with clinicians who use these technologies at the point of care. Yet clinicians rarely determine how AI systems are designed, trained, procured, implemented or governed. Those decisions are shared across governments, regulators, health service boards, executives, technology developers, researchers and numerous other stakeholders. The growing influence of AI therefore challenges traditional assumptions about where accountability should reside and whether existing governance arrangements remain fit for purpose.

This discussion paper argues that healthcare has reached a pivotal moment. The question is no longer whether AI will become part of healthcare—that future is already unfolding. The more important question is whether health systems are evolving their governance with the same pace and sophistication as the technologies they are adopting.

Rather than asking "How should we govern artificial intelligence?", we propose a broader and more enduring question:

How should healthcare systems be stewarded so that artificial intelligence strengthens, rather than weakens, safe, equitable and person-centred care?

To explore this question, the paper introduces The Integrity Chain™, a conceptual model that explains how patient reality is transformed into organisational knowledge and ultimately into AI-enabled insight. Building on this foundation, the paper argues for a model of shared accountability, recognising that responsibility for AI-enabled healthcare should reflect the influence that different participants have over its design, implementation and use. Finally, it proposes intelligent stewardship as a governance philosophy that places people, organisational capability and system integrity at the centre of AI-enabled healthcare.

This paper does not seek to provide definitive answers or a comprehensive governance framework. Instead, it aims to stimulate discussion among healthcare leaders, policymakers, clinicians, regulators, researchers and technology partners about how AI should be integrated into healthcare systems in ways that preserve the values upon which healthcare has always depended. We hope this discussion serves as a catalyst for further collaboration, research and innovation, recognising that the future of AI in healthcare will be shaped not by technology alone, but by the collective choices we make as stewards of the systems within which it operates.

1. Healthcare Is Still About People

Artificial intelligence is rapidly becoming one of the defining technologies of modern healthcare. Its ability to process vast quantities of information, recognise complex patterns and support increasingly sophisticated decision-making has generated considerable optimism about its potential to improve health outcomes. In many respects, that optimism is justified. AI offers genuine opportunities to enhance diagnosis, reduce unwarranted variation, improve operational efficiency and support more personalised models of care.

Yet amidst this enthusiasm, it is worth asking a more fundamental question.

What problem are we actually trying to solve?

If the answer is simply to introduce better technology, then AI may indeed represent a significant part of the solution. However, if the challenge is to improve healthcare itself, then technology is only one part of a much larger system.

Healthcare has never been defined by technology. It has always been defined by people.

Every day, millions of healthcare professionals make decisions that cannot be reduced to algorithms alone. They interpret uncertainty, balance competing risks, communicate difficult news, respond to changing circumstances and make ethical judgements in situations where there is rarely a single correct answer. They care for patients whose lives are influenced not only by disease, but also by family, culture, socioeconomic circumstances, personal values and individual preferences. These realities remain largely unchanged regardless of how advanced technology becomes.

Patients, too, rarely judge the quality of their care by the sophistication of the technology used to deliver it. They judge it by whether they were listened to, whether they felt safe, whether decisions were explained, whether clinicians worked together and whether they were treated with dignity and compassion. Trust is built through relationships, not algorithms.

This is not to diminish the importance of AI. On the contrary, its increasing capability makes these human characteristics even more important. As technology assumes greater responsibility for analysing information and generating recommendations, healthcare professionals become increasingly responsible for interpreting those recommendations within the broader context of individual patient care. The more intelligent AI becomes, the greater the need for professional judgement, ethical reasoning and compassionate leadership.

Healthcare therefore remains fundamentally a human system.

Like all complex adaptive systems, its performance emerges from the interactions between people, organisations, processes and cultures rather than from any single component. Clinical excellence depends not only on scientific knowledge but also on effective communication, psychological safety, multidisciplinary collaboration, organisational learning and leadership that creates environments where people can perform at their best. These characteristics cannot simply be programmed into technology because they arise from the behaviours and relationships that exist within organisations.

This distinction is particularly important because many of healthcare's most persistent challenges are organisational rather than technological. Workforce shortages, fragmented models of care, poor communication between services, inconsistent documentation, clinician burnout, inequitable access, competing organisational priorities and cultures that discourage speaking up have existed long before the emergence of artificial intelligence. While AI may help organisations respond to some of these challenges, it cannot resolve them independently. Technology cannot compensate for weak governance, ineffective leadership or organisational cultures that fail to support safe, high-quality care.

Indeed, there is a risk that healthcare begins to view AI as a solution to problems that are fundamentally human in nature. History suggests otherwise. Digital technologies have repeatedly demonstrated that successful implementation depends less on the technology itself than on the organisational capability to adopt, govern and continuously improve it. Organisations characterised by trust, collaboration and learning consistently derive greater value from innovation than organisations seeking technological solutions to systemic dysfunction.

Artificial intelligence is unlikely to be any different.

Rather than replacing human capability, AI will increasingly reflect it. It will learn from the decisions healthcare professionals make, the information organisations preserve and the cultures within which care is delivered. If those systems promote curiosity, learning, transparency and collaboration, AI is likely to reinforce those strengths. If they are characterised by fragmented communication, poor documentation, inconsistent governance or limited organisational learning, AI is equally likely to inherit and amplify those weaknesses.

This perspective challenges one of the prevailing narratives surrounding AI in healthcare. The question is not whether artificial intelligence will become more capable. It almost certainly will. The more important question is whether healthcare organisations are becoming equally capable of using it wisely.

Ultimately, the future of healthcare will not be determined by artificial intelligence alone. It will be determined by the people who design, govern, implement and use it. Technology may change the way healthcare is delivered, but it does not change what healthcare fundamentally is: a profession built on trust, judgement, compassion and the shared commitment to improve the lives of others.

Recognising this changes the conversation. Instead of asking how healthcare should adapt to artificial intelligence, we should first ask how artificial intelligence can strengthen the human systems upon which healthcare has always depended.

Answering that question requires looking beyond algorithms and data to something much more fundamental: how patient reality is transformed into the knowledge upon which artificial intelligence ultimately depends. Understanding that journey is the purpose of The Integrity Chain™.

2. The Integrity Chain™: Why Trustworthy AI Begins with Trustworthy Systems

Much of the contemporary discussion surrounding artificial intelligence begins with data. Questions of data quality, completeness, interoperability and governance have become central to conversations about trustworthy AI, and rightly so. AI can only perform as well as the information upon which it is trained and from which it continues to learn.

However, this perspective raises an important question.

Where does that data come from?

The answer is both simple and profound.

Data does not originate within information systems. It originates with people.

Before information becomes data, it is first experienced by patients, observed by clinicians, interpreted through professional knowledge, communicated between individuals and documented within healthcare organisations. Long before an algorithm analyses information, a series of human decisions determines what is noticed, what is considered important, what is communicated, what is recorded and, equally importantly, what is omitted. By the time information reaches an AI system, it has already travelled through a complex organisational journey that has either preserved or diminished its integrity.

This observation fundamentally changes where the governance of AI should begin.

Rather than viewing trustworthy AI as a product of trustworthy data alone, we propose that trustworthy AI is the product of trustworthy healthcare systems. Data quality is therefore not simply a technical characteristic; it is an organisational outcome. It reflects the capability of healthcare systems to faithfully preserve patient reality as it moves through multiple stages of interpretation, communication and organisational learning.

To describe this process, we introduce The Integrity Chain™.

The Integrity Chain™

Patient Reality

↓

Clinical Observation

↓

Professional Interpretation

↓

Communication

↓

Documentation

↓

Data

↓

Information

↓

Organisational Knowledge

↓

Artificial Intelligence

↓

Professional Judgement

↓

Decision

↓

Patient Outcomes

↓

Learning

The Integrity Chain™ begins with the patient rather than the technology. This is intentional. Every healthcare interaction starts with a reality that exists independently of any information system—a patient's symptoms, concerns, behaviours, physiology, social circumstances, family context and lived experience. This reality is inherently richer and more complex than anything that can ultimately be captured within a clinical record.

The first transformation occurs through clinical observation. Healthcare professionals continually decide what to notice, what requires further investigation and what may safely be discounted. Observation is influenced by clinical expertise, workload, experience, environmental pressures and cognitive capacity. If deterioration is not recognised, if concerns are not voiced or if subtle changes remain unnoticed, that information cannot be recovered later by technology. AI cannot learn from observations that were never made.

Observation is followed by professional interpretation. Healthcare is rarely characterised by certainty. Clinicians integrate evidence with experience, patient history, professional standards and contextual understanding to determine what information is meaningful and what actions may be required. This process is dynamic and inherently human. Two experienced clinicians may legitimately interpret the same clinical presentation differently because healthcare is an exercise in judgement as much as knowledge.

Interpretation must then be translated into communication. Safe healthcare depends upon information moving effectively between clinicians, teams, organisations and patients themselves. Every handover, referral, multidisciplinary discussion and clinical conversation represents an opportunity either to preserve or to lose meaning. Misunderstandings, assumptions, omissions and ambiguity can subtly alter the information that ultimately becomes organisational knowledge. The quality of communication therefore becomes a critical determinant of the quality of AI.

Communication is then transformed into documentation. Clinical records are often viewed as repositories of information or evidence of care. In reality, they perform a far more significant organisational function. Documentation determines what becomes institutional memory. It is the mechanism through which healthcare organisations preserve knowledge beyond the individuals who generated it. Information that is poorly documented, inconsistently recorded or stripped of its clinical context may still exist within the organisation, but it no longer exists in a form that can reliably support learning, quality improvement or artificial intelligence.

Only after these human processes does information become data.

This distinction is important. Data is not patient reality. It is not even clinical reality. It is a structured representation of reality that has already passed through multiple layers of human judgement and organisational process. Consequently, many of the challenges described as "data quality issues" are, in fact, manifestations of broader organisational issues. Incomplete documentation, inconsistent coding, fragmented information systems and variable clinical practice are not primarily failures of data; they are reflections of how healthcare organisations function.

As data is aggregated and analysed, it becomes organisational knowledge. Health services use this knowledge to monitor performance, evaluate quality, allocate resources, develop policy and increasingly train or operate AI systems. Importantly, organisational knowledge captures not only clinical activity but also organisational behaviour. AI therefore learns as much about the health system as it does about disease. It learns what organisations consistently record, what they fail to record, how they classify events, where variation exists and, potentially, where inequities become embedded within routine practice.

Artificial intelligence then contributes new insights by identifying patterns, generating predictions and presenting recommendations that may assist decision-making. However, AI is deliberately positioned within The Integrity Chain™, not at its conclusion. This reflects a fundamental principle of the model: AI does not replace professional judgement; it informs it.

Healthcare decisions remain the responsibility of people.

Clinicians continue to integrate AI-generated insights with patient preferences, ethical considerations, clinical expertise and situational awareness. Leaders continue to balance organisational priorities, community expectations and finite resources. Boards continue to exercise oversight of quality, safety and strategic direction. AI becomes another contributor to decision-making, but not its ultimate authority.

Those decisions subsequently influence patient outcomes, which in turn generate new experiences from which organisations learn. Learning closes the loop within The Integrity Chain™. Every outcome—whether positive, negative or unexpected—provides an opportunity to strengthen organisational capability. Learning organisations refine documentation practices, improve communication, invest in workforce capability and continuously evaluate how AI is influencing care. In doing so, they strengthen not only the performance of their AI systems but also the integrity of the healthcare system itself.

The Integrity Chain™ therefore reframes the governance challenge facing healthcare. Rather than asking whether an algorithm is trustworthy, it asks whether the organisational system that produced the knowledge informing that algorithm is itself trustworthy. This is a subtle but important distinction. It moves the conversation upstream—from governing algorithms to governing the organisational conditions that shape them.

The significance of this shift extends beyond technology. If AI inherits the integrity of the systems from which it learns, then trustworthy AI can never be achieved through technical excellence alone. It requires organisations that value accurate observation, thoughtful interpretation, effective communication, meaningful documentation, continuous learning and strong governance. In other words, trustworthy AI begins with trustworthy systems.

This perspective also has profound implications for accountability. If every stage of The Integrity Chain™ is shaped by different people, professions and organisations, then responsibility for AI-enabled healthcare cannot reasonably rest with any single individual. The integrity of the chain is a shared endeavour, requiring contributions from clinicians, leaders, boards, policymakers, regulators, researchers and technology developers alike.

It is this recognition that leads to the next proposition of this paper: 

if the creation of trustworthy AI is a collective endeavour, then accountability for AI-enabled healthcare must also become a collective responsibility.

3. Shared Accountability: Reframing Governance for the AI Era

The introduction of artificial intelligence into healthcare does more than change how decisions are informed. It changes how responsibility should be understood.

For generations, healthcare governance has been built on a relatively clear principle: those who make decisions are accountable for them. Professional accountability remains one of the foundations of safe healthcare and should continue to be so. Clinicians are, and should remain, accountable for exercising professional judgement, acting ethically and placing the interests of patients at the centre of every decision.

Artificial intelligence, however, is changing the environment in which those decisions are made.

Increasingly, clinical and organisational decisions are influenced by technologies that clinicians neither designed nor selected. They rely on data they did not collect, algorithms they did not develop, validation processes they did not oversee and governance arrangements they often had little opportunity to shape. Their interaction with AI represents only the final stage of a much larger ecosystem of decisions made by many different individuals and organisations.

This raises an important question.

Can accountability continue to rest predominantly with those who have the least influence over the systems that increasingly shape their decisions?

We suggest that it cannot.

This is not an argument for reducing professional accountability. Nor is it an attempt to transfer responsibility away from clinicians. Rather, it is an acknowledgement that AI-enabled healthcare represents a fundamentally different governance environment—one in which responsibility is distributed across a complex network of participants whose decisions collectively influence patient outcomes.

The Integrity Chain™ demonstrates that AI inherits the quality of the healthcare system from which it learns. Equally, the governance of AI reflects the quality of the relationships between those who design, regulate, implement, govern and use it. No single organisation creates trustworthy AI, and no single organisation can govern it alone.

Healthcare has entered an era of shared accountability.

Shared accountability recognises that responsibility should be proportionate to influence. Those who influence how AI is designed, implemented, governed and used should also accept responsibility for the consequences of those decisions. This principle extends well beyond the point of care.

Governments establish the legislative and policy environment within which AI develops. Regulators determine the standards required to demonstrate safety, quality and effectiveness. Professional colleges influence education, competency and ethical practice. Universities and researchers generate the evidence that informs implementation. Health service boards determine organisational risk appetite and governance expectations. Executive leaders decide how AI is procured, implemented, evaluated and integrated into clinical practice. Technology companies design the products, define the assumptions embedded within algorithms, determine how systems evolve and increasingly influence how clinicians interact with information. Data custodians shape the quality, accessibility and integrity of the information upon which AI depends.

Finally, clinicians apply professional judgement to the care of individual patients.

Every participant influences the outcome.

Every participant therefore carries responsibility.

This proposition represents an important shift in governance philosophy. Traditional accountability frameworks often resemble a hierarchy in which responsibility accumulates at the point of care. The clinician becomes the final safeguard and, frequently, the individual against whom decisions are ultimately judged. While this model has served healthcare for many decades, it becomes increasingly difficult to sustain when decisions are influenced by technologies whose design and operation extend far beyond the clinician's sphere of control.

Paradoxically, the individual with the greatest legal and professional accountability may be the individual with the least influence over the AI system itself.

Conversely, organisations that exercise significant influence over AI—including those responsible for its design, procurement, implementation and governance—may experience comparatively limited accountability when technology contributes to unintended consequences.

This imbalance creates risk for everyone.

It creates risk for clinicians who may become reluctant either to rely upon AI or to challenge it, uncertain where responsibility ultimately lies. It creates risk for organisations that may underestimate the importance of governance beyond technical implementation. It creates risk for technology developers whose products may be evaluated independently of the organisational contexts in which they operate. Most importantly, it creates risk for patients, whose safety depends upon the entire ecosystem functioning as intended rather than upon the actions of any single participant.

Shared accountability does not imply shared blame.

Healthcare has often responded to adverse events by seeking to identify where responsibility rests. While accountability remains essential, contemporary patient safety science has repeatedly demonstrated that adverse outcomes rarely arise from a single failure. They emerge through interactions between people, processes, technologies, organisational cultures and governance arrangements. AI-enabled healthcare is unlikely to be different.

When an AI-supported decision contributes to an adverse outcome, the most important question should not be, 

"Who is at fault?"

The more valuable question is, 

"What happened within the system that allowed this outcome to occur?"

This distinction is fundamental.

A culture focused primarily on blame encourages defensiveness, limits transparency and discourages learning. A culture founded on shared accountability encourages organisations to examine the entire system—from the quality of the underlying data and the assumptions embedded within algorithms to governance arrangements, implementation decisions, workforce capability and organisational culture. Such an approach is far more likely to produce meaningful learning and continuous improvement.

Shared accountability also strengthens public trust.

Communities increasingly understand that AI is influencing healthcare. They are less interested in the technical details of machine learning than they are in knowing that someone is responsible for ensuring these technologies are safe, ethical and used appropriately. Public confidence is strengthened when accountability is transparent, proportionate and shared across all those who influence patient outcomes, rather than being concentrated solely at the point of care.

Ultimately, the introduction of AI presents healthcare with an opportunity to rethink governance itself. Rather than viewing accountability as something that cascades down towards clinicians, we can begin to view it as something that extends across an interconnected ecosystem of responsibility. Every decision made by policymakers, regulators, boards, executives, technology developers, researchers, clinicians and patients contributes to the integrity of the healthcare system and, ultimately, to the integrity of AI.

This broader perspective moves the conversation beyond governing technology towards governing relationships, organisations and systems. It recognises that the success of AI will depend not only on technical capability but also on leadership, organisational maturity and the willingness of every participant to accept responsibility for the part of the system they shape.

That is the essence of stewardship. And it is to stewardship that we now turn.

4. Intelligent Stewardship: Leading Healthcare in the Age of Artificial Intelligence

Throughout this discussion paper, we have argued that trustworthy AI begins with trustworthy healthcare systems and that responsibility for AI-enabled healthcare should be shared across the ecosystem of organisations and professions that influence its design, implementation and use. These propositions naturally lead to a broader question. If artificial intelligence is becoming an increasingly influential participant in healthcare, what kind of leadership will ensure that it strengthens, rather than diminishes, the purpose of healthcare?

We suggest that the answer lies in intelligent stewardship.

Stewardship is not a new concept in healthcare. Every day, boards, executives, clinicians and policymakers make decisions that shape the quality, safety and sustainability of the services entrusted to their care. Good stewardship has always required balancing competing priorities, allocating finite resources, managing risk and maintaining the trust that communities place in their health systems. Artificial intelligence does not replace these responsibilities. It expands them. Leaders are now called upon to steward not only organisations and people, but also the increasingly complex relationships between technology, information and human decision-making.

This distinction is important because governing AI is not the same as stewarding healthcare. Governance provides the structures through which organisations establish accountability, manage risk and ensure compliance with regulatory and professional standards. Stewardship goes further. It asks whether organisations are creating the conditions in which people, technology and systems can work together to achieve better outcomes. It recognises that safe and effective healthcare depends not only on sound governance arrangements, but also on leadership, culture, trust, organisational learning and the willingness to adapt as new knowledge emerges.

Viewed through this lens, implementing AI is not primarily a technology project; it is an organisational endeavour. Success depends as much on the readiness of the organisation as it does on the capability of the technology. An AI system may be technically robust, but its value will always be influenced by the environment into which it is introduced. Organisations with strong clinical governance, collaborative cultures and a commitment to continuous learning are more likely to realise the benefits of AI than organisations hoping technology will compensate for fragmented systems or longstanding organisational challenges.

For healthcare leaders, this means asking different questions. Rather than focusing solely on whether an AI system is accurate, organisations should also consider whether they are ready to use it well. Do clinicians understand where AI can add value and where professional judgement must prevail? Is the information upon which AI depends reliable and representative of the care being delivered? Are governance arrangements sufficiently robust to identify unintended consequences and respond as technologies evolve? Most importantly, does the organisation have a culture in which people feel confident to question AI, challenge assumptions and learn from experience?

These are leadership questions rather than technical ones, and their answers will ultimately determine whether AI strengthens or weakens healthcare.

The same principle extends beyond individual organisations. Intelligent stewardship recognises that no single participant can create trustworthy AI in isolation. Health services, governments, regulators, professional bodies, researchers and technology companies each contribute knowledge, expertise and influence that others do not possess. The challenge is therefore not simply to define individual responsibilities, but to build partnerships characterised by transparency, mutual trust and a shared commitment to improving patient care. Trustworthy AI is sustained through ongoing collaboration, continuous evaluation and collective learning, not through procurement alone.

Perhaps the most important contribution of intelligent stewardship is that it repositions technology within the broader purpose of healthcare. Artificial intelligence should never become the objective of transformation. Better healthcare should. The value of AI should therefore be judged not by the sophistication of its algorithms or the speed of its adoption, but by whether it enables safer care, supports better decisions, reduces inequity, strengthens organisational effectiveness and allows healthcare professionals to spend more time doing what only people can do: caring for other people.

Ultimately, intelligent stewardship reminds us that healthcare is, and always will be, a profoundly human endeavour. Technology will continue to evolve, as it always has. Algorithms will become more capable, data more abundant and digital systems more sophisticated. Yet the enduring challenge for healthcare leaders will remain remarkably familiar—to build organisations that are trusted by their communities, support the people who work within them and continually learn from the care they provide. If we can achieve that, artificial intelligence will become more than another technological innovation. It will become a catalyst for strengthening the health systems upon which patients, communities and future generations depend.

5. Imagine…AI as a component of a Broader System and not just a Technological Solution

Imagine a healthcare system in which artificial intelligence is not viewed as a technological solution in its own right, but as one component of a broader system committed to delivering safe, equitable and person-centred care. A system in which technology strengthens, rather than replaces, the judgement, compassion and professionalism of the people who work within it.

Imagine organisations where the introduction of AI prompts leaders to invest not only in digital capability, but also in the foundations of high-quality healthcare: strong governance, effective communication, reliable information, a skilled workforce and a culture of continuous learning. Rather than asking how quickly new technologies can be deployed, organisations ask how thoughtfully they can be integrated into clinical practice and organisational decision-making.

Imagine clinicians working with confidence, supported by AI that is transparent, evidence-informed and implemented within systems they trust. Professional judgement remains central to decision-making, informed by technology but never replaced by it. Patients remain active participants in their own care, confident that decisions are shaped not only by data and algorithms, but by empathy, ethics and an understanding of their individual circumstances.

Imagine boards and executive leaders routinely considering the integrity of the systems from which AI learns. Conversations about technology extend beyond procurement and performance to include organisational readiness, workforce capability, data integrity, clinical governance and the long-term impact on patients and communities. Success is measured not only by efficiency or productivity, but by improvements in quality, safety, equity and public trust.

Imagine a healthcare ecosystem in which governments, regulators, researchers, technology developers, professional bodies and healthcare organisations recognise that they are collectively responsible for the success of AI-enabled healthcare. Rather than operating in isolation, they work in partnership to ensure that innovation is accompanied by appropriate governance, meaningful evaluation and continuous learning. Accountability is understood as a shared commitment to achieving better outcomes rather than simply attributing responsibility when outcomes fall short.

Perhaps the greatest opportunity presented by artificial intelligence is not that it enables machines to become more intelligent, but that it encourages healthcare systems to become more reflective, more connected and more intentional in the way they deliver care. In striving to build trustworthy AI, organisations are compelled to strengthen the very capabilities that have always underpinned high-performing health systems: leadership, governance, collaboration, learning and an unwavering commitment to the people they serve.

If this vision is realised, the legacy of artificial intelligence will extend far beyond technological advancement. It will have provided the catalyst for building healthcare systems that are more resilient, more transparent and more worthy of the trust placed in them by patients, clinicians and the communities they serve.

Conclusion

Artificial intelligence represents one of the most significant developments in modern healthcare. Its capacity to analyse vast quantities of information, recognise complex patterns and support increasingly sophisticated decision-making offers unprecedented opportunities to improve patient care, strengthen health system performance and address some of the longstanding challenges facing healthcare. Yet, as with every major transformation in healthcare, the technology itself is only part of the story.

Throughout this discussion paper, we have argued that the future of AI in healthcare will be determined less by the capability of algorithms than by the capability of the systems within which they operate. Artificial intelligence does not exist independently of healthcare; it is shaped by the quality of the observations, decisions, relationships and governance that precede it. Trustworthy AI is therefore not created through technology alone. It is the product of trustworthy healthcare systems.

The Integrity Chain™ provides a way of understanding this relationship. By tracing the journey from patient reality through observation, interpretation, communication, documentation, data and organisational knowledge to AI-supported decision-making, it illustrates that every stage influences the integrity of the next. Weakness at any point in the chain cannot be fully corrected downstream. Conversely, strengthening the integrity of healthcare systems strengthens the integrity of the intelligence they produce.

Recognising this reality also requires a broader understanding of accountability. As AI becomes increasingly embedded within clinical and organisational decision-making, responsibility can no longer be viewed solely through the actions of individual clinicians or organisations. Governments, regulators, boards, executive leaders, technology developers, researchers, professional bodies and healthcare professionals each influence how AI is designed, implemented and used. The governance of AI therefore becomes a shared endeavour, with accountability proportionate to the influence each participant exercises over the system.

This shared accountability is brought to life through intelligent stewardship. More than a governance framework, intelligent stewardship reflects a commitment to leading healthcare systems in ways that preserve trust, encourage learning and ensure that technological innovation remains firmly aligned with the purpose of healthcare. It recognises that while AI may transform how decisions are supported, people remain responsible for how decisions are made, interpreted and acted upon. Leadership, culture and organisational capability will continue to shape the quality of care long after today's technologies have evolved.

Ultimately, artificial intelligence presents healthcare with an opportunity that extends beyond digital transformation. It invites us to reflect on the kind of health systems we wish to build. If organisations strengthen governance, improve communication, invest in their workforce, enhance organisational learning and maintain an unwavering focus on patients and communities, they will not only create better conditions for AI—they will create better healthcare.

The future of healthcare will not be defined by artificial intelligence alone. It will be defined by the choices we make about how intelligence—human and artificial—is brought together in service of a common purpose. If we approach this future with thoughtful leadership, shared responsibility and a commitment to strengthening the systems upon which healthcare depends, AI has the potential to become more than a technological advancement. It can become a catalyst for more trustworthy organisations, more confident professionals and healthier communities.

Because in the end, the question is not whether artificial intelligence will change healthcare.

It is whether healthcare will strengthen itself enough to realise the full promise of Artificial Intelligence.(AI)

References

Adams, C., Allen, J. and Flack, F. (2018) ‘Data custodians and the decision-making process: Releasing data for research’, Journal of Law and Medicine, 26(2), pp. 433–453.

Albanese, A. (2026) ‘AI in Australia’s interests’, speech delivered at the University of Sydney, Sydney, 15 July.

Allen, J., Holman, C.D.A.J., Meslin, E.M. and Stanley, F. (2013) ‘Privacy protectionism and health information: Is there any redress for harms to health?’, Journal of Law and Medicine, 21, pp. 473–485.

Australian Alliance for Artificial Intelligence in Healthcare (AAAiH) (2023) A national policy roadmap for artificial intelligence in healthcare. Sydney: AAAiH.

Australian Commission on Safety and Quality in Health Care (2025) AI clinical use guide: Guidance for clinicians, version 1.0, August. Sydney: Australian Commission on Safety and Quality in Health Care.

Australian Government Department of Health, Disability and Ageing (2025) Safe and responsible artificial intelligence in health care—Legislation and regulation review: Final report. Canberra: Commonwealth of Australia.

Australian Health Practitioner Regulation Agency and National Boards (n.d.) Artificial intelligence in healthcare. Melbourne: Ahpra.

Dekker, S. (2016) Just culture: Restoring trust and accountability in your organization, 3rd edn. Boca Raton, FL: CRC Press.

Finlayson, S.G., Subbaswamy, A., Singh, K., Bowers, J., Kupke, A., Zittrain, J., Kohane, I.S. and Saria, S. (2021) ‘The clinician and dataset shift in artificial intelligence’, New England Journal of Medicine, 385(3), pp. 283–286.

Goddard, K., Roudsari, A. and Wyatt, J.C. (2012) ‘Automation bias: A systematic review of frequency, effect mediators, and mitigators’, Journal of the American Medical Informatics Association, 19(1), pp. 121–127.

Johnston, C. (2023) Digital health technologies: Law, ethics, and the doctor–patient relationship. Abingdon: Routledge.

Mendelson, D. (2004) ‘HealthConnect and the duty of care: A dilemma for medical practitioners’, Journal of Law and Medicine, 12, pp. 69–79.

National Institute of Standards and Technology (2023) Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1. Gaithersburg, MD: US Department of Commerce. 

Office of the Australian Information Commissioner (2024) Guidance on privacy and the use of commercially available AI products. Canberra: OAIC.

Parasuraman, R. and Riley, V. (1997) ‘Humans and automation: Use, misuse, disuse, abuse’, Human Factors, 39(2), pp. 230–253.

Raji, I.D., Smart, A., White, R.N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D. and Barnes, P. (2020) ‘Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing’, in Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. New York: Association for Computing Machinery, pp. 33–44.

Richards, B. (2020) ‘Health care, technology and innovation: What’s law got to do with it?’, Journal of Law and Medicine, 27, pp. 561–569.

Richards, B. and Scheibner, J. (2022) ‘Health technology and big data: Social licence, trust and the law’, Journal of Law and Medicine, 29, pp. 388–401.

Roski, J., Bo-Linn, G.W. and Andrews, T.A. (2014) ‘Creating value in health care through big data: Opportunities and policy implications’, Health Affairs, 33(7), pp. 1115–1122. 

Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P. and Aroyo, L.M. (2021) ‘“Everyone wants to do the model work, not the data work”: Data cascades in high-stakes AI’, in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. New York: Association for Computing Machinery, article 39, pp. 1–15. 

Sujan, M.A., Furniss, D., Grundy, K., Grundy, H., Nelson, D., Elliott, M., White, S., Habli, I. and Reynolds, N. (2019) ‘Human factors challenges for the safe use of artificial intelligence in patient care’, BMJ Health & Care Informatics, 26(1), e100081. 

Swannell, C. (2026) ‘Office of AI welcomed, but healthcare needs its own focus, says Enrico Coiera’, Health Services Daily, July.

Therapeutic Goods Administration (2026) Artificial intelligence and medical device software regulation. Canberra: Australian Government Department of Health, Disability and Ageing.

Topol, E. (2019) Deep medicine: How artificial intelligence can make healthcare human again. New York: Basic Books.

Varhol, R.J., Randall, S., Boyd, J.H. and Robinson, S. (2022) ‘Australian general practitioner perceptions to sharing clinical data for secondary use: A mixed method approach’, BMC Primary Care, 23, article 167.

World Health Organization (2021) Ethics and governance of artificial intelligence for health: WHO guidance. Geneva: World Health Organization.