Dr Alwin Tan, GAICD, MBBS, FRACS, EMBA (Melbourne Business School)

Senior Surgeon | Governance Leader | HealthTech Co-founder |
Harvard Medical School — AI in Healthcare |
Australian Institute of Company Directors — GAICD graduate |
University of Oxford — Sustainable Enterprise

Institute for Systems Integrity (ISI)

The most dangerous organisations in the AI era may not be technologically weak.

They may be humanly hollow.

For decades, organisational leaders have focused on improving efficiency, digitisation, automation, analytics and scale. The logic is understandable. Faster information flows, improved productivity and better decision-support systems have created extraordinary gains across healthcare, government, finance and industry.

Yet something else appears to be happening at the same time.

Organisations are becoming operationally smarter while becoming relationally poorer.

Communication has increased.

Connection has declined.

People are more reachable than ever before, yet many feel less recognised, less heard and less understood. Teams exchange thousands of messages while struggling to hold difficult conversations. Executives speak constantly about culture while becoming increasingly inaccessible to frontline staff. Organisations measure engagement, satisfaction and productivity while losing sight of whether people still trust one another enough to tell the truth.

Many leaders regard this as a cultural issue.

It is not.

It is a governance issue.

Because organisations do not function solely through technology, processes or strategy.

They function through human signals.

The Human Signal System

Every organisation operates as a signal-detection system.

Risk management depends upon signals.

Innovation depends upon signals.

Safety depends upon signals.

Adaptation depends upon signals.

The most important signals rarely emerge first in dashboards.

They emerge first in conversations.

A clinician quietly saying:

"Something doesn't feel right."

A frontline worker noticing a recurring customer concern.

A junior employee identifying an overlooked risk.

A patient expressing discomfort.

A manager observing growing fatigue within a team.

These are weak signals.

They are often ambiguous, emotional, informal and difficult to quantify.

Yet they frequently provide the earliest indication that something important is changing within the system.

The problem is that human signals depend upon human relationships.

People only communicate honestly when they believe it is safe to do so.

They only challenge assumptions when they trust that dissent will not be punished.

They only raise concerns when they feel recognised rather than processed.

This is where many modern organisations are becoming vulnerable.

As relational quality declines, signal quality declines with it.

When signal quality deteriorates, governance becomes blind.

The Hidden Cost of Digital Sophistication

Digital transformation has generated enormous value.

However, many organisations have quietly adopted a dangerous assumption:

That communication and connection are interchangeable.

They are not.

Communication transfers information.

Connection transfers meaning.

An organisation can dramatically improve communication volume while simultaneously reducing relational quality.

More emails do not create more trust.

More messaging platforms do not create more psychological safety.

More visibility does not create more authenticity.

More engagement metrics do not create more belonging.

The result is an increasingly common organisational condition:

People are communicating continuously while feeling progressively disconnected.

The consequences are often subtle at first.

Questions become less candid.

Concerns become softened.

Feedback becomes filtered.

Disagreement becomes performative.

People begin managing impressions rather than communicating reality.

Eventually, organisations stop hearing what they most need to hear.

Not because information is unavailable.

But because relationships no longer support its transmission.

Relational Atrophy

The greatest long-term risk associated with artificial intelligence may not be automation.

It may be relational atrophy.

Relational atrophy describes the gradual erosion of human capabilities that emerge through interpersonal interaction:

• empathy
• curiosity
• attentiveness
• listening
• psychological presence
• social courage
• constructive challenge
• emotional intelligence

Like any capability, these skills weaken when they are no longer exercised.

The concern is not that AI becomes more human.

The concern is that humans become less human.

As organisations increasingly automate communication, triage, customer engagement and administrative interaction, there is a risk that individuals become progressively detached from the human experiences that sustain trust and social understanding.

The system becomes faster.

But thinner.

More efficient.

But less connected.

More scalable.

But less trusted.

This may represent one of the most significant unintended consequences of digital transformation.

Why Healthcare Should Be Concerned

Healthcare provides an early warning of this phenomenon.

Healthcare systems are increasingly digitised, standardised and data-rich.

Yet patient trust remains fundamentally relational.

Patients rarely assess care based upon algorithmic sophistication.

They assess care through human experience.

Did someone listen?

Did someone notice?

Did someone care?

Did someone treat them as a person rather than a process?

Healthcare professionals increasingly report spending substantial portions of their day documenting care rather than experiencing care. Electronic medical records have improved access to information but have also contributed to administrative burden, burnout and reduced face-to-face engagement (National Academy of Medicine, 2020).

This creates an uncomfortable paradox.

Healthcare is becoming better at recording interactions while becoming worse at experiencing them.

The same dynamic is emerging across many industries.

Customer service.

Education.

Government.

Professional services.

Banking.

Human interaction is increasingly measured, scripted, automated and optimised.

Yet trust continues to decline.

Perhaps these trends are not unrelated.

The Governance Risk Nobody Measures

Boards routinely monitor:

• financial performance
• cybersecurity
• compliance
• operational risk
• technology risk
• workforce metrics

Far fewer monitor relational capacity.

Yet relational capacity determines whether organisations retain access to reality.

Questions boards should increasingly ask include:

• Do staff still feel safe enough to challenge leadership?
• Are concerns being filtered before reaching decision-makers?
• Is technology strengthening human attention or replacing it?
• Are patients, customers and stakeholders feeling recognised or processed?
• Are weak signals still reaching the board?
• Is psychological safety increasing or deteriorating?
• Are relationships supporting truth-telling?

These are not cultural questions.

They are governance questions.

Because every major organisational failure begins as a signal failure.

And many signal failures begin as relationship failures.

Human Connection as Strategic Infrastructure

For many years, organisations treated trust, relationships and human connection as soft assets.

That assumption is becoming increasingly difficult to defend.

Research on relational coordination demonstrates that shared goals, shared knowledge and mutual respect improve coordination and organisational performance (Gittell and Suchman, 2013; Bolton, Logan and Gittell, 2021).

Research on psychological safety demonstrates that teams perform better when individuals feel safe to speak honestly, challenge assumptions and report concerns (Edmondson, 1999; Grailey et al., 2021).

These findings suggest that relational quality is not merely a cultural preference.

It is a functional capability.

It improves learning.

It improves adaptation.

It improves safety.

It improves decision-making.

Most importantly, it improves signal integrity.

The Humanly Hollow Organisation

The most dangerous organisation is not necessarily the one with outdated technology.

It is the one where people no longer tell the truth.

The one where staff stop speaking up.

The one where concerns are softened before reaching leadership.

The one where patients feel unseen.

The one where communication is constant but trust is absent.

The one where AI scales interaction while human connection quietly disappears.

Such organisations often appear successful.

Their dashboards remain green.

Their reports remain polished.

Their systems remain efficient.

But beneath the surface, the human intelligence of the organisation is deteriorating.

And once human intelligence deteriorates, institutional blindness follows.

Conclusion

The defining challenge of the next decade may not be whether organisations successfully adopt artificial intelligence.

Most will.

The greater challenge may be whether organisations preserve the human capabilities that artificial intelligence cannot replace.

Attention.

Empathy.

Trust.

Listening.

Recognition.

Psychological safety.

Human presence.

Technology can scale information.

Only human relationships can scale trust.

The future may not belong to the organisations with the most sophisticated technology.

It may belong to the organisations that continue to protect and cultivate the human signals upon which every safe, adaptive and trustworthy institution ultimately depends.

References

Bolton, R., Logan, C. and Gittell, J.H. (2021) ‘Revisiting relational coordination: A systematic review’, The Journal of Applied Behavioral Science, 57(3), pp. 290–322.

Edmondson, A.C. (1999) ‘Psychological safety and learning behavior in work teams’, Administrative Science Quarterly, 44(2), pp. 350–383.

Gittell, J.H. and Suchman, A.L. (2013) An overview of relational coordination research. Accreditation Council for Graduate Medical Education.

Grailey, K.E., Murray, E., Reader, T. and Brett, S.J. (2021) ‘The presence and potential impact of psychological safety in the healthcare setting: An evidence synthesis’, BMC Health Services Research, 21, 773.

National Academy of Medicine (2020) Care-centered clinical documentation in the digital environment: Solutions to alleviate burnout. Washington DC: National Academy of Medicine.

Wilkinson, H., Whittington, R., Perry, L. and Eames, C. (2017) ‘Examining the relationship between burnout and empathy in healthcare professionals: A systematic review’, Burnout Research, 6, pp. 18–29.