Dr Alwin Tan, GAICD, MBBS, FRACS, EMBA (Melbourne Business School)

Senior Surgeon | Governance Leader | HealthTech Co-founder | Founder of Institute for Systems Integrity (ISI) |Harvard Medical School — AI in Healthcare| University of Oxford — Sustainable Enterprise | Bastas Academy for Healthcare Leadership - Triple Scholar

Organisations are celebrating how quickly AI can make junior employees perform like experienced professionals.

Reports are produced faster.

Analysis appears sharper.

Emails sound more executive.

Presentations look more sophisticated.

Productivity rises.

Everyone appears more capable.

But appearing capable is not the same as becoming capable.

And that difference may become one of the most dangerous workforce risks of the AI era.

For generations, professionals developed judgement by doing the work now being automated.

They reviewed the files.

Drafted the first version.

Checked the calculations.

Sat through difficult meetings.

Misread situations.

Received correction.

Observed consequences.

And slowly learned to recognise what the textbook could not teach them.

The routine work was not merely production.

It was apprenticeship.

Now organisations are removing it in the name of efficiency—without asking what developmental function it served.

THE WORK LOOKED ROUTINE.

THE LEARNING WAS NOT.

Junior work has often been dismissed as administrative, repetitive or low value.

But repetition develops more than speed.

It develops pattern recognition.

It teaches people which details matter, which signals are unusual and which apparently convincing answers do not survive contact with reality.

A junior lawyer learns by examining imperfect arguments.

A clinician learns by seeing variation across patients.

An engineer learns by investigating why apparently minor deviations matter.

A future executive learns by watching how decisions are negotiated, resisted, reframed and sometimes quietly undermined.

Professional judgement does not arrive through promotion.

It develops through situated participation, feedback and progressively greater responsibility (Lave and Wenger, 1991; Schön, 1983).

Remove the participation and the title may still arrive.
The judgement may not.

AI CAN COMPRESS THE OUTPUT.

IT CANNOT AUTOMATICALLY REPLACE THE FORMATION.**

Generative AI can improve productivity.

That is real.

Research has found that AI assistance can help people complete some professional tasks faster and to a higher assessed standard, with particularly strong immediate benefits for less-experienced workers

But output studies are not necessarily learning studies.

A polished document tells us what was produced.

It does not tell us:

• whether the employee understood the problem;
• whether they recognised the uncertainty;
• whether they detected what the AI omitted;
• whether they could defend the recommendation;
• whether they could adapt it to an unfamiliar situation; or
• whether they could reproduce the reasoning without assistance.

AI can narrow the visible performance gap while widening the invisible capability gap.

That is not merely a training issue.

It is a governance risk.

WE MAY BE CREATING SYNTHETIC COMPETENCE.

Synthetic competence occurs when the apparent quality of someone’s work exceeds their independently demonstrable understanding, judgement or ability.

The output looks senior.

The underlying capability remains junior.

The organisation sees the document, not the dependency behind it.

The manager sees fluency, not whether the employee can distinguish a strong argument from a confident fabrication.

The board sees improved productivity, not whether the future leadership pipeline is becoming cognitively dependent on systems it cannot adequately challenge.

This is especially dangerous because AI-supported work may not look deficient.

It may look excellent.

The risk is not that weak capability becomes obvious.
The risk is that weak capability becomes professionally formatted.

THE PEOPLE MOST IMPRESSED BY AI MAY BE THE LEAST ABLE TO CHALLENGE IT.

Experts can compare an AI-generated answer with years of accumulated knowledge.

They recognise when something feels wrong.

They know when an exception matters.

They understand which apparently reasonable recommendation would fail in practice.

Novices often do not possess that internal reference model.

They may therefore accept an answer because it is:

• articulate;
• comprehensive;
• confident;
• aligned with the requested format; and
• delivered without visible hesitation.

But confidence is not evidence.

Fluency is not judgement.

And plausibility is not safety.

Expert intuition itself develops reliably only under particular conditions, including repeated exposure to sufficiently regular patterns and meaningful feedback

If AI performs the task, selects the evidence, structures the argument and proposes the conclusion, where does the novice receive the repetitions and feedback needed to build that intuition?

An employee cannot learn to detect what they are never required to examine.

MENTORING MATTERS—BUT IT CANNOT REPAIR A WORK SYSTEM DESIGNED AGAINST LEARNING.

Mentoring is rightly receiving renewed attention.

Experienced professionals can make invisible thinking visible.

They can explain:

• what they noticed;
• which evidence they distrusted;
• how they interpreted silence;
• why they rejected the obvious option;
• what second-order consequences they anticipated; and
• what would cause them to change their view.

That is enormously valuable.

Mentoring can transfer tacit knowledge, professional identity, contextual understanding and ethical judgement that cannot be reduced easily to a procedure or prompt.

But mentoring cannot carry the entire burden of professional formation.

A monthly conversation cannot replace daily participation.

A leadership programme cannot compensate for years of removed developmental experience.

A mentor cannot teach someone to recognise every weak signal if the work system no longer allows that person to encounter weak signals.

You cannot automate the apprenticeship and then ask mentors to reconstruct it from memory.

Mentoring must sit inside a wider developmental architecture that includes:

• observation;
• supervised practice;
• protected experimentation;
• exposure to ambiguity;
• meaningful feedback;
• independent reasoning;
• progressively greater responsibility; and
• accountability for real consequences.

Without these conditions, mentoring risks becoming another well-intentioned conversation disconnected from how capability is actually formed.

NOT ALL TACIT KNOWLEDGE DESERVES TO SURVIVE.

There is another danger.

Calls to preserve experience can become calls to preserve the past uncritically.

Senior people do not carry only wisdom.

They may also carry:

• outdated assumptions;
• professional tribalism;
• normalised workarounds;
• institutional prejudice;
• defensive habits;
• hierarchy; and
• practices that succeeded in yesterday’s environment but are unsafe in today’s.

Experience is not automatically expertise.

Seniority is not automatically judgement.

And mentoring is not automatically safe.

Tacit knowledge can carry institutional wisdom.
It can also carry institutional contamination.

The purpose of mentoring cannot be to reproduce the mentor.

It must be to expose reasoning to examination.

Good mentors do not merely say:

“This is how I have always done it.”

They also ask:

“Where could my judgement be wrong?”

Professional formation requires challenge, not imitation.

AI COULD STRENGTHEN APPRENTICESHIP—IF WE DESIGN IT TO.

This is not an argument against AI.

AI can expand access to knowledge.

It can generate simulations, provide immediate formative feedback, reveal alternative interpretations and allow employees to practise difficult situations safely.

It can reduce administrative burden and release experienced professionals to spend more time teaching judgement rather than correcting formatting.

It can help make expert reasoning searchable and reusable.

But that requires deliberate design.

AI should sometimes answer.

At other times, it should be required to question.

It should help employees compare alternatives rather than bypass analysis.

It should reveal uncertainty rather than conceal it behind polished language.

And it should never become the only place where junior professionals encounter a problem before submitting an answer.

AI should scaffold judgement—not silently substitute for its development.

THE REAL PROBLEM IS WORKFORCE REGENERATION.

Every organisation has an incentive to automate routine work.

Every organisation wants greater productivity.

Every organisation would prefer to recruit people who are already experienced.

But where will those experienced people come from when nobody wants to fund the years in which experience is formed?

This creates a collective workforce problem.

One organisation automates junior work and gains efficiency.

Many organisations do the same and a profession begins to lose its entry pathway.

The short-term business case may look excellent.

The long-term system becomes unable to regenerate its own experts.

The organisation saves the cost of apprenticeship.
The profession inherits the cost of capability failure.

This should concern boards.

Not in 10 years.

Now.

BOARDS MUST STOP TREATING CAPABILITY AS AN HR ASSUMPTION.

Boards routinely ask:

• What productivity will AI create?
• How much cost will it remove?
• How quickly can it scale?
• What is the adoption rate?
• What are our competitors doing?

They must also ask:

• Which developmental tasks are being automated?
• What capabilities were previously formed through those tasks?
• Where will employees now acquire those capabilities?
• How do we distinguish AI-enabled output from independent competence?
• Which roles must retain unaided capability?
• How are weak signals and contextual exposure being preserved?
• What happens when AI is wrong, unavailable or outside its domain?
• Who is accountable for monitoring long-term deskilling?
• Can we still explain how a novice becomes an expert here?

If those questions cannot be answered, the organisation does not have an AI workforce strategy.

It has an automation strategy with an unmeasured succession risk.

EFFICIENCY IS NOT THE SAME AS INSTITUTIONAL FITNESS.

A system can become faster while becoming more fragile.

It can produce more while learning less.

It can standardise output while weakening independent thought.

It can give junior employees access to senior-looking performance without giving them the experience needed to become senior professionals.

The danger is not simply that AI will make mistakes.

The deeper danger is that, over time, fewer humans may retain the capability to recognise them.

When an organisation loses the ability to produce independent judgement, it has not merely lost a skill.
It has lost a layer of defence.

The governance challenge of AI is therefore not only to oversee the technology.

It is to preserve the human capability required to challenge, contextualise and overrule it.

That means protecting apprenticeship.

Redesigning mentoring.

Testing independent competence.

Preserving exposure to reality.

And treating professional formation as critical organisational infrastructure.

Because the most consequential AI failure may not be the answer the machine gets wrong today.

It may be the generation of professionals we failed to teach how to know the difference tomorrow.

Institute for Systems Integrity position

The Institute for Systems Integrity proposes that organisations recognise Professional Formation Integrity as a governance responsibility:

The demonstrable capacity of an organisation to develop, test, preserve and renew the human judgement required for consequential work in an AI-mediated environment.

This requires oversight of four connected risks:

1. Apprenticeship Displacement Risk

The loss of developmental experiences when entry-level and routine work is automated.

2. Synthetic Competence Risk

The appearance of capability created by AI-supported output that exceeds the user’s independent understanding.

3. Developmental Signal Loss

The disappearance of anomalies, contextual cues and operational exposure through which professional pattern recognition develops.

4. Epistemic Dependency

Reliance on external cognitive systems beyond the organisation’s ability to verify, challenge or function safely without them.

These are not reasons to slow responsible AI adoption.

They are reasons to govern it properly.

The future belongs neither to humans working without AI nor to organisations replacing human thought indiscriminately.

It belongs to organisations capable of using AI while continuing to produce people who can think, question, decide and accept responsibility.

That capacity will not preserve itself.

It must be designed.

It must be measured.

And it must be governed.

References

Allen, T.D., Eby, L.T., Poteet, M.L., Lentz, E. and Lima, L. (2004) ‘Career benefits associated with mentoring for protégés: A meta-analysis’, Journal of Applied Psychology, 89(1), pp. 127–136.

Brynjolfsson, E., Li, D. and Raymond, L.R. (2023) ‘Generative AI at work’, NBER Working Paper No. 31161. Cambridge, MA: National Bureau of Economic Research.

Eby, L.T., Allen, T.D., Evans, S.C., Ng, T. and DuBois, D.L. (2008) ‘Does mentoring matter? A multidisciplinary meta-analysis comparing mentored and non-mentored individuals’, Journal of Vocational Behavior, 72(2), pp. 254–267.

Edmondson, A.C. (1999) ‘Psychological safety and learning behavior in work teams’, Administrative Science Quarterly, 44(2), pp. 350–383.

Eraut, M. (2004) ‘Informal learning in the workplace’, Studies in Continuing Education, 26(2), pp. 247–273.

Ericsson, K.A., Krampe, R.T. and Tesch-Römer, C. (1993) ‘The role of deliberate practice in the acquisition of expert performance’, Psychological Review, 100(3), pp. 363–406.

Kahneman, D. and Klein, G. (2009) ‘Conditions for intuitive expertise: A failure to disagree’, American Psychologist, 64(6), pp. 515–526.

Kram, K.E. (1985) Mentoring at Work: Developmental Relationships in Organizational Life. Glenview, IL: Scott Foresman.

Lave, J. and Wenger, E. (1991) Situated Learning: Legitimate Peripheral Participation. Cambridge: Cambridge University Press.

Lyons, M. (2026) ‘Why mentoring matters more in the AI era’, Harvard Business Review, 24 July.

Nonaka, I. (1994) ‘A dynamic theory of organizational knowledge creation’, Organization Science, 5(1), pp. 14–37.

Noy, S. and Zhang, W. (2023) ‘Experimental evidence on the productivity effects of generative artificial intelligence’, Science, 381(6654), pp. 187–192.

Polanyi, M. (1966) The Tacit Dimension. Chicago: University of Chicago Press.

Schön, D.A. (1983) The Reflective Practitioner: How Professionals Think in Action. New York: Basic Books.

Weick, K.E. (1995) Sensemaking in Organizations. Thousand Oaks, CA: Sage.

Wenger, E. (1998) Communities of Practice: Learning, Meaning, and Identity. Cambridge: Cambridge University Press.