EKALAVYA ACADEMY | SHORT BOOKS

When AI gets it wrong

How to spot a mistake early, respond calmly, put it right with the people affected and stop it happening the same way twice.

Ekalavya Academy by Almost Magic
Short Book 9 of 10 | Free to read and share

How to read this book

Before You Start

1. Learn
A few slides explain one idea in full sentences.

2. See it
A short, made-up example gives the idea a face.

3. Try it
A ten-minute exercise in the tools you already use.

4. Check it
Three questions. Guidance is at the back, not next to the question.

5. Keep it
A four-sentence summary to come back to.

Two levels in every chapter. The main slides are written so a first-time reader can follow them. A "Going deeper" slide gives the same idea at the level an experienced reader would argue about.

This book works with any AI tool you already have. It never asks you to buy or sign up for a particular product.

Where this book sits in the series

The Plan

Book 1	What AI is, and what it is not	Foundations
Book 2	Productivity with AI: writing, summaries, research, meetings	Practical use
Book 3	From one task to a repeatable workflow, and AI deputies	Workflow automation, agents
Book 4	Handling data safely	Data handling
Book 5	When a data shortcut becomes the rule	Data quality and governance
Book 6	Staying secure with AI	Security
Book 7	Rules, risk and what you must know	Risk and compliance awareness
Book 8	Your AI rules on one page	Policy and accountability
Book 9	When AI gets it wrong	Recovering from AI mistakes
Book 10	When not to use AI, and who stays in charge	Judgement and oversight

Each book works on its own. Read them in order or pick the one you need.

In this book

Contents

Chapter 1
Spotting a mistake early

Chapter 2
The first hour

Chapter 3
Putting it right with people

Chapter 4
Learning from it

At the back
Guidance and answers for every question and exercise

Spotting a mistake early

Chapter 1

By the end of this chapter you will be able to:

Say how AI mistakes usually show up.

Recognise the signs that something is off.

Make it easy for people to raise a doubt.

Know why late discovery is the norm.

Time: about 40 minutes, including the exercise.

Mistakes you find late

Learn

Most AI mistakes are not dramatic. A wrong figure in a quote. A made-up detail in a client email. A summary that left out the one condition that mattered.

The costly ones are found late: by a customer, after the email went out, or weeks after an automation started. The earlier you find a mistake, the smaller it stays.

This book assumes mistakes will happen. What matters is how fast you see them and what you do next.

IN PLAIN WORDS

Plan for the mistake, not only the success.

Four ways mistakes show up

Learn

Wrong fact
A date, name, price or rule that is not true, said with confidence.

Wrong number
A sum, total or percentage that looks neat and is off.

Wrong promise or tone
A message that commits you to something, or reads badly for the person.

Wrong or unfair outcome
A decision or shortlist that treats some people worse, or cannot be explained.

Signs something is off

Learn

Too smooth
Every answer is confident and tidy, with no gaps or doubts.

Different from last time
Same job, very different result, and nobody changed anything.

Unfamiliar detail
A name, source or figure nobody on the team recognises.

A query
A customer or colleague asks, "Where did this come from?"

Numbers that do not tie
Totals do not match your records or another report.

The same quiet fix
People keep correcting the same output without telling anyone.

Make it easy to say "I think this is wrong"

Learn

People often spot a problem and say nothing. They assume the tool is right, or that raising it will look foolish.

Tell your team that a doubt is useful. Give them one named person to tell and one easy way to do it. Thank the first person who does.

Book 8 puts this on the one-page rule sheet as "how to report". This chapter is why it matters.

RULE OF THUMB

A doubt raised early costs minutes.

Why late discovery is the norm

Going Deeper

The check is the weak point
If the check was rushed or skipped, the first real check is the customer.

Volume hides errors
One wrong output in a hundred is hard to see by eye. Sampling finds patterns.

Familiarity dulls attention
After many good results people stop reading closely. Rotate who checks.

Nobody owns the result
A mistake lives longest where no named person owns the output (Book 1).

A short example

See It

Priya runs a small bookkeeping practice. An AI-drafted client email gave a due date a month earlier than the real one.

She did not find it. The client did, and rang her. Afterwards she added a line to her checklist: every date in a client email is checked against the source before it is sent.

She also told her team that a doubt about any AI output is welcome.

THE LESSON

The first check is a person, before the customer sees it.

This is a made-up example for teaching.

Find your last doubt

Try It

1. Think of one time an AI output turned out wrong, or nearly did.

2. Write how it was found, and how long after it was produced.

3. Write what would have caught it sooner.

4. Add that check to a routine you already have.

YOUR TOOLS

Use paper or any notes tool. Real examples beat made-up ones.

GUIDANCE

What to look for is in the Guidance section at the end.

Check yourself

Check It

Question 1
Why are AI mistakes often found late?

Question 2
Name three signs that an AI result may be off.

Question 3
Why should a business make it easy to raise a doubt, even a small one?

Answer in your own words before you read the Guidance section at the end.

Chapter 1 in four sentences

Keep It

KEEP THIS

Most AI mistakes are quiet: a wrong fact, number, promise or outcome. They are found late when no one checks and no one owns the result. Signs include answers that are too smooth, unfamiliar details and numbers that do not tie. Make raising a doubt easy, and treat the first report as useful.

The first hour: pause, size, tell, fix

Chapter 2

By the end of this chapter you will be able to:

Pause before you fix anything.

Size a mistake by who it touches and whether it can be undone.

Tell the right person quickly.

Keep the evidence you will need later.

Time: about 40 minutes, including the exercise.

Stop before you fix

Learn

The first instinct is to fix the output and move on. Wait a few minutes.

If the mistake came from a routine or an automation, it may keep happening. Pause whatever produced it. Then work out what went wrong.

Quiet fixes hide the problem from the people who need to know, and from you next time.

RULE OF THUMB

Pause first, then fix.

Four first steps

Learn

1. Pause
Stop the routine, the automation or the sending until you understand it.

2. Size it
Who is affected, how badly, and can it be undone?

3. Tell
Tell your named person straight away. Do not wait until you are sure.

4. Fix
Correct the output first, then the cause.

How big is it?

Learn

Small
Caught before it left the business. Fix it, note it, carry on.

Medium
It reached a customer or colleague with limited effect. Correct it and tell them.

Serious
Money, personal data, safety or an unfair decision about a person. Stop, tell the owner, ask an adviser.

Data involved
If personal or confidential data left the business, follow Book 4, Chapter 4.

Keep the evidence

Learn

Before you change anything, save what you have. It is hard to learn from a mistake you cannot see.

Keep the request you gave, the output, the tool and its version if shown, the time, and who saw or sent it. A screenshot is enough.

If the case turns out to be serious, this record is what an adviser will ask for first.

REMEMBER

No evidence, no lesson.

Speed and honesty

Going Deeper

Do not delete to hide
Removing the evidence turns a mistake into a trust problem.

Do not guess the scope
Say what you know and what you are still checking.

One person leads
Name who is in charge so five people do not fix five things.

Small is still worth noting
A log of near misses shows patterns before the serious one arrives.

A short example

See It

Dev manages a small removalist business. An AI scheduling flow booked two crews for the same truck on a Friday. He noticed when a driver phoned.

He paused the flow first, then checked which other bookings were affected: three. He saved the flow's log, rang the three customers himself and moved two jobs.

Only after that did he work out why: the tool had read a column that had been changed in the spreadsheet.

THE LESSON

Pause, size, tell, then fix. In that order.

This is a made-up example for teaching.

Draft your first-hour card

Try It

1. Write the four steps on a card: Pause, Size it, Tell, Fix.

2. Write the name and number of the person to tell.

3. List what to save: request, output, time, who saw it.

4. Put the card where the team will find it.

YOUR TOOLS

A phone note works. It must be findable in a hurry.

GUIDANCE

What to look for is in the Guidance section at the end.

Check yourself

Check It

Question 1
Why pause the routine before you fix the output?

Question 2
What three questions help you size a mistake?

Question 3
Why save the evidence before you change anything?

Answer in your own words before you read the Guidance section at the end.

Chapter 2 in four sentences

Keep It

KEEP THIS

Pause the thing that made the mistake, then size it: who is affected, how badly, and whether it can be undone. Tell your named person straight away. Save the request, the output, the time and who saw it before you fix anything.

Putting it right with people

Chapter 3

By the end of this chapter you will be able to:

Decide who needs to be told.

Write a plain correction.

Handle money and promises with care.

Know when to ask an adviser.

Time: about 40 minutes, including the exercise.

Tell the person who was affected

Learn

If a mistake reached a customer or a colleague, they should hear it from you, not find it.

Be early and plain. Say what was wrong, what is right and what you are doing about it. Do not wait for every detail.

People forgive an honest correction far more easily than a hidden one.

IN PLAIN WORDS

Better they hear it from you.

A good correction says four things

Learn

What was wrong
In one or two plain sentences, with no jargon.

What is right
The correct fact, number or decision, clearly marked.

What it affected
How it may have changed what they did or decided.

What you are doing
The fix, and what you will do to avoid a repeat.

Who may need to know

Learn

The affected person
A customer, client, applicant or colleague who received or was judged by the output.

Your owner or manager
The person who answers for AI use in your business.

Your tool provider
If the tool failed in a way others may meet, tell them.

Your team
So the same mistake is not repeated tomorrow. Remove names.

Your adviser
For serious harm, money, unfair treatment or legal exposure.

A regulator
Some data breaches must be reported. See Book 4 and Book 7.

Money, promises and unfair decisions

Learn

If a mistake cost someone money, made a promise you cannot keep or treated a person unfairly, put it right for that person first.

Honour a reasonable promise where you can. If you cannot, say so plainly and offer a fair alternative.

In a serious case, talk to an adviser before you admit fault in writing or offer payment. This book is awareness, not legal advice.

RULE OF THUMB

Put it right for the person first.

Saying sorry well

Going Deeper

Own it
"The tool did it" is not an answer (Book 1). The business sent it.

Be specific
A vague apology sounds like avoidance. Name what went wrong.

Keep it short
A long account of how AI works reads as an excuse.

Follow through
Do what you said, and tell them when it is done.

A short example

See It

Mei runs a small travel agency. An AI-drafted itinerary told a family their tour included airport transfers. It did not.

She found out two days before they flew. She rang the family that morning, said the transfers were not included, offered to book them at the agency's cost and followed up in writing.

The family stayed customers. She added "inclusions" to the list of things a person checks against the booking.

THE LESSON

Call early, say it plainly, fix it for them.

This is a made-up example for teaching.

Write a correction

Try It

1. Pick a made-up or real mistake that reached a customer.

2. Write the four parts: wrong, right, affected, what you are doing.

3. Read it as the customer. Cut anything that sounds like an excuse.

4. Decide who would send it, and by what channel.

YOUR TOOLS

Keep it under 120 words. Short corrections get read.

GUIDANCE

What to look for is in the Guidance section at the end.

Check yourself

Check It

Question 1
What four things does a good correction say?

Question 2
Why should the affected person hear it from you first?

Question 3
When should you talk to an adviser before offering money or admitting fault in writing?

Answer in your own words before you read the Guidance section at the end.

Chapter 3 in four sentences

Keep It

KEEP THIS

Tell the affected person early and plainly: what was wrong, what is right, what it affected and what you are doing. Put it right for them first. Own it, because "the tool did it" is not an answer. In a serious case, ask an adviser before you admit fault or offer money.

Learning from it

Chapter 4

By the end of this chapter you will be able to:

Find the cause, not just the culprit.

Choose a change that stops a repeat.

Decide whether to keep, change or stop the use.

Write a one-page recovery plan.

Time: about 40 minutes, including the exercise.

Find the cause

Learn

Once the people affected are looked after, ask why. Not who.

Most AI mistakes trace back to a few causes: a vague request, missing or wrong data, no check, too much trust, or a tool that changed. Name which one it was.

Blaming a person teaches others to hide the next mistake. Blaming "the AI" teaches nothing.

IN PLAIN WORDS

Ask why, not who.

Four common causes

Learn

The request
Too vague, or missing facts the tool needed (Book 1, Chapter 3).

The data
Wrong, old or incomplete material went in.

The check
Skipped, rushed, or done by someone who could not judge the result.

The tool
It changed, failed, or was used for a job it does not suit.

Fix the system

Learn

Add a check
A named person checks the part that failed, against the source.

Narrow the use
Keep AI to drafting, and take the final decision away from it.

Change the setting or tool
A different mode, or a tool with clearer data terms.

Teach and record
Share the lesson with names removed and update your rule sheet (Book 8).

Keep, change or stop

Learn

After a serious mistake, decide about the job itself.

Keep it if the cause is fixed and the checking holds. Change it if the process needs a new check or limit. Stop it if the risk is higher than the benefit, or you cannot check the result.

Write down the decision and the date. Look again in a month.

REMEMBER

Stopping is a valid fix.

Rebuilding trust

Going Deeper

Near misses count
A log of caught mistakes shows weak spots before a customer finds them.

Share without blame
Tell the team what happened and what changed, with names removed.

Test the fix
Run the same job again and see whether the new check catches it.

Review on a schedule
Look at the log each quarter and ask what keeps recurring.

A short example

See It

Tomas runs a small training provider. An AI-drafted course description promised a certificate the course did not give. A customer complained.

He found the cause: the tool had been given an old brochure. He fixed the description, emailed the customer and added a rule to his sheet: anything about outcomes, prices or certificates is checked against the current course page by one named person.

A month later he reviewed it and kept the job.

THE LESSON

Fix the cause, not only the sentence.

This is a made-up example for teaching.

Write a recovery plan

Try It

1. Pick one AI use in your business.

2. Write what you would do in the first hour, and who you would tell.

3. Write the check that would catch the likeliest mistake.

4. Write who reviews it, and when.

YOUR TOOLS

One page. Link it to your rule sheet from Book 8.

GUIDANCE

What to look for is in the Guidance section at the end.

Check yourself

Check It

Question 1
Why ask "why" rather than "who" after a mistake?

Question 2
Name three common causes of AI mistakes.

Question 3
When might stopping an AI use be the right fix?

Answer in your own words before you read the Guidance section at the end.

Chapter 4 in four sentences

Keep It

KEEP THIS

After a mistake, find the cause: the request, the data, the check or the tool. Fix the system with a check, a narrower use or a better setting. Decide to keep, change or stop, and write it down. Share the lesson without blame.

Guidance and answers

At The Back

Questions and exercises stay in the chapters, on their own slides. Guidance lives here, so you can try first and look afterwards.

Try the task or question in your own words, then compare. Where your answer differs, that is worth a note, not a correction.

WHERE TO FIND IT

The Guidance section below: answers and exercise guidance, one part for each chapter.

Chapter 1: answers and guidance

Guidance

Answer 1
Checks are rushed or skipped, volume hides errors and nobody owns the result, so the first real check is the customer.

Answer 2
Any three of: too smooth, different from last time, unfamiliar detail, a query, numbers that do not tie, the same quiet fix.

Answer 3
People notice early but stay quiet if they fear looking foolish. A named contact and thanks for reports keep problems small.

Exercise
A good answer names a specific check, such as comparing dates to the source, and the routine where it will live.

Chapter 2: answers and guidance

Guidance

Answer 1
The routine may repeat the mistake while you fix one output. Pausing keeps the problem the same size.

Answer 2
Who is affected, how serious it is, and whether it can be undone.

Answer 3
You need it to find the cause, and an adviser may ask for it. Fixing first can erase it.

Exercise
A good card is short enough to read in a hurry, with a real name and number, not just a role.

Chapter 3: answers and guidance

Guidance

Answer 1
What was wrong, what is right, what it affected, and what you are doing about it.

Answer 2
Hearing it from you keeps their trust. Finding it themselves looks like hiding.

Answer 3
In serious cases: real harm, a large sum, unfair treatment of a person or possible legal exposure.

Exercise
A good draft is under about 120 words, names the mistake plainly and has no excuses.

Chapter 4: answers and guidance

Guidance

Answer 1
Blame teaches people to hide problems. Looking for the cause teaches the business to fix them.

Answer 2
Any three of: a vague request, wrong data, a skipped check, a tool change, too much trust.

Answer 3
When the risk outweighs the benefit, or you cannot check the result reliably.

Exercise
A good plan has a named person, a first-hour list, one check and a review date.

About Ekalavya Academy

Find Out More

Ekalavya Academy is a learning series from Almost Magic Tech Lab. The books are written for people who teach themselves, using the tools they already have.

Sources used: National AI Centre, "Guidance for AI adoption: foundations" (ai.gov.au), as summarised in Book 7. Awareness, not legal advice.

Almost Magic builds tools that help people check and govern their AI use. The books never depend on those tools.

LINKS

Almost Magic: almostmagic.net.au

Case studies: ai-casestudies.almostmagic.net.au

Page for this book on the Academy site: link to be added when the page exists.

Ekalavya Academy by Almost Magic | almostmagic.net.au