EKALAVYA ACADEMY | SHORT BOOKS
When AI gets it wrong
How to spot a mistake early, respond calmly, put it right with the people affected and stop it happening the same way twice.
Ekalavya Academy by Almost Magic
Short Book 9 of 10 | Free to read and share
How to read this book
Before You Start
A few slides explain one idea in full sentences.
A short, made-up example gives the idea a face.
A ten-minute exercise in the tools you already use.
Three questions. Guidance is at the back, not next to the question.
A four-sentence summary to come back to.
Two levels in every chapter. The main slides are written so a first-time reader can follow them. A "Going deeper" slide gives the same idea at the level an experienced reader would argue about.
This book works with any AI tool you already have. It never asks you to buy or sign up for a particular product.
Where this book sits in the series
The Plan
| Book 1 | What AI is, and what it is not | Foundations |
| Book 2 | Productivity with AI: writing, summaries, research, meetings | Practical use |
| Book 3 | From one task to a repeatable workflow, and AI deputies | Workflow automation, agents |
| Book 4 | Handling data safely | Data handling |
| Book 5 | When a data shortcut becomes the rule | Data quality and governance |
| Book 6 | Staying secure with AI | Security |
| Book 7 | Rules, risk and what you must know | Risk and compliance awareness |
| Book 8 | Your AI rules on one page | Policy and accountability |
| Book 9 | When AI gets it wrong | Recovering from AI mistakes |
| Book 10 | When not to use AI, and who stays in charge | Judgement and oversight |
Each book works on its own. Read them in order or pick the one you need.
In this book
Contents
Spotting a mistake early
The first hour
Putting it right with people
Learning from it
Guidance and answers for every question and exercise
Spotting a mistake early
Chapter 1
By the end of this chapter you will be able to:
Say how AI mistakes usually show up.
Recognise the signs that something is off.
Make it easy for people to raise a doubt.
Know why late discovery is the norm.
Time: about 40 minutes, including the exercise.
Mistakes you find late
Learn
Most AI mistakes are not dramatic. A wrong figure in a quote. A made-up detail in a client email. A summary that left out the one condition that mattered.
The costly ones are found late: by a customer, after the email went out, or weeks after an automation started. The earlier you find a mistake, the smaller it stays.
This book assumes mistakes will happen. What matters is how fast you see them and what you do next.
IN PLAIN WORDS
Plan for the mistake, not only the success.Four ways mistakes show up
Learn
A date, name, price or rule that is not true, said with confidence.
A sum, total or percentage that looks neat and is off.
A message that commits you to something, or reads badly for the person.
A decision or shortlist that treats some people worse, or cannot be explained.
Signs something is off
Learn
Every answer is confident and tidy, with no gaps or doubts.
Same job, very different result, and nobody changed anything.
A name, source or figure nobody on the team recognises.
A customer or colleague asks, "Where did this come from?"
Totals do not match your records or another report.
People keep correcting the same output without telling anyone.
Make it easy to say "I think this is wrong"
Learn
People often spot a problem and say nothing. They assume the tool is right, or that raising it will look foolish.
Tell your team that a doubt is useful. Give them one named person to tell and one easy way to do it. Thank the first person who does.
Book 8 puts this on the one-page rule sheet as "how to report". This chapter is why it matters.
RULE OF THUMB
A doubt raised early costs minutes.Why late discovery is the norm
Going Deeper
If the check was rushed or skipped, the first real check is the customer.
One wrong output in a hundred is hard to see by eye. Sampling finds patterns.
After many good results people stop reading closely. Rotate who checks.
A mistake lives longest where no named person owns the output (Book 1).
A short example
See It
Priya runs a small bookkeeping practice. An AI-drafted client email gave a due date a month earlier than the real one.
She did not find it. The client did, and rang her. Afterwards she added a line to her checklist: every date in a client email is checked against the source before it is sent.
She also told her team that a doubt about any AI output is welcome.
THE LESSON
The first check is a person, before the customer sees it.This is a made-up example for teaching.
Find your last doubt
Try It
YOUR TOOLS
Use paper or any notes tool. Real examples beat made-up ones.GUIDANCE
What to look for is in the Guidance section at the end.Check yourself
Check It
Why are AI mistakes often found late?
Name three signs that an AI result may be off.
Why should a business make it easy to raise a doubt, even a small one?
Answer in your own words before you read the Guidance section at the end.
Chapter 1 in four sentences
Keep It
KEEP THIS
Most AI mistakes are quiet: a wrong fact, number, promise or outcome. They are found late when no one checks and no one owns the result. Signs include answers that are too smooth, unfamiliar details and numbers that do not tie. Make raising a doubt easy, and treat the first report as useful.
The first hour: pause, size, tell, fix
Chapter 2
By the end of this chapter you will be able to:
Pause before you fix anything.
Size a mistake by who it touches and whether it can be undone.
Tell the right person quickly.
Keep the evidence you will need later.
Time: about 40 minutes, including the exercise.
Stop before you fix
Learn
The first instinct is to fix the output and move on. Wait a few minutes.
If the mistake came from a routine or an automation, it may keep happening. Pause whatever produced it. Then work out what went wrong.
Quiet fixes hide the problem from the people who need to know, and from you next time.
RULE OF THUMB
Pause first, then fix.Four first steps
Learn
Stop the routine, the automation or the sending until you understand it.
Who is affected, how badly, and can it be undone?
Tell your named person straight away. Do not wait until you are sure.
Correct the output first, then the cause.
How big is it?
Learn
Caught before it left the business. Fix it, note it, carry on.
It reached a customer or colleague with limited effect. Correct it and tell them.
Money, personal data, safety or an unfair decision about a person. Stop, tell the owner, ask an adviser.
If personal or confidential data left the business, follow Book 4, Chapter 4.
Keep the evidence
Learn
Before you change anything, save what you have. It is hard to learn from a mistake you cannot see.
Keep the request you gave, the output, the tool and its version if shown, the time, and who saw or sent it. A screenshot is enough.
If the case turns out to be serious, this record is what an adviser will ask for first.
REMEMBER
No evidence, no lesson.Speed and honesty
Going Deeper
Removing the evidence turns a mistake into a trust problem.
Say what you know and what you are still checking.
Name who is in charge so five people do not fix five things.
A log of near misses shows patterns before the serious one arrives.
A short example
See It
Dev manages a small removalist business. An AI scheduling flow booked two crews for the same truck on a Friday. He noticed when a driver phoned.
He paused the flow first, then checked which other bookings were affected: three. He saved the flow's log, rang the three customers himself and moved two jobs.
Only after that did he work out why: the tool had read a column that had been changed in the spreadsheet.
THE LESSON
Pause, size, tell, then fix. In that order.This is a made-up example for teaching.
Draft your first-hour card
Try It
YOUR TOOLS
A phone note works. It must be findable in a hurry.GUIDANCE
What to look for is in the Guidance section at the end.Check yourself
Check It
Why pause the routine before you fix the output?
What three questions help you size a mistake?
Why save the evidence before you change anything?
Answer in your own words before you read the Guidance section at the end.
Chapter 2 in four sentences
Keep It
KEEP THIS
Pause the thing that made the mistake, then size it: who is affected, how badly, and whether it can be undone. Tell your named person straight away. Save the request, the output, the time and who saw it before you fix anything.
Putting it right with people
Chapter 3
By the end of this chapter you will be able to:
Decide who needs to be told.
Write a plain correction.
Handle money and promises with care.
Know when to ask an adviser.
Time: about 40 minutes, including the exercise.
Tell the person who was affected
Learn
If a mistake reached a customer or a colleague, they should hear it from you, not find it.
Be early and plain. Say what was wrong, what is right and what you are doing about it. Do not wait for every detail.
People forgive an honest correction far more easily than a hidden one.
IN PLAIN WORDS
Better they hear it from you.A good correction says four things
Learn
In one or two plain sentences, with no jargon.
The correct fact, number or decision, clearly marked.
How it may have changed what they did or decided.
The fix, and what you will do to avoid a repeat.
Who may need to know
Learn
A customer, client, applicant or colleague who received or was judged by the output.
The person who answers for AI use in your business.
If the tool failed in a way others may meet, tell them.
So the same mistake is not repeated tomorrow. Remove names.
For serious harm, money, unfair treatment or legal exposure.
Some data breaches must be reported. See Book 4 and Book 7.
Money, promises and unfair decisions
Learn
If a mistake cost someone money, made a promise you cannot keep or treated a person unfairly, put it right for that person first.
Honour a reasonable promise where you can. If you cannot, say so plainly and offer a fair alternative.
In a serious case, talk to an adviser before you admit fault in writing or offer payment. This book is awareness, not legal advice.
RULE OF THUMB
Put it right for the person first.Saying sorry well
Going Deeper
"The tool did it" is not an answer (Book 1). The business sent it.
A vague apology sounds like avoidance. Name what went wrong.
A long account of how AI works reads as an excuse.
Do what you said, and tell them when it is done.
A short example
See It
Mei runs a small travel agency. An AI-drafted itinerary told a family their tour included airport transfers. It did not.
She found out two days before they flew. She rang the family that morning, said the transfers were not included, offered to book them at the agency's cost and followed up in writing.
The family stayed customers. She added "inclusions" to the list of things a person checks against the booking.
THE LESSON
Call early, say it plainly, fix it for them.This is a made-up example for teaching.
Write a correction
Try It
YOUR TOOLS
Keep it under 120 words. Short corrections get read.GUIDANCE
What to look for is in the Guidance section at the end.Check yourself
Check It
What four things does a good correction say?
Why should the affected person hear it from you first?
When should you talk to an adviser before offering money or admitting fault in writing?
Answer in your own words before you read the Guidance section at the end.
Chapter 3 in four sentences
Keep It
KEEP THIS
Tell the affected person early and plainly: what was wrong, what is right, what it affected and what you are doing. Put it right for them first. Own it, because "the tool did it" is not an answer. In a serious case, ask an adviser before you admit fault or offer money.
Learning from it
Chapter 4
By the end of this chapter you will be able to:
Find the cause, not just the culprit.
Choose a change that stops a repeat.
Decide whether to keep, change or stop the use.
Write a one-page recovery plan.
Time: about 40 minutes, including the exercise.
Find the cause
Learn
Once the people affected are looked after, ask why. Not who.
Most AI mistakes trace back to a few causes: a vague request, missing or wrong data, no check, too much trust, or a tool that changed. Name which one it was.
Blaming a person teaches others to hide the next mistake. Blaming "the AI" teaches nothing.
IN PLAIN WORDS
Ask why, not who.Four common causes
Learn
Too vague, or missing facts the tool needed (Book 1, Chapter 3).
Wrong, old or incomplete material went in.
Skipped, rushed, or done by someone who could not judge the result.
It changed, failed, or was used for a job it does not suit.
Fix the system
Learn
A named person checks the part that failed, against the source.
Keep AI to drafting, and take the final decision away from it.
A different mode, or a tool with clearer data terms.
Share the lesson with names removed and update your rule sheet (Book 8).
Keep, change or stop
Learn
After a serious mistake, decide about the job itself.
Keep it if the cause is fixed and the checking holds. Change it if the process needs a new check or limit. Stop it if the risk is higher than the benefit, or you cannot check the result.
Write down the decision and the date. Look again in a month.
REMEMBER
Stopping is a valid fix.Rebuilding trust
Going Deeper
A log of caught mistakes shows weak spots before a customer finds them.
Tell the team what happened and what changed, with names removed.
Run the same job again and see whether the new check catches it.
Look at the log each quarter and ask what keeps recurring.
A short example
See It
Tomas runs a small training provider. An AI-drafted course description promised a certificate the course did not give. A customer complained.
He found the cause: the tool had been given an old brochure. He fixed the description, emailed the customer and added a rule to his sheet: anything about outcomes, prices or certificates is checked against the current course page by one named person.
A month later he reviewed it and kept the job.
THE LESSON
Fix the cause, not only the sentence.This is a made-up example for teaching.
Write a recovery plan
Try It
YOUR TOOLS
One page. Link it to your rule sheet from Book 8.GUIDANCE
What to look for is in the Guidance section at the end.Check yourself
Check It
Why ask "why" rather than "who" after a mistake?
Name three common causes of AI mistakes.
When might stopping an AI use be the right fix?
Answer in your own words before you read the Guidance section at the end.
Chapter 4 in four sentences
Keep It
KEEP THIS
After a mistake, find the cause: the request, the data, the check or the tool. Fix the system with a check, a narrower use or a better setting. Decide to keep, change or stop, and write it down. Share the lesson without blame.
Guidance and answers
At The Back
Questions and exercises stay in the chapters, on their own slides. Guidance lives here, so you can try first and look afterwards.
Try the task or question in your own words, then compare. Where your answer differs, that is worth a note, not a correction.
WHERE TO FIND IT
The Guidance section below: answers and exercise guidance, one part for each chapter.Chapter 1: answers and guidance
Guidance
Checks are rushed or skipped, volume hides errors and nobody owns the result, so the first real check is the customer.
Any three of: too smooth, different from last time, unfamiliar detail, a query, numbers that do not tie, the same quiet fix.
People notice early but stay quiet if they fear looking foolish. A named contact and thanks for reports keep problems small.
A good answer names a specific check, such as comparing dates to the source, and the routine where it will live.
Chapter 2: answers and guidance
Guidance
The routine may repeat the mistake while you fix one output. Pausing keeps the problem the same size.
Who is affected, how serious it is, and whether it can be undone.
You need it to find the cause, and an adviser may ask for it. Fixing first can erase it.
A good card is short enough to read in a hurry, with a real name and number, not just a role.
Chapter 3: answers and guidance
Guidance
What was wrong, what is right, what it affected, and what you are doing about it.
Hearing it from you keeps their trust. Finding it themselves looks like hiding.
In serious cases: real harm, a large sum, unfair treatment of a person or possible legal exposure.
A good draft is under about 120 words, names the mistake plainly and has no excuses.
Chapter 4: answers and guidance
Guidance
Blame teaches people to hide problems. Looking for the cause teaches the business to fix them.
Any three of: a vague request, wrong data, a skipped check, a tool change, too much trust.
When the risk outweighs the benefit, or you cannot check the result reliably.
A good plan has a named person, a first-hour list, one check and a review date.
About Ekalavya Academy
Find Out More
Ekalavya Academy is a learning series from Almost Magic Tech Lab. The books are written for people who teach themselves, using the tools they already have.
Sources used: National AI Centre, "Guidance for AI adoption: foundations" (ai.gov.au), as summarised in Book 7. Awareness, not legal advice.
Almost Magic builds tools that help people check and govern their AI use. The books never depend on those tools.
LINKS
Almost Magic: almostmagic.net.au
Case studies: ai-casestudies.almostmagic.net.au
Page for this book on the Academy site: link to be added when the page exists.
Ekalavya Academy by Almost Magic | almostmagic.net.au