Playbook
Don’t Fix the Output — Fix What’s Underneath
How to recognise when repeated AI mistakes are symptoms of a deeper problem in the data, rules or source of truth underneath.
One of the easiest traps with AI is to keep correcting what you can see.
A date is wrong, so you fix the date.
A coding agent has taken a feature in the wrong direction, so you steer it back.
Sometimes that is exactly the right thing to do.
But if you keep fixing the same kind of problem and it keeps coming back, the output may not be the thing that is broken.
The problem may be underneath it.
A bad decision is normal
AI agents make bad decisions.
That does not automatically mean your project has a structural problem.
Sometimes Codex takes a wrong turn, you point it back in the right direction, and it carries on perfectly well.
That is normal.
The warning sign is when you keep steering and correcting, but you are either coming back round in loops or making things bit by bit worse.
At that point, I think it is worth stopping.
The Route 26 lesson
I learnt this properly while building Route 26 for MyTourneyTime.
It was a football world cup fixture planner/tool. The feature needed tournament fixtures, kickoff times, locations and all the possible routes teams could take through the competition.
Early on, I asked an AI agent to find much of that information from the internet and build around it.
On the surface, that seemed reasonable. The information existed.
But I was asking the agent to chew through too much at once.
The result was fixture errors everywhere!!!
At first, the obvious ones were easy enough to correct. Then the output improved and looked increasingly convincing.
Later, more problems appeared.
A kickoff time was wrong. Then a venue. Then a date. Then another route through the tournament.
I could have kept tweaking the output forever.
Eventually, the real answer was to stop patching and build a proper structured database underneath the feature.
That database became the dependable source for fixtures, venues, time zones and progression routes.
Once the foundation was reliable, the output had something reliable to build on.
That is a very different thing from making a broken result look slightly better.
Fixing one problem can move the error somewhere else
I saw the same pattern in a completely different project while building a digital twin of a property.
Suppose one wall appears to be 30 mm out.
You could simply move that wall 30 mm and declare the problem solved.
But the wall exists as part of a wider geometry.
Move it to satisfy one measurement and you may immediately make another relationship wrong somewhere else.
You have not necessarily fixed the model.
You may just have moved the error.
That is where repeated surface corrections become dangerous. Each individual fix can look sensible while the system underneath becomes less coherent.
Know when to stop steering
The distinction I now try to make is fairly simple.
One-off bad decision? Steer and correct.
Repeated interconnected failures? Inspect the foundation.
A few warning signs are:
- the same type of problem keeps returning;
- correcting one area creates another problem elsewhere;
- each prompt needs another exception added to it;
- the output looks plausible, but you cannot trace it back to a dependable source;
- or you are spending more time patching symptoms than improving the thing producing them.
That last one is usually the giveaway.
If you find yourself going round in circles, there is a good chance the problem is no longer the latest AI response.
Find the source of truth
The answer is not always "rebuild everything".
It is to work backwards.
What is this output actually based on?
A database?
A geometry model?
An API response?
A project instruction?
A calculation?
An assumption made several stages ago?
Find the layer that is meant to represent the truth and validate that first.
If that layer is wrong, no amount of clever prompting on top of it will make the system dependable.
And if the thing you are building has any real-world consequence, "looks about right" is not enough.
Sometimes going backwards is faster
Stopping to rebuild the foundation can feel painful.
You may lose work. You may burn time. You may have to throw away a sequence that looked almost finished.
Sometimes it feels less like one step backwards and more like ten.
But continuing to patch a bad foundation is usually slower.
Once the underlying source is sound, progress tends to become much more straightforward because you are no longer fighting the same hidden problem repeatedly.
That is the habit I now try to keep:
Steer the model when it makes a bad decision.
Question the foundation when the same failure keeps coming back.
The output is only the visible layer.
Sometimes the most useful thing you can do is stop touching it.