A known failure is still a failure
If you build software with AI agents, sooner or later you will get a daily summary that says something like: “All clear, apart from the usual ones.”
“The usual ones” are failures that everyone has seen so often they no longer register. They have moved from the problem list to the background. Sometimes someone is quietly working on them. Often, as in the case below, nobody is, and the label makes it hard to tell which.
The two checks nobody read
On one software project, an AI job ran every hour and wrote a short health report: which changes were waiting, which tests were failing, what needed a person. It was also, every day for weeks, saying the same thing about two scheduled checks: still failing, already known, nothing new.
The first check watched a list that tracked changes to the database. Each change was supposed to be marked as applied once it had actually been run. The check went red when entries sat as “pending” for too long. A review found it had failed on every recent run.
The second check watched for new releases of an outside data file that the product was meant to refresh regularly. It went red when a newer file was out than the one the product was using.
Both had been failing for so long that the daily reports described them as “the known scheduled reds”, meaning the failing checks everyone already expected. One morning’s task list included the line “Noted, not a task.” That is exactly how a failure gets filed away without anyone deciding anything about it.
Then a review of the whole system flagged it directly: a check that is always red is a check nobody reads. Either the check is broken and should be fixed, or something real is wrong and nobody is acting on it. An AI session was asked to find out which.
Both checks were right.
The database list had more than twenty entries stuck at “pending”, some for over two months. The cause was simple. When a person ran a database change by hand, nobody went back and marked it applied. So the list had stopped meaning anything, and if one of those changes had genuinely never been run, that real gap would be invisible among all the stale ones.
The data file check was right too. A newer file had been released about a month earlier, and the refresh had simply never been run.
Neither problem was hidden. Both had a red light on them every day. The red light just had a label on it that made it easy to step past.
Why “known” is a dangerous word
A failing check usually means one of three things:
- The check is wrong. It is broken, too strict, or watching something that no longer matters.
- The thing it watches is wrong. The alarm is doing its job.
- It needs a decision from a person. Fixing it costs money, touches live data, or needs a judgment call.
“Known” is none of these. It is a way of not choosing. And it tends to stick, because each day’s report is written by a session that sees “this was failing yesterday too” and reasonably carries the label forward. In practice, AI reports often repeat the framing they were handed, unless something asks them to question it.
The cost goes beyond the two failures themselves. While a check is always red, its red status alone stops telling you whether anything new has gone wrong. The details inside the check may still change, but a report that only says “still failing” hides them. In this case, a genuinely missed database change would have shown up as one more stale entry in a long list, easy to miss. A related case, where a check could never fail at all, is in A check that can’t fail is not a check. An always-red check can have a similar effect from the other side: the colour stops carrying news, so people stop looking at it.
The habit: no failure stays “known” without a verdict
Any failure that shows up in your reports more than a couple of days in a row needs a verdict, an owner and a date. Ask your AI assistant for this in plain words:
- “List every check or test that has failed on each of the last several runs, and how long each has been failing.” This gives you the standing failures, not just today’s new ones.
- “For each one, tell me which of three it is: the check is wrong, the thing it watches is wrong, or it needs a decision from me. Show me the evidence for each answer.” Ask for evidence you can look at, such as the entries that are stuck or the date of the newer file. A verdict with no evidence is a guess.
- “For each one, give me the next step, who does it, and by when.” If the check is wrong, fix it or remove it on purpose. If the thing is wrong, fix the thing, and ask what stops it happening again (in the story above, that meant whoever hands over a database change also updates the list once the run is confirmed). If it needs you, put it on your own list with a date.
Then change what your reports are allowed to say. Ask the AI that writes them to never use “known” or “as before” on its own. Every repeat failure should come with its verdict, its owner, and how many days it has been open.
How to check it worked
Two things to look for:
- The repeat-failure section shrinks. A week after you start, the list of long-running failures should be shorter, and each item left on it should name an owner and a date. If the same items are still there with the same wording, the habit has not taken.
- Your always-red checks can turn green. When a fix is done, the check that was red should go green on its next run, and you should see that run yourself. If it never goes green even after the fix, the check itself is suspect, and that is now a verdict you can act on.
The aim is not zero failures. It is that every failure on your screen is something somebody has decided about. If you want a second set of eyes on the changes that come out of this, our free review skill can review them before they go in.