Built is not the same as connected
On one software product, a progress report listed two items as done. The first was an automatic post to a team chat channel whenever a user completed a certain action. The second was a checkbox that let a user opt out of that post for one particular action.
Both had code. Both had been merged. The database had a new column to store the opt-out choice, and that column had already been added to the live database.
Then someone checked the code before releasing it, and found that neither feature could ever do anything.
The piece of code that wrote the chat post existed, in its own file, and nothing else in the app referred to it. The app was never told to use it, and no part of the app ever asked it to run. So no post had ever been sent, and none ever would be. The new database column was never written to and never read by any code. And on one of the screens where a user could tick the opt-out box, the app read the user’s choice and then dropped it.
Nothing was broken in the usual sense. There were no errors, because code that never runs cannot fail. The report was not lying on purpose either. The parts had been built, and building the parts is what it reported.
Why this happens
A feature in real software is usually two things: the new part, and the connection that makes the existing app use the new part. The new part is the interesting work. The connection is often one or two lines in a different file: a button that calls the new code, a step added after a save, a line that tells the app the new piece exists.
In practice, an AI working on a long task tends to finish the interesting part, describe it well, and sometimes miss the connection. That is easy to miss for a reader too. When you look at a summary or even a list of changed files, a large, well-written new file looks like progress. The absent line in some other file does not show up anywhere. You cannot see what is not there.
Tests do not always catch it either. A test that checks the new piece directly, by calling it on purpose, can pass. At best it shows the part works when someone calls it. It says nothing about whether anyone does.
The habit: trace it from the user’s click
For every feature marked done, ask one question before you accept it: starting from what a real user does, what is the path to this new code?
You do not need to read the code yourself to do this. Ask your AI, ideally in a fresh session that did not build the feature (The author can’t grade the paper explains why), to answer these in writing:
- Who calls it? “List every place in the app that calls the new code for this feature, with the file name and line. If there are none, say so.”
- What starts it? “Starting from the user action (the button, the form, the event), list each step until the new code runs.”
- Is every new setting both saved and read? “For every new setting, checkbox, or database field this feature added, show me where its value is saved, if it needs to be kept, and where it is read. A setting that is saved but never read does nothing. A choice the user makes on screen has to reach the code that acts on it, whether it is used right away or stored for later.”
The third question is the one that would have caught the checkbox in the story. The box appeared on screen. The column existed in the database. The thread between them, and from the database to the code that decides whether to post, was missing at both ends.
If any answer comes back as “none” or “I could not find one,” treat the feature as not done until someone looks closer, whatever the report says. Code can sometimes be started indirectly, for example by a schedule or a settings file, so “none found” is a reason to dig, not final proof. And an answer that names a file and line is still a claim. The check at the end of this post is what confirms it.
Make the test start where the user starts
Once the connection is made, ask for one test that begins at the user’s end rather than at the new code. In the story above, the fix included a test that created the action through the app’s normal front door and then checked that exactly one chat post was queued, without calling the posting code directly. A test shaped like that fails if the wiring is ever removed again, which a test of the part alone does not.
A plain-English way to ask for it: “Write one test that does what a user does, through the normal entry point, and then checks the visible result. Do not call the new code directly in this test.”
Then try it yourself
The final check is the one no report can replace. Do the thing a user does, in the real product, and look for the result yourself. If the feature posts a message, find the message. If it adds an opt-out, tick the box and confirm the thing you opted out of did not happen.
In the story, the corrected work was itself reported carefully: the code was merged and passing its tests, and the report still said plainly that it was not live, because nobody had yet seen it working in production. That is the right line to hold. Built, merged, and tested are all real steps. None of them means a user can reach it.
If you already use a separate reviewer, such as our free review skill, you can add the three questions above to what you ask it to check.