The light said yes and did nothing

I built a replacement for my commercial smart home controller. Python, FastAPI, MQTT, SQLite, a React dashboard. Local-first, no cloud, no subscription. I did not write it by hand. I directed it, reviewed it, argued with it, and made the calls on architecture.

It worked. The dashboard loaded, the rules engine hot-reloaded, the tests were green.

And for a stretch of that build, not one command it sent actually reached a device.

How a system says yes and does nothing

The outbound command path is the bit that turns “turn on the kitchen light” into a message a device acts on. Somewhere between the API and the device adapter, that path was silently dead. The API accepted the command. It logged it. It returned success. Then nothing happened.

Every test passed, because every test asked the same question: did the API respond? It always did. No test asked the only question that mattered: did the light turn on?

That is the failure mode I would warn any non-engineer about first. AI will happily write you a test suite that confirms the system acknowledges requests. It is much less likely, unprompted, to write the test that confirms the system does the thing.

Why it got found

Not by the tests. By being annoying.

The first review of the codebase came back clean and a bit quick. It felt like a pass, not an audit, so I said so and asked for a deeper one with no stones unturned. That second pass traced a command end to end and found the break.

I have no special technical instinct here. What I have is a career in commercial roles, which is mostly practice at noticing when someone is telling me what I want to hear. That skill transfers to AI completely. If an answer arrives fast and tidy on a hard problem, push.

The other three things that broke

None of these were dramatic. All of them cost time, and all of them are the kind of thing nobody warns you about.

Background processes that did not survive. The end-to-end test needed a message broker and the backend running at the same time. Started in one step, they were gone by the next. The fix was to run the whole thing as one script: start the broker, wait, start the backend, wait, test, shut everything down. Boring. Reliable.

A setting that got cached before the test could change it. The API key was read once when the code loaded, so tests that set their own key were quietly using a different one. The fix was to force the value in the test setup rather than trust the environment.

Folders with brackets in their names. A shortcut meant to create several subfolders at once created one folder with the literal name {types,hooks,...} instead. Now every folder gets created explicitly before anything is copied into it.

A constraint I gave that got misread

I set a rule early: no build step. I meant the server. It got applied to the frontend too, which pushed the dashboard in a worse direction for a while. When you give AI a constraint, give it a scope as well. “No build step on the server” and “no build step” are different instructions, and it will take you literally.

What I would tell you to do

• Test the outcome, not the acknowledgement. For anything that acts on the world (a device, an email, a payment) ask for a test that proves the action happened.
• Push back on fast answers to hard questions. A deeper pass costs you one message.
• Scope every constraint. Say what it applies to, and what it does not.
• Make the boring version the default. One process, one database, one script that runs the whole test. Clever setups fail in clever ways.

Read next: I audited my own AI agent. Almost everything was open. The six-agent fleet that runs my day, and the audit that found I had built it the wrong way round.