Imagine settling into a dentist’s chair, breathing through the laughing-gas mask, when your dentist announces he’ll also be performing brain surgery.
“Relax,” he says, opening a fictional app called Brain.ai. “I’ve upgraded to Pro.”
You might have a few concerns. His enthusiasm would probably do very little to address them.
That is roughly how I feel about business-critical software built by someone whose entire support plan is “the AI said it worked.”
“AI slop grenades” is a phrase circulating on LinkedIn for AI-generated work handed to other people without enough thought or review. It captures the unpleasant experience of inheriting something its creator never properly checked. Landon Ledford’s LinkedIn post describes that handoff.
Software adds a particularly expensive twist: people can start depending on it before anyone discovers the problem.
Someone outside the engineering team builds a customer portal over a weekend. A designer generates an internal workflow. A manager connects a few business systems because the demo looked straightforward.
These can be excellent starting points. AI gives people a practical way to explore ideas they previously couldn’t build.
Then the prototype gets promoted.
Someone uploads customer information. Another department starts using it every morning. The person who built it moves on, having received considerable praise for saving the company money.
Six weeks later, an ordinary change breaks it.
Now the engineering team has inherited a system it didn’t design, with undocumented dependencies and no clear explanation of how the pieces fit together. The original creator can show everyone the conversation that generated it. Unfortunately, nobody can explain why it is sending invoices twice.
The launch announcement aged better than the software.
This is where experience still earns its keep.
Give an AI a vague instruction to “build a production-ready system,” and you have left an enormous amount open to interpretation. It has no automatic knowledge of your business rules, operating constraints or the mistakes your team has already paid to understand.
Give it the working practices developed through 30 years of software engineering, and you can set a much more useful direction.
That experience becomes concrete through established architecture, maintained examples, documented decisions and clear expectations for failure. An experienced engineer can tell the AI how access should work, how a failed operation should recover, and which existing patterns it needs to follow.
They can also recognize when the generated answer looks convincing but makes a bad decision.
Thirty years of maturity includes remembering why the apparently simpler approach caused trouble last time.
You can bring that knowledge into an AI-assisted build. You still need someone capable of checking that the result actually follows it. A folder full of standards helps only if someone understands their purpose and checks the work against them.
The same applies to design. A polished screen can conceal a workflow that fails the first time a user makes a mistake. A designer’s judgment about accessibility and interaction matters just as much after the AI produces the first version.
Job titles alone don’t settle this. Engineers can ship careless AI output too. Anyone building beyond their expertise needs someone with the relevant knowledge involved before other people depend on the result.
AI is a great tool. It can accelerate useful work and make experienced people considerably more productive. I want people using it.
I also want a name beside the system when someone asks who supports it.
Who understands how it works? Who checks changes? Who can restore service when it fails?
If the answer is “we’ll find someone when that happens,” include that future rescue in the budget. The cleanup can cost more than the guidance you skipped, especially when someone has to reverse-engineer the system while the business waits.
Build the prototype. Explore the idea. Bring in the people who can help turn it into something dependable, and give them time to do that work.
And if your dentist offers the Brain.ai upgrade, perhaps stick with the cleaning.