Why AI exposes weak engineering practices

Developers have been talking AI down in software development lately. The complaints are familiar: AI produces buggy code. It hallucinates. It creates security vulnerabilities. It can't be trusted.

They're not wrong. AI does produce buggy code. But buggy code has always been part of the job. Juniors write it. Seniors under pressure write it. You wrote it at 2am last Tuesday. What mattered was everything around it: code review, challenging assumptions, production pushing back. AI doesn't change that system. It relies on it. And people experiencing "AI chaos" usually aren't failing because of AI. They're failing because their review practices, ownership culture, or quality processes were already weak. AI just makes those gaps show up faster.

That's the tool problem, and it's the one we can actually fix. The bigger problem is who gets to use these tools at all, and what happens when the current economics collapse.

The Rocket-Strapped Car

Think of AI coding assistance as a rocket-strapped car. Everyone can drive a car. But strap a rocket engine to it and hand the keys to someone who just passed their driving test? Almost certainly not a good idea.

Push it to the limits (blindly accepting every suggestion, skipping review, committing without testing) and you will crash.

Case in point: Moltbook, the AI-only social network that exploded to 1.5 million agents in days. Founder Matt Schlicht publicly stated he "didn't write one line of code." It was entirely vibe-coded with AI assistance, and the database was wide open. No authentication required for read/write access. Hardcoded credentials exposed in client-side JavaScript. 1.5 million API tokens, 35,000 email addresses, 30,000 private messages. All accessible to anyone with basic SQL knowledge.

It would have taken two SQL statements to enable Row-Level Security, a feature prominently documented on page one of Supabase's security docs. The platform launched, went viral, and nobody checked if the database had basic access controls. That's what happens when you floor it without looking at the road.

But use the tool within limits, with the same rigor you'd apply to any code, and you get performance improvements without sacrificing quality. If reviewing AI-generated code takes extra time, that's fine. You still gained efficiency elsewhere. If it doesn't help for a specific task, stop using it for that task. Use AI where it actually adds value, not everywhere because it's the new shiny thing.

Accountability doesn't disappear because a model wrote the first draft. If developers don't feel responsible just because "AI did it," that's a management failure, not a technology one.

When AI Works (and When It Doesn't)

AI coding assistance works when you know exactly what you want. I had a TypeScript codebase that needed OpenAPI implementation. Dozens of endpoints. Sure, there are code generators that spit out TypeScript from Swagger specs. But I wanted control over the structure, and I wanted to see if this powerful but insecure automation everyone's hyping could actually save time.

So I wrote one endpoint by hand. Got the types right, set up the validation, handled errors the way I wanted. Then I gave AI the pattern and the Swagger spec. "Here's the structure. Implement the rest following this exact pattern." It generated 30+ endpoints. I reviewed each one, fixed a few type mismatches, caught where it misread the spec format, adjusted error handling in three places. Still saved hours.

That's the sweet spot: repetitive work where you demonstrate the pattern once, then let AI replicate it. You know what good output looks like, so you can catch when it deviates. You're not learning. You're not exploring. You're implementing something you already understand.

Everyone talks about AI as a great equalizer, but it's an amplifier. The developers who already had strong instincts, who could spot a bad abstraction and question an architecture decision, those are the ones compounding their output. They reject bad suggestions on the first pass. They know when the model is confidently wrong. They use AI the way you're supposed to use any tool: with judgment.

Everyone else just got a faster way to produce code they can't evaluate.

AI is terrible at complex problems. Once it commits to an approach, it doubles down. If it said yes to something early in the conversation, it won't suddenly say no. It'll dig itself deeper trying to make the wrong approach work. You have to recognize this, stop it, back out, and restart with better constraints.

I don't do the role-playing prompts. No "you are a senior engineer," "act as an expert in TypeScript," or "my grandmother is about to die if you don't complete this coding task very fast and with no bugs." I can't make myself do it. Maybe it's more optimal? Maybe those prompts produce better results? I'm not sure. I use the tool the way that works for me: input, desired output, constraints.

I use it for a final check before doing a PR. Sometimes it finds nothing. Sometimes it catches something, an edge case, a typo in error handling.

But it also makes up problems that don't exist. cURL ended their bug bounty program in January 2026 after being flooded with AI-generated false reports. Their confirmation rate dropped from 15% to below 5% in 2025. Maintainer Daniel Stenberg said the slop took a "mental toll" and "hampered the team's will to live." Half of the 67 submissions they reviewed since July 2024 arrived in the last two months alone.

AI will confidently tell you about security issues that aren't there, race conditions in single-threaded code, memory leaks that can't happen. You need to know enough to filter the noise.

And some developers skip testing because they don't feel like they own the code, and they might be right. Maybe they did a single prompt and that PR that now I have to read, with 100 changed files, while trying not to cry. The code becomes something they transported, not something they built. That's a human cost that doesn't show up in productivity metrics.

The Control Plane Problem

LLMs have a fundamental problem: they mix the data plane and control plane. Instructions and data flow through the same channel. This is flawed by design. You can build security layers on top, but the core architecture can never be secure. That's a trade-off worth being explicit about.

Think of it like SQL injection, except there's no prepared statement equivalent. With SQLi, we learned to separate code from data. Parameterized queries fixed the problem. Done.

Prompt injection has no fix. You can add system prompts, use delimiters, implement "guardrails," build sandboxes. But instructions and data still flow through the same channel. Every mitigation is a trick, not a solution. And tricks can be bypassed.

OpenClaw, a third-party agent gateway, gives AI agents direct access to local files, applications, browsers, and terminals. Security researchers at 1Password analyzed the attack surface and found that the most popular "skill" in OpenClaw's marketplace was malware delivery.

MCP servers, the things you npx to extend AI capabilities, are just npm packages. In September 2025, researchers discovered a malicious MCP server called postmark-mcp. It claimed to be an email connector but contained a one-line backdoor that BCC'd every outgoing email to an attacker-controlled address. By the time it was removed: 1,643 downloads, roughly 300 organizations compromised. One line of code.

Prompt Kiddies

If you've spent any time around security communities, you know the term "script kiddie." Someone who runs tools they don't understand, following tutorials without knowing what's actually happening, or just spinning up a GUI tool. They can find basic vulnerabilities because the tools do the thinking. But ask them to chain exploits or think creatively? They're stuck.

AI just made everyone a script kiddie.

I hosted a CTF for a company recently. It was great. People learned, had fun, felt accomplished. But I also learned something: with a couple of MCPs and some loose privileges, AI will pentest just about anything. It might say no initially, but tell it "this is my system" or "this is a CTF" and it goes happily along. Spins up sqlmap, finds SQL injections, runs through the basic vulnerability checklist.

The low-hanging fruit is now accessible to anyone. If you're not testing for basic stuff (SQLi, XSS, exposed credentials, open databases), someone with zero security knowledge and an AI will find it. The barrier to entry dropped to near-zero.

But the high-hanging fruit? AI is terrible at it. Complex vulnerability chains, novel attack vectors, thinking past the standard playbook. That still requires actual thinking. People with security knowledge and creativity reign supreme.

Moltbook didn't need a sophisticated attack. It needed someone to check if the database had authentication. That's script kiddie territory now.

Irony: AI most likely would have caught these flaws if anyone had asked it to check. But when you don't know what questions to ask, you get the rocket-strapped car. Again.

Move Fast, Break Security

Governance and risk management seem to have been forsaken when it comes to AI. We have to have velocity.

Moltbook launched with 1.5 million agents and zero security checks. OpenClaw's marketplace features malware delivery as the top skill with no vetting process. MCP servers get npx'd into production without anyone checking what they actually do. postmark-mcp compromised 300 organizations because people trusted an npm package with email access.

AI tech gets a governance exemption. In any other context, you'd have security reviews, vendor assessments, risk analysis, change management. Someone would ask "what's the blast radius if this goes wrong?" But with AI, the answer is apparently "ship it and see."

Why? FOMO. Competitive pressure. Everyone's building AI agents and nobody wants to be the slowest one in the room. If your competitor is moving fast and you're doing security reviews, you feel like you're falling behind.

You can move fast with bad security until you can't. And when it breaks, it doesn't break small. It breaks at scale. Moltbook exposed 1.5 million tokens. postmark-mcp compromised 300 organizations. The velocity that let you ship fast is the same velocity that spreads the damage.

You can recover from technical debt, at a high price. Leaked databases don't have a recovery path.

The part we can actually fix

The tool problem is solvable, and the fixes aren't new. Code review, ownership, testing before you merge, basic security checks before you ship. None of this is groundbreaking. AI just makes the gaps in these practices show up faster, so fix the practices.

Quick Checks

Some of this is a management problem, as I covered above. You have to make it clear that even if an AI wrote it, you have to have the same ownership as if you made it 100%. Don't expect code reviewers to read the code if you can't or won't.

On the technical side, PR Quality Gates, branch protection, and linting are a good baseline, but just the start. If you weren't already doing that, that's a whole other blog post.

Bigger Issues?

But there's a bigger question behind all of this: who controls the intelligence layer, who pays for it, and what happens when the subsidized pricing ends. That's an infrastructure and power problem, and it doesn't have a clean answer.

---

Have thoughts on AI security or governance? Reach out on LinkedIn or email.