⬇ Download PDF ⬇ LinkedIn Carousel

Executive Summary

This article is written for technology leaders — CIOs, engineering directors, and delivery executives — who are adopting AI coding agents in enterprise environments and want to understand where the real risks lie.

After building production systems with AI coding agents, one pattern emerges with uncomfortable clarity: AI does not hallucinate randomly. It hallucinates into the gaps you left — missing requirements, stale documentation, undesigned decisions, and ambiguous architecture.

When the spec is precise, AI agents front-load work into cheap, correct increments. When the spec is absent or wrong, they generate confident, wrong code — and the cost of that confidence is paid later, in the most expensive phase of any project: rework, real-target testing, regression debugging, and deployment failures.

The economic argument is simple: requirements and design are cheap. Rework is not.

The Inversion Most Teams Miss

The standard narrative around AI development goes like this: AI makes you faster, so you can worry less about upfront planning and iterate your way to the right answer.

That narrative is backwards — and it is costing teams real money.

Traditional development, the manual kind, has a natural correction mechanism: developers notice when their mental model of the system is wrong. They pause, they re-read the code, they ask questions. This friction is inefficient, but it is also a quality gate.

AI agents do not pause. They extrapolate from whatever context they have been given, fill the gaps with plausible-sounding assumptions, and generate code that compiles, passes the tests you wrote for it, and looks completely credible — right up until it hits the real system.

The result is an inversion of where errors are caught. In manual development, missing requirements surface during implementation, when a developer hits a decision point they cannot resolve. In AI-native development, missing requirements surface in production, when a perfectly well-written function turns out to have been written against a model of the world that was never true.

The corrective is not to use AI less. It is to invest more deliberately in the cheap phase — and to understand exactly what is cheap and what is not.

The Economics: Cheap Phase vs Expensive Phase

Every software project has two cost phases, but AI dramatically changes their relative weight.

The cheap phase is everything that happens before code: requirements documents, architecture decision records (ADRs), sequence diagrams, test vectors, explicit interface contracts, agreed terminology. A sentence in an ADR costs almost nothing. A correct test vector — the concrete input-output pair that pins a behaviour — can be written in minutes.

The expensive phase is everything that happens after a wrong decision has been encoded: rework loops, hallucinated API calls against outdated interfaces, confidently-wrong edits that propagate across a codebase, debugging sessions where the tests pass but the real target fails.

In manual development, these two phases are already imbalanced — but developers compensate by asking questions and building intuition over time. With AI agents, the imbalance becomes acute:

  • AI makes the cheap phase nearly free. An agent can synthesise a draft ADR from a conversation, enumerate edge cases from a spec, generate twenty test vectors from three examples. The cost of good upfront work collapses.
  • AI makes the expensive phase brutally repeatable. An agent will re-make the same wrong assumption across every file it touches, without fatigue, without doubt, and at speed that a human rework cycle cannot match.

Every team I speak with focuses on how fast AI can generate code. The real question is: how fast does it generate the wrong code, and how much does that cost to unwind?

Six Lessons From Practice

These patterns emerged from building AI-native systems in production. No project names — these failure modes are universal.

1. Stale documentation is worse than no documentation.

AI agents trust what they are given. A specification or configuration file that is out of date is not a minor inconvenience — it is a lie the agent will act on with full confidence.

Here is the scenario in concrete form. A service configuration file listed an API endpoint that had been migrated three months earlier. The agent read the config, generated correct authentication code against the old endpoint, and wrote unit tests against a mock of that same endpoint. All tests passed. The first real-target integration test failed immediately — the endpoint no longer existed. The code was well-structured, well-reviewed, and pointed at the wrong place.

The fix was not better documentation. It was a live orientation script that queries the running system at session start and reports its actual state — current endpoints, active service versions, real deployment targets. Not what the config says, but what the system is. When the truth and the docs disagree, source wins.

2. A green test suite that skips the real path is the most dangerous failure mode.

An AI agent generates tests that are logically consistent with the code it just wrote. If that code is subtly wrong — wrong interface, wrong data format, wrong assumption about state — the tests will be wrong in exactly the same way, and they will pass.

In one case, a suite of unit tests ran green for weeks while a real-target integration test revealed that the core authentication flow was broken. The unit tests had been written against a mock that did not reflect the actual behaviour of the service being called. Twenty-four passing tests, one broken user journey.

The discipline: at least one integration path must exercise the real system — not a mock, not a stub, not an in-memory stand-in. For mobile or embedded systems this means the device; for server-side services it means a real deployed environment. If the only thing standing between you and a production incident is a test that cannot fail because it never touches the real system, you do not have a test suite. You have expensive reassurance.

3. Architecture-first, with verification vectors, beats trial-and-error measurably.

Teams new to AI-native development often discover a seductive pattern: ask the agent to try something, see what it generates, iterate. This works for simple, well-understood problems. It does not work for complex systems where the design space is wide and the interactions are non-obvious.

On one workstream that involved a significant architectural decision — choosing between two cryptographic approaches with different performance, compatibility, and security tradeoffs — a week of trial-and-error generated working code that was architecturally wrong. The decision was re-made with an ADR that specified the approach, documented the tradeoffs, and included explicit test vectors for the behaviours that needed to be preserved. The same problem was then solved in a day.

The difference was not the speed of the agent. It was the clarity of the brief.

4. Two competing mental models will produce one incoherent codebase.

When a system’s architecture is documented in one place but partially implemented another way — through legacy code, an old design, or an unresolved debate — an AI agent will mix them. It has no way to know which model is canonical. It will draw on whichever is most prevalent in the context it is reading.

This manifests as code that is correct within each file but inconsistent across files. Refactoring it is expensive because no single change makes it coherent — you are not fixing a bug, you are reconciling two designs that were never properly resolved.

The corrective is to make the canonical architecture explicit, in one place, before the agent starts. If the architecture has changed, update the ADR before touching the code.

5. Silently-swallowed failures turn five-minute bugs into hour-long expeditions.

AI agents tend to generate optimistic code. They handle the happy path cleanly and often handle errors by catching them, logging a message, and continuing as if nothing happened. This is not a character flaw — it reflects the implicit instruction in most codebases, which is “make it work.”

The result is that a failure that should surface immediately — a missing file, an unreachable endpoint, a bad response — instead disappears into a log line and the system continues in a degraded state. When something eventually fails visibly, the root cause is several steps removed from the original error.

The discipline: treat silent error handling as a bug, not a feature. Errors should fail fast, loudly, and with enough context to be actioned. An agent that swallows a failure is not being robust — it is hiding a problem.

6. Small verifiable truths drift constantly, and someone must own their accuracy.

AI agents work from a snapshot of the system at the time of context loading. They do not know that the API endpoint changed last Tuesday, that the authentication token format was updated, or that the configuration directory was moved. These small truths — file paths, environment names, service endpoints, schema shapes — are individually trivial but collectively critical.

When they drift, the agent does not know. It continues operating against the old truth, generating plausible code that is quietly wrong. The errors are hard to diagnose because the code itself is correct — only the constants are wrong.

The practice that worked: a machine-readable source-of-truth catalogue, updated as part of every change, queried by the agent before it touches any integration point. Not a wiki. Not a README. A file that is wrong only if someone deliberately made it wrong.

What Actually Worked

These six failure modes are not theoretical. They were encountered, diagnosed, and resolved. The practices that resolved them share a common thread: they move uncertainty from the agent’s execution to the human’s upfront decisions.

Source-derived orientation over documentation. Before any agent session, run a script that queries the live system — build outputs, deployment targets, active configuration, real endpoints. Feed the results into the context. The agent works from truth, not memory.

Root-cause analysis before fix. When something fails, do not ask the agent to fix it. Ask it to explain why it failed, to enumerate what assumptions it made, and to identify which assumption was wrong. Fix the assumption first. Then fix the code. This prevents the same wrong assumption from being re-encoded in the fix.

Architecture-first for integration and security components. For any component that involves cryptography, authentication, or third-party integration, write the ADR before writing any code. Specify the algorithm, the format, the version, and the failure mode. The agent’s implementation is then a mechanical translation of a human decision, not an autonomous architectural choice.

Real-target verification as a hard gate. Unit tests are necessary but not sufficient. Before any feature is considered complete, it must pass on the real target — not a simulator, not a mock, not a local integration. For mobile teams this means a physical device; for backend teams it means a real deployed environment, not localhost. This gate is deliberately slow, which is exactly why it must be a gate rather than a best-effort check.

Machine-checked guards turning incidents into pre-push failures. Every incident that makes it to production contains a fact that could have been checked automatically before deployment. Build that check. A lint rule, a pre-commit hook, a CI validation step. Incidents are expensive. The check is free.

The Economic Case, Restated

AI-native development does not reduce the need for engineering discipline. It raises the stakes for it.

A disciplined team using AI will outperform a disciplined team without it. The cost of requirements, design, and verification stays roughly constant; the cost of implementation collapses. The economic leverage is real and significant.

An undisciplined team using AI will underperform an undisciplined team without it. The cost of rework multiplies because the agent encodes wrong decisions faster than a human team would, at greater scale, with greater consistency. The errors are not obvious — they are credible, well-structured, and fully tested against the wrong spec.

The teams that are already pulling away from the field are not the ones that gave their agents more autonomy. They are the ones that invested in being precise about what they wanted, before they asked for it.

The Right Mental Model

An AI agent is a brilliant contractor with no memory of yesterday and total faith in whatever you hand it.

Give it a clear blueprint and it builds fast, correctly, and at a cost that would have been unthinkable five years ago.

Give it a lie — a stale doc, an ambiguous requirement, an unresolved architectural question — and it builds that lie faithfully, completely, and at speed. Then it bills you to tear it down.

The brief is not overhead. The brief is the work.

Senthil Parameswaran is Managing Director at SKA Global Partners. SKA Global advises boards, CIOs, and technology leadership on AI-native transformation, software architecture, and technology strategy.