Shipping production code with coding agents, and the harnesses that keep it honest

The honest baseline

Coding agents used without discipline introduce defects faster. That was the starting point I gave the developers at Grant ApS, and it is the assumption I work under myself.

In 2026 I delivered production code on two engagements: the end-to-end test layer for Nominal Systems' open-source DMI platform, and the AI service behind Grant ApS' grant-management product GrantOne. On both, most of the code I delivered was written by Claude Code agents. That did not make the engineering smaller. It moved it into the environment the agents work against: the instructions they read, the review every change must pass and the tests that decide whether a change is correct.

Guardrails before speed

The working norms are deliberately plain. Each repository carries an agent instruction file, so conventions are written down once rather than repeated in every session. Larger pieces of work get a branch roadmap, and sessions are archived so the reasoning behind a change can be traced afterwards.

The gate is just as plain. On GrantOne, no change landed without a human review of the diff and a full regression run. The agent proposes; the tests and I decide.

Blind pre-merge review

Reviewing a diff written by an agent has a trap: the diff and its description frame what you look for. If the agent missed a place that should have changed, nothing in the diff points to it.

So at Nominal Systems I introduced a blind pre-merge review for AI-authored pull requests. A fresh agent reads the task and freezes a written prediction of what the change should touch, before it sees the diff. Only then does it grade the diff against that prediction.

On its first use it caught a missed seventh change site and a wrong claim in the PR text. It has since become the standard pre-merge step, and I run it on every PR.

End-to-end harnesses as ground truth

Review catches intent errors. It does not prove the system works. For that I rely on end-to-end harnesses that run the real thing.

DMI. DMI is an open-source API that connects veterinary practice software to diagnostic labs and in-clinic analysers, run as one shared platform with 1,000+ connected organisations. I built its first real-services harness, dmi-e2e, and its first CI test workflow. The harness is black-box: it runs the real API, MySQL, MongoDB and the MQTT broker in Docker against mocked provider APIs.

On top of it sit five full order-to-result-to-report loops, one per lab-provider integration. Each has its own CI workflow, every suite runs nightly, and parallel harness slots let suites run side by side.

A few rules keep the harness honest:

• Probe before you mock. Before building a provider's mock, I ran live-endpoint probe rounds against its development API. The harness then proves the real contract, not the integration's own assumptions.
• Tripwires, then upstream fixes. Each confirmed defect becomes a failing-marked test, then an issue, then a fix merged upstream. Eleven tripwires were live at peak, and the tests that caught the defects stay on as regression guards.
• Every assertion must be able to fail. A long-green suite was shown to miss 6 of 10 deliberate breaks. Since then, mocks and fixtures are deliberately mutated before merge to prove each assertion catches what it claims to.

The harness found real defects in production code. I fixed them upstream in the API and the provider integrations, including credential redaction in stored provider requests, species and breed mapping, timeout and transport-error handling, and result-set isolation.

GrantOne. The grant-management service handles applications full of sensitive personal data, so its fixtures are synthetic and PII-free. The safety net is 644 automated tests plus 80 live end-to-end scenarios over 130+ synthetic fixtures. It is what makes a prompt or model change visible before it reaches a caseworker.

The same discipline applied to the model itself. Replaying a corpus of already-granted applications cut false automatic rejections from 24 to 0 and manual escalations from 28 to 4.

I also scoped a browser-test proof of concept spanning the applicant portal, the backend API and the caseworker board, to catch cross-application regressions that per-repository tests cannot see.

Scoped AI bug hunting

Asking an agent to "find bugs" produces a long list of plausible noise. The method I developed at Grant ApS has three rules:

1. Scope each hunt to one bug class.
2. Require a concrete, reachable path as evidence for every finding.
3. Adversarially verify every finding before a human sees it.

On the production codebase, 55 raw candidates became 10 confirmed findings, which the team could triage in 20 minutes.

Sharing the method

At Grant ApS I designed and ran a hands-on workshop introducing coding agents to the in-house developers. It started from the honest baseline above and ended with the norms the team now uses: repo-level agent instruction files, branch roadmaps and archived sessions.

At Nominal Systems I built the team's shared Claude Code plugin: skills, conventions, a knowledge layout and a daily brief posted to Teams. The method then lives in the tooling rather than in one person's habits.

What this adds up to

Coding agents write code quickly. Whether that code is correct is decided elsewhere: in written guardrails, in review that does not trust the diff, and in end-to-end tests that run the real services and have been proven able to fail. That is where I put the engineering effort, and it is what made agent-written code safe to ship on a platform with 1,000+ connected organisations and on a service screening sensitive applications in production.

For the earlier groundwork on testing non-deterministic AI behaviour, see Why 'AI unit tests' are non-negotiable for enterprise-grade AI.