The Agent Factory

Between July 10 and August 2, 2026, an agentic pipeline planned and shipped 20 production stories of a real product. My role was not “programmer.” It was closer to plant manager: approve plans, review what the machines couldn’t judge, arbitrate deviations, and read the gauges.

The gauges are the point. Anyone can tell you agents wrote their code. Almost nobody can tell you what a story cost, how many attempts it burned, or whether the tests that went green actually proved anything — because almost nobody measured. I did, and some of what the telemetry revealed was uncomfortable, starting with the fact that the telemetry itself lied to me until I forced it to fail visibly.

This is the hub for a six-post series about that experiment. It’s written for people deciding whether — and how — to trust agentic delivery, not just for people who enjoy watching agents work.

The Product

A factory needs to build something real, with real users who complain when it breaks. Mine is Azeroth Pocket: a mobile-first Progressive Web App that lets my family play World of Warcraft: Wrath of the Lich King — on our own private AzerothCore server — from a phone on the couch. Not a port of the desktop client: a thumb-first instrument panel for an existing world. Chat with the guild in portrait; rotate to landscape and pilot your character with a virtual joystick and a radar.

Chat is live today. The next milestone is fighting and questing through that radar view; a real 3D viewport is a post-V1 ambition, deliberately deferred.

Two things worth stating plainly. First, the legal posture: no Blizzard asset ever touches the project’s infrastructure. The app ships with original placeholder art, and icons are extracted client-side from the player’s own game files into the browser’s private storage. Second, this isn’t a toy problem: the stack spans a browser client, a WebSocket-to-TCP gateway speaking the 2008 WoW wire protocol (SRP6 authentication, RC4 header crypto), and an identity bridge wired into Keycloak behind Cloudflare Access — running on the same Raspberry Pi that hosts everything else in my homelab. If you read my earlier piece on building a PWA with push notifications, this is that interest taken to its logical extreme.

What I Mean by “Factory”

I surveyed the tooling landscape in Agentic Coding Systems Landscape 2026; this project is what happened when I stopped surveying and committed. The factory has three parts:

1. A planning line. The full BMad-METHOD corpus — brief, PRD, UX, architecture, epics, stories — produced by agents, stress-tested by adversarial review passes, and ratified by me. The entire planning phase metered at $177.80.
2. A production line. Unattended dev iterations with Claude Code: pick a ready story, implement, test, review, halt with a machine-readable status. I intervene at defined checkpoints, not continuously.
3. A quality department that assumes the workers cheat. The behavioral reference is the real desktop game client — an oracle the agent’s output is diffed against, packet by packet. Three gate layers stand between code and merge, the last one run by CI precisely because an agent grading its own homework will occasionally be generous. Any approved difference from the oracle lives in a signed deviation register; an unmatched difference blocks the merge.

The premise of the whole series: replace “trust the agent” with mechanisms. Where that worked, I can show it working. Where it failed — vacuous tests, defects only production could reveal, gauges that read healthy while wrong — I can show that too, because it’s all in the same metrics store.

Measured, Not Remembered

Retrospectives about AI-assisted development are usually written from memory, and memory is a flattering narrator. So this series follows one editorial rule: numbers I can’t regenerate are numbers I can’t publish. Every figure traces to a query against the project’s own telemetry — and where the data doesn’t exist, the posts say not measured rather than estimating.

A note on freshness: these figures are a snapshot taken August 2, 2026 — and the factory hasn’t stopped. Treat this hub as live documentation: totals will be refreshed as later epics ship, and the series may grow beyond the six posts below. That same day, the factory’s instrumentation itself got an overhaul — session-level cost attribution, genuine time-on-task, test-suite health, bundle-size trends. None of it backfills, which means the later posts can answer questions (cost per line, real hours per story) that the first three epics structurally cannot. Where a chart spans that boundary, it will say so.

The project’s queries.md — the exact queries and the reset-aware aggregation script behind every published figure — ships with the repo when it goes public. Until then, each post inlines the queries behind its own numbers, like the provenance notes above.

The Series

Six posts to start, one every two weeks, each standing on its own:

1. What it cost — the full bill, story by story: $2,479 of metered compute for 20 stories, the model policy as measured rather than as remembered, cache economics — and the meta-finding that the telemetry itself lied (dead gauges, a 2.6× undercount) until it was driven red. Ships alongside this hub.
2. Planning a product with an agent factory — the BMad phases, the adversarial review passes that reversed real decisions, and what a complete planning corpus costs when agents write it: $177.80.
3. The oracle is the spec — the verification model: behavioral oracles, gate classes, the three-layer gate stack, and the deviation register that turns “close enough” into a signed, machine-checkable artifact.
4. Greens that prove nothing — the vacuous-guard problem: tests that pass because they test nothing, how they slipped past review, and the harness changes that made emptiness itself a failure.
5. What only broke live — the defect class no repository test can see: CDNs caching your service worker, credential-less manifest fetches, browsers quietly retiring install UI.
6. Governance in regulated environments — translating the factory’s mechanisms (deviation registers, ceremony budgets, procedure exceptions) into the language of organizations that answer to auditors.

The deep-engineering middle (posts 3–5) is where practitioners should linger; posts 1, 2 and 6 are the ones to forward to whoever owns your engineering budget.

Final Thoughts

I went into this expecting to learn whether agents could build a real product. That question turned out to be the boring one — they can, and by post 1 you’ll see exactly what it costs. The interesting questions are the ones factories have always had to answer: how do you verify work you didn’t do yourself, how do you know your instruments aren’t lying, and how much ceremony is worth its price?

Start with What it cost. And if you only remember one thing from this hub, make it the editorial rule: numbers I can’t regenerate are numbers I can’t publish.