The Agent Factory

Between July 10 and August 2, 2026, an agentic pipeline planned and shipped the first 20 production stories of a real product — 49 by this hub’s September 4 refresh, with the line still running. My role was not “programmer.” It was closer to plant manager: approve plans, review what the machines couldn’t judge, arbitrate deviations, and read the gauges.

The gauges are the point. Anyone can tell you agents wrote their code. Almost nobody can tell you what a story cost, how many attempts it burned, or whether the tests that went green actually proved anything — because almost nobody measured. I did, and some of what the telemetry revealed was uncomfortable, starting with the fact that the telemetry itself lied to me until I forced it to fail visibly.

This is the hub for a series about that experiment — six posts to begin with, eight now, and still growing with the factory. It’s written for people deciding whether — and how — to trust agentic delivery, not just for people who enjoy watching agents work.

The Product

A factory needs to build something real, with real users who complain when it breaks. Mine is Azeroth Pocket: a mobile-first Progressive Web App that lets my family play World of Warcraft: Wrath of the Lich King — on our own private AzerothCore server — from a phone on the couch. Not a port of the desktop client: a thumb-first instrument panel for an existing world. Chat with the guild in portrait; rotate to landscape and pilot your character with a virtual joystick and a radar.

Chat is live today. The next milestone is fighting and questing through that radar view; a real 3D viewport is a post-V1 ambition, deliberately deferred.

Two things worth stating plainly. First, the legal posture: no Blizzard asset ever touches the project’s infrastructure. The app ships with original placeholder art, and icons are extracted client-side from the player’s own game files into the browser’s private storage. Second, this isn’t a toy problem: the stack spans a browser client, a WebSocket-to-TCP gateway speaking the 2008 WoW wire protocol (SRP6 authentication, RC4 header crypto), and an identity bridge wired into Keycloak behind Cloudflare Access — running on the same Raspberry Pi that hosts everything else in my homelab. If you read my earlier piece on building a PWA with push notifications, this is that interest taken to its logical extreme.

What I Mean by “Factory”

I surveyed the tooling landscape in Agentic Coding Systems Landscape 2026; this project is what happened when I stopped surveying and committed. The factory has three parts:

1. A planning line. The full BMad-METHOD corpus — brief, PRD, UX, architecture, epics, stories — produced by agents, stress-tested by adversarial review passes, and ratified by me. The entire planning phase metered at $177.80.
2. A production line. Unattended dev iterations with Claude Code: pick a ready story, implement, test, review, halt with a machine-readable status. I intervene at defined checkpoints, not continuously.
3. A quality department that assumes the workers cheat. The behavioral reference is the real desktop game client — an oracle the agent’s output is diffed against, packet by packet. Three gate layers stand between code and merge, the last one run by CI precisely because an agent grading its own homework will occasionally be generous. Any approved difference from the oracle lives in a signed deviation register; an unmatched difference blocks the merge.

The premise of the whole series: replace “trust the agent” with mechanisms. Where that worked, I can show it working. Where it failed — vacuous tests, defects only production could reveal, gauges that read healthy while wrong — I can show that too, because it’s all in the same metrics store.

Measured, Not Remembered

Retrospectives about AI-assisted development are usually written from memory, and memory is a flattering narrator. So this series follows one editorial rule: numbers I can’t regenerate are numbers I can’t publish. Every figure traces to a query against the project’s own telemetry — and where the data doesn’t exist, the posts say not measured rather than estimating.

A note on freshness: these figures were first taken August 2, 2026, refreshed August 10, and refreshed again September 4 — and the factory hasn’t stopped. Treat this hub as live documentation: totals get refreshed as later epics ship, and the series may grow beyond the eight posts below. On August 2 the factory’s instrumentation itself got an overhaul — session-level cost attribution, genuine active-time tracking, test-suite health, bundle-size trends — and the refresh carries that pipeline’s first week of real data: the new stories above have exact session counts, honest active-time figures, and machine-readable review tallies (post 1’s second reading has the details). None of it backfills, which means later posts can answer questions (cost per line, real hours per story) that the first three epics structurally cannot. Where a chart spans that boundary, it will say so.

The project’s queries.md — the exact queries and the reset-aware aggregation script behind every published figure — ships with the repo when it goes public. Until then, each post inlines the queries behind its own numbers, like the provenance notes above.

The Series

Six posts to start, one every two weeks, each standing on its own — and two more since, because the factory kept teaching:

1. What it cost — the full bill, story by story: $2,479 of metered compute for the first 20 stories, the model policy as measured rather than as remembered, cache economics, the meta-finding that the telemetry itself lied (dead gauges, a 2.6× undercount) until it was driven red — plus a dated second reading from the factory’s next week. First out, two weeks after this hub.
2. Planning a product with an agent factory — the BMad phases, the adversarial review passes that reversed real decisions, and what a complete planning corpus costs when agents write it: $177.80.
3. The oracle is the spec — the verification model: behavioral oracles, gate classes, the three-layer gate stack, and the deviation register that turns “close enough” into a signed, machine-checkable artifact.
4. Greens that prove nothing — the vacuous-guard problem: tests that pass because they test nothing, how they slipped past review, and the harness changes that made emptiness itself a failure.
5. What only broke live — the defect class no repository test can see: CDNs caching your service worker, credential-less manifest fetches, browsers quietly retiring install UI.
6. Governance in regulated environments — translating the factory’s mechanisms (deviation registers, ceremony budgets, procedure exceptions) into the language of organizations that answer to auditors.
7. The factory learns — the register for friction in the factory’s own machinery: agents propose a change, a human-governed register decides owner, trigger and scope; why repeated vigilance is evidence of missing tooling; and what happened when this blog interviewed the factory.
8. The testing spine — which verification rungs actually find defects, at what cost, and what each cannot see, measured per story and per phase since every session started saying who it was; plus the instrument that had been under-counting the tester for five stories.

The deep-engineering middle (posts 3–5 and 8) is where practitioners should linger; posts 1, 2, 6 and 7 are the ones to forward to whoever owns your engineering budget.

Final Thoughts

I went into this expecting to learn whether agents could build a real product. That question turned out to be the boring one — they can, and by post 1 you’ll see exactly what it costs. The interesting questions are the ones factories have always had to answer: how do you verify work you didn’t do yourself, how do you know your instruments aren’t lying, and how much ceremony is worth its price?

Start with What it cost. And if you only remember one thing from this hub, make it the editorial rule: numbers I can’t regenerate are numbers I can’t publish.