Deciding and Checking Cost More Than Building

There’s a question I should have been able to answer months ago, and it’s the first one anyone with a budget asks: of the roles in this factory — the analyst who plans a story, the developer who builds it, the tester who plays it, the reviewer who refuses to believe it — which costs the most, and which takes the longest?

For five epics the honest answer was not measured. Every story had one cost lane. Planning dissolved into a no-issue bucket, review rode on the developer’s meter, and the tester didn’t exist yet. On August 22 every session started carrying two labels, the phase it was working and the story it belonged to, and nothing was backfilled. Epic 6 — combat, loot, death, buffs, the whole evening loop of the game — is therefore the first epic this project can state end to end rather than estimate. Nineteen stories, the last one merged on September 21.

The short answer: building costs the most and takes the longest, and that is the least interesting thing in the table. Development is 42% of the bill. The two phases that write no product code, planning and review, are 52% between them. In this factory, deciding and checking cost more than building, and they have done so in every snapshot since the labels existed.

(Series rules apply: running totals, never final ones. Every figure is metered at API list rates; the work ran on a ~$100/month subscription plus credits, so real cash is roughly 10× lower. Where an instrument doesn’t exist, the cell says not measured.)

The Bill, by Phase

• Share of cost
• Share of wall-clock
• Share of active hours

Plan

17.1 %

9.4 %

17.2 %

Dev

41.6 %

58.0 %

42.1 %

QA agent

5.6 %

11.3 %

12.6 %

Review

34.6 %

19.9 %

26.0 %

Epic prep

1.1 %

1.4 %

2.1 %

Shares of $13,716 · 447 wall-clock hours · 285 active hours. Shares rather than raw values, because dollars and hours don't belong on one axis. September 21 reading, API list rates.

Table view

	Share of cost	Share of wall-clock	Share of active hours
Plan	17.1 %	9.4 %	17.2 %
Dev	41.6 %	58.0 %	42.1 %
QA agent	5.6 %	11.3 %	12.6 %
Review	34.6 %	19.9 %	26.0 %
Epic prep	1.1 %	1.4 %	2.1 %

Read the three bars per phase as a sentence. Dev is 58% of the elapsed time but only 42% of the cost and of the active hours: development sessions stay open a long time. Review is the inverse — a fifth of the elapsed time, over a third of the money. The QA agent is the cheapest phase by a wide margin: about an eighteenth of the bill. Planning is a sixth — and on four of the nineteen stories, planning cost more than the development it planned.

Who Are These Agents, Actually?

I asked the question in terms of roles — analyst, developer, tester — and the first thing the product repo’s own session told me, when I interviewed it for this post, was that the framing doesn’t survive contact with the launcher scripts. The factory’s own documents call the label “the persona axis”, and that’s loose talk. What the label records is a phase, stamped by whichever launcher started the session.

Phase	What actually runs	Who that is
Plan	the story-creation workflow, plus a fresh-context validation pass	the developer persona’s menu
Dev	the story-implementation workflow	the developer persona
Review	the code-review workflow, in a separate session pinned to the strongest model	the same persona — deliberately with no memory of having written the code
QA	a lean tester brief and a running build	no persona at all; the launcher’s own comment reads “no tester persona”

There is no business analyst in the story loop. The analyst persona did its work in the planning corpus post 2 covered, before the labels existed. “Plan” here means turning an epic’s line item into a story a developer can’t misread.

So here is the role-shaped answer, offered as an analogy and not a measurement. If you think in a classic team: the story-writer is 17% of the bill, the tester is 6%, and the developer is 76% — because the developer shows up twice, once to build (42%) and once, as a stranger to their own code, to review (35%). The design principle behind that second appearance is one line in the factory’s own rules: a reviewer with no memory of writing the code, cross-instance, not cross-model.

Two Clocks, and Neither Is “Hours of Work”

“Which one takes the most time” has two answers, because the factory keeps two clocks, and I’d rather publish both with their disagreement than pick the flattering one.

Phase	Wall-clock	Active hours	Active / wall	Median story (wall)	Longest (wall)
Plan	42 h	49	over 100% — see below	1.3 h	15.1 h
Dev	259 h	120	46%	13.0 h	49.7 h
QA agent	51 h	36	71%	0.6 h	13.8 h
Review	89 h	74	83%	2.1 h	17.9 h
Epic prep	6 h	6	over 100%	—	—
Epic 6	447 h	285	64%

Development wins on both clocks. Wall-clock is last telemetry sample minus first, per session: it includes every minute a session sat open. Active hours counts the hour-buckets in which a session produced any traffic: it drops the idle nights, but a forty-minute session that straddles the top of the hour counts as two. That over-count is why planning and prep show more active hours than wall-clock hours — short sessions, many boundaries. If I had to headline one, it’s active hours, with that caveat attached; neither column is hours of work.

The ratio between them is the useful number. Across the epic, 64%: a little over a third of all elapsed session time was a session holding its context with no traffic. Development is lowest at 46%, review highest at 83% — which is what a phase that reads a fixed diff and then stops ought to look like. I’d love to tell you dev sessions idle because they wait on a gate, a deploy or me, and the dated facts file this post draws on said exactly that. It’s an inference. Telemetry can’t tell a session waiting on CI from one left open overnight, and the interviewed session withdrew the sentence when I asked how it knew.

One clock doesn’t exist at all. My own time is not measured — the approvals, the rulings, the evenings walking a build on a phone. Two proxies exist and neither is a duration: the review workflow staged 191 questions for me across sixteen of the stories (thirty on the last one alone), which counts rulings requested, not minutes; and the final story’s walk bounds about an hour of app time from its own latency telemetry, which is play time, not effort. The factory’s instruments record what the agents did. A number nobody counted is worse than a question mark, so: question mark.

Nineteen Stories

• Plan
• Dev
• QA agent
• Review

6.1 Combat domain

$675

6.2 Plates

$904

6.3 Context layout

$1,266

6.9 HUD rework

$1,008

6.10 The instruments lie

$217

6.4 Casting

$964

6.11 Create a character

$468

6.15 Ground everywhere ¹

$531

6.5 Engage

$1,671

6.12 The holster

$466

6.6 Loot and consume

$1,477

6.16 The loot capture

$247

6.17 The mount key

$184

6.13 Auras and buff tiles

$1,050

6.7 Death and recovery

$818

6.19 Proportional buff drain

$217

6.14 Spell static data

$629

6.18 The aura panel

$313

6.8 The daily loop proven

$465

A 2px tick is a real zero: no QA session ran on that story. ¹ 6.15's planning session carried no label, so its plan is not measured and its bar is a lower bound.

Table view

	Plan	Dev	QA agent	Review	Total
6.1 Combat domain	$325	$174	$0	$176	$675
6.2 Plates	$113	$465	$52	$274	$904
6.3 Context layout	$111	$555	$92	$507	$1,266
6.9 HUD rework	$55	$543	$52	$358	$1,008
6.10 The instruments lie	$65	$69	$0	$83	$217
6.4 Casting	$153	$420	$52	$339	$964
6.11 Create a character	$64	$218	$33	$152	$468
6.15 Ground everywhere ¹	not measured	$190	$61	$280	$531 (lower bound)
6.5 Engage	$230	$666	$42	$733	$1,671
6.12 The holster	$59	$195	$33	$179	$466
6.6 Loot and consume	$295	$514	$179	$489	$1,477
6.16 The loot capture	$72	$51	$0	$124	$247
6.17 The mount key	$78	$32	$15	$59	$184
6.13 Auras and buff tiles	$265	$572	$30	$183	$1,050
6.7 Death and recovery	$95	$410	$84	$229	$818
6.19 Proportional buff drain	$58	$78	$14	$66	$217
6.14 Spell static data	$95	$281	$0	$254	$629
6.18 The aura panel	$128	$109	$0	$76	$313
6.8 The daily loop proven	$88	$165	$31	$182	$465

Three things I didn’t expect, all visible in that chart.

Review out-cost development on seven of nineteen stories. Not on the small ones only: the most expensive story the project has recorded, Engage at $1,671, spent $733 on review against $666 on development. That one review cost more than eleven of the epic’s nineteen complete stories. (The facts file said eight; when I asked which basis gives eight, the answer was none — it is a slip, please quote the correction. So: quoted.)

Diff size predicts nothing. The two largest diffs in the epic are +111,699 and +85,528 lines. The first is a packet capture converted to fixtures, and the story cost $247 — the fourth-cheapest of the nineteen. The second is dominated by generated codec and faction tables. Lines changed has never been a cost driver in this project, and if your pipeline’s dashboards lead with it, they’re leading with noise.

The bill falls across the epic, and I can’t call it learning. The first nine stories have a median of $904, the last ten $466, and the rank correlation between merge position and cost is −0.36. But four of those last ten are narrow follow-ups appended at a course-correction — a capture, a single key, a panel, a drain rule. “Stories got cheaper” and “stories got smaller” aren’t separable on nineteen points. I’d want a second instrumented epic before claiming the factory is getting cheaper at anything.

Why Review Burns the Meter

Review is a fifth of the clock and a third of the money, and the token counters say why. Three measured facts, Epic 6 only:

• It is all on the strongest model, by policy. The review launcher pins it. Planning is the only phase with a model mix, which matters later in this post.
• It isn’t repeated reviews. Twenty review sessions for nineteen stories.
• It writes more than development does: 19.4 million output tokens against 15.2 million. It also creates the most cache (110 million tokens against 91 million) and reads 29× development’s uncached input.

That last signature is what a fan-out looks like. A review here is not one reader; it’s an orchestrator dispatching reviewing subagents, each starting from a fresh context — sixteen on one story, eight layers plus a ninth over the fix delta on another, four in isolated worktrees on the last. I’m inferring the link: subagent count isn’t a telemetry label, it’s prose in the story files. But post 7 already carried the register item that names the cost — subagents re-deriving the same context sixteen times — and it’s still open, ranked last on purpose, because sixteen independent readings is part of why the layers find different things.

I don’t read 35% as waste, and I’d be suspicious of anyone who did without looking at what the checking found. Across the epic, review filed 682 findings and 531 of them were fixed before merge. The QA agent filed 87 more on an eighteenth of the budget. Post 8 has the per-instrument view — including why I won’t publish a cost-per-finding ranking even though the division is easy: a review finding that gets dismissed as noise and a behavioral defect reproduced on a running build are not the same unit, and a ratio built on them would rank the instruments backwards.

What I do read from it is where optimization belongs. Shaving development’s 42% is the intuitive move and the wrong one. The expensive, fast-burning, widest-variance phase is review, and it remains the least instrumented machine in the building.

Structural, Not Drift

The headline ratio deserves its own paragraph, partly because the factory got it wrong first. The facts file originally called September 21 “the first snapshot where” planning plus review exceeded development. It isn’t. The session that wrote that sentence re-derived the earlier snapshots, found the ratio above one in all of them, and corrected itself in print — the wrong sentence quoted, not deleted, under the rule this series applies to itself.

The correction makes the claim stronger. A ratio that climbs is a trend you can argue with. A ratio that reads 1.56×, 1.25×, 1.28× from the first day it could be measured is the shape of the thing. This is what a factory built on distrusting its workers costs: for every dollar of building, a dollar and a quarter of deciding what to build and checking that it was built.

What the Money Bought

A bill without a product is just a bill. Epic 6 closed when its last story — literally titled The Daily Loop Proven — was walked on the production realm, from a phone, unassisted: sign in, enter the world, fight, loot, eat and drink, talk to a quest giver, accept, complete, hand in, die and recover. The same walk measured the one product number the requirements actually commit to, the player’s own action latency:

Network	Samples	p50	p95	Max
Wi-Fi, the whole loop	598	33 ms	58 ms	2,006 ms
Cellular: a fight, loot and a meal	47	79 ms	163 ms	225 ms

The requirement is p95 under 300 ms. Wi-Fi clears it by about 5×, cellular by about 1.8×, and cellular costs roughly 2.8× the Wi-Fi latency — a fact no earlier milestone could supply, because the earlier one measured a single action for 68 seconds on Wi-Fi. My own note from the walk was shorter: nothing felt slow. This is one player on one evening; it says nothing about the realm under load, and the four-player reading is a deferral I’ve ruled on rather than forgotten.

The loop is proven. V1 is not declared, and that distinction is deliberate: a proven loop is a measurement, a release is a decision. Talents, professions, selling and the 3D view are all still ahead — at least one more epic, probably more — and I closed this one intending to overhaul the factory itself first.

What Isn’t in the Bill

Five holes, named rather than filled. The first one is the reason this post’s numbers changed between drafts.

• $182 that the query threw away. The facts file published Epic 6 at $13,534. While answering my questions, the session that wrote it noticed its cost recipe grouped by session but not by model — so a session that switched models mid-run was billed for its most expensive model only. Planning sessions are the ones that switch, so planning was low by 8.4% and everything else by pennies. Corrected: $13,716. I re-ran both recipes myself and both reproduce to the cent, which is the only reason either number is in this post. The same recipe fed the two previous snapshots, so the series’ cumulative total moved too: $23,478 became $23,661 across 60 completed stories. Post 10 is about why this keeps happening, and why it always errs low.
• One planning session. Story 6.15 was planned from a plain shell instead of the planning launcher, so no labelled series exists for it. That’s a measured gap, not $0; the plan lane is a lower bound twice over.
• The retrospective. It hasn’t run, so its lane doesn’t exist yet.
• Me. Above.
• Everything before August 22. The labelled era holds $15,162; the cumulative across all 60 stories is $23,661. The first is a subset of the second by construction — it cannot see stories that merged before the labels existed — and reading the smaller number as “spend falling” would be exactly the kind of wrong sentence post 1 was written to prevent. A third, unaudited whole-era sum reconciles with neither and is not a headline.

Regenerate This

Cost per phase per story is one query, pinned to the snapshot’s evaluation time. Two details are load-bearing. max_over_time, because a resumed session keeps its id and restarts its counter at zero. And model in the inner grouping, because a session’s series splits by model and the outer max would otherwise keep one of them:

sum by (story, phase) (
  max by (story, phase, session_id, model) (
    max_over_time(claude_code_cost_usage{project="pwa-wotlk", phase!=""}[120d])))

Wall-clock wants the opposite treatment — collapse the models, and take the earliest first-sample per session:

sum by (story, phase) (
  (max by (story, phase, session_id) (tlast_over_time(claude_code_cost_usage{project="pwa-wotlk", phase!=""}[120d]))
 - min by (story, phase, session_id) (tfirst_over_time(claude_code_cost_usage{project="pwa-wotlk", phase!=""}[120d]))) / 3600)

Active hours counts hour-buckets with any traffic, over a subquery window checked to be non-truncating (45 and 60 days return the same 285). Token totals per phase are the interviewed session’s reading of the token counters, grouped the same way. The story count is a grep for done rows in the sprint tracker. Player latency is the app’s own action-latency histogram, scoped to the run ids of the evening — several of them, because a death, a backgrounding and a reconnect each mint a new one. All recipes ship with the product repo’s query file when it goes public, beside the dated delta and the interview this post is drawn from.

Final Thoughts

If you own a budget and someone shows you a business case for agentic delivery that prices code generation, ask what it prices the other 58% at. In the one epic I can state end to end, building was the largest single line and still the minority of the bill. The planning that decides what a story is and the review that refuses to believe it’s done together cost more — every time I’ve been able to look.

And if you run one of these pipelines: label the sessions. One label for which phase and one for which story, stamped by the launcher, not remembered by the session. It’s an afternoon of work, it backfills nothing, and until you do it every question in this post has the same answer mine had for five epics.

The hub carries the freshest totals. When the retrospective runs, this post gets a dated second reading — with the lane it’s currently missing. Next: why every one of this factory’s measurement errors has pointed the same way. That’s Failing in the Safe Direction, in two weeks.