Jonathon's AI Wiki

Anthropic Cookbook — Coordinator-Pattern Cost (Big Models Plan, Small Models Execute)

7 min read

Source: raw/anthropic-watch-anthropic-cookbook-commit-a485090d4c3af1a59f6071aa13d388d48e0c9bf3.md, raw/anthropic-watch-anthropic-cookbook-commit-7aaa17fa217f27ff4b7d97fcfaba1ba75f328523.md, raw/anthropic-watch-anthropic-cookbook-commit-5d6d633b6a746e9302946d3ef5b82b99e99a1966.md, raw/anthropic-watch-anthropic-cookbook-commit-bbfab1bbbe5d4c353241a6df4e7d9a112a1ba356.md (2026-08-28 correction)

Cost and speed figures withdrawn (2026-08-28)

Anthropic removed the “roughly 2.5x cheaper and 3x faster” figures from this notebook on 2026-08-28 (cookbook commit bbfab1bb, PR #851) and reworded the matched-rigor caveat so it “no longer asserts a cost win”. The measured token split — 84–98% of the team’s input tokens billed at the worker rate — was kept. Every 2.5x / 3x figure below records what the notebook said before that date; do not cite it as a current result.

A first-party Anthropic cookbook notebook (added to anthropics/claude-cookbooks, merged to main via PR #754 on 2026-07-06) that puts a hard number on the managed_agents coordinator configuration: a frontier coordinator (Fable 5) plans and synthesizes while cheap workers (Sonnet 5) do the token-hungry reading in parallel threads. Its distinguishing feature is a rigor-matched comparison — the control isn’t a weaker product, it’s one frontier agent held to the exact same standard — so the originally reported “2.5x cheaper, 3x faster” was presented as an apples-to-apples result rather than a favorable framing (those figures were removed on 2026-08-28 — see the callout above). The notebook is unusually candid about the ways this measurement can be faked, which is the reason to read it.

Key Takeaways

  • The pattern. A tool-less frontier coordinator (Fable 5) plans the work and synthesizes the answer; cheap workers (Sonnet 5), scoped to web_search + web_fetch only, do the expensive reading in parallel threads.
  • Rigor-matched control. The comparison baseline is one frontier agent with the same tools, held to the same two-source verification standard, answering the same 20-fact coverage question. Both arms read a similar volume — so the delta is attributable to the architecture, not to one arm doing less work.
  • Headline result (withdrawn 2026-08-28). As first published, the notebook reported the team coming out 2.5x cheaper and 3x faster than the rigor-matched solo frontier agent at similar read volume. Anthropic has since removed those figures and the notebook no longer asserts a cost win. What it still reports is the token split: 84–98% of the team’s input tokens were billed at the worker rate.

Coordinator-pattern cost and speed ratio

Existing claim: (from anthropic-cookbook-coordinator-cost.md, compiled 2026-07-09 from commits a485090d / 7aaa17fa / 5d6d633b) — “On the committed run, the team came out 2.5x cheaper and 3x faster than the rigor-matched solo frontier agent at similar read volume.” New source says: (from raw/anthropic-watch-anthropic-cookbook-commit-bbfab1bbbe5d4c353241a6df4e7d9a112a1ba356.md, 2026-08-28) — “Drop the ‘roughly 2.5x cheaper and 3x faster’ figures from the intro and the results discussion, and reword the matched-rigor caveat so it no longer asserts a cost win. The measured token split (84-98% of the team’s input tokens at the worker rate) stays.” Status: unresolved

  • Trap (a) — the fake win. An unconstrained solo agent reads less and therefore only looks cheaper. That is a different product (less coverage), not a fair comparison. The whole point of the rigor match is to refuse this comparison.
  • Trap (b) — the per-thread floor. Over-narrow briefs spin up many tiny worker threads, and each thread pays a fixed floor cost. Slicing the work too finely pays that floor repeatedly and erodes the savings.
  • Trap (c) — the invisible team contract. The coordinator cannot see its roster agents’ prompts or descriptions. The team contract — what each worker is for, how to brief it — must live in the coordinator’s own prompt, not in the worker definitions.
  • Cost attribution is typed, not estimated. Use the per-thread usage rollups: sessions.threads.list → thread.usage for each worker; session.usage is the team total. The notebook also points to where per-request span detail lives.
  • Model IDs. validate_notebook.py’s model allowlist was extended with the current IDs (fable-5, sonnet-5, opus-4-8) so the notebook passes the repo’s audit check.

How the Coordinator Config Works

The architecture diagram (added in the internal-review-fixes commit) sits right after the intro and reads left to right: the recorded 20-fact question feeds a tool-less frontier coordinator, which fans out to ten scoped web-reading workers. Every arrow is two-way — workers return findings to the coordinator, which synthesizes and can re-dispatch.

  • Coordinator = brain. Fable 5. Plans the decomposition, holds the team contract, synthesizes the final answer. Has no reading tools of its own.
  • Workers = hands. Sonnet 5, each scoped to only web_search + web_fetch. They absorb the token-hungry reading that would otherwise be billed at the frontier rate.
  • Why it’s cheaper. The bulk of the tokens (reading source pages) are consumed by the cheap model in parallel; the expensive model is reserved for planning and synthesis. This is the same “big models plan, small models execute” lever documented for the advisor and orchestrator strategies, applied to managed_agents with measured cost attribution.

The Rigor-Matched Comparison

The notebook’s core methodological contribution is refusing the easy win. Both arms must:

  • Answer the same 20-fact coverage question.
  • Use the same tools (web_search + web_fetch).
  • Meet the same two-source verification standard (each fact confirmed against two sources).

Only then is cost compared. Because both arms read a similar volume of source material, the notebook originally attributed its 2.5x / 3x deltas to the division of labor, not a coverage shortcut. This is the guard against Trap (a). Since 2026-08-28 the notebook keeps the rigor match and the token split but no longer states a cost ratio or asserts a cost win.

Honest Caveats (from the internal-review commit)

The polish commit added review fixes to comments and prose only — the recorded outputs were left untouched — including a fourth honest caveat:

  • The committed run’s own “park list” is wrong at #10. The facts were verified, but the question decomposition was not — which the notebook calls its sharpest lesson. (The term “park list” is the source’s own wording.
  • Sonnet 5 introductory pricing. The PRICES comment now names the fact that Sonnet 5’s figures reflect an introductory pricing window — so the absolute cost ratio may shift once that window closes. (That window no longer closes: 10 is now permanent — see Open Questions.)
  • Tool scoping is a security boundary. Restricting workers to web_search + web_fetch isn’t only about cost — the comment now names the prompt-injection boundary that narrow tool scoping provides. A worker that can only search and fetch can’t be talked into doing anything more dangerous.

Open Questions

  • Why were the 2.5x / 3x figures removed? Commit bbfab1bb states what changed, not why. Whether the ratio failed to reproduce, depended on the Sonnet 5 introductory pricing, or was judged too task-specific is not stated.
  • Even before removal, the 2.5x / 3x figures were for the single committed 20-fact research question; the notebook did not claim they generalize to other task shapes.
  • The absolute cost advantage depends on the Sonnet 5 introductory pricing window remaining open. Superseded: the claude-api skill states “Sonnet 5 pricing is the permanent 10 list price” (skills commit 53048666, 2026-09-01); see Claude Sonnet 5.

Try It

  • Open the notebook in anthropics/claude-cookbooks under the managed_agents directory and run it against your own coverage question.
  • Copy the rigor match, not just the pattern. Before trusting any “multi-agent is cheaper” claim (your own included), pin the solo baseline to the same tools, the same verification standard, and the same question. If the solo agent is allowed to read less, you’re measuring the wrong thing.
  • Put the team contract in the coordinator’s prompt, never only in the worker definitions — the coordinator can’t see them.
  • Attribute cost with typed rollups: iterate sessions.threads.list → thread.usage per worker and reconcile against session.usage (the team total) rather than eyeballing spend.
  • Right-size the briefs. Watch for the per-thread floor cost — if you’re spawning many tiny workers, consolidate until the reading-per-thread justifies the thread.

Graph View

Backlinks