Source: wiki synthesis: Tulsee Doshi (Google DeepMind), Vijay Raji (OpenAI), Muse and the Case for Another Swing (Nate B Jones), Andrew Ambrosino (OpenAI Codex), Canva AI 2.0, Factory, Maintain the Harness, How the Claude Code Team Uses Claude Code, Boris Cherny on Lenny’s Podcast, Boris Cherny at AI Ascent 2026, Engineering Leverage in the Agent Era, The New Rules of Context Engineering for Claude 5, Prompting Claude Opus 5.5, Prompting Claude Fable 5, The Capability Curve, The Expanding Toolkit, Code with Claude London 2026 Keynote, The Detox Skill, The 2026 Claude Code AIOS Pattern, Where You Inject an Instruction Decides Whether It Survives
Product leads at Google DeepMind and OpenAI, the teams behind Canva AI, Factory and Claude Code, Anthropic’s own prompting guides, and Nate B Jones writing about Meta’s Muse all make the same claim: plan what you build around the model that ships next, and treat whatever you added to cover the current model’s weaknesses as temporary. The claim has two halves.^[inferred] Build ahead: ship what works now and keep the ambitious version ready for the next release. Delete behind: after each upgrade, re-test your prompts, skills and harness, and remove what the new model no longer needs.
Key Takeaways
- One claim, three scopes. Product leads (Doshi, Ambrosino, Raji, Boris Cherny, Jones) apply it to what to build. Canva, Factory and the Claude Code team apply it to harness code. Anthropic’s guides and talks (Thariq Shihipar, Jeremy, Lucas, Lisa, the Fable 5 and Opus 5.5 guides) apply it to prompt and context text.
- The only measured number is a deletion. Anthropic removed over 80% of Claude Code’s system prompt for Opus 5 and Fable 5 “with no measurable loss on our coding evaluations.” Every other figure in this article is a speaker’s anecdote or estimate.
- Speakers give the shelf life in months. Lucas: code that compensates for model unreliability has “a half-life of just months.” The Claude Code team: the technology “shifts underneath you every two months.” Factory’s founder: “Every month, we need to rethink.” Canva: about three harness rewrites in three years.
- One rule makes the sources agree. Lucas’s split: code that compensates for model weakness expires; code and context that connect the model to your world compound. Canva’s durable evals, Jones’s maintained harness and Boris Cherny’s encoded team knowledge all fall on the connecting side.
- Scaffolding changes shape; it does not only shrink. The same Anthropic guides that tell you to delete lines also add new ones for new behaviour (elapsed-time budgets, progress grounding, memory files), and Canva’s harness grew from a handful of tools to hundreds. The instruction is to re-test after each upgrade, not to minimise.
- Building too far ahead also costs. Boris says Claude Code had poor product-market fit for about six months. Ambrosino says Codex was “too AGI-pilled for the moment.” Factory started in April 2023, when “the models weren’t really there yet.” Doshi’s 60/20/20 rule hedges this by keeping about 60% working today.
- For an operator it becomes a routine: on each release, re-run a small eval of real tasks, run once with skills off, delete what made no difference, and re-try the automations you shelved.
How each source arrives at it
Product strategy: what to build
- Tulsee Doshi, head of product for Google’s frontier models (Lenny’s Podcast).
- The 60/20/20 rule: about 60% of what you build works now; about 20% works “well enough” and the models look close; the last 20% is where you want models to be, where you see “glimmers of light.” The numbers are “fungible.”
- She cannot say where a model will be in three months, so she uses her own product’s signals rather than a fixed “build three months ahead” rule.
- Customers who built tightly around an older model got stuck when it improved.
- When Lenny relays Boris Cherny’s advice to remove your skills now and then and see if the model does as well without them, she answers “consistently retest”, because your structure may be holding a new model back.
- Andrew Ambrosino, Codex product and engineering lead at OpenAI (Lenny’s Podcast).
- The Codex app that shipped in February would have failed in November; the only difference was the models.
- His method: list every feature you might want, prototype all of them, ship the ready ones, let the rest “bake,” and re-try each when models leap. Operator, then Atlas, then the Codex/ChatGPT browser is the same feature re-released with more intelligence.
- Nine-month plans stay “deliberately hazy” because precision that far out is false precision.
- Boris Cherny, head of Claude Code (Lenny’s Podcast, February 2026; AI Ascent, May 2026).
- “Build for the model 6 months from now, not for the model of today” was Claude Code’s founding bet, accepting poor product-market fit for about six months.
- Citing Rich Sutton’s bitter lesson: scaffolding buys about 10-20% “but often these gains just get wiped out with the next model — it’s almost better to just wait for the next one.”
- At AI Ascent: as the model improves, the harness matters less, and within a year today’s safety scaffolding (prompt-injection defences, permission modes, human-in-the-loop) will matter less “cuz the model will just do the right thing.”
- Vijay Raji, CTO of Applications at OpenAI (interviewed by Nate B Jones).
- The founder test: if each model release worries you because it may absorb what you built, “you’re not building the right product”; if it excites you because your product gets better, you are.
- His example is OpenAI’s ads team: a $1 billion ARR run rate within seven or eight months, with about 50 people by his estimate, built with LLM calls at each pipeline stage, so “the model keeps getting better, which means… the entire ads system keeps getting better.”
- He says the smallest model available now is ten times more powerful than the biggest one of a year ago.
- Nate B Jones on Meta’s Muse. Jones applies Raji’s test to personal ambition: pick work you would get more excited about when the next model ships, not more worried, and revisit ideas you shelved because the first version needed a team you could not afford. Most of the episode is about small teams and product polish; the next-model point is his closing advice, not a harness rule.
Harness engineering: what to throw away
- Canva (Danny Wu, Code with Claude Tokyo). “Today’s clever harness is tomorrow’s dead code.” Canva rewrote its agent harness about three times in three years: internal chain-of-thought, then native model capabilities, then a handful of tools, then hundreds of tools. What survived every rewrite was the end-to-end eval suite, which grew from about a dozen cases to hundreds.
- Factory (Matan, founder). “A model today could do what a model plus a harness could do 2 years ago.” “If you are lazy, and you don’t change your harness, it’s going to be superseded.” “We need to always have in one hand a sledgehammer to whatever we thought was the best solution. Every month, we need to rethink.” The monthly harness-review checklist in the Factory article is the wiki’s own Try It built on this, marked as inferred there.
- The Claude Code team (official Anthropic roundtable).
- Product technology used to last years; with models “it fundamentally shifts underneath you every two months.”
- To-do lists fixed a Sonnet 3.5-era failure (given five things, the model did three and gave up). A year later “you don’t really need the to do list anymore.” Much of the harness exists “to cover for some of the failure modes” of the current model and is removed as models improve.
- AskUserQuestion went the same way: its designer now mostly asks Claude for an HTML artifact that asks the questions.
- The “delete a scaffold” exercise in that article is the wiki’s Try It derived from these statements, not the team’s own instruction.
- Nate B Jones, Maintain the Harness. Agents break in two directions, and one of them is model improvement: a tool that helped a weaker model can confuse a stronger one, and a guardrail built for a clumsy model can trap a better one. His example is Vercel’s sales-inbox agent improving after about 80% of its tools were deleted (a figure the wiki has not corroborated).
Prompt and context text: what to delete
- Thariq Shihipar, The New Rules of Context Engineering for Claude 5 (Anthropic blog, 2026-07-24).
- Over 80% of Claude Code’s system prompt removed for Opus 5 and Fable 5 “with no measurable loss on our coding evaluations.”
- Guidance written to stop older models failing now costs quality, because the model spends effort reconciling overlapping and conflicting instructions.
- Tool-usage examples, once “the number one rule,” now “constrain them to a certain exploration space.”
- Ben AI, relaying Thariq. The first thing to try with a better model is to remove instructions, and harnesses go out of date quickly. Ben’s own audit found 156 skills loaded and 8 used in the last 30 days; he says his first cleanup saved about 15,000 tokens per chat.
- Anthropic’s model-specific prompting guides.
- Fable 5: “Prompts designed for prior models can actively degrade Fable 5 output; review and trim.”
- Opus 5.5: Opus 5 prompts “should perform well without changes,” and most recommended changes delete or narrow instructions. Removing a “think carefully” line made chat replies start sooner with no clear quality drop. For dense charts and screenshots, “re-test old visual scaffolding first.”
- Anthropic speakers at Code with Claude, May 2026.
- Jeremy (Research PM): “Frankenstein prompts,” 3,000-line system messages built up to patch old failures, hurt new models; audit and cut prompts on every upgrade. A saturated eval shows about 1% improvement and hides real gains.
- Lisa (London keynote): “As models get smarter, the scaffolding that used to help can hold Claude back.” Build ambitious prototypes that don’t quite work today, and automate evals so you can drop in a new model fast.
- Lucas (Research PM): “Any code you’re writing that is compensating for model unreliability will have a half-life of just months… Code that connects your model to your world… compounds.” Tool-selection accuracy is now high enough that routers and prefiltering “usually make things worse, not better.”
Where the sources disagree or differ in scope
- What is durable. Canva says the harness is disposable and the evals are durable. Jones says the harness is the layer you own and maintain. Lucas draws the line inside the harness: code that compensates for the model expires; your tools, data, auth and context compound. Read through Lucas’s line, Canva’s rewrites replaced compensating layers, while Jones’s five-point audit (sources, reach, job, proof, value) maintains connecting ones.^[inferred] The Maintain the Harness article lists this reconciliation as an open question; this is a proposed answer, not a finding.
- Delete versus add. The Anthropic guides mostly delete, but not only. The Opus 5.5 guide adds an elapsed-time line for agent teams, an “explore broadly before acting” sentence, and a harness reminder after about five silent steps. The Fable 5 migration checklist adds a progress-grounding instruction, a memory file and a
send_to_usertool alongside “trim.” Canva’s harness grew in tool count while Vercel’s improved by cutting tools. - Encode versus remove. Boris also says to write each repeated mistake into
CLAUDE.mdor a skill, and calls that core engineering work. Theo Browne’s operating rule in the same article is to send prompts with as little context as possible first and add only what breaks. The two fit if what you encode is knowledge about your codebase or business rather than a workaround for the model.^[inferred] Removal also has a recorded field cost: after Anthropic replaced its “no comments” default with a judgement rule, a Reddit thread reported Opus 5 adding comments against an explicitCLAUDE.mdprohibition, once breaking a bash script. A rule you still need may have to move to a stronger layer rather than stay or go. - How far ahead. Boris: six months. The common advice Lenny cites: three months. Doshi: no fixed horizon; use your product’s signals. Ambrosino: keep nine-month plans hazy. Factory shows the cost of being early: it started in April 2023 when “the models weren’t really there yet,” and Matan says that had the timing been right then, “I was the wrong founder.”
- Cadence. Factory: every month. The Claude Code team: the technology shifts every two months. Ben AI: a setup audit every one or two weeks. Doshi, Jeremy, Lisa, and Daniel Miessler (relayed in the Fable 5 guide): on every model release. Release-triggered and calendar-triggered reviews catch different drift, matching Jones’s two directions: model improvement and changes in the world around the agent.
- Evidence quality. Only Anthropic’s 80% deletion was measured, and it was measured on their system prompt against their coding evals. Factory’s 90% drop in human involvement is one comparison between two migrations. Raji’s ads figures are his on-air estimates. The remove-your-skills advice reaches the wiki secondhand, through Lenny.
Sorting a Claude setup: compensating or connecting
The classification below is this article’s, applying Lucas’s rule.^[inferred] The right-hand column names the source each row draws on.
| Item in a skills or agents setup | Likely class | What to do after an upgrade | Source basis |
|---|---|---|---|
| ”Think carefully” and “step by step” lines | Compensating | Delete | Opus 5.5 guide |
| ”Show your reasoning” lines | Compensating, and now risky | Replace with a request for a short explanation; on Fable 5 and Opus 5.5 they can trigger a reasoning_extraction refusal | Fable 5 and Opus 5.5 guides |
| Forced to-do lists and numbered step checklists in skills | Compensating | Run once without; keep only if output drops | Claude Code team (to-do lists) |
| Worked examples of how to call a tool | Compensating | Replace with clearer parameters | Thariq, New Rules |
| Blanket NEVER/ALWAYS rules | Mixed | Keep those guarding a real failure and move them to a stronger layer; delete the rest | New Rules; Instruction Injection Layers |
| Routers, retry wrappers and prefilter steps in automations | Compensating | Test letting the model choose tools directly | Lucas |
| Brand voice, ICP, offers and pricing, house-style gotchas | Connecting | Keep; update when the business changes | Lucas; Boris (encode team knowledge); Ben AI, “good context beats a bigger model” (AIOS pattern) |
| Connectors to CRM, analytics, CMS | Connecting | Keep the used ones; switch off ones unused for 30 days | Lucas; Ben AI’s detox (19 of his 25 connectors unused) |
| An eval set of real tasks and past rejections | Durable | Add cases; replace ones every model passes | Canva; Jeremy |
Try It
For a marketer or operator running Claude skills and agents:
- Build a five-prompt upgrade eval this week. Pick five jobs your skills do for real, for example a blog draft in brand voice, a monthly performance summary from an analytics export, an ad-copy batch with character limits, a competitor page teardown and a nurture email. Save your best current output for each with a one-line note on what “good” means. Grow it to about 20 prompts once the rating is clear. (Doshi’s five-to-20 prompt method; Jeremy’s warning that an eval every model passes hides real gains.)
- On each model release, run the eval twice. Once with your skills,
CLAUDE.mdand project instructions on, once vanilla. Where vanilla matches, delete or slim the skill. (Doshi, and the Boris advice she responds to; the paired run as a method is this article’s. - Search your skills and instructions for old-model phrasing: “think carefully,” “step by step,” “show your reasoning,” all-caps NEVER/ALWAYS. Follow the Opus 5.5 article’s checklist: delete the first two and replace the third with a request for a short explanation. In Claude Code,
/doctorand theclaude-apiskill’sprompt-auditsubcommand cover part of this. - Label every rule and skill compensating or connecting using the table above, and put new effort on the connecting side: client and brand context, clean data connectors, more eval cases.
- If a deleted rule turns out to matter, move it up instead of pasting it back. Put it where the model cannot argue with it: an output style, a hook, or an explicit check step inside the skill. See Where You Inject an Instruction Decides Whether It Survives.
- Keep a 60/20/20 automation backlog. List your automations as works-now, nearly-there or bet (for example, a fully unattended weekly SEO report, or an agent that drafts and schedules a month of social posts). Keep the bets to about a fifth. Re-run every bet on each release and promote what now works. (Doshi; Ambrosino’s “let it bake.”)
- Schedule the review. A recurring scheduled task that runs your setup audit monthly (Factory’s cadence) or every two weeks (Ben AI’s), plus the eval run from step 2 on release day.
- Run Raji’s test on your service list. For each thing you sell or run, ask whether a better model makes it better or makes it redundant, and move effort toward the first group. Where a pipeline has several steps, make each step a model call so an upgrade improves the whole pipeline, as in Raji’s ads stack.
Open Questions
- Boris Cherny’s remove-your-skills advice is secondhand. It reaches the wiki through Lenny on the Doshi episode. Boris’s own recorded statements here are the bitter-lesson point and “build for the model 6 months from now.” Ben AI also relays a Boris quote (“my setup might be surprisingly vanilla”) that the wiki has not checked.
- No source measures removal outside coding. Anthropic’s 80% figure is on coding evals. Whether marketing skills (voice, formatting, compliance wording) lose quality when trimmed is untested in the wiki.
- No source gives a way to classify a rule without running the test. The compensating-versus-connecting split here is a heuristic, not a measured rule.
- Is a release-day review enough? Jones’s world-drift direction (a renamed CRM field, a stale source) happens between releases, which argues for both triggers; no source compares them.
Related
- The Expanding Toolkit (Lucas, Anthropic) — the compensating-versus-connecting rule this article sorts by.
- The New Rules of Context Engineering for Claude 5 — the one measured deletion: over 80% of a system prompt, no coding-eval loss.
- Maintain the Harness (Nate B Jones) — the counter-view that the harness is the layer you own and maintain.
- Canva AI 2.0 — three harness rewrites in three years, with the eval suite as the part that survived.
- 20 Rule — the hedge between building ahead and shipping what works now.
- Where You Inject an Instruction Decides Whether It Survives — where to put a rule you still need after a cleanup.
- The Verification Frontier — why the eval set is the investment that lasts.
- The Detox Skill (Ben AI) — a recurring setup audit to run between releases.