Jonathon's AI Wiki

Managing an Agent Fleet as a Solo Founder (Ryan Carson × Claire Vo)

11 min read

Source: raw/How_I_manage_15_AI_agents_24_7_as_a_solo_founder_Ryan_Carson.md — How I AI (Claire Vo), episode with Ryan Carson, ~45 min (https://www.youtube.com/watch?v=zPfxlcVpFgs)

Two solo founders compare their actual agent stacks a year after both were “very handholdy” in an IDE. Ryan Carson runs Untangle (B2B family-law software) on roughly 10–15 concurrent Devin threads; Claire Vo runs Eve Legal out of Slack. The episode is unusual because both of them spend most of it arguing against maximal agent output — and because their two triage systems solve the same problem in opposite directions.

Key Takeaways

  • The job changed from writing code to managing throngs of agents. Carson’s framing: “how do you effectively manage 10, 20, 100, 500 employees? This is the whole reason organizations were built as pyramids.” The claim that follows is that management experience, not coding experience, is the transferable skill right now.
  • Two opposite triage systems, both working. Carson buckets tasks into Devin folders (bugs, P0, P1, P2, new features, investors) because humans can only track four or five concurrent threads. Vo buckets time — blocks of the day for co-piloting versus blocks for spinning off and checking in — and just Slacks Devin ad hoc. Neither claims the other is wrong.
  • The anchor layer is a piece of paper. Carson keeps weekly priorities on a physical to-do card next to eight screens. It is the only thing in the stack that does not scale, which is the point: the folders hold the work, the paper holds the goal.
  • Both push back hard on token-maxxing. Vo: “I really don’t ship more than I think the market wants… I don’t get multiples of quality off of multiples of output.” Carson: “we’re nowhere near any frontier model having the intelligence to know what to ship,” and self-improvement loops on product “are kind of a joke.” Vo’s structural reason: all this intelligence “has not magically created markets where there were none before.”
  • The corrective is leaving your chair. Carson found product-market fit by cold-emailing one lawyer, taking one call, and then asking to sit in her office. His diagnosis of the output/value mismatch: “because we can ship so much and build so much digital product, we’re not getting out of our chairs enough.”
  • Background agents are not coding agents. Vo has Devin running deal desk, custom quoting, ops documentation, and customer triage. Her framing question: “what would you do with somebody who both knew your codebase and could write code to solve anything in your business?”
  • Both keep a human-review gate on merge, and both automated the approach to the gate, not the gate itself. Vo’s Merge Mommy scores risk and auto-approves only the low band; Carson’s Land PR loops review twice and then makes him watch a video.
  • Design is where the model choice is least fungible. Carson runs Devin and Codex but goes back to Claude specifically for carrying a design system forward — “Devin is terrible at design,” Codex “hilariously bad” at it. Vo independently confirms Claude Design “creates a design system… is unmatched right now” while adding that it is bad at coming up with new design.

Carson’s system: folders, playbooks, and a paper card

Folders as the concurrency limiter. Devin threads are bucketed into bugs, P0, P1, P2, new features, and investors. The rationale is explicitly cognitive — Carson cites a prior episode’s claim that a human can genuinely track four or five concurrent tasks, so the folder is what lets 15 threads exist without 15 things being in his head.

Watchdog — the highest-leverage playbook in the episode. It runs per customer firm and answers “what is going on in my business right now” on demand, triggered by the feeling of not knowing:

  1. For each firm, what activity has happened since the last Watchdog run?
  2. Where are the errors, including Sentry errors?
  3. Filter and sort — surface the top three things going wrong, not the raw data.
  4. For each: is it already fixed, currently being fixed, or is there an open PR that never got merged?

Step 4 is the part that makes it work at ~40 PRs a day. A list of open problems is useless when the fix rate is that high; the playbook has to reconcile problems against in-flight work.

Land PR — the merge gate. Triggered when an agent reports a PR done:

  1. Run a fresh Devin review on the PR.
  2. Allow up to two review loops — find bugs, confirm every comment resolved, repeat once.
  3. Record a narrated browser video walkthrough with captions and a red/green test list.
  4. Carson watches the video and says “video approved, land it.” Only then does it merge.

He is explicit that he tried to build this himself and gave up: “I don’t have time to do this,” which is his argument for picking a code factory with a real built-in review product rather than assembling one.

Investor updates are a skill. Carson’s aside — “if you’re not using an agent to help draft your investor updates and help you keep track of the business, clearly do that. But here’s the key: build a skill to do it.” Devin calls these playbooks; the shape is the same.

Vo’s system: Slack, Merge Mommy, and time buckets

Why Slack. Not because the UI is better — she says the Slack view is “just a reflection of the web app.” The reason is organisational, learned deploying background agents at LaunchDarkly: putting agents in public Slack threads was “such an effective change-agent move,” and the only place she had seen real-time coding work happen in public. It changes what colleagues believe AI can do. Devin also handles multiplayer chat without colleagues tripping over each other in a thread.

Merge Mommy — an Eve agent deployed on Vercel. When a PR opens and passes all CI checks including Bugbot, it scores the PR across roughly five risk aspects (blast radius and security among them). Low risk gets an approval stamp and merges without human approval; medium or high pings a named human in Slack. The design constraint is compliance: SOC 2 controls require alternative review, so the agent’s job is to route to the human, not to replace them.

Codex for the grind and the verification. Her Codex use is specific: large feature initiatives spanning front end and back end, long-running refactors, and verification. The verification recipe is worth copying literally —

write user stories, use Chrome, pull up the preview branch, go through it as a user, list which user stories pass, identify bugs, fix them until you’re good.

She credits /goal plus subagents plus browser use, and calls the low-latency local pairing “unmatched” for front-end work that needs a hand on the wheel.

Codex runs her email. It “ran through 2,000 unread emails in my Gmail the other day and just changed my life.”

Decision fatigue as a prioritisation heuristic. A feature she lets die on the vine is her signal that she did not believe in it. This is a deliberately passive filter, and it is the counterpart to Carson’s paper card.

The design pipeline both converged on

Vo’s most recent brand build, described step by step:

  1. Claude Design decomposes an example site or a Figma file into non-slop, reusable design tokens — “unmatched right now” at this, per Vo; Carson independently calls it “magical” for carrying a design system forward.
  2. Download the design file, hand it to Codex along with the Figma, and ask for a technical design system — shared components, interlinked in a monorepo.
  3. Use Codex with the Figma plugin to push the design tokens back into Figma as components.

Both are explicit about the boundary: Claude Design is strong at systematising a design and weak at originating one. Carson: “I don’t mean in coming up with new design. I think it’s bad.”

The verbosity complaint, on the record

Carson, unprompted: “My problem with Claude — infinite tokens, very intelligent — my blood boils reading Claude output. It actually raises my blood pressure… I’m so sick of Opus 5.” He frames it as a harness-experience problem rather than a capability one, and says he finds Codex “just a lot more effective and enjoyable.” Vo qualifies that Claude remains “kind of unbeatable on the execution side” for very specific deep spikes — she names cyber security and design.

This is the same output-quality complaint Anthropic addressed with the built-in Concise output style in v2.1.237 — see Claude Opus 5 and Week 34. Recording it here because it is a named practitioner attaching the complaint to a specific model and a specific decision (routing his work to a competitor’s harness), not an anonymous vote.

Hiring by watching the agent output

Carson’s process for his first engineering hire, in progress at recording time:

  1. Public call on X: be AI-forward, location irrelevant.
  2. No interview, no meeting. “Record a video, full screen, everything, your whole desktop, of you building a new feature for an app that already exists.”
  3. Phase two: give finalists access to Devin, have them build a real thing, record that too.
  4. Then a short “we should meet” call, hire, “and then I’m just going to watch their agent output and that’s how I’m going to manage them.”

Vo’s observation lands the point: “You’re almost hiring a human like you would hire an agent… you’re doing a verification loop.” Carson also notes the free version of step 3 — if the candidate works in Devin, you get the replay video anyway.

Try It

  1. Build Watchdog before you build anything else. Its trigger is a feeling (“I don’t know what’s happening”), and its output is three ranked problems reconciled against in-flight fixes — not a dashboard. Adapt it per customer, per project, or per repo.
  2. Pick a concurrency limiter and commit. Folders by priority (Carson) or blocks of your day (Vo). The failure mode is neither — 15 live threads and no bucketing.
  3. Put one non-scaling anchor next to the fleet. A weekly priority card on paper is the cheapest version and the one both of them respect.
  4. Score PR risk before you automate approval. Merge Mommy’s design — auto-approve only the low band, route medium and high to a named human — is the version that survives a compliance audit.
  5. Make the review produce an artifact you can check in 60 seconds. A narrated video with a red/green test list is a stronger gate than a longer text review, because you will actually consume it.
  6. Give a background agent one non-code job this week. Quoting, ops docs, customer triage, or the inbox. The coding-agent framing is leaving most of the value on the table.
  7. Before you scale output, go talk to a customer in person. Both founders name this as the binding constraint, and neither is arguing from a shortage of tokens.
  • Grok Bot — the same fleet-management problem solved with a chief-of-staff topology rather than folders; also a non-technical on-ramp
  • Checker — the org-chart framing Carson and Vo are both instances of
  • Claude Opus 5 — the verbosity complaint recorded above, and Anthropic’s Concise response
  • Claude Design — the design-system step both founders route to Claude for
  • Subagents — the delegation primitive underneath Vo’s Codex verification loop
  • Verifier-First Loops — Land PR and Merge Mommy are two verifier designs; this is the theory
  • The Loop Is the Unit of Work — and the counter-argument both founders make against loops on product decisions
  • Agent Guardrails — human-in-the-loop gates and where to place them
  • Cheap-Executor Delegation — the cost-side companion to this management-side view
  • Agents & Agentic Systems — topic index

Open Questions

  • The “16 paid hours of Devin access” figure. A secondary write-up of this hiring process cites a specific paid-hours number for phase two; the transcript does not state one. Treat the number as unverified.
  • Devin spend. Carson describes going from ~20,000 in one month before negotiating free credits on the strength of a large X following. Whether $20k/month is representative of the workload or of an unoptimised period is not addressed — and the credits mean his current usage is not price-disciplined.
  • Video verification reliability. Vo cannot get Devin’s video verification to work (“it taps my credits for the session and then just falls over”) while Carson relies on it. The difference is unexplained; both suspect configuration.
  • Does the folder system survive a second human? Carson plans to require all multi-user interaction to happen in Devin threads rather than Slack, and floats banning DMs entirely, but has not yet hired anyone to test it.
  • No numbers on the anti-token-maxxing claim. “I don’t get multiples of quality off of multiples of output” is stated as experience by two operators, not measured. It is a hypothesis the wiki should keep looking for evidence on.

Graph View

Backlinks