Jonathon's AI Wiki

Browser and Computer Use in Practice (Claire Vo, How I AI)

8 min read

Source: raw/I_let_Codex_control_my_browser_so_I_don_t_have_to.md Creator: Claire Vo (ChatPRD founder) — How I AI URL: https://www.youtube.com/watch?v=lk63Sl-LRKE Platform: YouTube (yt-podcast feed how-i-ai)

Browser and computer use is a capability the wiki documents at the harness level (Computer Use, browser-harness) but has thin coverage of as a daily working habit. This episode is the habit: Vo runs Codex’s browser and computer control “every minute of every day” and walks through four concrete use cases — agentic QA, persona-driven UX research, inbox triage, and shopping — plus the invocation model and the one prompting rule she says matters most. Her verdict on tool choice is blunt and worth recording as an operator data point rather than a benchmark: Codex (the ChatGPT desktop app) is “just the best at controlling my computer.”

Key Takeaways

  • Codex exposes three distinct control surfaces, and picking the right one matters. @browser opens Codex’s own side browser window; @chrome drives your Chrome through the extension (so it inherits your logged-in sessions); @computer controls the whole machine — mouse, keyboard, any app. Prerequisites are the desktop app plus the official Chrome extension. Vo reaches for @browser when the target is already open on localhost and @chrome when the task needs her real logged-in state.
  • Under-prompt frontier models on browser tasks. Her explicit tip: “QA the onboarding flow” produces a better result than “QA these 25 things in the onboarding flow.” Over-specification caps the agent at your checklist; letting it plan gets you coverage you wouldn’t have written down. ^[Vo’s stated finding from repeated use, not a controlled comparison]
  • Agentic QA’s real advantage is exhaustiveness, not speed. Humans QA the happy path — fill every required field, click next, don’t get tired. The agent tests failure states, error states, both signup paths, viewport resizes, accessibility, overflow, and touch targets. On her own long-shipped onboarding flow it surfaced 11 issues including one high-severity navigation blocker — a required field with no validation code, invisible to her because she always clicked the required buttons.
  • Make the QA output a spreadsheet, and the spreadsheet becomes the fix queue. The prompt shape is “test [flow] for usability and mobile responsiveness, screenshot as you go, and make a Google spreadsheet of any issues you identify.” The resulting sheet carries repro steps, viewport, severity, remediation, and the embedded screenshots — everything a fixing subagent needs, and a tracker she can compare against after-fix screenshots.
  • Persona-driven UX research is the more unusual use case. Rather than “you are a genius PM” in a text prompt, put the persona in the browser: have the agent walk the real product as a PM, then an engineer, then a team lead, and write a research-style critique with friction points, moments of delight, and improvements. It surfaced a genuine structural gap in ChatPRD (a document created in one thread can’t be referenced from another) that the team knew about abstractly but hadn’t felt from the user’s side.
  • Browser use is the channel for surfaces with no API or MCP. LinkedIn has no official API or MCP, so Vo drives it through the browser: “go through my recent unread messages and reply to any that are super critical for ChatPRD or the podcast; leave me notes on the others about how I should reply.”
  • Calibrate effort down for mechanical browser work. She noted she was running LinkedIn triage on GPT-5.6 Sol at high effort and that Terra at medium effort would likely be both sufficient and faster — the same model-vs-effort discipline the wiki documents for coding applies to browser agents, where the bottleneck is usually page latency rather than reasoning.
  • Bot detection is the hard wall, and the human becomes the verification layer. A retail shopping run tripped an are-you-a-bot check. Her framing: “sometimes we put AI to use for us, and then sometimes we need AI to put us to use” — expect to be the hands for the agent at the CAPTCHA step rather than expecting the run to be fully unattended.
  • Computer use can drive your phone via macOS iPhone mirroring. Her worked example: out of state, needing to reach Macs at home, she had Codex use iPhone mirroring to open her Wi-Fi app, change router settings to open firewall ports, SSH into the machines, and close the ports again when done. This is the sharpest capability claim in the episode — the phone becomes just another window computer use can operate.
  • Computer use is also the fallback when an MCP or plugin path fails. When an integration hits an access error, “please just open my browser and do it” often works — she uses this routinely for creating Google Docs and Sheets.
  • The standing cost is latency. Browser use is slow; her habit is to launch it and work on something else in parallel rather than watch it.

The four prompt shapes

Use caseSurfacePrompt shapeOutput
Agentic QA@browser (localhost)“Test the [flow] for usability and mobile responsiveness. Screenshot as you go and make a Google spreadsheet of any issues you identify.”Prioritized issue sheet w/ repro steps, viewport, screenshots — feeds a subagent fix loop
Persona UX research@browser”Go through [product] as three personas: [A], [B], [C]. I’m giving you no more instructions than that. Then create a doc with a research-style critique — friction points, places of delight, improvements for each persona.”Research doc grounded in an actual session, not a synthetic-user simulation
Inbox triage (no API)@chrome (logged-in session)“Go through my recent unread messages and reply to any that are critical for [X]. Leave me notes on the others about how I should reply. If it’s just a friendly thank-you, reply in a friendly way.”Sent replies + a decision queue for the rest
Web chores@chrome”[Constraints]. Put N items in my cart that I can pick from and buy — don’t check out.”A staged, reviewable end state rather than a completed transaction

The pattern across all four: give the agent a goal and a deliverable artifact, not a step list — and for anything consequential, stop it one step short of the irreversible action (cart not checkout, draft not send).

Try It

  1. Run the QA prompt against a flow you’ve shipped and stopped looking at. The bug Vo found existed precisely because she had tested it a hundred times as a human on the happy path.
  2. A/B your own prompting. Run one flow with a 25-item checklist and one with “QA the onboarding flow,” and compare issue counts. Her under-prompting claim is cheap to falsify on your own product.
  3. Pick the right @ surface deliberately. If the task needs your logins, @chrome; if it’s localhost or a throwaway session, @browser; only reach for @computer when the target isn’t a web page.
  4. Wire the spreadsheet into a fix loop. Have the QA run emit the sheet, then dispatch subagents against rows and write status back — that’s the loop the artifact is designed for.
  5. Drop an effort tier on mechanical runs. Page loads dominate the wall clock on browser work; see Model vs. Effort for the general heuristic.
  6. Stage, don’t commit. Add “don’t check out / don’t send / stop before the final step” to any browser-use prompt that touches money or someone else’s inbox.

Open Questions

  • No head-to-head data behind “Codex is best at controlling my computer.” It’s a practitioner preference stated without a comparison run against Claude’s Computer Use or the AI-native browsers (Comet, Atlas) she mentions but doesn’t test.
  • Bot-detection failure rate is unquantified — one observed trip on one retailer. How often browser use stalls on anti-bot checks across common sites isn’t established here; the wiki’s CloakBrowser coverage is the adjacent data point on that problem.
  • Security and privacy get raised and not answered. Vo explicitly invites audience questions about both at the end. Driving a logged-in Chrome session and a mirrored phone is a wide blast radius, and this episode offers no guardrail practice.
  • Does the persona-research output hold up against real user research? She flags herself as “not a big synthetic user girl” — the critique it produced was directionally useful, but nothing validates it against findings from actual users.

Graph View

Backlinks