Source: raw/I_tested_4_AI_agents_on_9_real_tasks._Here_s_who_won..md

Creator: Claire Vo, How I AI · URL: https://www.youtube.com/watch?v=hnCoQ5wB-GU · Duration: not recorded · Platform: YouTube (auto-captions, fetched 2026-11-01). Sponsors: WorkOS and Firecrawl.

Claire Vo ran four personal agents (OpenAI’s Dot, SpaceX AI’s Grok Bot, Meta’s Muse and Anthropic’s Claude Duet) through the same nine tasks from her own week, using other agents to drive them. Grok Bot came first (93%) and Claude Duet last (79%), but the per-task results matter more than the total: each agent won something. It is one user’s benchmark with non-blind, live judging, so read it as a method you can copy and a set of data points, not a ranking.

Key Takeaways

  • Use the agents to write the benchmark. She asked each agent for “the top tasks that you do for me” (ten each), then had Claude pick the tasks most common across all four, plus ones that test different components.
  • Use an operator agent to run the agents. Codex, with computer use, drove Duet, Grok Bot and Muse; Claude with computer use drove Dot (Codex couldn’t drive itself). She calls Claude computer use “hot garbage” in this role and still had to babysit and nudge.
  • Baseline access first. She had Codex use the browser to check that every agent was connected to roughly the same accounts and skills. She still missed Muse, which had no Slack (Slack “isn’t in the connector list”) and no GitHub, so it could do only 7 of 9 tasks. Duet also finished 7 of 9; Dot and Grok Bot finished all nine.
  • One-line prompts in her own voice, plus a run sheet saying what to answer if an agent asks a question and how to nudge it.
  • Speed: Muse was fastest on 4 of 9 tasks despite needing the most steering. Duet was slowest overall, matching her earlier review. Dot was also slow on many tasks.
  • Permissions: Duet asked for permission four times; each of the others asked once. Her verdict: “Claude was just way too conservative.”
  • Final score (Claude scored her recorded commentary): Grok Bot 93%, then Muse, then Dot, then Duet at 79%. She thinks Dot “got kind of sandbagged” and sits closer to Duet than to Muse.
  • Her rule of thumb: work jobs to Grok Bot, life jobs to Muse. Dot and Duet were best at coding, but she would code in the underlying tools (Codex, Claude Code), not through the personal agents.

Results by task

TaskBestNotes
Reply in Slack to a friend at Lovable asking her to speakDot (accuracy), Grok Bot (voice)Dot alone asked whether it was paid and remote. Duet’s reply was shortest and “most inaccurate”. Muse had no Slack.
Cut a short video from a raw file in DropboxMuseDuet failed: it “would not download a video”. Muse had the cleanest design and punchy cuts; Dot second; Grok Bot worst (overlay on the guest’s forehead) and slowest, though it did zoom cuts.
Fix a Sentry bugDot and DuetAbout 10 minutes each; Claude Opus 5.5 graded the fixes. Dot was closest to the real fix. Duet found a second bug and noticed Dot had already shipped the first fix. Grok Bot “loosened validation”. Muse had no GitHub and thought Devin, an agent, was a person.
Make an episode thumbnail (Figma MCP, existing templates)Grok BotDot and Duet filled the templates; Grok Bot proposed new layouts and headlines. Muse’s was “creepy”. She says Grok Bot “is like 100% the best at using Figma”.
Triage inbox into to-dosGrok BotDelivered as a Grok document (new that week) with checkboxes. Muse did well with Gmail only. Duet put them in its task list. Dot sent “a message with like five things”.
Family newspaper for the weekMuseGenerated a cover image and a two-page layout. Grok Bot good. Dot’s design was “Ash Trash”. Duet, with only her work calendar, returned a markdown outline of how to do it.
Explain a velocity-over-time homework graphMuseAll four were correct (answer C); Muse answered in about half a second. She is moving quick look-up questions from ChatGPT to Muse.
Shop for red Adidas Tokyo shoes on a budgetGrok BotMuse close with some steering. Duet was “blocked by almost every shopping site”, searched the wrong colourway and found the most expensive pair. Dot “barely tried”.
Launch video for a ChatPRD feature, with her video skillDuetDuet got the value proposition and branding right with no layout errors, and used the ElevenLabs MCP for a voiceover. Grok Bot used the skill but overflowed the layout; Dot made “a PowerPoint”; Muse had no repo access and invented content.

Try It

  1. Generate your task list from your agents. Ask each agent you use for the ten tasks it does most for you, then have a model pick the overlap.
  2. Check access parity before you compare. Have one agent with browser control list the connectors and skills each agent can reach. Missing access was the single biggest cause of failed tasks here.
  3. Write a run sheet: a one-line prompt per task, the answer to give if the agent asks for something, and when to nudge. Keep the prompts in your own words.
  4. Record time to a finished result and the number of permission requests, not only quality. Speed and interruptions decided several of her rankings.
  5. Test the specific capability you need (video files, Figma, shopping sites) rather than trusting a total score. Duet won the launch video and lost the short clip.
  6. For Duet specifically, expect it to stall on downloading media and on retail sites, and give it the same personal calendar access as your other agents before judging its personal tasks.

Open Questions

  • The judging was not blind, and the percentage scores came from Claude reading her spoken commentary; she says “I don’t actually know how these scores work.”
  • She says “Codex was the fastest” at the video and shoe tasks. It is unclear whether she means Dot (ChatGPT’s agent, linked to Codex) or the operator.
  • Muse’s two failures and Duet’s newspaper failure were partly access gaps she set up, so they do not measure the agents alone.
  • Duet ran under a different operator (Codex) from Dot (Claude computer use); she notes “a little bit of diff in the operator agent”.