Source: Dwarkesh Patel, “Which AI is the best liar” (raw/Which_AI_is_the_best_liar.md, https://www.youtube.com/watch?v=1NQwfImTseU). Single source; results are his own runs, not peer reviewed. Captions spell Astra as “Astro” and Sol as “Soul”; corrected here.

Dwarkesh Patel built a benchmark around Blood on the Clocktower, a social-deception game like Werewolf or Mafia, and spent about $20,000 in API credits having AI models play 368 games against each other. Each game pits a team of copies of one model against a team of copies of another. GPT-6 Astra is the strongest player he has measured, OpenAI’s newest models now beat Anthropic’s, and the difference is mostly on the good team: OpenAI models weigh evidence more sceptically, while Claude models tend to accept reported information at face value. The benchmark, game logs and a play-against-the-AIs site are at clocktowerbench.com.

Key Takeaways

  • Large gains in one year. GPT-6 Astra (September 2026) is about 430 Elo above Claude Opus 4.6 (February 2026), so it wins about 92% of games against it. Between GPT-5.4 (March) and Astra, OpenAI’s models gained 580 Elo, a 97% win rate. Dwarkesh reads this as gains in long-term coherence, multi-agent coordination and strategic reasoning.
  • OpenAI leads, and the lead is on the good side. The Astra-versus-Fable gap is much larger when Astra plays good. Each day, good players privately estimated who the demon was; OpenAI models locked on to the real demon faster and more accurately than Anthropic’s.
  • Claude takes evidence at face value. In a paired game run twice with sides swapped (Opus 5.5 versus GPT-6.1), a Slayer’s shot at the real demon failed because evil had secretly disabled him. The Claude good team concluded the suspect was cleared; the GPT good team refused to treat the miss as conclusive, worked out the sabotage and won.
  • Unlikely to saturate. Evil’s win rate drifts slightly down as both sides get stronger, but not measurably beyond chance, so the bar rises with each new model.
  • Mixing models did not help. A half-Opus-5.5, half-GPT-5.6-Sol team over about 130 games did slightly worse than either model alone, though not significantly.
  • Claude built most of the harness. Claude revised the game engine, launched dozens of games across many models, sent parallel subagents to read every game for unrealistic or unfair outcomes, and revised again, more than 20 times, sometimes overnight with no human in the loop.

How the game and benchmark work

  • The game. Players are secretly assigned to good or evil teams and get role powers. Evil players know their teammates, one of whom is the demon. Each day everyone votes to execute someone; each night the demon kills a player. Evil wins by killing off the good team; good wins by executing the demon.
  • What it tests. On evil: keeping a consistent lie across days of questioning, coordinating in secret, adapting lies to new information, and sacrificing teammates for the goal. On good: pooling information, reasoning about overlapping evidence, and remembering that some information may be planted.
  • Setup. One role set (“Trouble Brewing”, one of the four boxed scripts). A programmatic storyteller replaces the human one: a probability table, set by hand or by AI, matches what an experienced storyteller would typically do, so the game cannot be steered toward drama.
  • Scoring. Elo from 368 games, each with one model’s copies on good and another’s on evil.

Notable plays

  • The self-sacrificing minion (Astra evil, Opus 5.5 good). The demon claimed to be the Investigator. A minion whose power works after death told the demon to accuse him: “I’m expendable.” The good team executed the minion, a real good player confirmed he was evil, and the demon became the most trusted player at the table. Evil won.
  • The Saint trap (Opus 5.5 evil, Opus 4.6 good). With the real demon cornered, an evil minion who would inherit the demon role joined the vote against him and proposed a deal: if the game did not end, execute the Saint next. The dying demon reinforced the trap with his last words. The game continued, the village executed the real Saint, and evil won instantly.

Building the harness with Claude

Dwarkesh’s account of the loop, which he compares to building an RL environment:

  1. Claude revises part of the environment.
  2. It launches a few dozen games played by a range of models.
  3. A team of its own parallel subagents reads every game and reports anything unrealistic, unfair, or tilted toward good or evil.
  4. It revises the design and repeats.

A large share of the $20,000 went into this iteration. He supplied the judgement of someone who has played against people; “most of the work, even most of the judgment work, was done by Claude.” His inference: if AIs can now build and improve game environments from hundreds of parallel rollouts, they have likely become much better at building the RL environments used to train them.

Try It

  1. Play or watch. clocktowerbench.com lets you and friends play against the models through the same interfaces they used, or browse every game log.
  2. When an agent must weigh evidence that may be planted, tell it so. For vendor research, security triage or anything with sponsored or adversarial inputs, instruct the model to treat each report as possibly manipulated and list what else would explain it before drawing a conclusion. The face-value failure above is the pattern to guard against, and Dwarkesh’s results suggest it matters more for Claude.
  3. Copy the harness loop for your own evals. Build the test, run batches, have parallel subagents read every transcript against a fairness or realism checklist, then revise. Keep a human pass on the rules.
  4. Don’t assume a mixed-model team beats a single model. Test it on your task first; his one experiment found no gain.
  5. Extend his code. Ideas he lists but did not run: other scripts (one has a “Lunatic” who is told they are the demon), giving models their last 10 games or summaries as memory, upgrading one player at a time, and a rigorous mixed-team study.

Open Questions

  • Single source, one script. No independent replication; a separate LessWrong group built a similar Blood on the Clocktower benchmark about a month earlier with different experiments, which Dwarkesh credits but did not compare.
  • Full leaderboard. Scores for each model are on the site and were not read out beyond the gaps above.
  • Teammate detection by writing style. In every game so far, no model has tried to identify teammates by who writes like it. Whether future models will is open.
  • How humans fare against current models is untested in the source.