Source: raw/How_I_fight_token_fraud_as_an_AI_founder.md (auto-caption transcript)

Creator: Claire Vo, How I AI (solo episode) · URL: https://www.youtube.com/watch?v=ZDpP7JXPlhc · Duration: not recorded · Platform: YouTube (fetched 2026-11-17) · Sponsors: WorkOS and a second sponsor captioned “Oros”. Several vendors she praises (WorkOS, Persona) are current or past sponsors of the show, which she states. Caption fixes: “Chat urity / Chat Purity / Chat Party / Chaparral / Chat Proxy” are all ChatPRD; “Opus 55” is Opus 5.5; “Sonic” is Sonnet.

Claire Vo runs ChatPRD, a bootstrapped AI product for product managers with seat-based pricing. She says people who are not customers have used free trials, promo codes and custom MCP servers to run their own workloads on her model bill, costing her “tens of thousands of dollars” over about a year and taking most of her engineering time. The episode lists six abuse patterns, nine business controls, seven technical controls and four agent loops. All figures and incidents are her own account; none was checked against a second source.

Key Takeaways

  • If you sell an AI product on seats, you are selling compute. Her pricing assumes average use per seat. Anyone who gets a free or cheap seat and runs it flat out on the most expensive model breaks that margin.
  • Every free door gets used. Trials with no card, trials with a card (card cycling, bad cards, chargebacks) and paid trials were all abused. Her conclusion: “I just think trials are over.” She turned them off and sends prospects to sales, at the cost of her self-serve pipeline.
  • Expensive models are the target. She turned off Opus 5.5 in a few countries and token theft from those countries stopped.
  • Monitor spend hourly or live, in dollars. Her “big mistake” was not realizing how fast an exploit runs up cost. She counts dollars, not requests, because one request to an expensive model is not one request to a cheap one.
  • Only serve models on an allowlist. If your backend forwards whatever model name the client sends to the provider’s messages endpoint, a caller can ask for Opus or Fable 5.2 when you meant to give them Sonnet.
  • Verify the client, not only the login. Valid credentials do not prove the request came from your app. One reseller sat authenticated in a browser and relayed their own customers’ requests through ChatPRD.
  • Custom MCP servers are an abuse path. Someone installed a large custom MCP that turned a ChatPRD account into a 24/7 worker doing external tool calls. She now restricts which MCP hosts are allowed and blocks free tunnels.
  • Classify suspect chats with a cheap decision model, then ban or silently downgrade. A flagged chat goes through a zero-data-retention Jev loop that sorts it into real customer or a fraud category.
  • Your support agent is an attack surface. People try to get the AI support desk to unban them or issue promo codes. Give it fraud playbooks and lists; do not give it tools to ban, unban or grant codes.
  • Every control costs real business. Blocking a country, ending trials and capping usage each remove legitimate users. She banned one real customer in a sweep and reversed it.

The six abuse patterns

PatternWhat she saw
Free-tier and trial farmingAutomated signups using the most expensive models for work unrelated to product management. Signup loops with sequential names (firstname lastname 12345, ...12346) are the easy ones to catch.
Promo-code abusePartner codes resold (she names Reddit resellers), stolen, and used in bulk to run other AI businesses. She will not issue promo codes again.
Bad cardsPrepaid cards, stolen cards, chargebacks, card testing. One morning showed about $200,000 in attempted purchases, all failed. She calls this the least of the problems because payment controls are mature.
Resale and relaysA reseller wrapped the app in a proxy and sold access by the hour. Others ran reverse proxies and relays from their own AI product.
MCP abuseA custom MCP used to bypass message-level controls and run an always-on worker.
Model probingMany international accounts asking “what model are you?” and for the training cutoff. She does not know whether this was distillation or farming.
  • Worst case she describes: an overseas studio building Roblox and Fortnite maps by looping free trials and promo codes and tunnelling to her agent, using the coding capacity ChatPRD offers for prototypes.
  • Abusers combine techniques to stay under per-account thresholds. Because ChatPRD does not retain or read customer data, she can only investigate extreme cases.
  • Same actors, other products. From the abusive chats she concluded the same people were rotating reverse proxies across several well-known AI products. She says she could tell from how they prompted her tool. Not independently confirmed.
  • Users wrote PRDs in ChatPRD describing how to scam ChatPRD.

Business controls

She credits part of this list to a post by Gareth at Atlassian (not in the source; the post was not fetched).

  1. End free trials, or expect to. Card-backed and paid trials did not stop abuse for her.
  2. Stop or tightly police promo codes.
  3. Bill up front. No grace period on failed payments, no flexible terms.
  4. Hard usage caps per account and per organization. She used to run high caps to learn from heavy users.
  5. Restrict trials to business email domains.
  6. Put trials and other open doors behind feature flags so you can switch them off the moment something looks wrong.
  7. Write terms of service that allow bans for abuse and suspected abuse.
  8. Decide which countries you serve. Blocking whole countries at the firewall cut fraud “by a significant amount”; she is considering a trusted-country list based on fraud signals.
  9. Report organized abuse. She filed cybercrime reports with the FBI (Codex drafted them) and posted about it publicly, which brought in help from Stripe, Vercel, Clerk, WorkOS, Persona and Raindrop.

Technical controls

ControlWhat it doesWhat she uses
AI gateway with spend limitsOne place to cap spend and to restrict which models can be calledVercel AI Gateway; she notes you can build your own
Model allowlistRejects any model name outside the set you sellIn application code
Penalty modelRoutes suspected-fraud calls to a cheaper modelSilent downgrade to “like a GLM” (her example)
Spend alerts, caps and failoversHourly or live monitoring with automationsNot specified
Payment fraud rulesBlock countries, require card address to match location, stop card testing, fingerprint cards to catch one card across many trialsStripe Radar; the fingerprint check took code plus Radar
Client attestationProves each request came from your app, even with valid credentials. Goes on every chat route; run in monitor mode, tune, then enforceVercel BotID
FirewallCountry and VPN blocks, IP blocks, rate limitsVercel Firewall
Auth-provider bot protectionFraud and bot controls that are usually off by defaultClerk; she suggests turning them all on and asking the provider for tuning help
MCP host restrictionsAllow only MCP servers from real companies; block free tunnels as custom MCP hostsIn application code
  • Stripe Radar tip: she has Codex write and apply the complex Radar rules through Stripe’s API, then reviews and switches them on in the dashboard. Radar previews how many payments a rule would have blocked and how many real ones would have passed.
  • None of these needs a new product, she says; a coding agent can implement most of them. The same tools are available to the people committing the fraud.

Agent loops

  1. Fraud classification with a decision model. When a chat’s metadata crosses an abuse threshold, a zero-data-retention Jev run labels it: real customer, trial abuse, promo abuse, card fraud, resale, agent farming, and so on. Depending on category and tier, the account is banned or silently downgraded to a cheaper model.
  2. Hourly review agent. Looks at recent signups and out-of-bounds chat threads. High-signal cases are banned automatically; the rest ping her or go to the AI support desk.
  3. AI support desk with fraud tooling. It has fraud lists, allow lists, playbooks and canned responses for banned users. It cannot unban, ban or issue promo codes.
  4. Feed every incident back in. Each new pattern becomes a rule, an ops step and an agent check.
  • Two kinds of appeal arrive after a ban wave: badly written messages from throwaway addresses, and polite, well-argued ones that she says are often written with ChatGPT. Both get the same answer when the account used a fake code or stolen card.

Try It

  1. Check your model allowlist today. Find every place your backend calls a model provider and confirm the model name comes from your own config, not from the request.
  2. Add an hourly spend alert in dollars at the gateway or provider level, with a hard cap that fails closed.
  3. List your zero-dollar doors (free tier, trials, promo codes, referral credits, early access) and put each behind a feature flag you can turn off without a deploy.
  4. Turn on your auth provider’s bot protection and your payment processor’s fraud rules, and add a check for one card fingerprint across multiple trials.
  5. If you allow custom MCP servers, restrict the hosts and block tunnel domains.
  6. Add a classifier step for flagged conversations. Pick the categories that match your abuse, run them through a cheap decision model with zero data retention, and decide per category whether to ban, downgrade or review.
  7. Audit your support agent’s tools. Remove anything that grants access, credits or codes.
  8. Set per-account and per-organization usage caps based on what a real customer at that tier uses.

Implementation

Tool/Service: Vercel AI Gateway, Vercel BotID, Vercel Firewall, Stripe Radar, Clerk (auth), a decision model (Jev) for classification, a cheaper model such as GLM as the downgrade target, Codex for writing Radar rules and FBI reports. Setup: BotID must be added to each chat route; she ran it in monitoring first, tuned on real user profiles, then enforced. Auth-provider fraud settings are off by default. Cost: No product prices given. Her stated loss is “tens of thousands of dollars” over about a year. She says ChatPRD has had about a million users and has enterprise customers. Integration notes: She shows example code on screen and says it is illustrative; the transcript contains no code. Her setup is specific to a Vercel-hosted app with Stripe billing; the control list transfers, the vendor choices may not.

Open Questions

  • No independent figures. Loss totals, the $200,000 card-testing day and the country-block effect are her statements.
  • Gareth’s Atlassian post and her own X post are referenced but not in the source.
  • Abuse threshold. She does not say which metadata flags or values trigger the classifier.
  • Cross-product proxy rings. Her claim that the same actors hit other large AI products rests on prompt content she saw; no other company is named.
  • The second sponsor’s name (“Oros”) is a caption spelling and was not confirmed.