Ollama + Claude Code — Cutting Token Costs with Local and Open Models

Source: Ollama + Claude Code = 99% Cheaper (YouTube video, https://youtu.be/sboNwYmH3AY) + raw/GLM_5.2_in_Claude_Code_is_Blowing_My_Mind.md (z.ai / GLM 5.2 engine-swap path)

Claude Code is the harness; the model is the engine. By default that engine is Opus, Sonnet, or Haiku via Anthropic’s API — which is what generates token charges and rate limits. This source walks through swapping the engine for an open-weight model running either locally via Ollama or routed through OpenRouter’s free tier, keeping the Claude Code agent harness intact while collapsing per-token cost. Anthropic ToS allows it because you keep using their harness; you just stop hitting their inference endpoint.

Key Takeaways

• Two routes to “free” Claude Code: local Ollama models on your own hardware, or OpenRouter’s free model tier in the cloud. Both swap the engine without changing the harness.
• Open-weight models have caught up to mid-tier closed models. On SWE-bench Verified, top open models now beat Sonnet 3.7. Opus 4.6 still leads for high-stakes work.
• Hardware constrains local quality. A 9B Qwen 3.5 (~6.6 GB) runs on a typical laptop but feels significantly slower than Anthropic’s API; larger models require more RAM/GPU.
• Context windows are smaller and lie by default. Ollama models often advertise 200k context but ship with a much smaller default — explicit Modelfile overrides (e.g., 64k) are required to make Claude Code’s system prompt fit.
• Tool-call visibility degrades on smaller models. A 9B local model “spins forever then responds”; larger cloud-hosted opens (e.g., MiniMax M 2.7 via Ollama Cloud) restore the streaming tool-call view.
• OpenRouter’s free tier requires the four-variable override. Setting only ANTHROPIC_MODEL silently leaves Haiku/Sonnet on Anthropic’s paid endpoints for tool calls — you must override every model slot.
• Cost framing: even when not strictly “free,” Gemma 4 31B via OpenRouter is ~$0.14/MinputvsOpus4.6^{\prime }s$5/M — roughly 35x cheaper per token, which the source frames as the more realistic win.
• z.ai (GLM) is a third engine-swap path. Set via the same env override (ANTHROPIC_BASE_URL -> z.ai’s API, ANTHROPIC_AUTH_TOKEN -> z.ai key, ANTHROPIC_API_KEY blank, all model slots -> the GLM model). Unlike Ollama/OpenRouter, z.ai offers Claude-Code-style subscription plans (~$16/$64/$144permonth,cheaperyearly)with5-hour+weeklyquotasandaseparateweb-searchquota,alongsideper-tokenpricing(creatorcitesGLMinput/outputfarbelowOpus4.8^{\prime }s$5/$25). ^[inferred — exact GLM 5.2 price points are from a single creator video; treat as approximate]

Implementation

Tool/Service: Ollama (local) + Claude Code, with optional Ollama Cloud and OpenRouter as alternates.

Setup:

Local-only path (free, slow, private):

1. Download Ollama from ollama.com for your OS.
2. ollama pull qwen3.5:9b (or pick a size your RAM/GPU can handle).
3. Optional but recommended — create a context-extended variant via a Modelfile so Claude Code’s system prompt fits (default Ollama context is often <8k even when the model card advertises 200k).
4. Launch Claude Code through Ollama’s helper: ollama launch claude then pick the local model.

Cloud-hosted open model via Ollama Cloud (faster, paid above free tier):

1. From Ollama, use the same ollama launch claude flow but pick a cloud-only model (e.g., MiniMax M 2.7) — sign in at ollama.com to authorize.

OpenRouter path (cheapest cloud, free tier with rate limits):

1. Create an OpenRouter account at openrouter.ai. Add ~$10 to lift the free-model rate limit from 50/day to 1,000/day.
2. Create an API key.
3. Edit .claude/settings.local.json and override the API base + every model slot:

   {
     "env": {
       "ANTHROPIC_BASE_URL": "https://openrouter.ai/api/v1",
       "ANTHROPIC_AUTH_TOKEN": "<openrouter-key>",
       "ANTHROPIC_API_KEY": "",
       "ANTHROPIC_MODEL": "qwen/qwen3.5-coder:free",
       "ANTHROPIC_SMALL_FAST_MODEL": "qwen/qwen3.5-coder:free",
       "ANTHROPIC_DEFAULT_HAIKU_MODEL": "qwen/qwen3.5-coder:free",
       "ANTHROPIC_DEFAULT_SONNET_MODEL": "qwen/qwen3.5-coder:free"
     }
   }

4. Launch claude — header should read “OpenRouter free, API billing usage.”

z.ai (GLM) path (subscription option, GLM 5.2 the model the creator used):

1. Create a z.ai account and API key.
2. Edit .claude/settings.local.json and override the API base + every model slot:

   {
     "env": {
       "ANTHROPIC_BASE_URL": "<z.ai-api-endpoint>",
       "ANTHROPIC_AUTH_TOKEN": "<z.ai-key>",
       "ANTHROPIC_API_KEY": "",
       "ANTHROPIC_MODEL": "<glm-model>",
       "ANTHROPIC_SMALL_FAST_MODEL": "<glm-model>",
       "ANTHROPIC_DEFAULT_HAIKU_MODEL": "<glm-model>",
       "ANTHROPIC_DEFAULT_SONNET_MODEL": "<glm-model>"
     }
   }

3. Pricing is dual-mode, unlike Ollama/OpenRouter: subscription plans run ~$16/$64/$144permonth(cheaperbilledyearly)with5-hour+weeklyusagequotasandaseparateweb-searchquota,\ast \ast or\ast \ast per-tokenpricingwithGLMinput/outputratesthecreatorcitesasfarbelowOpus4.8^{\prime }s$5/$25. ^[inferred — GLM 5.2 price points are from a single creator video; treat as approximate]
4. The creator used GLM 5.2 as the model for this swap.

Cost: Local Ollama: $0/token,hardware-bound.OpenRouterfreemodels:$0/token within 50/day (no credits) or 1,000/day ($10deposit).OpenRouterpaidopens(e.g.,Gemma431B):$0.14/M input, ~$0.40/MoutputvsOpus4.6^{\prime }s$5/M and $25/M.Anthropiconboardingstillrequiresaone-time$5 credit purchase to activate the API key, but the credit is never consumed if every model slot routes elsewhere.

Integration notes: Anthropic ToS explicitly allows this — you’re using the agent harness, just pointing it at a different inference endpoint. Tool calling is the main fragility: smaller open models may not match Claude’s expected JSON tool-call protocol, and web-search tools that ship as native Claude capabilities (not MCP) often fail on swapped engines — fall back to Brave/Tavily/Perplexity MCP servers explicitly.

Try It

• Pick a tier: local for privacy + zero ongoing cost, OpenRouter for speed at near-zero cost.
• For local, ollama pull a model sized to your RAM (rule of thumb from the source: ask Claude Code itself “here are my specs, which model size fits?”).
• If context errors appear, build a Modelfile variant with PARAMETER num_ctx 64000 (or larger) and re-launch — Ollama’s default context is the silent failure point.
• Route low-stakes operations to the cheap engine: file reads, grep-style searches, scaffolding, classification, triage. Keep Opus for high-stakes architectural decisions and security-sensitive code.
• When using OpenRouter, double-check your usage logs after the first session — if Haiku charges appear, you missed one of the four model env vars.
• Route by directory: put the override block in a project’s .claude/settings.local.json so that folder runs the alt model (GLM) while folders without it fall back to your default Claude plan — letting you keep one project on GLM and another on Opus.

Two corrections and an update to the z.ai path (2026-08-25)

GLM 5.3 is the current model, and z.ai’s coding plan is now quoted at **from $18/month\ast \ast —against$200 for Claude Max or Codex Pro. It is documented as officially working inside both Claude Code and Codex. Note what the entry tier is not: the $18plancarriessmaller\ast \ast 5-hour\ast and\ast weekly\ast \ast limitsand,perthesource,is"notthesametokencountorsameproblem-solvingabilityasthe$200 plan.” The opportunity is task routing, not replacement.

Codex has its own path, and it is cleaner than the Claude Code one. z.ai publishes a Responses-compatible endpoint specifically for Codex. Register it as a model provider in your personal Codex config (address plus the name of the env var holding the key), define a named profile, and launch with codex --profile glm. Ordinary codex keeps your OpenAI setup, and both can run at once — no env-var override of every model slot.

A cost trap this article did not carry: switching model mid-conversation with /model keeps the conversation, but per Anthropic’s own warning the next response rereads the whole history without the old prompt caches. A late-in-the-job switch can be slower and materially more expensive than the token price suggests, even when the new model is cheaper. The rule that follows — start a substantial job on the model you expect to finish it — and the handoff-file discipline for when you must move a job anyway are in Running a Cheap Model Beside Your Expensive One, which is the “which work, and how do you hand it over” companion to this article’s “how do you configure it.”

Local models moved too. The Qwen 3.5 9B figures above are now two generations old. Qwen 3.8 27B scores 52 on the Artificial Analysis Intelligence Index at 18–23GB quantized, and there is a measured Apple Silicon configuration reaching ~0.25s to first token on warm cache — see Qwen 3.8 27B.

Related

• Running a Cheap Model Beside Your Expensive One — the routing and handoff discipline that sits on top of these mechanics

• Qwen 3.8 27B — The Local-Model Inflection — the current local option, with measured numbers

• GLM-5.2 — Z.ai’s Open-Weight Agentic-Coding Frontier — primary-source profile of the GLM model itself (official 744B-A40B MoE, 1M context, benchmarks vs Opus 4.8); this article is the how-to-plug-it-into-Claude-Code companion.

• Opus 4.7 Best Practices

• Claude Code CLI Reference

• Essential MCP Servers for 2026

• Claude Code Plugins and Marketplaces

• The Advisor Strategy (advisor_20260301)