Jonathon's AI Wiki

AI Video Tools

6 min read

Practical AI video production: avatar models, automation pipelines, composition frameworks, and motion graphics. Each article is compiled from primary sources (product pages, technical reports, tutorials, and open-source repos) rather than press summaries.

AI Avatars & Generation Models

  • HeyGen Avatar V — Production-scale video-reference avatar model. 15s webcam clip → unlimited-duration 1080p twins with preserved identity, talking rhythm, and gestures. 175+ language lip-sync. State-of-the-art vs Kling, Veo, OmniHuman, Seedance.
  • HeyGen Studio Automation with Claude Code — Three-tool production pipeline (ElevenLabs + HeyGen + Remotion) orchestrated by Claude Code. Script-to-finished-video overnight. Open-source Python template, 6-stage pipeline, Avatar V Playwright workaround, cost breakdown.

Editing & Assembly

  • video-use (browser-use) — Claude Code skill for conversational video editing. Transcript-driven cuts (ElevenLabs Scribe word-level + diarization), parallel animation sub-agents (PIL / Manim / Remotion), self-evaluating render loop. 12 hard rules for production correctness, artistic freedom elsewhere. Python + ffmpeg, “100% open source.” 3,151 stars.

Composition & Motion Graphics

  • HeyGen Hyperframes — Open-source HTML-based video composition framework (Apache 2.0). Ships /hyperframes, /hyperframes-cli, /gsap skills for Claude Code, Cursor, Codex, Gemini CLI. Deterministic rendering, Frame Adapter pattern, 50+ prebuilt blocks.
  • Remotion Motion Graphics — AI motion graphics generator converting natural language prompts into React-based Remotion animations. Constants-first code generation, in-browser Babel compilation, live preview.
  • Claude Code Video Toolkit (Digital Samba) — Open-source AI-native video production workspace for Claude Code (MIT, 890 stars). 10 skills, 13 slash commands, templates, brand profiles, transitions library. OSS model stack (Qwen3-TTS, FLUX.2, LTX-2, ACE-Step, SadTalker) on your Modal/RunPod account. Typical cost $1–2/month.

Higgsfield (API-first generative video)

  • Higgsfield Overview — API platform for generative AI (images + videos). Async queue: submit → poll-or-webhook → fetch. Credit-based billing. API key + secret auth. Base URL platform.higgsfield.ai.
  • skills (Official Skills Bundle) — Vendor-published Markdown SKILL.md bundle for Claude Code, Cursor, Codex (MIT, 102★, v0.3.0). Four skills: higgsfield-generate (30+ models + Marketing Studio), higgsfield-soul-id (face training → reusable reference_id), higgsfield-product-photoshoot (10 modes with backend prompt enhancement on gpt_image_2), higgsfield-marketplace-cards (Amazon-style listing assets — main + secondaries + A+ modules with hidden marketplace-compliant templates). Three install paths (npx skills add / gh skill install / Claude Code /plugin marketplace add); CLI install + auth handled automatically. The productized version of the Nate Herk CLI-over-MCP thesis; supersedes Higgsfield MCP as the recommended primary agent surface. COOKBOOK ships three end-to-end recipes (founder-photo brand campaign, URL-to-4-ad-modes UGC batch, recurring founder team-update video).
  • Higgsfield MCP — Conversational surface. Custom connector at https://mcp.higgsfield.ai/mcp drops the full image + video stack inside Claude / OpenClaw / Hermes / NemoClaw. No API keys; sign in with your Higgsfield account. 16+ image models, 17+ video models, 9 video presets, Soul Characters, multi-model side-by-side, Product Ads from a URL.
  • Higgsfield + Claude Code Ad Agency Workflow — End-to-end DTC campaign in one Claude Code conversation. Firecrawl MCP scrapes brand brief → GPT Image 2.0 hero static (4 variations, Claude grades) → copy overlay → Seedance 2.0 hero animation → UGC creator generation → 2 UGC video clips. Claude as orchestration layer + creative director. ~170 credits per campaign before iteration. Mike Futia / SCALE AI tutorial.
  • Higgsfield MCP Tutorial — Brand Book, Storyboard, Landing Page (Robo Nuggets) — Sister tutorial to the Mike Futia ad-agency workflow but optimized for a brand-launch use case. End-to-end Spiderhead AI demo: logo → side-by-side brand books from Nano Banana 2 vs GPT Image 2 → 6-panel logo-animation storyboard → Seedance 2.0 video → mockup landing page → Claude Code (VS Code fork-conversation) translating mockup to a working localhost landing page. Two-path setup (~30s, desktop-app or /mcp for Antigravity-class IDEs). Operator gotcha: per-call credit cost not yet returned by MCP. Cost-aware confirmation pattern before high-credit ops (e.g., 720p Seedance). Higgsfield-vs-fal.ai decision call: existing subscribers should use the MCP; new users should evaluate fal.ai for pay-as-you-go on the same models.
  • Higgsfield MCP — 50-Ad Instagram Campaign from One Product Image (Claude Desktop) — Third Higgsfield-MCP tutorial, narrower than the other two: pure ad-campaign-at-scale use case. Connects MCP via Claude Desktop’s Connectors UI; runs a four-phase chain (research with Playwright MCP scraping Meta Ads Library → 5×5×2 ad matrix with human-approval gate → batched generation across Nano Banana 2 (product) + Soul 2 (humans) → local download organized by batch/aspect-ratio). Ends by saving the entire workflow as a /ad-creator skill via /skill-creator, turning the multi-prompt build into a one-command tool. Operator gotchas captured: Claude misreporting Playwright availability (verify, don’t trust); generations land in Higgsfield Community tab → My Generations not the main image gallery; pre-flight list_workspaces + balance check before 50-image runs (~949 credits used). Settings: Opus 4.7 extra-high, bypass-permissions, project folder mounted. Pairs with Meta Ads CLI for end-to-end generate-and-upload.
  • Higgsfield as a Creative Agency in Claude (Nate Herk) — Fourth Higgsfield+Claude tutorial, three new dimensions vs the prior trio: (1) CLI-over-MCP architectural call for agentic work on token-cost grounds (“the MCP has all those tools, so from a token perspective it’s actually more expensive — the CLI is just better for agents”); (2) skill reverse-engineering workflow turning a single winning prompt into a reusable .claude/skills/hypermotion-video/SKILL.md recipe that compounds across runs; (3) two-routine scaling pattern (Sunday-plan + Monday-generate) that grows asset bank from 50 → 100 → 200 ads per week while operator sleeps, with a Google Workspace CLI–created Sheet acting as the cross-routine asset database. Demoed on fictional headphone brand “Murmur” + a sleep-supplement bottle with Higgsfield Marketing Studio’s Hypermotion preset. Operator gotchas: skill written mid-session needs Claude restart to register; reference-image fidelity needs explicit “must appear exactly as shown” wording; rejected generations diagnose-and-retry via Claude reading its own prompt. Companion to the same author’s ElevenLabs voice-agents tutorial (same direct-vendor-CLI architecture for ElevenLabs).
  • Higgsfield Image-to-Video — Three featured models (higgsfield-ai/dop/preview, Bytedance Seedance Pro, Kling v2.1 Pro). Motion-prompt template: describe movement + set pace + specify camera moves. Technical checklist.
  • Higgsfield SDK (Python) — pip install higgsfield-client. Auth via HF_KEY or HF_API_KEY+HF_API_SECRET env vars. Four usage patterns (submit-and-wait, polling, callback, request management). File uploads for bytes/paths/PIL. JS/TS coming soon.
  • Higgsfield Webhooks — Add hf_webhook query param to submit URL. Delivers completed/failed/NSFW final states. 2-hour retry window. HTTPS + 2xx in 10s required. Idempotency via request_id.
  • Higgsfield Training Framework (OSS Origin) — Historical context. Apache-2.0 distributed-training framework at higgsfield-ai/higgsfield (3.6k stars, last push 2024-05-25). Pre-pivot artifact — the company moved from trillion-param LLM training infra to consumer creative AI. Listed for vendor-lineage awareness, not for use.

18 items under this folder.