Jonathon's AI Wiki

HyperFrames Video Components, HTML-in-Canvas & Variables

8 min read

Source: Video Components, HTML-in-Canvas, Variables — HeyGen HyperFrames docs

Three HyperFrames primitives that turn one-off compositions into a reusable system. Video components are the catalog/registry layer — 50+ production-ready blocks and components you install with npx hyperframes add <name> instead of rebuilding scenes from scratch. HTML-in-Canvas (drawElementImage) captures live, rendered DOM into a canvas at GPU speed so any HTML — dashboards, app UIs — becomes a WebGL texture for shaders and 3D. Variables are the templating primitive: typed, named slots declared on the composition <html> root that let the same source render different content from a parent comp, the CLI, or an API call. Together they answer “install a block, drop your brand colors and clips into it, and render it a hundred ways.”

Key Takeaways

  • The catalog ships two install shapes, wired differently. Blocks are full standalone scenes with their own size + duration — install to compositions/<name>.html, wire with data-composition-src + a timeline position on a host element. Components are reusable snippets/effects that adapt to the host — install to compositions/components/<name>.html, wired by pasting their HTML/CSS/JS into your composition. One command installs either: npx hyperframes add <name>.
  • The registry is agent-discoverable. The /hyperframes-registry skill (from npx skills add heygen-com/hyperframes) discovers, installs, and wires a catalog item from a plain request (“add a chart block”); every item lists name, description, tags, dimensions, and duration.
  • Catalog contribution bar is a hard contract. Each item is one self-contained HTML file with a paused GSAP timeline, must be deterministic (seeded randomness only — no Date.now() / Math.random()), must register its timeline on window.__timelines, and must pass hyperframes lint + validate before npx hyperframes publish + PR.
  • HTML-in-Canvas needs a Chrome flag — auto-enabled at render. drawElementImage is experimental; live preview in Studio needs chrome://flags/#canvas-draw-element → CanvasDrawElement: Enabled. HyperFrames sets --enable-features=CanvasDrawElement automatically during render (and inside the Docker container), so final output needs no manual setup.
  • Why HTML-in-Canvas beats html2canvas. It uses the browser’s own compositor (not a JS DOM re-parse), so it is pixel-perfect across every CSS feature (backdrop-filter, complex shadows, web fonts), GPU-accelerated at 60fps, supports live/animated/scrolling content, and allows multiple simultaneous <canvas layoutsubtree> captures with no nesting restriction.
  • Variables are typed and required-field-strict. Declare a JSON array on data-composition-variables; every entry needs id (unique), type, label, and default. Five types: string, number, color, boolean, enum (each with type-specific extra options like min/max/unit or options).
  • A string URL variable is the escape hatch for media — swap images, video clips, or audio tracks per render by assigning the URL to the element’s src in your composition script. Pass URL references, not inline base64 (a 256 KiB execution-input cap applies on distributed/Lambda renders).
  • Video/audio media rules when parameterized. Keep the timing attributes (data-start, data-duration, data-track-index, data-has-audio) on the <video>/<audio> element itself; the probe phase scans video[data-start] after your script runs and reads the resolved src for pre-extraction. data-duration is optional on <video>/<audio> — omit it and the renderer ffprobes the source’s natural length. The HyperFrames-wide rule that <video> must be muted (with separate <audio> elements for sound) is documented on the HTML schema / hub, not these three pages. ^[inferred — these pages show muted playsinline on the example <video> but don’t restate the rule]
  • Three-layer variable precedence: declared default (lowest) → per-instance data-variable-values on a sub-comp host (middle) → CLI --variables (highest). A missing key falls through to the next lower layer.
  • Batch + strict validation are built in. --batch rows.json renders one output per data row ({key}/{index} output placeholders, a written manifest.json); --strict-variables turns undeclared/type-mismatch/enum-out-of-range warnings into a non-zero exit for CI.

Video Components — the catalog/registry

  • What it is: a registry of 50+ production-ready video components — captions, code animations, social overlays, shader transitions, data viz, liquid-glass/VFX, CSS transitions, code snippets — each installable in one command and rendering to a deterministic MP4.
  • Categories called out in the docs: Code Animations, Captions (15 styles), Social Overlays (X / TikTok / Instagram / Reddit / Spotify / lower thirds), Shader Transitions (14 WebGL), Liquid Glass & VFX (HTML-in-canvas), Data (charts + US/world/region maps), CSS Transitions (13 zero-dependency), Code Snippets (24 cards/terminal themes), plus Effects, Text Effects, and Showcases.

Blocks vs components:

BlocksComponents
What it isA full standalone sceneA reusable snippet / effect
Own size & durationYesNo — adapts to the host
Installs tocompositions/<name>.htmlcompositions/components/<name>.html
Wired bydata-composition-src on a host elementPasting its HTML / CSS / JS into your composition
Examplesx-post, data-chart, code-morphgrain-overlay, caption-highlight, shimmer-sweep

Install + wire a block:

npx hyperframes add data-chart
<!-- index.html — blocks are standalone compositions, included by src + timeline position -->
<div
  data-composition-id="data-chart"
  data-composition-src="compositions/data-chart.html"
  data-start="0"
  data-duration="15"
  data-track-index="1"
  data-width="1920"
  data-height="1080"
></div>

A component instead has its HTML pasted into your markup, its CSS into your styles, and any JS into your script, with its timeline calls hooked into yours.

Contributing back: fork the repo → write one HTML file with a paused GSAP timeline → add registry-item.json → hyperframes lint + validate → npx hyperframes publish → open a PR. No build step, no framework; deterministic + window.__timelines registration + production-quality bar required.

HTML-in-Canvas — DOM as a WebGL texture

The four-step flow: put HTML inside a <canvas layoutsubtree>, let the browser render the children as normal DOM, capture the pixels with ctx.drawElementImage(element, x, y, w, h), then use the canvas as a Three.js texture (apply shaders, map to 3D geometry).

<!-- 1. HTML content lives inside the canvas -->
<canvas id="capture" layoutsubtree width="1920" height="1080">
  <div class="my-dashboard">
    <h1>Revenue: $4.2M</h1>
    <div class="chart">...</div>
  </div>
</canvas>
 
<!-- 2. WebGL canvas for 3D rendering -->
<canvas id="theater" width="1920" height="1080"></canvas>
// 3. Capture HTML to canvas
var capCanvas = document.getElementById("capture");
var ctx = capCanvas.getContext("2d");
ctx.drawElementImage(capCanvas.querySelector(".my-dashboard"), 0, 0, 1920, 1080);
 
// 4. Use as Three.js texture
var texture = new THREE.CanvasTexture(capCanvas);
var material = new THREE.MeshBasicMaterial({ map: texture });
  • Always feature-detect first. Check "layoutSubtree" in canvas and typeof ctx.drawElementImage === "function"; fall back gracefully (draw text directly, use a static image) for browsers without the flag.
  • Animated content re-captures per frame. For scrolling/transitions/counters, call drawElementImage inside the render loop, set texture.needsUpdate = true, then renderer.render(scene, camera).
  • Nine catalog VFX blocks install via npx hyperframes add html-in-canvas (all) or individually: ios26-liquid-glass, macos-tahoe-liquid-glass, liquid-glass-notification, vfx-iphone-device (real 3D GLTF devices with live HTML screens), vfx-text-cursor, vfx-portal, vfx-shatter, vfx-magnetic, vfx-liquid-background.

Variables — the templating primitive

Declare on the <html> root; read at runtime with __hyperframes.getVariables() (destructure with defaults matching the declared defaults). Works identically in top-level and sub-composition scripts — the runtime scopes each sub-comp instance to its own resolved values.

<html data-composition-variables='[
  {"id":"title","type":"string","label":"Title","default":"Untitled"},
  {"id":"color","type":"color","label":"Color","default":"#111827"}
]'>
  <body>
    <div data-composition-id="card" data-width="1920" data-height="1080">
      <h1 class="card-title"></h1>
      <script>
        const { title = "Untitled", color = "#111827" } = __hyperframes.getVariables();
        const root = document.querySelector('[data-composition-id="card"]');
        root.querySelector(".card-title").textContent = title;
        root.style.setProperty("--card-color", color);
      </script>
    </div>
  </body>
</html>

Variable types:

TypedefaultExtra options
string"some text"placeholder?, maxLength?
number0min?, max?, step?, unit?
color"#rrggbb"—
booleantrue / false—
enumone of the option valuesoptions: [{value, label}]

Override layers (lowest → highest precedence):

  • Per-instance — data-variable-values='{"title":"Pro","color":"#ff4d4f"}' on a sub-comp host element; the same card.html runs with different content simultaneously.
  • CLI (top-level only) — --variables '{...}' or --variables-file ./vars.json; add --strict-variables to fail CI on undeclared/mistyped keys.
  • Batch — --variables/--variables-file per the precedence table; one render per row via --batch rows.json with {key}/{index} output paths, optional --batch-concurrency 2.

What can’t be a variable (read once at compile time / outside the comp): composition dimensions (data-width/data-height), frame rate (--fps), output format/codec/quality, and a sibling/parent comp’s variables. The rule: variables drive anything the renderer reads from the live DOM after your script runs; they can’t change compile-time or CLI-level inputs.

  • Color grading by reference: inside data-color-grading, use $name or ${name} as a whole field value and the runtime resolves it from the comp’s variables before applying the shader grade.
  • Static inspection: extractCompositionMetadata(html).variables from @hyperframes/core reads declarations without rendering (same API Studio uses to build the variables panel).
  • Lint (npx hyperframes lint) statically catches malformed JSON, missing required fields, and type-vs-default mismatches.

Try It

  1. In a HyperFrames project, npx hyperframes add data-chart, then paste the printed data-composition-src host snippet into index.html and npx hyperframes preview.
  2. Add data-composition-variables (a title string + color color) to a card comp, read them with __hyperframes.getVariables(), then re-render two ways: --variables '{"title":"Pro","color":"#ff4d4f"}' and again with different values.
  3. Drive a batch: write rows.json with 2–3 rows and run npx hyperframes render --batch rows.json --output "renders/{name}.mp4" --strict-variables; confirm the manifest.json written next to the outputs.
  4. Try one HTML-in-Canvas block — npx hyperframes add vfx-shatter — and preview it (the flag is auto-enabled at render; for Studio live preview, enable chrome://flags/#canvas-draw-element).

Open Questions

  • The <video>-must-be-muted + separate-<audio> rule and the no-async/await-in-GSAP-setup rules are HyperFrames-wide conventions referenced on the hub/schema but are not restated on these three pages — verify exact wording against the HTML-schema and core-concepts docs before relying on them here.
  • These pages cite “50+” catalog items while the block catalog inventory counts ~109 — the gap is likely blocks-vs-(blocks+components) counting, but the precise split is not stated.
  • The 256 KiB execution-input cap is referenced for “large variables” on Lambda; the per-page docs link out to a Templates-on-Lambda deploy page not ingested here for the full distributed-render variable-size handling.

Graph View

Backlinks