Jonathon's AI Wiki

Venice AI — Private LLM Inference with Verifiable TEE Attestation

5 min read

Source: raw/How_Venice_AI_Achieves_Privacy_in_AI_And_Watch_Me_Prove_It.md (YouTube, Tonbi’s AI Garage, youtube.com/watch?v=i3dSlJs2oww, published 2026-06-04)

A technical walkthrough plus a live cryptographic proof of how Venice AI delivers private LLM inference — from metadata-stripping proxies up to hardware-sealed enclaves whose attestations the creator verifies on camera with a Python script. Relevant here because private inference is a drop-in concern for agent stacks: the demo wires Venice into Hermes Agent as a custom OpenAI-compatible endpoint, trading some capability (web search, memory) and a price premium for provable confidentiality.

Creator: Tonbi’s AI Garage (the same creator behind the Hermes Masterclass series and the multi-agent Kanban workflow) | Platform: YouTube | Published: 2026-06-04

Key Takeaways

  • Four escalating privacy tiers:
    1. Anonymous — a proxy strips identity, IP, and metadata; gives access to frontier models (the quality play), but Venice itself can still see prompt content.
    2. Private (default) — runs on Venice/partner zero-retention GPUs: a contractual no-logs guarantee, not hardware-enforced. Serves open models (DeepSeek, Kimi, GLM-class).
    3. TEE — inference inside trusted-execution-environment enclaves (Intel TDX CPUs + Nvidia confidential GPUs): the operator physically cannot read prompts, and the claim is provable via attestation, not just policy.
    4. E2EE (beta) — the prompt is encrypted on-device to the enclave’s key via ECDH before it ever leaves your machine; plaintext exists only inside the enclave. Strongest guarantee, slower responses.
  • Attestation is the load-bearing difference. The chip signs a report with root keys only genuine Intel/AMD/Nvidia silicon holds, proving (a) real hardware, (b) the exact measured code, (c) execution inside the sealed enclave. The enclave generates its own keypair internally and ECDSA-signs responses; a fresh client-generated nonce echoed back proves liveness (not a replay). All of it verifies client-side against the vendors’ public keys — the creator runs the verification script in the video.
  • The infra is decentralized confidential compute: Phala Network and NEAR AI cloud orchestrate the TEEs across independent hardware hosts. The creator’s read (which he explicitly invites correction on): the blockchains secure keys and trust (e.g., Phala can gate enclave key-release via an on-chain registry), while inference itself runs off-chain in the enclaves.
  • The honest trade-offs: TEE/E2EE modes disable web search and memory — both require reading prompt data outside the enclave. And privacy costs money: comparing the same open models on the Private tier vs OpenRouter, Llama 3.3 was 0.10, Kimi K2.6 0.57, DeepSeek only slightly higher — “you’re not just buying tokens, you’re buying attestation.” (Spot prices at recording; expect drift.)
  • Agent integration is plain OpenAI-compatible: the demo configures Venice in Hermes Agent as a custom endpoint — base URL https://api.venice.ai/api/v1 + API key, Chat Completions format — and 85 models show up. Any agent runtime that accepts a custom OpenAI-compatible provider can do the same.
  • Account floor is low: free account (Google sign-in, no card); some TEE/E2EE models require the paid tier.

Where this fits for agent builders

Local models are the usual answer to AI privacy, but they cap quality at what your hardware runs. Venice’s pitch is the middle path: cloud-class models with cryptographically verifiable confidentiality — a different trust model from both “trust our privacy policy” (every mainstream API) and “trust your own GPU” (local). For agent workloads that touch client data (the recurring concern in this wiki’s marketing-agency context), the TEE tier turns “the vendor promises not to look” into “the vendor cannot look, and here’s the proof” — at the cost of the search/memory features agents often lean on.

Implementation

  • Tool/Service: Venice AI (venice.ai) — private-inference API + chat app
  • Setup: free account via Google sign-in → generate API key → in Hermes Agent (or any OpenAI-compatible runtime) add a custom provider with base URL https://api.venice.ai/api/v1 → pick a model per privacy tier
  • Cost: free tier to start; per-token API pricing carries a consistent premium over OpenRouter for the same open models (see numbers above); some TEE/E2EE models gated to paid plans
  • Integration notes: OpenAI Chat Completions-compatible; 85 models listed at recording; TEE/E2EE tiers disable web search + memory; E2EE adds latency

Try It

  1. Create a free Venice account (Google sign-in) and generate an API key.
  2. Add Venice as a custom endpoint in Hermes Agent (https://api.venice.ai/api/v1) and confirm the model list loads.
  3. Run one prompt per tier (Private → TEE → E2EE) and note the capability/latency differences.
  4. For the trust-but-verify step: request the enclave attestation with a fresh nonce and check the signature chain against the Intel/Nvidia public keys, as demonstrated in the video.

Open Questions

  • The creator explicitly flags uncertainty about the exact on-chain role of Phala/NEAR (“please let me know if anyone from NEAR or Phala [can confirm] what I’m saying here”) — the keys-and-trust-on-chain / compute-off-chain split is his best reading, not vendor-confirmed.
  • Pricing premiums are spot observations at recording time; no SLA/throughput comparison vs OpenRouter was made.
  • How Venice’s attestation UX compares to other confidential-inference offerings (e.g., cloud-vendor confidential-computing endpoints) is untested here.

Graph View

Backlinks