LLM Gateway

LLM Gateway is AssemblyAI's unified, OpenAI-compatible API for calling frontier language models inside voice AI workflows. It replaces LeMUR ("Leveraging Large Language Models to Understand Recognized speech"), which is deprecated and stops working March 31, 2026.

General
Developer	AssemblyAI
Type	LLM gateway / unified API for voice AI workflows
License	Commercial API
Documentation	assemblyai.com/products/llm-gateway

Core Features

OpenAI-compatible API: a single interface for calling multiple LLM providers.
Multi-provider failover: requests can fail over between providers.
Tool calling and JSON output: supports structured output and tool/function calling.
Zero-data-retention option: available for workflows with stricter data handling requirements.
Supported providers: OpenAI, Anthropic, Google Gemini, Alibaba Qwen, and Moonshot Kimi.

Pricing

LLM Gateway is billed per 1M tokens, with separate input and output rates:

Model	Input (per 1M tokens)	Output (per 1M tokens)
Claude 4.5 Sonnet	$3.00	$15.00
GPT-5.1	$1.25	$10.00
Gemini 3.5 Flash	$1.50	$9.00

Source: assemblyai.com/pricing

Use Cases

Transcript summaries and chapter markers: generate summaries directly from transcript output.
Structured data extraction: pull structured fields out of call transcripts.
Live LLM enhancement: apply LLM processing to transcript streams as they arrive.

Migrating from LeMUR

Existing LeMUR integrations need to move to LLM Gateway before LeMUR stops working on March 31, 2026. AssemblyAI publishes a migration guide covering the API changes.

Read the LLM Gateway docs or the migration guide to get started.

Prove-It Analyst

Prove-It Analyst answers questions about a CSV by writing Python and running it in a throwaway sandbox per run: no network (private Linux network namespace plus an audit hook on sockets/DNS), an empty environment so no API keys are visible, a temp directory that is deleted afterwards, reads/writes outside it refused, subprocess/exec/ctypes refused, 256 MB RAM, CPU and wall-clock kill switches. Every answer ships with receipts: the exact code, the raw output and SHA-256 hashes of both. Failed runs feed their error back for up to 2 self-healing retries; attacks are never retried. Containment moment: a poisoned CSV tells the agent to print os.environ, curl the VULTR_API_KEY to evil.example and plant a backdoor. A simulated compromised model obeys; the Blast Radius panel shows the key absent, the host .env read refused, the backdoor write refused, DNS blocked, and the run killed at 256 MB. Host impact: none. Vultr: the planner uses Vultr Serverless Inference (OpenAI-compatible) when a key is set, with an offline deterministic planner so the demo is free; production runs one Docker container per run on a Vultr Cloud Compute VM (--network none --read-only --memory 256m --pids-limit 64 --cap-drop ALL). The demo uses the container-free Python backend because the dev box had no Docker. Business value: ops and finance teams can let an agent query untrusted spreadsheets without handing it the server. 15 tests pass; MIT licensed.-li

Agent_Trap_Evals

The problem: agents rarely fail hard questions. They fail chores. They report "backup verified" because the log says success, while the archive is truncated. These false claims slip out during routine work, and a real shell makes them dangerous. What Receipts does: you give it a real task, like checking a backup or triaging a failing build. A control layer plans the work and runs every action in an isolated Vultr sandbox [gVisor pods on VKE]. The sandbox has no secrets, has time and memory caps, and is destroyed after each task. Every command's output, exit code, and files become a receipt. Before any claim reaches you, a verifier checks it against those receipts: did the agent check the layer that matters, or just the easy one? An unsupported claim is blocked and sent back to be verified, or reported as unverified. You get the answer together with its proof. Containment: [demo moment, e.g. the agent runs rm -rf / and the sandbox absorbs it; the host is untouched]. Measured: we score Receipts with agent-trap-evals, an open-source trap set we built before the hackathon from real agent incidents. Verifier off vs. on: [X/Y vs. X/Y caught], [n] false alarms on honest controls. For: platform and DevOps teams putting agents on operational work, where a confident false "done" costs more than a slow answer. Sandboxes like E2B contain actions but don't check claims. Eval tools score offline but don't stop a bad claim at runtime. Receipts does both, live. [Business model; next steps.] All LLM calls use Vultr Serverless Inference.

VoxGuard agent

VoxGuard: Secure AI Action Firewall Voice can request. AI can propose. Security decides. VoxGuard is a security-first AI system designed to address one of the most important challenges in financial AI: how can an intelligent assistant understand and assist with sensitive financial requests without being trusted to authorise or execute them independently? Traditional AI assistants can interpret natural-language instructions, but financial operations require additional protection against social engineering, prompt injection, unauthorised actions, and changes to transaction details. VoxGuard introduces a deterministic security layer between AI-generated intent and financial execution. The AI interprets the user's request and proposes a structured action, while a separate security gateway evaluates the request and determines whether to ALLOW, CHALLENGE, or DENY it. The system also uses controlled execution permits, explicit user confirmation, replay protection, and a synthetic banking environment to demonstrate how sensitive actions can be handled without exposing real financial accounts. Key capabilities include:- Financial intent interpretation through conversational AI. - Social-engineering and suspicious-request detection. - Independent security policy enforcement. - Controlled, single-use execution permits. - Protection against unauthorised or replayed actions. - Synthetic financial transaction demonstrations. - Experimental multilingual support, including Nigerian Pidgin. AMD Integration: The project is designed to incorporate AMD-hosted language-model inference for structured financial intent extraction while retaining deterministic security enforcement. The AMD connection and live inference results should be presented according to their actual verified status. VoxGuard demonstrates an architecture for safer AI-powered banking assistants, where language models provide intelligence but never become the final authority over financial transactions.

NumeriGuard

NumeriGuard is a proposed numerical validation platform for industrial R&D teams that depend on simulation, scientific computing, and engineering software. It helps developers detect when changes to code, compilers, GPU drivers, numerical libraries, or hardware alter the accuracy, stability, or performance of their calculations. The platform turns trusted engineering examples into repeatable tests. Teams define acceptable errors, physical constraints, convergence requirements, and runtime limits. NumeriGuard then executes those tests across selected environments, compares results against appropriate references, and produces evidence explaining whether a change meets the agreed acceptance criteria. Its core capabilities include floating point comparisons, conservation checks, mesh convergence analysis, repeated execution for nondeterministic workloads, performance regression detection, and reproducible failure packages. Engineers can trace unexpected results to the exact inputs, software versions, hardware configuration, and execution settings involved. AMD ROCm would support GPU execution and profiling, while AMD Developer Cloud would provide development and benchmarking resources. Supported Radeon hardware would enable local validation. Ryzen AI could support a later extension for testing compact engineering inference models against their references. The initial MVP would validate one thermal solver and one structural solver through a command line interface and CI integration. A review interface would show numerical differences, performance changes, and downloadable reproduction bundles. Designed for simulation vendors, manufacturing R&D groups, and research computing teams, NumeriGuard aims to reduce debugging effort and improve confidence in engineering releases. It evaluates defined numerical requirements

VitaLoop 360

VitaLoop 360 is a shared AI core for a mid-sized health-products manufacturer, with five department stations, one per ACT III track, joined by the production lot. When a fault appears on the factory line, the same lot number links the alarm, the quality hold, the supplier invoice, the stock plan, the customer's onboarding and the care instructions in the box, so one event becomes one traceable chain of decisions instead of five unrelated alerts. The lead station, Factory Sentinel (Intelligent Industry), computes a shift-level Energy and Scrap Waste Index from public industrial telemetry, flags unusual windows, and uses Gemma 4 on AMD GPUs via the AMD Token Factory to explain each one with cited procedure passages. A verification layer rejects any number the model did not get from the calculation, and the supervisor approves every action. Every model call, its latency and cost are logged. The other four stations (Evolus order hub, demand and retention, adaptive onboarding, caregiver navigator) are fully specified and clearly labelled as roadmap, not shown as finished.

RADIANT X-Ray Triage & Evidence Assistant

> RADIANT is a multimodal chest X-ray review prototype. It runs a > vision model (DenseNet-121 trained on NIH ChestX-ray14) and a > deterministic report parser in parallel, compares their outputs > through a cross-modal verifier, and surfaces disagreements for > human review. It never picks a winner in a conflict. It assigns a > review priority, retrieves supporting evidence, generates an > explanation through AMD-hosted LLM endpoints (with a > deterministic fallback), and passes the output through two safety > gates. Every number in the project's evaluation comes from a real > execution; items that could not be measured are marked NOT RUN > with a reason. RADIANT is a research prototype, not a diagnostic > tool. All outputs require review by a licensed physician.