voice-agent-kit · packages

A scaffolding CLI that generates a complete voice-agent-kit project with pipeline configuration, STT/TTS provider setup, telephony or WebRTC transport, an MCP client, and a ready-to-run server.

A Zod-validated configuration system and pipeline orchestrator for building voice-enabled AI agents, providing a `createPipeline()` function that coordinates STT, MCP, and TTS stages with latency enforcement, session management, and OpenTelemetry observability.

A JSON-RPC 2.0 client that connects to any MCP server endpoint, providing tool discovery, conversation history management, retry with backoff, and TTS-safe response sanitization. It exposes an `MCPClient` class with `connect()`, `sendRequest()`, `discoverTools()`, and `close()` methods, and requires only a `fetch`-compatible runtime.

A CLI and programmatic simulator that runs a voice agent pipeline (STT → MCP → TTS) locally from a WAV file or live microphone, reporting per-turn latency without requiring Twilio or cloud telephony. Exports `createSimulator`, `Simulator`, `renderLatencyWaterfall`, `readWavFile`, `writeWavFile`, `captureMicrophone`, and `playAudio`.

Provider-agnostic speech-to-text interface with a unified `STTProvider` class and seven adapter implementations (Deepgram, AWS Transcribe, Google Cloud Speech-to-Text, OpenAI Realtime, OpenAI Whisper, AssemblyAI, Groq Whisper), plus built-in audio format conversion between mulaw and linear16 with sample rate resampling.

A WebSocket handler for voice AI agents that normalizes bidirectional streaming protocols (start/media/stop/mark/DTMF) across Twilio, Telnyx, SignalWire, and Vonage, providing barge-in detection, audio buffering, and base64 encoding/decoding. Exports a `TwilioMediaStreamHandler` class (EventEmitter) with methods like `acceptConnection`, `sendAudio`, `clearAudio`, and `sendMark`, plus static `encodeForTwilio`/`decodeFromTwilio` utilities.

Provider-agnostic text-to-speech interface with five adapter implementations (Deepgram Aura, AWS Polly, Google Cloud TTS, ElevenLabs, Cartesia), returning streaming audio as `AsyncIterable<AudioChunk>` with cancelable synthesis and Twilio-ready audio formatting.

A WebSocket-based transport class (`WebRTCTransport`) for browser voice AI agents that handles Opus encode/decode, PCM resampling, and barge-in detection, paired with standalone PCM utility functions. Requires a WebSocket connection and optionally `@discordjs/opus` or `prism-media` for Opus support.