A complete, working implementation of this recipe — downloadable as a zip or browsable file by file. Generated by our build pipeline; tested with full coverage before publishing.
This tutorial walks you through building a voice agent that lets diners manage OpenTable reservations over the phone using natural conversation. The agent uses Twilio to handle inbound calls, Deepgram for speech-to-text, OpenAI’s GPT model for intent classification, Cartesia for text-to-speech, and a custom OpenTable API client for reservation operations. You’ll wire everything together using the @reaatech/voice-agent-core pipeline orchestrator, with Langfuse tracing and OpenTelemetry for observability.
Prerequisites
Node.js >= 22 and pnpm 10.x
A Twilio account with a phone number that supports Media Streams
Deepgram API key (speech-to-text)
Cartesia API key (text-to-speech)
OpenAI API key with access to gpt-5.2
OpenTable API access (base URL and API key)
Langfuse account (optional — for tracing)
Familiarity with TypeScript, Next.js App Router, and Zod schemas
Step 1: Create the project and install dependencies
Create a new Next.js project with TypeScript, then install all the dependencies you’ll need.
cd openai-voice-agent-for-opentable-reservation-management
Now install the runtime dependencies. The @reaatech/* packages provide the voice pipeline, STT/TTS providers, session continuity, and structured output repair.
Expected output:package.json now contains all these packages at exact pinned versions. The node_modules/ directory is populated.
Step 2: Configure environment variables
Create .env.example as a template, then copy it to .env and fill in your real API keys.
env
# Env vars used by openai-voice-agent-for-opentable-reservation-management.# The builder adds entries here as it wires up each integration.# Keep placeholders only — never commit real values.NODE_ENV=developmentOPENAI_API_KEY=<your-openai-api-key>TWILIO_ACCOUNT_SID=<your-twilio-account-sid>TWILIO_AUTH_TOKEN=<your-twilio-auth-token>TWILIO_PHONE_NUMBER=<your-twilio-phone-number>DEEPGRAM_API_KEY=<your-deepgram-api-key>CARTESIA_API_KEY=<your-cartesia-api-key>OPENTABLE_API_KEY=<your-opentable-api-key>OPENTABLE_BASE_URL=https://platform.opentable.comLANGFUSE_PUBLIC_KEY=<your-langfuse-public-key>LANGFUSE_SECRET_KEY=<your-langfuse-secret-key>LANGFUSE_BASE_URL=https://cloud.langfuse.comOTLP_ENDPOINT=http://localhost:4318/v1/tracesPORT=8080MCP_ENDPOINT=<mcp-server-url>PUBLIC_URL=<publicly-routable-url-for-twilio-webhooks>
Expected output: Your .env.example file is committed to version control so other developers know what to configure. Your .env file (gitignored) holds the real values that the application reads at startup.
Step 3: Create the Zod-validated application config
This module reads environment variables and validates them with Zod. It also builds the pipeline configuration object used by @reaatech/voice-agent-core.
Expected output:pnpm typecheck passes without errors. The config validates env vars at runtime and throws descriptive Zod errors when required values are missing.
Step 4: Define OpenTable operation schemas
You’ll define the Zod schemas that describe every OpenTable operation the agent can perform — checking availability, creating, updating, and cancelling reservations. These schemas also power the tool definitions that the LLM sees.
Expected output: The Zod schemas enforce that party sizes are positive integers, reservation IDs are non-empty strings, and unknown actions are rejected at parse time.
Step 5: Build the OpenTable API client
This HTTP client wraps the OpenTable REST API. Each method corresponds to one of the four reservation operations, with proper error handling for HTTP errors, network failures, and timeouts.
ts
// src/agent/opentable-tool.tsimport type { TableAvailabilityQuery, TableAvailabilityResult, ReservationCreateParams, ReservationUpdateParams, ReservationCancelParams } from "./opentable-schemas.js";import { TableAvailabilityQuerySchema, ReservationCreateParamsSchema, ReservationUpdateParamsSchema, ReservationCancelParamsSchema } from "./opentable-schemas.js";import type { MCPTool } from "@reaatech/voice-agent-mcp-client";function schemaToRecord(schema: object): Record<string, unknown> { return schema as Record<string, unknown>;}export class ApiError extends Error { public status: number; constructor
Expected output:OpentableApiClient uses AbortController for timeouts, throws typed errors (ApiError for non-2xx, ApiConnectionError for network/timeout), and supports the full CRUD cycle.
Step 6: Create the tool execution dispatcher
This module takes the LLM’s parsed tool call and dispatches it to the right method on the OpenTable API client. It validates parameters against the Zod schemas before executing and returns TTS-friendly result objects.
ts
// src/agent/tool-executor.tsimport { OpentableApiClient } from "./opentable-tool.js";import type { AgentToolCall } from "./opentable-schemas.js";import { ReservationCreateParamsSchema, ReservationUpdateParamsSchema, ReservationCancelParamsSchema, TableAvailabilityQuerySchema } from "./opentable-schemas.js";export type ToolResult = { type: string; payload: Record<string, unknown> };export async function executeToolAction( client: OpentableApiClient, toolCall: AgentToolCall): Promise<ToolResult> { try { const action = toolCall.action; if (action.action === "checkAvailability") { const params = TableAvailabilityQuerySchema.parse(action.parameters); const result = await client.checkAvailability(params); return { type: "availability", payload: { slots: result.slots } }; } if (action.action === "createReservation") { const params = ReservationCreateParamsSchema.parse(action.parameters); const result = await client.createReservation(params); return { type: "reservation_created", payload: result }; } if (action.action === "updateReservation") { const params = ReservationUpdateParamsSchema.parse(action.parameters); const result = await client.updateReservation(params); return { type: "reservation_updated", payload: result }; } const params = ReservationCancelParamsSchema.parse(action.parameters); const result = await client.cancelReservation(params); return { type: "reservation_cancelled", payload: { success: result.success } }; } catch (err) { const error = err as Error; if ("issues" in error) { return { type: "error", payload: { error: `Invalid parameters: ${error.message}` } }; } return { type: "error", payload: { error: error.message } }; }}
Expected output: Unknown or invalid actions return an error result instead of throwing. Each result has a type string suitable for your TTS formatting logic.
Step 7: Wire up the LLM orchestrator and structured repair
The LLM orchestrator sends the caller’s transcript and conversation history to OpenAI and parses the JSON response into a tool call. Because LLM JSON output is often unreliable, you’ll pass it through @reaatech/structured-repair-core to handle markdown fences, truncated JSON, type coercion, and hallucinated fields.
ts
// src/agent/llm-orchestrator.tsimport OpenAI from "openai";import type { AgentToolCall } from "./opentable-schemas.js";const SYSTEM_PROMPT = `You are a reservation management assistant for OpenTable. You help diners manage their reservations.You must output a JSON object with the following structure:{ "action": { "action": "<action_type>", "parameters": { ... } }, "confidence": <number between 0 and 1>}Supported actions:1. checkAvailability — Check table availability Parameters: { "partySize": <number>, "dateTime": "<ISO datetime>", "durationMinutes": <number> }2. createReservation — Create a new reservation Parameters: { "restaurantId": "<string>", "partySize": <number>, "dateTime": "<ISO datetime>", "customerName": "<string>", "customerPhone": "<string>", "specialRequests": "<string>" }3. updateReservation — Update an existing reservation Parameters: { "reservationId": "<string>", "partySize": <number>, "dateTime": "<ISO datetime>", "specialRequests": "<string>" }
Now create the structured repair wrapper that handles malformed LLM output:
ts
// src/agent/structured-repair.tsimport { repair, repairOutput, isValid } from "@reaatech/structured-repair-core";import { AgentToolCallSchema } from "./opentable-schemas.js";import type { AgentToolCall } from "./opentable-schemas.js";export async function repairToolCall(raw: string): Promise<AgentToolCall> { try { return await repair(AgentToolCallSchema, raw); } catch { return { action: { action: "none" as const, parameters: {} }, confidence: 0, }; }}export function repairToolCallWithDiagnostics( raw: string): import("@reaatech/structured-repair-core").RepairResult<AgentToolCall> { return repairOutput({ schema: AgentToolCallSchema, input: raw, debug: true, strategies: ["strip-fences", "fix-json-syntax", "coerce-types", "fuzzy-match-keys", "remove-extra-fields"], });}export function isValidToolJson(data: unknown): data is AgentToolCall { if (typeof data === "string") { return isValid(AgentToolCallSchema, data); } try { return isValid(AgentToolCallSchema, JSON.stringify(data)); } catch { return false; }}
Expected output: When the LLM wraps JSON in ```json...``` fences, repairToolCall strips them. When it’s completely garbled, the fallback returns { action: "none" } so the pipeline stays alive.
Step 8: Set up session continuity and conversation management
Use @reaatech/session-continuity to store session data and conversation history. The in-memory storage adapter implements the IStorageAdapter interface, and the token counter approximates token counts for budget enforcement.
ts
// src/services/conversation-store.tsimport type { IStorageAdapter, Session, SessionId, Message, MessageId, MessageQueryOptions, SessionFilters, UpdateSessionOptions, TokenCounter, HealthStatus,} from "@reaatech/session-continuity";export class InMemoryStorageAdapter implements IStorageAdapter { private sessions = new Map<SessionId, Session>(); private messages = new Map<SessionId, Message[]>(); private messageSeq = 0; createSession
Now create the conversation manager that wraps SessionManager with voice-session-specific helpers:
Expected output:createConversationManager() returns a SessionManager with a 4096-token budget, sliding-window compression, and in-memory storage.
Step 9: Build the observability module
Langfuse traces every voice turn, while OpenTelemetry provides infrastructure-level spans. A cost tracker accumulates Deepgram and Cartesia usage per session.
Expected output: Langfuse and OTel initialize without crashing even when API keys are missing. initLangfuse returns null when keys aren’t provided, and the rest of the app handles that gracefully.
Step 10: Create the pipeline orchestration service
This is the heart of the application. The pipeline service wires together STT (Deepgram), TTS (Cartesia), the MCP client, the LLM orchestrator, structured repair, and the OpenTable tool executor into a single event-driven pipeline using @reaatech/voice-agent-core.
ts
// src/services/pipeline-service.tsimport { loadAppConfig } from "../config.js";import { createPipeline, createLatencyBudget, initializeSessionManager, LatencyBudgetEnforcer } from "@reaatech/voice-agent-core";import { createSTTProvider } from "@reaatech/voice-agent-stt";import { createTTSProvider, type TTSProvider } from "@reaatech/voice-agent-tts";import { MCPClient } from "@reaatech/voice-agent-mcp-client";import { generateResponse } from "../agent/llm-orchestrator.js";import { repairToolCall } from "../agent/structured-repair.js";import { executeToolAction } from "../agent/tool-executor.js";import type { ToolResult } from "../agent/tool-executor.js";import { OpentableApiClient, createOpentableToolDefinitions }
Expected output: The pipeline registers handlers for stt:final, error, and turn:end events. On the stt:final event, it runs the full chain — LLM → repair → tool execute → TTS — and streams audio back via the registered callbacks.
Step 11: Set up the Fastify WebSocket server and Next.js API routes
The Fastify server handles the WebSocket connection for Twilio Media Streams. Twilio sends raw audio from the caller’s phone call over this WebSocket, and the server streams TTS responses back.
ts
// src/server.tsimport Fastify from "fastify";import websocket from "@fastify/websocket";import { createTwilioHandler } from "@reaatech/voice-agent-telephony";import { loadAppConfig } from "./config.js";import { initializeServices, startSession, processAudio, endSession, cancelTTS, setTTSChunkCallback, setTTSStartCallback, setTTSEndCallback, getActiveSessionCount,} from "./services/pipeline-service.js";import { shutdown } from "./services/observability.js";import type { AudioChunk } from "@reaatech/voice-agent-core";
Now create the Next.js API route that Twilio calls when a call comes in. It returns TwiML that instructs Twilio to open a Media Stream WebSocket to your Fastify server. Place this file at the project root under app/, not under src/:
// app/api/health/route.tsimport { NextResponse } from "next/server";import { getActiveSessionCount } from "../../../src/services/pipeline-service.js";export function GET() { return NextResponse.json({ status: "ok", timestamp: new Date().toISOString(), activeSessions: getActiveSessionCount(), });}
Expected output: When you call POST /api/twilio/webhook with a host header, it returns TwiML containing a <Stream> element pointing to ws://<host>/media-stream. The health endpoint at GET /api/health returns { "status": "ok", "activeSessions": 0 }.
Step 12: Create the bootstrap entry point and wire up tests
The entry point initializes Langfuse, OTel, and the cost tracker before starting the Fastify server:
Now write the test suite. Each module gets happy-path, error-path, and boundary tests. Here’s a test for the config module:
ts
// tests/config.test.tsimport { describe, it, expect, beforeEach, afterEach } from "vitest";import { vi } from "vitest";import { loadAppConfig } from "../src/config.js";const ORIGINAL_ENV = { ...process.env };beforeEach(() => { vi.unstubAllEnvs();});afterEach(() => { process.env = { ...ORIGINAL_ENV };});describe("loadAppConfig", () => { it("returns valid AppConfig when all required env vars are set", () => {
Write similar tests for the OpenTable schemas, API client, tool executor, LLM orchestrator, structured repair, conversation store, pipeline service, observability module, and server routes. The vitest config enforces >= 90% coverage on lines, branches, functions, and statements.
Run the full test suite:
terminal
pnpm test
Expected output:pnpm test runs vitest run --coverage and outputs a JSON report to vitest-report.json. All tests pass and coverage meets the 90% threshold across all four categories.
Step 13: Run the application
Start the development server:
terminal
pnpm dev
Expected output: The console shows [server] listening on :8080 and [bootstrap] env: with your non-secret environment variables logged.
Configure your Twilio phone number’s voice webhook to POST https://<your-public-domain>/api/twilio/webhook. When someone calls the number, Twilio fetches the TwiML, opens a Media Stream WebSocket to your server, and the voice agent takes over — transcribing speech, calling OpenAI for intent classification, executing OpenTable operations, and streaming TTS responses back to the caller.
Next steps
Add a database-backed session store — Replace InMemoryStorageAdapter with a Postgres or Redis adapter for persistence across server restarts.
Support multiple languages — Configure Deepgram’s language option and the LLM system prompt to handle Spanish, French, or other languages.
Add conversational summaries — After a session ends, use the LLM to generate a summary of the reservation conversation and store it alongside the session data.
Deploy with HTTPS — For production deployment, front the WebSocket server with an HTTPS/TLS termination proxy and set PUBLIC_URL to your domain.
(status
:
number
, message
:
string
) {
super(message);
this.name = "ApiError";
this.status = status;
}
}
export class ApiConnectionError extends Error {
constructor(message: string) {
super(message);
this.name = "ApiConnectionError";
}
}
export class OpentableApiClient {
private baseUrl: string;
private apiKey: string;
private timeoutMs: number;
constructor(config: { baseUrl: string; apiKey: string; timeoutMs: number }) {
