Small practices like dental clinics lose revenue because nobody answers the phone after hours. Missed appointment calls lead to empty slots and frustrated patients who expect 24/7 booking.
A complete, working implementation of this recipe — downloadable as a zip or browsable file by file. Generated by our build pipeline; tested with full coverage before publishing.
You’ll build a Twilio-powered voice agent that answers after-hours phone calls, transcribes the caller’s speech with OpenAI Whisper, classifies their intent, and books appointments through Calendly — all without human intervention. Along the way you’ll work with the REAA agent-mesh ecosystem for request orchestration, intent classification, session management, budget enforcement, and observability. By the end you’ll have a production-style Next.js application with full TypeScript coverage, a test suite, and two working API routes: one for inbound Twilio voice webhooks and one for health checks.
Prerequisites
Node.js >= 22 (the engines field requires it)
pnpm 10.x (the project uses pnpm@10.6.3 as its package manager; install it with corepack enable && corepack prepare pnpm@10.6.3 --activate)
Accounts and API keys:
A Twilio account with an active phone number (TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN, TWILIO_PHONE_NUMBER from console.twilio.com)
Run the package manager install. This pulls in Next.js, Twilio, OpenAI, all six REAA packages, and every dev dependency.
terminal
pnpm install
The postinstall script runs automatically (it calls sh bin/postinstall.sh) — the artifact provides a no-op hook. If you haven’t created that file yet the script will fail harmlessly; you can add an empty bin/postinstall.sh with #!/bin/sh\n# noop or skip the postinstall in package.json for now. All dependencies land in node_modules/ with exact pinned versions.
Step 3: Configure environment variables
Create .env.local at the project root. This file is ignored by git (you added it to .gitignore), so it’s safe for local secrets.
Fill in the empty values with your actual keys. AGENT_MESH_GATEWAY_API_KEY defaults to dev-key for local development. The voice middleware validates that TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN, OPENAI_API_KEY, and CALENDLY_API_KEY are set on every request — if any are missing the route returns a spoken error TwiML.
Step 4: Create the environment validator and logger
The application needs a startup guard that validates environment variables, and a logging re-export from the observability package.
Create src/lib/env.ts:
ts
/** * Validates that all required environment variables are set at startup. */export function validateEnv(): void { const required = [ 'TWILIO_ACCOUNT_SID', 'TWILIO_AUTH_TOKEN', 'OPENAI_API_KEY', 'CALENDLY_API_KEY', ] as const; const missing: string[] = []; for (const key of required) { if (!process.env[key]) { missing.push(key); } } if (missing.length > 0) { throw new Error( `Missing required environment variables: ${missing.join(', ')}. ` + 'Set them in .env or your deployment environment.', ); }}
Create src/lib/logger.ts:
ts
/** * Re-exports logger and child logger factory from agent-mesh-observability. * * The observability package provides Winston-based structured JSON logging * with automatic PII redaction (emails, phone numbers, SSNs, credit cards). */export { logger, createChildLogger } from '@reaatech/agent-mesh-observability';
The logger module re-exports from @reaatech/agent-mesh-observability, which gives you structured JSON logging with automatic PII redaction. The rest of the codebase imports from ./logger instead of reaching directly into the REAA package, keeping the dependency surface clean.
Step 5: Build the voice middleware
Every inbound Twilio webhook needs request tracing, env validation, and error recovery that returns spoken TwiML instead of crashing. The withVoiceMiddleware wrapper handles that.
Create src/lib/voice-middleware.ts:
ts
/** * Voice middleware — wraps Next.js route handlers with request tracing * and error recovery. * * Generates a request_id via crypto.randomUUID(), creates a scoped child * logger, wraps the handler in try/catch, and records session lookup * duration on success. */import { createChildLogger, recordSessionLookupDuration } from '@reaatech/agent-mesh-observability';import { validateEnv } from './env';/** * Wraps a route handler with voice middleware — request tracing, logging, * error handling, and env validation. */export function withVoiceMiddleware( handler: (req: Request) => Promise<Response>,): (req: Request) => Promise<Response> { return async (req: Request): Promise<Response> => { const requestId = crypto.randomUUID(); const logger = createChildLogger({ request_id: requestId }); try { validateEnv(); const start = Date.now(); const response = await handler(req); const durationMs = Date.now() - start; recordSessionLookupDuration(durationMs, false); return response; } catch (error) { const message = error instanceof Error ? error.message : String(error); logger.error('Voice handler error', { error: message }); const twiml = `<?xml version="1.0" encoding="UTF-8"?><Response> <Say>Sorry, we're experiencing a technical issue. Please try again later.</Say></Response>`; return new Response(twiml, { status: 200, headers: { 'Content-Type': 'text/xml' }, }); } };}
On success the middleware records the call duration as a metric via the observability package. On any thrown error it returns valid TwiML with a spoken apology — the caller hears a friendly message instead of silence.
Step 6: Create session management and intent classification
Session management wraps @reaatech/agent-mesh-session for the Twilio voice call lifecycle — creating and retrieving sessions by CallSid, appending conversation turns, and finalizing sessions.
Create src/lib/session.ts:
ts
/** * Session management wrapping @reaatech/agent-mesh-session. * * Provides getOrCreateVoiceSession, appendCallTurn, and finalizeSession * for the Twilio voice call lifecycle. */import { getActiveSession, createSession, appendTurn, closeSession,} from '@reaatech/agent-mesh-session';import { createChildLogger } from './logger';const log = createChildLogger({ module: 'session' });/** Turn entry shape for session history. */export interface TurnEntry { role: string; content: string; timestamp: string; intent_summary?: string;}/** Session record shape from agent-mesh-session. */export interface SessionRecord { session_id: string; user_id: string; employee_id: string; status: 'active' | 'completed' | 'abandoned' | 'error'; active_agent: string; turn_history: TurnEntry[]; workflow_state: Record<string, unknown>; created_at: string; updated_at: string; ttl: Date;}/** * Retrieves an active session for a call SID, or creates a new one. */export async function getOrCreateVoiceSession( callSid: string,): Promise<SessionRecord> { const existing = await getActiveSession(callSid); if (existing) { return existing as SessionRecord; } const session = await createSession({ userId: callSid, employeeId: callSid, activeAgent: 'scheduler', }); return session as SessionRecord;}/** * Appends a conversation turn to a session. */export async function appendCallTurn( sessionId: string, role: 'user' | 'agent', content: string, intentSummary?: string,): Promise<void> { await appendTurn(sessionId, { role, content, timestamp: new Date().toISOString(), intent_summary: intentSummary, });}/** * Finalizes a session with a terminal status. */export async function finalizeSession( sessionId: string, status: 'completed' | 'abandoned' | 'error',): Promise<void> { await closeSession(sessionId, status); log.info('Session closed', { session_id: sessionId, status });}
Now create the classifier. It registers two agents (scheduler and general) with example utterances, then delegates classification to the @reaatech/agent-mesh-classifier service — which uses Gemini Flash when available or falls back to keyword matching.
Create src/lib/classifier.ts:
ts
import { classifierService } from '@reaatech/agent-mesh-classifier';/** * Classifier output shape matching the agent-mesh-classifier return type. */export interface ClassifierOutput { agent_id: string; confidence: number; ambiguous: boolean; detected_language: string; intent_summary: string; entities: Record<string, unknown>;}const defaultRegistry = [ { agent_id: 'scheduler', display_name: 'Appointment Scheduler', description: 'Books, reschedules, or cancels appointments', endpoint: 'http://localhost:8080/v1/request', type: 'mcp' as const, is_default: true, confidence_threshold: 0.6, clarification_required: false, examples: [ 'I need to book a dental cleaning', 'Can I reschedule my appointment?', 'I want to cancel my visit', ], }, { agent_id: 'general', display_name: 'General Inquiry', description: 'Answers general questions and routes to staff', endpoint: 'http://localhost:8080/v1/request', type: 'mcp' as const, is_default: false, confidence_threshold: 0.6, clarification_required: false, examples: [ 'What are your business hours?', 'Do you accept my insurance?', 'I have a question about billing', ], },];export async function classifyIntent( transcript: string,): Promise<ClassifierOutput> { const result = await classifierService.classify(transcript, defaultRegistry); return result as ClassifierOutput;}export function isClassifierMock(): boolean { return classifierService.isMock();}
The registry entries include display names, descriptions, confidence thresholds, and example utterances. The classifier matches incoming speech to the most likely agent and returns a confidence score — the handoff layer uses this to decide whether to route, clarify, or fallback.
Step 7: Build handoff routing and gateway dispatch
The handoff module converts classifier output into a structured payload that follows the @reaatech/agent-handoff protocol, then decides which agent should handle the request.
Create src/lib/handoff.ts:
ts
/** * Handoff routing helpers — builds handoff payloads and routes to agents * based on classifier output. */import type { HandoffPayload, Message,} from '@reaatech/agent-handoff';import type { ClassifierOutput } from './classifier';/** * Custom routing decision type for this application. */export interface AppRoutingDecision { action: 'route' | 'clarify' | 'fallback'; targetAgent?: string; reason: string;}/** * Builds a HandoffPayload from classification output and session context. */export function buildHandoffPayload( intent: ClassifierOutput, sessionId: string, transcript: string,): HandoffPayload { const now = new Date(); const message: Message = { id: crypto.randomUUID(), role: 'user', content: transcript, timestamp: now, }; return { handoffId: crypto.randomUUID(), sessionId, conversationId: sessionId, sessionHistory: [message], compressedContext: { summary: intent.intent_summary, keyFacts: [], intents: [ { intent: intent.intent_summary, confidence: intent.confidence, entities: Object.keys(intent.entities), }, ], entities: Object.entries(intent.entities).map(([name, value]) => ({ name, type: typeof value === 'string' ? 'string' : 'unknown', value: value as string, resolved: true, })), openItems: [], compressionMethod: 'identity', originalTokenCount: transcript.length, compressedTokenCount: intent.intent_summary.length, compressionRatio: transcript.length > 0 ? intent.intent_summary.length / transcript.length : 1, }, handoffReason: { type: 'specialist_required' as const, requiredSkills: [intent.agent_id], currentAgentSkills: [], }, userMetadata: { userId: sessionId, }, conversationState: { resolvedEntities: intent.entities as Record<string, unknown>, openQuestions: [], contextVariables: {}, }, createdAt: now, };}/** * Routes a HandoffPayload to the appropriate agent based on classification. */export function routeToAgent( _payload: HandoffPayload, intent: ClassifierOutput,): AppRoutingDecision { if (intent.ambiguous) { return { action: 'clarify', reason: 'Intent ambiguous across agents. Need clarification.', }; } if (intent.agent_id === 'scheduler') { return { action: 'route', targetAgent: 'scheduler', reason: 'High-confidence scheduler intent', }; } if (intent.agent_id === 'general') { return { action: 'route', targetAgent: 'general', reason: 'General inquiry', }; } return { action: 'fallback', reason: 'No matching agent for intent', };}
Now create the gateway dispatch module. This is the bridge between the application and the agent-mesh orchestration layer — it sends transcripts to @reaatech/agent-mesh-gateway’s handleInternalRequest and shapes the response into a structured object.
Every LLM call costs money. The budget module prevents runaway spend with a BudgetController from @reaatech/agent-budget-engine, an in-memory spend store, and a static pricing provider that knows the per-call cost of whisper-1 ($0.006), gpt-4o ($0.005), and gpt-4o-mini ($0.00015).
Create src/lib/budget.ts:
ts
/** * Budget management for per-call AI cost tracking. * * Provides an InMemorySpendStore compatible with the BudgetController's * SpendStore interface, a PricingProvider for known models, and factory * functions to wire up budget enforcement per phone call. */import { BudgetController } from '@reaatech/agent-budget-engine';import { SpendStore } from '@reaatech/agent-budget-spend-tracker';import type { SpendEntry, BudgetScope } from '@reaatech/agent-budget-types';import { createChildLogger } from './logger';const log = createChildLogger({ module: 'budget' });/** Per-request USD cost for known models. */const MODEL_COSTS: Record<string, number
Each call gets a $1.00 hard cap with a soft warning at $0.80. The controller emits threshold-breach and hard-stop events that are logged through the observability layer. The pricingProvider knows the cost of each model — if you ask for a model it doesn’t recognize, it throws immediately rather than silently overspending.
Step 9: Create the scheduler and general agents
The scheduler agent is the workhorse: it extracts appointment details from a transcript via OpenAI’s gpt-4o-mini with JSON structured output, checks Calendly availability, and creates a scheduled event.
Create src/agents/scheduler.ts:
ts
/** * Scheduler Agent — parses appointment details from transcripts and * interacts with the Calendly API to find availability and book slots. */import OpenAI from 'openai';import { updateWorkflowState } from '@reaatech/agent-mesh-session';import { createChildLogger } from '../lib/logger';const log = createChildLogger({ module: 'scheduler-agent' });const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY,});/** * Parsed appointment details from the transcript. */export interface AppointmentDetails { patient_name: string;
The general agent handles non-appointment inquiries (billing questions, hours, insurance) by returning a polite message promising a human callback.
Create src/agents/general.ts:
ts
/** * General fallback agent — returns a polite message for non-appointment * inquiries. */export class GeneralAgent { /** * Handles a general inquiry by returning a human-transfer message. */ async handleInquiry(_input: string, _sessionId: string): Promise<string> { return "I'll have a staff member call you back during business hours to assist with your inquiry."; }}
Step 10: Build the media stream handler
When Twilio connects a call, it streams audio via a WebSocket. The media stream handler accumulates transcribed segments, then dispatches the completed transcript through the agent-mesh gateway.
Create src/lib/media-stream.ts:
ts
/** * Twilio Media Stream handler — processes audio chunks from a Twilio * Media Stream WebSocket, transcribes them via OpenAI Whisper, accumulates * the transcript, and dispatches to the agent-mesh gateway. */import OpenAI from 'openai';import { logger } from '@reaatech/agent-mesh-observability';import { dispatchToAgentMesh } from './gateway';const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });/** * Accumulates partial transcript segments as they arrive from Whisper. */export class AccumulatingTranscript { private segments: string[] = []; /** * Adds a transcribed segment to the accumulation buffer.
The AccumulatingTranscript class collects partial transcription segments as they stream in from Whisper. transcribeAudioChunk takes a base64-encoded audio payload, wraps it in a File object, and sends it to the Whisper API. handleCompletedTranscript dispatches the full text through the agent-mesh gateway and returns the spoken response — or a graceful fallback message on failure.
Step 11: Create the API routes
Now you’ll create the two API routes. The main one is the Twilio voice webhook at POST /api/twilio-voice. It validates the Twilio signature, creates or retrieves a session, builds a TwiML response that connects the call to a media stream, and fires off an initial dispatch to the agent mesh.
Create src/app/api/twilio-voice/route.ts:
ts
import twilio from 'twilio';import { withVoiceMiddleware } from '@/lib/voice-middleware';import { createChildLogger, recordSessionLookupDuration } from '@reaatech/agent-mesh-observability';import { getOrCreateVoiceSession, appendCallTurn } from '@/lib/session';import { dispatchToAgentMesh } from '@/lib/gateway';const log = createChildLogger({ module: 'twilio-voice' });function parseFormBody(text: string): Record<string, string> { const params: Record<string, string> = {};
The health check endpoint probes gateway reachability and confirms the OpenAI key is set.
Create src/app/api/healthz/route.ts:
ts
/** * Health check endpoint for monitoring readiness. * Returns 200 with service status or 503 if dependencies are unreachable. */import { initOtel } from '@reaatech/agent-mesh-observability';import { handleInternalRequest } from '@reaatech/agent-mesh-gateway';// Initialize OpenTelemetry at module load (fails silently if no endpoint)try { initOtel();} catch { // No OTLP endpoint configured — observability operates without tracing}/** * GET handler — health check. */export async function GET(): Promise<Response> { const checks: Record<string, string> = {}; // Check gateway reachability try { await handleInternalRequest({ input: 'health', user_id: 'health-check', session_id: crypto.randomUUID(), }); checks.gateway = 'ok'; } catch { checks.gateway = 'unreachable'; } // Check OpenAI config checks.openai = process.env.OPENAI_API_KEY ? 'ok' : 'missing'; const allOk = Object.values(checks).every((v) => v === 'ok'); return new Response( JSON.stringify({ status: allOk ? 'ok' : 'degraded', checks, }), { status: allOk ? 200 : 503, headers: { 'Content-Type': 'application/json' }, }, );}
GET requests to the Twilio webhook return a plain OK. The healthz endpoint returns {"status":"ok","checks":{"gateway":"ok","openai":"ok"}} when everything is healthy, or {"status":"degraded",...} with a 503 status code if a dependency is down.
Step 12: Write the tests and run them
The test suite uses vitest with mocked externals. Create the test setup file that mocks Twilio, OpenAI, and all REAA packages so tests never make real network calls.
Your Twilio voice agent is ready. Point your Twilio phone number’s voice webhook URL to https://<your-domain>/api/twilio-voice (use a tunnel like ngrok for local testing) and start handling calls.
Next steps
Configure a real Twilio Media Stream WebSocket handler to replace the placeholder — the artifact’s media-stream.ts is designed to be wired into a WebSocket route that the TwiML <Connect><Stream> element connects to.
Replace the in-memory spend store with a persistent store (Postgres, Redis) so budget enforcement survives server restarts.
Deploy to Vercel: push to GitHub, import the project, set all environment variables in the Vercel dashboard, and update the Twilio webhook URL to your production domain.
>
=
{
'whisper-1': 0.006,
'gpt-4o': 0.005,
'gpt-4o-mini': 0.00015,
};
/**
* In-memory spend store compatible with BudgetController's SpendStore interface.
* Extends the real SpendStore class for full type compatibility.
*/
export class InMemorySpendStore extends SpendStore {
private data = new Map<string, number>();
/**
* Records a spend amount for a scope and returns the updated total.
return `Great news, ${details.patient_name}! Your appointment for ${details.reason} has been booked on ${details.date_time}. You'll receive a confirmation shortly.`;
