SMB employees lose time logging into Freshservice to report IT problems; phone calls are faster but create manual data entry. Without voice integration, small IT teams face delays.
A complete, working implementation of this recipe — downloadable as a zip or browsable file by file. Generated by our build pipeline; tested with full coverage before publishing.
Employees who need IT help shouldn’t have to log into a portal, find the right form, and type out their issue. This recipe builds an Azure AI-powered voice agent that answers a Twilio phone number, listens to the caller describe their IT problem, and automatically creates a Freshservice ticket — no typing, no portal login. You’ll wire up Deepgram for speech-to-text and text-to-speech (via the @reaatech/voice-agent-stt and @reaatech/voice-agent-tts wrappers), Azure AI Inference for LLM chat, the @reaatech/voice-agent-core pipeline for orchestration, and @reaatech/structured-repair-core to turn spoken descriptions into valid Freshservice ticket data. By the end, you’ll have a fully testable Next.js 16 project that handles real Twilio media streams.
Prerequisites
Node.js 22+ and pnpm 10 installed on your machine
A Twilio account with a phone number that has Voice capabilities (trial works)
A Deepgram API key (used as the STT and TTS backend via @reaatech/voice-agent-stt and @reaatech/voice-agent-tts)
A Freshservice instance with an API key (admin-generated under your profile settings)
An Azure AI Inference resource endpoint and key (deploy a model such as GPT-4o-mini)
Basic familiarity with TypeScript and Next.js App Router conventions
Step 1: Scaffold the project and install dependencies
Start from an empty directory and initialize a Next.js 16 project with TypeScript and App Router support:
terminal
pnpm create next --typescript
--app
.
Then install the exact dependency versions this recipe uses. Every version is pinned — no range operators:
Expected output: Your package.json should list all dependencies with exact semver numbers (no ^ or ~ prefixes). The scaffolded Next.js files — tsconfig.json, eslint.config.mjs, next.config.ts, vitest.config.ts, .gitignore — are already in place.
Step 2: Configure environment variables
Copy the provided example file to .env and fill in your own values. Every service this recipe calls has its own variable:
env
# Env vars used by azure-ai-voice-agent-for-freshservice-itsm-ticket-creation.# Keep placeholders only — never commit real values.NODE_ENV=developmentTWILIO_ACCOUNT_SID=<your-twilio-account-sid>TWILIO_AUTH_TOKEN=<your-twilio-auth-token>DEEPGRAM_API_KEY=<your-deepgram-api-key>FRESHSERVICE_API_KEY=<your-freshservice-api-key>FRESHSERVICE_DOMAIN=<your-subdomain>AZURE_AI_INFERENCE_ENDPOINT=<https://your-resource.services.ai.azure.com/models>AZURE_AI_INFERENCE_KEY=<your-azure-key>WS_PORT=8080
Expected output: The file compiles without type errors when read via process.env.X. The src/lib/config.ts module (which you’ll create next) validates every required variable at boot time and throws a descriptive error if any are missing.
Step 3: Define TypeScript types for Freshservice and voice config
Create the type definitions your services will share. Start with Freshservice domain types in src/types/freshservice.ts:
Expected output: Both files compile cleanly. The types re-export @reaatech/voice-agent-core and @reaatech/agent-handoff types so consumers don’t need to import from those packages directly.
Step 4: Build the configuration loader
Create src/lib/config.ts to read all environment variables, validate them, and return a typed VoiceAgentConfig. Every missing required variable produces a clear error:
ts
import { VoiceAgentConfig } from '../types/voice.js';export function createVoiceAgentConfig(): VoiceAgentConfig { const twilioAccountSid = process.env.TWILIO_ACCOUNT_SID; const twilioAuthToken = process.env.TWILIO_AUTH_TOKEN; const deepgramApiKey = process.env.DEEPGRAM_API_KEY; const freshserviceApiKey = process.env.FRESHSERVICE_API_KEY; const freshserviceDomain = process.env.FRESHSERVICE_DOMAIN; const azureEndpoint = process.env.AZURE_AI_INFERENCE_ENDPOINT ?? ''; const azureKey = process.env.AZURE_AI_INFERENCE_KEY; const wsPortStr = process.env.WS_PORT; if (!twilioAccountSid) throw new Error('TWILIO_ACCOUNT_SID is required'); if (!twilioAuthToken) throw new Error('TWILIO_AUTH_TOKEN is required'); if (!deepgramApiKey) throw new Error('DEEPGRAM_API_KEY is required'); if (!freshserviceApiKey) throw new Error('FRESHSERVICE_API_KEY is required'); if (!freshserviceDomain) throw new Error('FRESHSERVICE_DOMAIN is required'); if (!azureKey) throw new Error('AZURE_AI_INFERENCE_KEY is required'); const wsPort = wsPortStr ? parseInt(wsPortStr, 10) : 8080; if (isNaN(wsPort)) throw new Error('WS_PORT must be a number'); return { twilio: { accountSid: twilioAccountSid, authToken: twilioAuthToken }, deepgram: { apiKey: deepgramApiKey }, freshservice: { apiKey: freshserviceApiKey, domain: freshserviceDomain }, azure: { endpoint: azureEndpoint, apiKey: azureKey }, wsPort, };}export interface TwilioConfig { accountSid: string; authToken: string;}export function getTwilioConfig(): TwilioConfig { const accountSid = process.env.TWILIO_ACCOUNT_SID; const authToken = process.env.TWILIO_AUTH_TOKEN; if (!accountSid) throw new Error('TWILIO_ACCOUNT_SID is required'); if (!authToken) throw new Error('TWILIO_AUTH_TOKEN is required'); return { accountSid, authToken };}
Expected output: When all env vars are set, createVoiceAgentConfig() returns a fully populated config object. Missing any required key throws immediately, so misconfiguration is caught at startup rather than mid-call.
Step 5: Create the Azure AI inference client
This client wraps the @azure-rest/ai-inference SDK with a typed error class and a chat() method that sends messages to your deployed model. Create src/lib/azure-ai-client.ts:
ts
import * as AzureAI from '@azure-rest/ai-inference';export class AzureAiError extends Error { public status: number; public details: unknown; constructor(message: string, status: number, details: unknown) { super(message); this.name = 'AzureAiError'; this.status = status; this.details = details; }}export class AzureAiClient { private endpoint: string; private apiKey: string; private client: AzureAI.ModelClient; constructor(endpoint: string, apiKey: string) { this.endpoint = endpoint.replace(/\/$/, ''); this.apiKey = apiKey; this.client = AzureAI.default(this.endpoint, { key: apiKey }); } async chat(messages: { role: string; content: string }[]): Promise<string> { try { const response = await this.client.path('/chat/completions').post({ body: { messages, }, }); if (AzureAI.isUnexpected(response)) { throw new AzureAiError( `Azure AI inference returned ${response.status}`, Number(response.status), response.body, ); } const choice = response.body.choices[0]; if (choice?.message.content === undefined) { throw new AzureAiError('No content in Azure AI response', 200, response.body); } return choice.message.content ?? ''; } catch (error) { if (error instanceof AzureAiError) { throw error; } throw new AzureAiError( 'Network error connecting to Azure AI', 0, error, ); } }}
Expected output: The AzureAiClient sends a chat completion request via @azure-rest/ai-inference and returns the text content. Network failures and non-OK responses throw AzureAiError with the HTTP status and response body for debugging.
Step 6: Build the Freshservice API client
The Freshservice REST API authenticates via Basic Auth with your API key. Create src/lib/freshservice-client.ts with createTicket() and getTicket() methods:
ts
import type { FreshserviceTicket, FreshserviceTicketResponse } from '../types/freshservice.js';export class FreshserviceApiError extends Error { public code: string; public readonly status: number; public readonly details: unknown; constructor(message: string, status: number, details?: unknown, code?: string) { super(message); this.name = 'FreshserviceApiError'; this.code = code ?? 'API_ERROR'; this.status = status;
Expected output: Both methods authenticate with Basic Auth (apiKey:X), send the request, and return parsed response data. Non-2xx responses raise FreshserviceApiError with the status and response body.
Step 7: Build the ticket creation handler with structured repair
When the LLM produces ticket data from the caller’s spoken description, the output is often noisy — markdown fences, misspelled keys, string priorities instead of numbers. The @reaatech/structured-repair-core package handles this with a graduated repair pipeline. Create src/handlers/ticket-creation.ts:
ts
import { z } from 'zod';import { repair, repairOutput, isValid, UnrepairableError } from '@reaatech/structured-repair-core';import { withRetry } from '@reaatech/agent-handoff';import { FreshserviceClient } from '../lib/freshservice-client.js';import type { FreshserviceTicket } from '../types/freshservice.js';const _ticketSchema = z.object({ subject: z.string().min(5).max(200), description: z.string().min(10), priority: z.coerce.number().pipe(z.union([z.
Expected output: The create() method first tries repair() for a fast fix. If that fails with UnrepairableError, it falls back to repairOutput() with all six strategies. The ticket is then posted via FreshserviceClient.createTicket() wrapped in exponential-backoff retry from @reaatech/agent-handoff.
Step 8: Assemble the voice agent pipeline
This is the heart of the recipe. The VoiceAgent class initializes the full @reaatech/voice-agent-core pipeline: STT provider, TTS provider, session manager, latency budget, VAD, cost tracking, and an MCP client that routes through Azure AI. Create src/voice/agent.ts:
ts
import { createPipeline, createLatencyBudget, initializeSessionManager, LatencyBudgetEnforcer, defineConfig, createVADProvider, createCostTracker,} from '@reaatech/voice-agent-core';import type { STTProvider, TTSProvider, MCPClient, VoiceAgentKitConfig, AudioChunk, AgentResponse, PipelineEvent } from '@reaatech/voice-agent-core';import { createSTTProvider, STTProviderInterface } from '@reaatech/voice-agent-stt';import { createTTSProvider, TTSProviderInterface } from '@reaatech/voice-agent-tts';import type { DeepgramTTSConfig } from '@reaatech/voice-agent-tts';import { AzureAiClient } from '../lib/azure-ai-client.js';import { TicketCreationHandler } from '../handlers/ticket-creation.js';import
Expected output: The VoiceAgent creates an STT provider (Deepgram nova-2), TTS provider (Deepgram aura), a latency budget (800ms target), a session manager (1-hour TTL), and a pipeline that connects them. When the MCP response arrives, the pipeline emits pipeline:mcp:response, which triggers handleMCPResponse() — it calls ticketCreationHandler.create() and streams the result back to the caller as TTS audio.
Step 9: Wire the Twilio webhook and media stream handler
Twilio sends call events to your server. The handleIncomingCall() function generates TwiML that instructs Twilio to open a media stream WebSocket to your app. The handleMediaStreamConnection() function sets up the @reaatech/voice-agent-telephony Twilio handler and routes audio to the voice agent. Create src/routes/webhook.ts:
ts
import type { AudioChunk } from '@reaatech/voice-agent-core';import { createTwilioHandler, TwilioMediaStreamHandler } from '@reaatech/voice-agent-telephony';import twilio from 'twilio';import { VoiceAgent } from '../voice/agent.js';import { createVoiceAgentConfig } from '../lib/config.js';import { FreshserviceClient } from '../lib/freshservice-client.js';import type { WebSocket as WsWebSocket } from 'ws';export type MinimalWebSocket = Pick<WsWebSocket, 'on' | 'close'>;export function encodeAudioForTwilio(buffer: Buffer): string { return TwilioMediaStreamHandler.encodeForTwilio(buffer);}export function decodeAudioFromTwilio(base64: string): Buffer { return TwilioMediaStreamHandler.decodeFromTwilio(base64);}export function handleIncomingCall(from: string, to: string, host: string): string { return `<Response><Connect><Stream url=\"wss://${host}/media-stream\" /></Connect></Response>`;}export async function handleMediaStreamConnection(ws: MinimalWebSocket): Promise<void> { const config = createVoiceAgentConfig(); const freshserviceClient = new FreshserviceClient(config.freshservice.apiKey, config.freshservice.domain); const agent = new VoiceAgent(config, freshserviceClient); const handler = createTwilioHandler({ bargeInEnabled: true, minSpeechDuration: 300, confidenceThreshold: 0.7, }); await handler.acceptConnection(ws as WsWebSocket); let sessionId = ''; handler.on('call:start', (data: { callSid: string }) => { sessionId = data.callSid; agent.startSession(sessionId).catch(console.error); }); handler.on('audio:received', (chunk: AudioChunk) => { agent.processAudio(sessionId, chunk).catch(console.error); }); handler.on('barge-in:detected', () => { agent.bargeIn(sessionId); handler.setTTSPlaying(false); void handler.clearAudio(); }); handler.on('call:end', () => { agent.endSession(sessionId).catch(console.error); void handler.close(); }); handler.on('error', (error: Error) => { console.error('Twilio handler error:', error); void handler.close(); });}export function validateTwilioRequest(authToken: string, signature: string | null, url: string, body: string): boolean { if (!signature) return false; try { return twilio.validateRequestWithBody(authToken, signature, url, body); } catch { return false; }}
Expected output:handleIncomingCall() returns TwiML pointing to a wss:// media stream. handleMediaStreamConnection() accepts the WebSocket, creates a VoiceAgent, and wires up call lifecycle events. When the caller speaks, the Twilio handler emits audio:received, which feeds audio into VoiceAgent.processAudio(). Barging in stops TTS, marks setTTSPlaying(false), and clears the audio buffer.
Step 10: Start the WebSocket server via instrumentation
Next.js instrumentation runs at server startup. Use instrumentation.ts to start a WebSocket server that Twilio connects to for the media stream. Create src/instrumentation.ts:
ts
import { WebSocketServer } from 'ws';import type { WebSocket } from 'ws';export async function register() { if (process.env.NEXT_RUNTIME !== 'nodejs') return; const { handleMediaStreamConnection } = await import('./routes/webhook.js'); const { createVoiceAgentConfig } = await import('./lib/config.js'); const config = createVoiceAgentConfig(); const server = new WebSocketServer({ port: config.wsPort }); server.on('connection', (ws: WebSocket) => { handleMediaStreamConnection(ws).catch(console.error); });}
You must also enable instrumentation in your Next.js config. Make sure next.config.ts includes the experimental instrumentation hook:
ts
import type { NextConfig } from "next";const nextConfig = { experimental: { instrumentationHook: true, },} as NextConfig;export default nextConfig;
Expected output: When the Next.js dev server starts, the register() function runs (only in the Node.js runtime). It reads the config, creates a WebSocketServer on the configured port (default 8080), and each incoming WebSocket connection triggers handleMediaStreamConnection().
Step 11: Create the Next.js API routes
The inbound call webhook and ticket lookup endpoints live in the App Router. Create app/api/webhook/route.ts:
ts
import { NextRequest, NextResponse } from 'next/server';import { handleIncomingCall, validateTwilioRequest } from '../../../src/routes/webhook.js';import { getTwilioConfig } from '../../../src/lib/config.js';export async function POST(req: NextRequest) { const bodyText = await req.text(); const twilioConfig = getTwilioConfig(); const signature = req.headers.get('x-twilio-signature') ?? ''; if (!validateTwilioRequest(twilioConfig.authToken, signature, req.url, bodyText)) { return new NextResponse('Forbidden', { status: 403 }); } const params = new URLSearchParams(bodyText); const from = params.get('From') ?? ''; const to = params.get('To') ?? ''; const host = req.headers.get('host') ?? ''; const twiml = handleIncomingCall(from, to, host); return new NextResponse(twiml, { headers: { 'Content-Type': 'text/xml' }, });}export function GET() { return NextResponse.json({ status: 'ok' });}
Create app/api/tickets/[id]/route.ts for looking up a ticket by ID:
Expected output: Configure your Twilio phone number’s Voice webhook URL to https://your-domain.com/api/webhook (method POST). When a call comes in, Twilio POSTs to that endpoint, validateTwilioRequest() verifies the Twilio signature, and the response TwiML tells Twilio to connect a media stream to your WebSocket server. The GET /api/tickets/:id endpoint lets you check ticket status after a call.
Step 12: Create the homepage and barrel exports
Replace the scaffolded app/page.tsx with a simple landing page:
tsx
export default function Home() { return ( <div style={{ maxWidth: 600, margin: '40px auto', fontFamily: 'sans-serif' }}> <h1>Azure AI Voice Agent for Freshservice</h1> <p>Call your dedicated Twilio number, describe your IT issue, and a Freshservice ticket is created automatically — no typing, no portal login.</p> <h2>Setup</h2> <ol> <li>Configure your Twilio phone number to send incoming call webhooks to <code>/api/webhook</code></li> <li>Set all environment variables in <code>.env</code></li> <li>Run <code>pnpm dev</code></li> <li>Call the Twilio number and describe your issue</li> </ol> </div> );}
Create src/index.ts to export the public API:
ts
export const SCAFFOLD_VERSION = "0.1.0" as const;export { VoiceAgent } from './voice/agent.js';export { FreshserviceClient, FreshserviceApiError } from './lib/freshservice-client.js';export { TicketCreationHandler } from './handlers/ticket-creation.js';export { createVoiceAgentConfig } from './lib/config.js';
Expected output: The homepage displays setup instructions. The barrel exports let consumers import VoiceAgent, FreshserviceClient, TicketCreationHandler, and createVoiceAgentConfig from a single path.
Step 13: Run the tests
The recipe includes a comprehensive test suite. Run it to confirm everything works:
terminal
pnpm vitest run --coverage --reporter=json --outputFile=vitest-report.json
Expected output: All tests pass. The suite covers:
Config loader — validates every required env var, defaults WS_PORT to 8080, throws descriptive errors for missing values
Ticket creation handler — creates tickets from valid JSON, repairs LLM output with markdown fences, returns field-level errors for partial input, handles UnrepairableError gracefully, coerces string priorities to numbers
Azure AI client — handles successful chat completions, non-OK responses, and network errors
Freshservice client — creates and retrieves tickets, handles API errors
Voice agent — starts and ends sessions, processes audio chunks, handles barge-in and destroy lifecycle
Instrumentation — registers the WebSocket server on startup
Coverage should be at or above 90% on lines, branches, functions, and statements for runtime code in src/ and app/**/route.ts.
Next steps
Add a Slack notification — emit a webhook to Slack when a ticket is created so your IT team sees it instantly
Implement priority classification — use the Azure AI model to classify urgency from the caller’s description, mapping to Freshservice’s priority levels automatically
Build a ticket status IVR — extend the GET /api/tickets/:id endpoint into a call-and-response flow: call back to hear your ticket’s current status read aloud
