Files · Vertex AI Voice Agent for Twilio Field Service Scheduling

# Vertex AI Voice Agent for Twilio Field Service Scheduling
 
> Automate field service appointment booking and dispatch over phone calls with a Vertex AI voice agent that integrates Twilio and Google Calendar.
 
A tutorialized reference solution from [reaatech.com](https://reaatech.com), demonstrating how to build a production-grade voice agent with the `@reaatech/*` package family.
 
## Problem
 
Small field-service businesses miss after-hours calls and spend hours manually coordinating technician schedules, leading to lost jobs and double-bookings.
 
## Architecture
 
```
Inbound Call (Twilio)
       │
       ▼
┌─────────────────┐     ┌──────────────────────────────────┐
│  Express Server  │────▶│  LiveKit Room + Vertex AI Agent │
│  (src/server.ts) │     │  (src/voice/)                   │
│  POST /twilio/*  │     │  ├── AgentCoordinator           │
└─────────────────┘     │  ├── IntentClassifier            │
       │                │  ├── Booking Specialist          │
       ▼                │  ├── Rescheduling Specialist     │
┌─────────────────┐     │  └── FAQ Specialist              │
│  Next.js API     │     └──────────────────────────────────┘
│  (app/api/)      │                    │
│  ├── calls/*     │                    ▼
│  ├── metrics/*   │     ┌──────────────────────────────────┐
│  └── appointments│     │  BudgetGuard + CalendarService   │
└─────────────────┘     │  (src/lib/)                       │
       │                │  ├── agent-coordinator.ts         │
       ▼                │  ├── budget-guard.ts              │
┌─────────────────┐     │  ├── calendar.ts                  │
│  Dashboard       │     │  ├── store.ts                    │
│  (/dashboard)    │     │  └── structured-output-repair.ts │
└─────────────────┘     └──────────────────────────────────┘
```
 
## Features
 
- **Inbound Voice Agent** — Twilio calls are answered by a real-time voice agent powered by Vertex AI (Gemini 2.5 Flash via LiveKit)
- **Smart Routing** — Caller intents (booking, rescheduling, FAQ) are classified and routed to specialist agents using `@reaatech/agent-handoff`
- **Budget Enforcement** — Per-call cost caps enforced by `@reaatech/agent-budget-engine` with automatic model downgrades
- **Idempotent Bookings** — Duplicate appointment creation prevented via `@reaatech/idempotency-middleware`
- **Google Calendar Integration** — Appointments created and managed through the Google Calendar API
- **Structured Output Repair** — LLM output cleaned and validated before writing to external systems
- **Observability** — Langfuse tracing across all LLM calls and agent handoffs
- **Dashboard** — Next.js dashboard showing call logs, cost metrics, and appointments
 
## Prerequisites
 
- Node.js >= 22, pnpm >= 10
- [Twilio account](https://www.twilio.com) with a phone number
- [Google Cloud project](https://console.cloud.google.com) with Vertex AI API enabled
- [LiveKit server](https://livekit.io) (self-hosted or LiveKit Cloud)
- Google Calendar API enabled
 
## Setup
 
```bash
pnpm install
cp .env.example .env
# Fill in your credentials
pnpm dev              # Next.js dashboard (localhost:3000)
pnpm dev:server       # Express webhook server (localhost:3001)
pnpm test             # vitest run with coverage
```
 
## Project Structure
 
```
src/
├── api/twilio/voice.ts       — Twilio voice webhook handler
├── lib/
│   ├── agent-coordinator.ts  — Agent handoff orchestration
│   ├── budget-guard.ts       — Budget enforcement
│   ├── calendar.ts           — Google Calendar integration
│   ├── observability.ts      — Langfuse tracing
│   └── store.ts              — In-memory data store
├── types/                    — TypeScript types and Zod schemas
├── voice/                    — LiveKit voice agent files
│   ├── agent.ts              — Vertex AI LLM setup
│   ├── realtime.ts           — Realtime speech model
│   ├── intent-classifier.ts  — Caller intent classification
│   ├── conversation-router.ts— Specialist routing
│   └── specialists/          — Booking, rescheduling, FAQ specialists
│── server.ts                 — Express server entry point
└── instrumentation.ts        — Next.js instrumentation
app/
├── api/                      — Next.js API routes
│   ├── calls/*               — Call log endpoints
│   ├── metrics/*             — Cost metrics endpoints
│   ├── appointments/         — Appointment creation
│   └── health/               — Health check
└── dashboard/                — Dashboard pages
    ├── calls/                — Call log views
    ├── metrics/              — Cost analytics
    └── appointments/         — Appointment list
```
 
## API Endpoints
 
- `POST /twilio/voice` — Twilio voice webhook (Express)
- `POST /twilio/status` — Twilio status callback (Express)
- `GET /health` — Health check
- `POST /api/calls/incoming` — Record an incoming call
- `GET /api/calls/logs` — List call logs
- `GET /api/calls/[id]` — Get call detail
- `GET /api/metrics/cost` — Get cost metrics
- `POST /api/appointments` — Create appointment
- `GET /api/health` — Next.js health check
 
## Testing
 
```bash
pnpm test
```
 
Runs vitest with coverage reporting. Coverage threshold: 90% on lines, branches, functions, and statements for runtime code (services, lib, route handlers).
 
## Environment Variables
 
Copy `.env.example` to `.env` and configure:
 
| Variable | Description |
|---|---|
| `TWILIO_ACCOUNT_SID` | Twilio account SID |
| `TWILIO_AUTH_TOKEN` | Twilio auth token |
| `TWILIO_PHONE_NUMBER` | Twilio phone number |
| `GOOGLE_CLOUD_PROJECT` | GCP project ID |
| `GOOGLE_CLOUD_LOCATION` | GCP region (default: us-central1) |
| `GOOGLE_GENAI_USE_VERTEXAI` | Set to "true" for Vertex AI |
| `GOOGLE_APPLICATION_CREDENTIALS` | Path to service account JSON |
| `LIVEKIT_URL` | LiveKit server URL |
| `LIVEKIT_API_KEY` | LiveKit API key |
| `LIVEKIT_API_SECRET` | LiveKit API secret |
| `LANGFUSE_PUBLIC_KEY` | Langfuse public key |
| `LANGFUSE_SECRET_KEY` | Langfuse secret key |
| `LANGFUSE_HOST` | Langfuse host URL |
| `GOOGLE_CALENDAR_ID` | Google Calendar ID |
| `EXPRESS_PORT` | Express server port |
 
## Packages Used
 
- `@livekit/agents` — Voice agent runtime
- `@livekit/agents-plugin-google` — Vertex AI Gemini integration
- `twilio` — Telephony integration
- `googleapis` — Google Calendar API
- `@reaatech/agent-handoff` — Agent handoff protocol
- `@reaatech/agent-budget-engine` — Budget enforcement
- `@reaatech/idempotency-middleware` — Idempotent booking creation
- `langfuse` — LLM observability
- `zod` — Runtime schema validation
 
## License
 
MIT — see [LICENSE](./LICENSE).