Files · Vertex AI Voice Agent for Field Service Dispatch

# Vertex AI Voice Agent for Field Service Dispatch
 
Automate incoming service calls, classify needs, and route to the right team with human-like voice interaction powered by Vertex AI.
 
## Description
 
A real-time voice agent that answers calls, classifies service requests (emergency repair, maintenance, estimate) using Google Vertex AI and REAA agent-handoff-routing, then dispatches to the appropriate on-call technician. REAA agent-mesh-session manages call context, agent-memory retains customer history, and agent-budget-engine keeps LLM costs predictable. The result is a 24/7 dispatchable AI receptionist that learns from past interactions.
 
## Architecture
 
```
Caller (PSTN)
    |
    v
Twilio (Inbound Voice Webhook)
    |
    v
Next.js API Route (src/api/calls/route.ts)
    |
    +---> LiveKit Room Service (create room, generate token)
    |
    +---> Agent Dispatcher (src/agents/dispatcher.ts)
    |         |
    |         +---> Vertex AI (transcript classification)
    |         +---> Agent Memory (customer history lookup)
    |         +---> Handoff Router (capability-based agent selection)
    |         +---> Mesh Session (session state management)
    |
    +---> TwiML Response (<Connect><Sip> to LiveKit)
              |
              v
        Voice Pipeline (src/agents/pipeline.ts)
              |
              +---> Deepgram (real-time STT)
              +---> Gemini via LiveKit plugin (response generation)
              +---> ElevenLabs (TTS)
              |
              v
        Caller hears AI agent response
```
 
The dashboard UI at `/dashboard` provides live call logs, budget tracking, and system status monitoring.
 
## Prerequisites
 
- Node.js >= 22
- pnpm (v10.14.0)
- A Twilio account with a phone number (voice-enabled)
- A LiveKit server (self-hosted or cloud)
- A Google Cloud project with Vertex AI enabled
- Deepgram API key
- ElevenLabs API key
- Firebase project (for Firestore budget storage and call log persistence)
 
## Environment Variables
 
| Variable | Description |
|---|---|
| `TWILIO_ACCOUNT_SID` | Twilio Account SID |
| `TWILIO_AUTH_TOKEN` | Twilio Auth Token |
| `TWILIO_PHONE_NUMBER` | Twilio phone number for incoming calls |
| `LIVEKIT_API_KEY` | LiveKit API key |
| `LIVEKIT_API_SECRET` | LiveKit API secret |
| `LIVEKIT_URL` | LiveKit server URL (wss://...) |
| `GOOGLE_CLOUD_PROJECT` | Google Cloud project ID |
| `GOOGLE_CLOUD_LOCATION` | Google Cloud location (e.g., us-central1) |
| `GOOGLE_GENAI_USE_VERTEXAI` | Set to "true" to use Vertex AI |
| `DEEPGRAM_API_KEY` | Deepgram API key for speech-to-text |
| `ELEVENLABS_API_KEY` | ElevenLabs API key for text-to-speech |
| `BUDGET_MONTHLY_LIMIT_USD` | Monthly budget limit in USD (default: 50.0) |
| `FIREBASE_SERVICE_ACCOUNT_PATH` | Path to Firebase service account key JSON |
| `DASHBOARD_API_KEY` | API key for dashboard access via `x-api-key` header |
 
## Setup
 
1. Clone the repository and install dependencies:
 
```bash
pnpm install
```
 
2. Copy the environment file and fill in your values:
 
```bash
cp .env.example .env
```
 
3. Run the development server:
 
```bash
pnpm dev
```
 
4. Configure your Twilio phone number's voice webhook URL to point to:
 
```
https://your-domain.com/api/calls
```
 
5. Access the dashboard at `/dashboard` with your `x-api-key` header set to `DASHBOARD_API_KEY`.
 
## Testing
 
Run all tests:
 
```bash
pnpm test
```
 
Run tests with coverage:
 
```bash
pnpm test:coverage
```
 
Run a specific test file:
 
```bash
pnpm vitest run src/components/__tests__/BudgetCard.test.tsx
```
 
Tests use:
- **Vitest** as the test runner
- **@testing-library/react** for React component rendering and assertions
- **MSW (Mock Service Worker)** for HTTP request mocking
- **vi.mock / vi.spyOn** for module-level mocking
 
All external services (Twilio, LiveKit, Vertex AI, Deepgram, ElevenLabs, Firestore) are mocked in tests. No live API calls are made during test runs.
 
## Project Structure
 
```
src/
  agents/
    dispatcher.ts          # Call classification and agent routing
    pipeline.ts            # Voice pipeline (STT, LLM, TTS)
  api/
    calls/route.ts         # Twilio webhook handler
    health/route.ts        # Health check endpoint
  app/
    dashboard/
      layout.tsx           # Dashboard layout
      page.tsx             # Dashboard page (BudgetCard + CallLogTable)
  components/
    BudgetCard.tsx         # Monthly spend progress card
    CallLogTable.tsx        # Call logs table
  lib/
    budget.ts              # Budget engine (Vertex AI pricing, spend tracking)
    firestore-admin.ts     # Firestore admin client
  mocks/
    factories.ts           # Test data factory functions
    handlers.ts            # MSW request handlers
  middleware.ts            # Dashboard API key authentication
  health.ts               # Health check definitions
  test-setup.ts           # Global test setup (MSW, env stubs)
```
 
## Key Features
 
- **24/7 Call Answering**: Answers after-hours calls automatically
- **Intent Classification**: Uses Vertex AI Gemini to classify service requests
- **Smart Routing**: Routes calls to the right agent based on capabilities
- **Budget Management**: Tracks LLM costs with configurable monthly limits and auto-downgrade
- **Customer Memory**: Remembers past interactions via Agent Memory
- **Health Monitoring**: Built-in health checks for all service dependencies
- **Dashboard UI**: Real-time call logs and budget overview
 
## License
 
MIT