Files · OpenAI Voice Agent for Appointment Scheduling

# OpenAI Voice Agent for Appointment Scheduling
 
A Twilio-powered voice agent that answers after-hours calls, classifies caller intent, and books appointments without human intervention. Built with Next.js, OpenAI Whisper, and the REAA agent-mesh ecosystem.
 
## Architecture
 
```
Caller → Twilio → Voice Webhook → Whisper Transcription → agent-mesh-gateway
  → agent-mesh-classifier → agent-handoff → Scheduler Agent → Calendly API
```
 
The call flow:
1. **Twilio Voice Webhook** receives inbound calls and streams audio via Media Streams
2. **OpenAI Whisper** transcribes caller speech in real-time
3. **agent-mesh-gateway** orchestrates the request lifecycle
4. **agent-mesh-classifier** determines caller intent (new appointment, reschedule, general question)
5. **agent-handoff** routes to the appropriate agent
6. **Scheduler Agent** extracts appointment details, checks Calendly availability, and books the slot
7. **agent-budget-engine** tracks per-call AI costs with hard/soft caps
8. **agent-mesh-observability** provides structured logging, metrics, and PII redaction
 
## Getting Started
 
```bash
pnpm install
pnpm dev
pnpm test
pnpm test:coverage
```
 
## Environment Variables
 
| Variable | Description | Where to Get It |
|---|---|---|
| `TWILIO_ACCOUNT_SID` | Twilio Account SID | console.twilio.com |
| `TWILIO_AUTH_TOKEN` | Twilio Auth Token | console.twilio.com |
| `TWILIO_PHONE_NUMBER` | Twilio phone number (E.164) | console.twilio.com |
| `OPENAI_API_KEY` | OpenAI API key | platform.openai.com |
| `CALENDLY_API_KEY` | Calendly Personal Access Token | calendly.com/integrations/api |
| `CALENDLY_WEBHOOK_SECRET` | Calendly webhook verification secret | calendly.com/integrations/webhooks |
| `AGENT_MESH_GATEWAY_API_KEY` | Gateway API key (dev-key in dev) | Set manually |
 
## Twilio Setup
 
1. Buy a Twilio phone number
2. Configure the voice webhook URL to `https://<your-domain>/api/twilio-voice`
3. Set the HTTP method to POST
4. Ensure the phone number's voice configuration points to your webhook
 
## Calendly Integration
 
1. Generate a Personal Access Token at https://calendly.com/integrations/api
2. Set `CALENDLY_API_KEY` in your environment
3. Optionally set up a webhook to receive event notifications
4. The scheduler uses your default event type for slot availability
 
## Scripts
 
| Command | Description |
|---|---|
| `pnpm dev` | Start Next.js development server |
| `pnpm typecheck` | Run TypeScript type checking |
| `pnpm lint` | Run ESLint |
| `pnpm test` | Run vitest tests |
| `pnpm test:coverage` | Run tests with coverage report |
 
## Deployment (Vercel)
 
1. Push to GitHub
2. Import project in Vercel dashboard
3. Set all environment variables in Vercel project settings
4. Configure a custom domain for the Twilio webhook URL
5. Deploy
 
## License
 
MIT