Files · Azure AI Phone Agent for Cal.com Appointment Scheduling

# Azure AI Phone Agent for Cal.com Appointment Scheduling
 
AI-powered phone agent that answers calls, transcribes speech, classifies intents, and books appointments via Cal.com — all over a real-time voice pipeline.
 
## Overview
 
The voice agent pipeline flows as follows:
 
1. **Twilio** receives an inbound call and bridges it to a **LiveKit** room
2. **Deepgram** (STT) transcribes the caller's speech in real time
3. **Azure OpenAI** classifies the caller's intent and extracts booking details (date, time, service)
4. **Cal.com API** creates or looks up available slots and books the appointment
5. **Cartesia** (TTS) generates a natural-sounding voice response
6. The synthesized audio is played back to the caller via Twilio
 
## Architecture
 
```
       ┌─────────┐     ┌────────────┐     ┌──────────┐
       │ Twilio  │────▶│ LiveKit    │────▶│ Deepgram │
       │ (Phone) │     │ (Room)     │     │ (STT)    │
       └─────────┘     └────────────┘     └──────────┘
            ▲                                    │
            │                                    ▼
            │                             ┌──────────────┐
            │                             │ Azure OpenAI │
            │                             │ (classify +  │
            │                             │  extract)    │
            │                             └──────────────┘
            │                                    │
            │                                    ▼
            │                             ┌──────────────┐
            │                             │  Cal.com API │
            │                             │  (book slot) │
            │                             └──────────────┘
            │                                    │
            │                                    ▼
            │                             ┌──────────────┐
            │◀────────────────────────────│  Cartesia    │
                                          │  (TTS)       │
                                          └──────────────┘
```
 
## Prerequisites
 
- **Azure OpenAI** — deployment with GPT-4o-mini (or compatible model)
- **Cal.com** — account with API key and an event type configured
- **LiveKit** — cloud or self-hosted server for media streaming
- **Twilio** — phone number with voice webhook capability
- **Deepgram** — API key for real-time speech-to-text
- **Cartesia** — API key for text-to-speech synthesis
 
## Getting Started
 
```bash
git clone <repo-url>
cd azure-ai-phone-agent-for-calcom-appointment-scheduling
pnpm install
cp .env.example .env
# Fill in your credentials (see table below)
pnpm dev
```
 
Configure your Twilio phone number's voice webhook to point at `https://your-host/api/twilio/call`.
 
## Environment Variables
 
| Variable | Description |
|---|---|
| `AZURE_OPENAI_ENDPOINT` | Azure OpenAI resource endpoint |
| `AZURE_OPENAI_API_KEY` | Azure OpenAI API key |
| `AZURE_OPENAI_DEPLOYMENT` | Deployment name (default: `gpt-4o-mini`) |
| `AZURE_OPENAI_API_VERSION` | API version (default: `2024-08-01-preview`) |
| `CALCOM_API_KEY` | Cal.com API key |
| `CALCOM_BASE_URL` | Cal.com API base URL (default: `https://api.cal.com/v2`) |
| `CALCOM_EVENT_TYPE_ID` | Event type ID for bookings |
| `LIVEKIT_HOST` | LiveKit server host |
| `LIVEKIT_API_KEY` | LiveKit API key |
| `LIVEKIT_API_SECRET` | LiveKit API secret |
| `DEEPGRAM_API_KEY` | Deepgram API key |
| `CARTESIA_API_KEY` | Cartesia API key |
| `TWILIO_ACCOUNT_SID` | Twilio account SID |
| `TWILIO_AUTH_TOKEN` | Twilio auth token |
| `TWILIO_PHONE_NUMBER` | Twilio phone number (E.164 format) |
| `ALLOWED_ORIGINS` | CORS origins (default: `http://localhost:3000`) |
| `SESSION_BUDGET_LIMIT_USD` | Per-call budget cap in USD (default: `0.50`) |
| `AZURE_LLM_PRICING_JSON` | JSON map of model pricing (input/output per 1K tokens) |
 
## Testing
 
```bash
pnpm test
```
 
Runs vitest with coverage. Target threshold: **≥90%** for branches, functions, lines, and statements.
 
## License
 
MIT — see [LICENSE](./LICENSE).