Files · Anthropic Voice Agent for Toast Restaurant Order Taking

# Anthropic Voice Agent for Toast Restaurant Order Taking
 
> 24/7 AI phone agent that takes food orders, answers menu questions, and submits orders directly to Toast POS via phone call.
 
A tutorialized reference solution from [reaatech.com](https://reaatech.com), demonstrating how to build production-grade AI systems with the `@reaatech/*` package family.
 
## Description
 
This project implements an AI-powered voice agent that integrates with Twilio telephony to handle inbound restaurant phone calls. Callers can speak naturally to place food orders, ask about menu items, and complete transactions. The agent uses Anthropic's Claude for natural language understanding, a confidence-based router for menu intent disambiguation, and the Toast POS API for order submission.
 
## Architecture
 
```
Twilio PSTN → WebSocket media stream → Voice Pipeline (STT / Anthropic MCP / TTS) → Toast POS API
```
 
1. **Twilio inbound call** triggers a webhook that establishes a WebSocket media stream
2. **Voice Pipeline** handles speech-to-text (Deepgram), LLM reasoning (Anthropic Claude via MCP), and text-to-speech (ElevenLabs)
3. **Order Agent** manages multi-turn conversation, menu routing, session context, and handoff logic
4. **Toast POS API** receives finalized orders via tab creation and submission
 
## Tech Stack
 
- **Next.js** 16 (App Router) — API routes and server-side rendering
- **React** 19 — UI components
- **@anthropic-ai/sdk** — Claude LLM integration
- **Twilio** — PSTN telephony and media streaming
- **Redis** — Session storage and caching
- **@reaatech/agent-handoff** — Human handoff protocol
- **@reaatech/confidence-router** — Menu intent routing with keyword classifiers
- **@reaatech/llm-cost-telemetry** — Token usage and cost tracking
- **@reaatech/session-continuity** — Multi-turn session management with compression
- **@reaatech/session-continuity-storage-redis** — Redis-backed session persistence
- **@reaatech/voice-agent-core** — Voice pipeline orchestration
- **@reaatech/voice-agent-telephony** — Telephony adapter
- **LiveKit Agents** — WebRTC media transport
- **Langfuse** — LLM observability
- **Zod** — Schema validation
 
## Setup
 
### Environment Variables
 
Copy `.env.example` to `.env` and configure:
 
```env
ANTHROPIC_API_KEY=sk-ant-...
TWILIO_ACCOUNT_SID=AC...
TWILIO_AUTH_TOKEN=...
TWILIO_PHONE_NUMBER=+1555...
TOAST_API_KEY=...
TOAST_RESTAURANT_GUID=...
TOAST_BASE_URL=https://api.toast.com
REDIS_URL=redis://localhost:6379
AGENT_BASE_URL=https://your-ngrok-url.ngrok.io
```
 
### Dependencies
 
```bash
pnpm install
```
 
### Run Locally
 
```bash
pnpm dev              # Next.js dev server on port 3000
pnpm test             # vitest run with coverage
pnpm lint             # ESLint
pnpm typecheck        # TypeScript type checking
```
 
### Twilio + Ngrok
 
Expose your local server with ngrok:
 
```bash
ngrok http 3000
```
 
Configure your Twilio phone number's voice webhook URL to `https://your-ngrok-url.ngrok.io/api/calls/webhook`.
 
## Project layout
 
```
app/                  Next.js App Router pages + API routes
src/                  services, lib, adapters
tests/                vitest suite (mirrors src/)
packages/             API references for every dependency
DEV_PLAN.md           build plan for this recipe
```
 
## License
 
MIT — see [LICENSE](./LICENSE).