Files · vLLM Agent Mesh for E-commerce Order Management

# vLLM Agent Mesh for E-commerce Order Management
 
> A multi-agent system that handles order inquiries, shipment tracking, and returns for SMB e‑commerce stores, powered by a vLLM‑hosted model and orchestrated with REAA agent-mesh.
 
A tutorialized reference solution from [reaatech.com](https://reaatech.com), demonstrating how to build production-grade AI systems with the `@reaatech/*` package family.
 
## Architecture
 
The system uses a **vLLM endpoint** as the LLM backend, accessed through the **OpenAI SDK** (`Chat Completions` API). User messages flow through an **agent mesh classifier** (`@reaatech/agent-mesh`) that determines intent, then a **router** (`@reaatech/agent-mesh-router`) dispatches requests to one of three specialist agents:
 
1. **Order Specialist** — order inquiries, status checks, and order modifications
2. **Shipping Specialist** — shipment tracking, delivery estimates, and shipping address updates
3. **Returns Specialist** — return requests, refund status, and exchange policies (default fallback)
 
Each agent is defined in a YAML file under `agents/` and served via MCP endpoints. The router selects the best-matching agent based on confidence thresholds and example-based classification.
 
## Setup
 
```bash
pnpm install
```
 
Copy `.env.example` to `.env` and configure the following variables:
 
| Variable | Description |
|---|---|
| `VLLM_API_KEY` | API key for your vLLM endpoint |
| `VLLM_BASE_URL` | Base URL of the vLLM endpoint |
| `VLLM_MODEL` | Model name (e.g., `qwen2.5-7b-instruct`) |
| `UPSTASH_REDIS_URL` | Upstash Redis REST URL for session storage |
| `UPSTASH_REDIS_TOKEN` | Upstash Redis REST token |
| `AGENT_REGISTRY_DIR` | Directory containing agent YAML files (`./agents`) |
| `MCP_REQUEST_TIMEOUT_MS` | Timeout for MCP requests (default `30000`) |
| `MCP_MAX_RETRIES` | Max retries for MCP requests (default `3`) |
| `SESSION_TTL_SECONDS` | Session time-to-live in seconds (default `3600`) |
| `BUDGET_DAILY_LIMIT` | Daily LLM cost budget in USD (default `10.00`) |
 
```bash
pnpm dev
```
 
## Agent YAML Format
 
Each specialist agent is defined in a YAML file under `agents/`. Here is a complete example from `agents/returns-agent.yaml`:
 
```yaml
agent_id: "returns-agent"
display_name: "Returns Specialist"
description: "Handles return requests, refund status, and exchange policies"
endpoint: "${RETURNS_AGENT_ENDPOINT:-http://localhost:9083}"
type: mcp
is_default: true
confidence_threshold: 0
clarification_required: false
examples:
  - "I want to return item #ITEM-555 from order #ORD-12345"
  - "What is your return policy?"
  - "How long do refunds take?"
  - "I received a damaged item, what do I do?"
  - "Can I exchange this for a different size?"
```
 
- **`agent_id`** — unique identifier for the agent (e.g., `returns-agent`)
- **`endpoint`** — MCP endpoint URL; supports `${ENV_VAR:-default}` expansion
- **`is_default`** — when `true`, this agent is the fallback if no other agent matches with sufficient confidence
- **`confidence_threshold`** — minimum confidence score required to route to this agent (0 = always matches as default)
- **`examples`** — list of example queries used by the classifier to match intent to this agent
 
## API Usage
 
Classify and respond to a user message:
 
```bash
curl -X POST http://localhost:3000/api/chat \
  -H "Content-Type: application/json" \
  -d '{"input":"Where is my order?","session_id":"abc"}'
```
 
Stream responses (SSE):
 
```bash
curl -X GET "http://localhost:3000/api/chat?input=Where+is+my+order%3F&session_id=abc"
```
 
## How It Works
 
1. **Classification** — user input is sent to the vLLM model via the OpenAI SDK; the agent mesh classifier determines the intent
2. **Routing** — `@reaatech/agent-mesh-router` evaluates confidence scores against each agent's `confidence_threshold` and `examples`
3. **Dispatch** — the best-matching agent's MCP endpoint is called with the classified request
4. **Repair** — `@reaatech/structured-repair-core` handles malformed responses or missing fields
5. **Handoff** — `@reaatech/agent-handoff` manages agent-to-agent transitions when a query requires multiple specialists
 
## Redis
 
**Upstash Redis** is used for session storage, maintaining conversation history across requests. Get a free database at [upstash.com](https://upstash.com). Configure via `UPSTASH_REDIS_URL` and `UPSTASH_REDIS_TOKEN` in your `.env`.
 
## Streaming UI
 
The frontend uses **`@assistant-ui/react`** to provide a streaming chat interface. Messages are rendered in real-time as the LLM generates tokens, with support for session continuity and agent switching.
 
## Project layout
 
```
app/                  Next.js App Router pages + API routes
src/                  services, lib, adapters
tests/                vitest suite (mirrors src/)
packages/             API references for every dependency (read these first)
DEV_PLAN.md           build plan for this recipe
```
 
## License
 
MIT — see [LICENSE](./LICENSE).