Files · Azure AI Multi-Agent Handoff for Linear Issue Triage

# Azure AI Multi-Agent Handoff for Linear Issue Triage
 
## What this does
 
Automatically triage incoming Linear issues by routing them to specialized AI agents, escalating only complex cases to human engineers.
 
## Architecture
 
The triage pipeline follows these stages:
 
1. **Webhook** — Linear sends an `Issue.create` webhook to the Next.js API route
2. **Verification** — The route verifies the HMAC-SHA256 signature and parses the payload
3. **Budget Check** — A cost guard checks whether the monthly budget allows processing
4. **Orchestrator Classification** — Azure OpenAI classifies the issue into a category (bug, feature_request, docs_issue, question, urgent_bug)
5. **Confidence Routing** — The confidence router evaluates prediction scores and routes or falls back
6. **Agent Handoff** — If a specialist is required, the handoff protocol transfers context to the target agent
7. **Specialist Analysis** — The specialist agent analyzes the issue and produces labels, priority, and a summary
8. **Linear Update** — The analysis is applied to the Linear issue (labels, priority, comment, assignee)
9. **Observability** — Every step is traced via Langfuse OTLP for debugging and cost analysis
 
## Prerequisites
 
- Node.js 22+
- pnpm 10.0.0
- An Azure OpenAI deployment with GPT-4o or compatible model
- A Linear workspace with API key and webhook signing secret
- A Langfuse instance (cloud or self-hosted) for telemetry
- Temporal server (optional — falls back to direct execution)
 
## Setup
 
```bash
git clone <repo-url>
pnpm install
cp .env.example .env
# Fill in your environment variables
pnpm dev
```
 
## Environment Variables
 
| Variable | Required | Default | Description |
|---|---|---|---|
| `AZURE_OPENAI_ENDPOINT` | Yes | — | Azure OpenAI endpoint URL |
| `AZURE_OPENAI_API_KEY` | Yes | — | Azure OpenAI API key |
| `AZURE_OPENAI_DEPLOYMENT_ORCHESTRATOR` | No | `gpt-4o` | Orchestrator model deployment name |
| `AZURE_OPENAI_DEPLOYMENT_SPECIALIST` | No | `gpt-4o` | Specialist model deployment name |
| `LINEAR_API_KEY` | Yes | — | Linear API key |
| `LINEAR_WEBHOOK_SIGNING_SECRET` | Yes | — | Linear webhook HMAC secret |
| `LANGFUSE_PUBLIC_KEY` | Yes | — | Langfuse public key |
| `LANGFUSE_SECRET_KEY` | Yes | — | Langfuse secret key |
| `LANGFUSE_HOST` | Yes | — | Langfuse host URL |
| `TEMPORAL_HOST` | Yes | — | Temporal server address |
| `TEMPORAL_NAMESPACE` | No | `default` | Temporal namespace |
| `TEMPORAL_TASK_QUEUE` | No | `linear-triage` | Temporal task queue |
 
## API Routes
 
### `POST /api/linear/webhook`
 
Accepts Linear webhook events. Only `Issue.create` events trigger triage.
 
**Request body:**
```json
{
  "type": "Issue",
  "action": "create",
  "data": {
    "id": "ISS-1",
    "title": "Login page crash",
    "description": "Steps to reproduce...",
    "teamId": "team-1",
    "labels": [{ "name": "bug" }]
  }
}
```
 
**Headers:** `x-signature-256: <hmac-sha256-hex>`
 
**Response codes:**
- `202` — Accepted, workflow started (`{ status: "accepted", workflowId: "..." }`)
- `400` — Invalid JSON body
- `401` — Missing or invalid signature
- `500` — Workflow start failed
 
### `GET /api/linear/webhook`
 
Health check endpoint.
 
**Response:** `{ status: "ok", service: "linear-triage-webhook" }`
 
## How It Works
 
1. A Linear webhook delivers new issue data to the API route
2. The signature is verified using HMAC-SHA256
3. Budget is checked before any AI or API calls
4. The orchestrator classifies the issue via Azure OpenAI
5. The confidence router maps the classification to a specialist agent
6. If the specialist is not generic, the handoff protocol transfers context
7. The specialist analyzes the issue and generates labels, priority, and a summary
8. The analysis is applied directly to the Linear issue
9. Circuit breakers protect against cascading failures in Azure OpenAI and Linear APIs
10. All spans and costs are recorded in Langfuse
 
## Agent Specialists
 
| Agent | Prompt Focus | Skills |
|---|---|---|
| **Debug Specialist** | Error reproduction, stack traces, severity | debugging, typescript, testing |
| **Docs Specialist** | API gaps, readability, help content | documentation, writing, api-design |
| **Scheduling Specialist** | Sprint impact, dependencies, estimation | project-management, estimation, prioritization |
| **Generic Specialist** | Categorization, routing, fallback | general, triage |
 
## Observability
 
All pipeline steps emit OpenTelemetry spans exported to Langfuse via the OTLP HTTP exporter. Each workflow execution produces a trace with spans for:
 
- `orchestrator.classify` — LLM classification call
- `orchestrator.decide` — Routing decision
- `handoff.*` — Agent handoff lifecycle
- `specialist.analyze` — Specialist LLM analysis
- `linear.update` — Linear API updates
 
Cost tracking is enforced via a budget guard with configurable soft and hard caps.
 
## Testing
 
```bash
pnpm test
```
 
Runs vitest with coverage reporting (target: 90%+ lines, branches, functions, statements).