Files · xAI Grok Observability for SMB AI Workflow Monitoring

# xAI Grok Observability for SMB AI Workflow Monitoring
 
> Single-pane monitoring for token usage, latency, errors, and cost across all Grok-powered features in your SMB app.
 
A tutorialized reference solution from [reaatech.com](https://reaatech.com), demonstrating how to build production-grade AI systems with the `@reaatech/*` package family.
 
## Problem
 
SMBs lack visibility into Grok usage, cost, and errors across their workflows. Without centralized observability, teams face unpredictable AI bills, undetected error spikes, and no way to measure per-feature or per-tenant token consumption.
 
This recipe solves that by wrapping xAI Grok API calls with OpenTelemetry spans and cost tracking, then exporting traces to Langfuse.
 
## Getting Started
 
### Environment Setup
 
Copy `.env.example` to `.env` and fill in your credentials:
 
```bash
cp .env.example .env
```
 
Required environment variables:
 
| Variable | Description |
|----------|-------------|
| `XAI_API_KEY` | xAI Grok API key |
| `XAI_BASE_URL` | xAI API base URL (default: `https://api.x.ai/v1`) |
| `LANGFUSE_PUBLIC_KEY` | Langfuse project public key |
| `LANGFUSE_SECRET_KEY` | Langfuse project secret key |
| `LANGFUSE_HOST` | Langfuse host URL (default: `https://cloud.langfuse.com`) |
| `OTEL_EXPORTER_OTLP_ENDPOINT` | Langfuse OTLP ingestion URL |
| `OTEL_SERVICE_NAME` | OpenTelemetry service name |
| `LOG_LEVEL` | Pino log level (`debug`, `info`, `warn`, `error`) |
 
### Run the app
 
```bash
pnpm install
pnpm dev            # next dev on http://localhost:3000
pnpm test           # vitest run with coverage
pnpm typecheck      # tsc --noEmit
pnpm lint           # eslint
```
 
## Architecture
 
```
User Request → API Route → InstrumentedGrok → wrapOpenAI + OTel spans → Langfuse
                              ↓
                          CostTracker → CostSpan[]
                              ↓
                          OpenTelemetry → OTLP Exporter → Langfuse
```
 
The `InstrumentedGrok` class wraps the OpenAI SDK (configured against xAI's compatible endpoint) with `@reaatech/llm-cost-telemetry-providers` (`wrapOpenAI`). Every chat completion call produces:
 
- An **OTel span** with GenAI semantic convention attributes (`gen_ai.request.model`, `gen_ai.response.finish_reasons`, etc.)
- A **CostSpan** with token counts, timing, and cost calculation
- A **trace ID** that links the API response to the OTel trace in Langfuse
 
## API Endpoints
 
### `POST /api/chat`
 
Submit a chat completion request to xAI Grok.
 
**Request body:**
 
```json
{
  "messages": [{ "role": "user", "content": "Hello" }],
  "model": "grok-3",
  "temperature": 0.7,
  "max_tokens": 1024,
  "tenant": "acme-corp",
  "feature": "support-chat"
}
```
 
**Response (200):**
 
```json
{
  "completion": { "id": "chatcmpl-xxx", "choices": [...] },
  "traceId": "abc123...",
  "costUsd": 0.00015
}
```
 
**Error responses:** 400 for validation errors, 502 for API failures.
 
### `GET /api/metrics`
 
Returns aggregate metrics from all Grok calls in the current process.
 
```json
{
  "totalCost": 0.00042,
  "totalRequests": 3,
  "errorCount": 1,
  "averageLatencyMs": 245.7,
  "spanCount": 3
}
```
 
### `POST /api/reset-metrics`
 
Resets all accumulated metrics to zero.
 
```json
{ "ok": true }
```
 
## OpenTelemetry & Langfuse
 
The recipe uses `@opentelemetry/sdk-node` with the OTLP protobuf exporter to send traces to Langfuse. The `src/instrumentation.ts` hook bootstraps the SDK at server startup.
 
Each GenAI span includes:
- `gen_ai.request.model` — the model name (e.g., `grok-3`)
- `gen_ai.request.temperature` — temperature parameter
- `gen_ai.request.max_tokens` — max tokens parameter
- `gen_ai.response.finish_reasons` — completion finish reasons
- `gen_ai.usage.input_tokens` / `gen_ai.usage.output_tokens` — token counts
- `llm.cost.total` — calculated cost in USD
 
## Cost Tracking
 
Grok pricing per 1M tokens (as of this writing):
 
| Model | Input ($/1M) | Output ($/1M) |
|-------|-------------|--------------|
| `grok-2` | $2.00 | $8.00 |
| `grok-2-mini` | $0.30 | $1.20 |
| `grok-3` | $3.00 | $12.00 |
| `grok-3-mini` | $0.50 | $2.00 |
| default (fallback) | $2.00 | $8.00 |
 
Costs are calculated via `calculateCostFromTokens` from `@reaatech/llm-cost-telemetry` and stored as `CostSpan` objects.
 
## Testing
 
```bash
pnpm test           # Run all tests with coverage
pnpm typecheck      # TypeScript type checking
pnpm lint           # ESLint
```
 
Tests cover:
- API route validation (happy paths, error cases, boundary conditions)
- Cost calculation accuracy
- MSW-based HTTP mocking for xAI API
- E2E instrumentation flow (chat → metrics → reset)
 
## Project layout
 
```
app/                  Next.js App Router pages + API routes
src/                  services, lib, adapters
  lib/
    instrumented-grok.ts    Wrapped Grok client with OTel + cost telemetry
    cost-tracker.ts         Grok pricing and cost span creation
    langfuse-exporter.ts    OTel SDK bootstrap for Langfuse export
    request-normalizer.ts   OpenAI params → LLMRequest mapping
tests/                vitest suite (mirrors src/)
packages/             API references for every dependency (read these first)
DEV_PLAN.md           build plan for this recipe
```
 
## License
 
MIT — see [LICENSE](./LICENSE).