llm-router · packages

A CLI that provides four commands—`route`, `benchmark`, `cost-report`, and `validate-config`—for routing prompts through configurable LLM strategies, benchmarking model performance with latency and cost statistics, generating cost summaries per budget period, and validating router configuration files.

Core TypeScript types, Zod schemas, and input validation for the llm-router ecosystem, providing 30+ domain types and 11 runtime validation schemas for model definitions, routing requests, configurations, budgets, quality scores, and cost telemetry. Exports composable enums, circuit breaker types, and dual ESM/CJS modules with zero runtime dependencies beyond `zod`.

A config-driven `LLMRouter` class that orchestrates model selection, fallback chains, cost tracking, A/B testing, quality scoring, and observability into a single routing decision for LLM calls. It requires an `executeModel` callback or registered provider clients to actually dispatch requests.

A function that creates ordered fallback chains for LLM calls, with per-model circuit breakers and exponential-backoff retry logic, returning a `FallbackChain` instance whose `executeFrom` method automatically tries the next model when one fails.

An MCP server that exposes three tools—`route_request`, `get_model_info`, and `get_cost_report`—for routing LLM requests through a pluggable router implementation via the Model Context Protocol. It provides a `createMCPServer` factory function that returns an `MCPServer` class instance, which requires a `RouterInterface` implementation and auto-detects stdio or Streamable HTTP transport.

A set of pluggable routing strategies (CostOptimized, LatencyOptimized, JudgmentBased, CapabilityBased) and a priority-based `StrategyOrchestrator` that evaluates them in order to select the best LLM model for each request based on cost, latency, or capability constraints.

A set of classes (`CostTracker`, `BudgetManager`, `CostReporter`) and a `MetricsCollector` facade for tracking per-request LLM costs, enforcing daily budget limits with configurable soft/hard enforcement and alert callbacks, and exposing OpenTelemetry-compatible metrics. Designed to pair with `@reaatech/llm-router` but usable standalone.