Skip to content
reaatechREAATECH

@reaatech/llm-router-strategies

npm v1.0.0

Provides a collection of routing strategies and an orchestrator class to select the optimal LLM for a request based on cost, latency, capability, or judgment. It exposes a `StrategyOrchestrator` and several strategy classes that implement a common interface for evaluating and selecting models from a provided pool.

npm →source →

@reaatech/llm-router-strategies

npm version License: MIT CI

Status: Pre-1.0 — APIs may change in minor versions. Pin to a specific version in production.

Pluggable routing strategies for cost-aware, multi-model LLM routing. Four strategies plus a priority-based orchestrator that evaluates them in order, selecting the best model for each request.

Installation

terminal
npm install @reaatech/llm-router-strategies
# or
pnpm add @reaatech/llm-router-strategies

Feature Overview

  • Cost-optimized routing — selects the cheapest model that meets budget constraints
  • Latency-optimized routing — selects the fastest model based on historical P50/P95 data
  • Judgment-based routing — two-tier escalation from cheap workhorses to premium judges
  • Capability-based routing — matches models to required capabilities (code, reasoning, vision, etc.)
  • Strategy orchestrator — evaluates all registered strategies in priority order, picks the best fit
  • Pluggable interface — implement RoutingStrategy to add custom logic
  • Config-driven construction — StrategyOrchestrator.fromConfig() builds from YAML/JSON

Quick Start

typescript
import {
  CostOptimizedStrategy,
  LatencyOptimizedStrategy,
  StrategyOrchestrator,
} from "@reaatech/llm-router-strategies";
 
const orchestrator = new StrategyOrchestrator();
 
orchestrator.register(
  new CostOptimizedStrategy({
    workhorsePool: ["glm-edge", "kat-coder-pro"],
    budgetPerRequest: 0.05,
  }),
);
 
orchestrator.register(
  new LatencyOptimizedStrategy({
    modelPool: ["glm-edge", "kat-coder-pro"],
    targetP99Ms: 2000,
  }),
);
 
const evaluation = orchestrator.evaluate(request, context, availableModels);
console.log(evaluation?.model.id, evaluation?.strategy.name);

API Reference

BaseRoutingStrategy (abstract)

All strategies extend this base class which implements the RoutingStrategy interface.

MemberDescription
nameUnique strategy identifier (set by subclass)
priorityEvaluation order — lower numbers evaluated first
applies(request, context)Returns true if this strategy should be considered for the request
select(request, context, availableModels)Returns a StrategySelectionResult with the chosen model and reason

StrategySelectionResult

FieldTypeDescription
modelModelDefinitionThe selected model
confidencenumber (0–1)How confident the strategy is in its selection
reasonstringHuman-readable explanation
alternativesModelDefinition[]Other models that were considered

CostOptimizedStrategy

Selects the cheapest model that fits within the budget constraint.

typescript
import { CostOptimizedStrategy, type CostOptimizedConfig } from "@reaatech/llm-router-strategies";
 
const strat = new CostOptimizedStrategy({
  workhorsePool: ["glm-edge", "kat-coder-pro", "kimi-chat"],
  budgetPerRequest: 0.05,
});

CostOptimizedConfig

FieldTypeDescription
workhorsePoolstring[]Model IDs available for selection
budgetPerRequestnumberMaximum cost in USD for a single request

LatencyOptimizedStrategy

Selects the fastest model based on historical latency from the RoutingContext.

typescript
import { LatencyOptimizedStrategy, type LatencyOptimizedConfig } from "@reaatech/llm-router-strategies";
 
const strat = new LatencyOptimizedStrategy({
  modelPool: ["glm-edge", "kat-coder-pro"],
  targetP99Ms: 2000,
  defaultTimeoutMs: 3000,
});

LatencyOptimizedConfig

FieldTypeDescription
modelPoolstring[]Model IDs available for selection
targetP99MsnumberTarget P99 latency in milliseconds
defaultTimeoutMsnumberFallback timeout when no history exists

JudgmentBasedStrategy

Two-tier routing: cheap workhorse models handle routine requests, premium judge models handle complex reasoning.

typescript
import {
  JudgmentBasedStrategy,
  type JudgmentBasedConfig,
} from "@reaatech/llm-router-strategies";
 
const strat = new JudgmentBasedStrategy({
  workhorsePool: ["kat-coder-pro", "kimi-chat"],
  judgePool: ["claude-opus", "gpt-4-turbo"],
  escalationThreshold: 0.7,
  maxJudgeInvocations: 2,
  consensusRequired: false,
});

JudgmentBasedConfig

FieldTypeDefaultDescription
workhorsePoolstring[]—Low-cost models for routine tasks
judgePoolstring[]—Premium models for complex tasks
escalationThresholdnumber0.7Confidence threshold below which escalation triggers
maxJudgeInvocationsnumber1How many judge models to consult per escalation
consensusRequiredbooleanfalseWhether all judges must agree

CapabilityBasedStrategy

Routes based on required model capabilities (code, reasoning, vision, etc.).

typescript
import {
  CapabilityBasedStrategy,
  type CapabilityBasedConfig,
} from "@reaatech/llm-router-strategies";
 
const strat = new CapabilityBasedStrategy({
  preferredModels: {
    code: ["kat-coder-pro", "gpt-4-turbo"],
    "complex-reasoning": ["claude-opus"],
  },
});

CapabilityBasedConfig

FieldTypeDescription
preferredModelsPartial<Record<ModelCapability, string[]>>Capability-to-preferred-model mapping
preferredModelIdsstring[]Fallback preferred models regardless of capability
defaultModelstringDefault model when no capability match

StrategyOrchestrator

Evaluates all registered strategies in priority order and returns the best match.

typescript
import { StrategyOrchestrator } from "@reaatech/llm-router-strategies";
 
const orchestrator = new StrategyOrchestrator();
 
// Register strategies individually
orchestrator.register(new CostOptimizedStrategy());
 
// Or build from config
const orchestrator2 = StrategyOrchestrator.fromConfig(
  {
    default: { type: "cost-optimized", workhorsePool: ["glm-edge"] },
    complex: { type: "judgment-based", workhorsePool: ["kat-coder-pro"], judgePool: ["claude-opus"] },
  },
  { workhorsePool: ["glm-edge", "kat-coder-pro"], judgePool: ["claude-opus"] },
);
 
const evaluation = orchestrator.evaluate(request, context, availableModels);
if (!evaluation) {
  throw new Error("No strategy could select a model");
}

StrategyEvaluationResult

FieldTypeDescription
modelModelDefinitionThe model selected by the winning strategy
strategyRoutingStrategyThe strategy that made the selection
selectionResultStrategySelectionResultThe detailed result including confidence and alternatives

Creating a Custom Strategy

Implement the RoutingStrategy interface or extend BaseRoutingStrategy:

typescript
import { BaseRoutingStrategy, type StrategySelectionResult } from "@reaatech/llm-router-strategies";
import type { ModelDefinition, RoutingRequest, RoutingContext } from "@reaatech/llm-router-core";
 
class MyCustomStrategy extends BaseRoutingStrategy {
  readonly name = "my-custom";
  readonly priority = 5;
 
  constructor() {
    super(5);
  }
 
  applies(request: RoutingRequest, _context: RoutingContext): boolean {
    return request.strategy === "my-custom" || request.metadata?.useCustom === true;
  }
 
  select(
    request: RoutingRequest,
    context: RoutingContext,
    availableModels: ModelDefinition[],
  ): StrategySelectionResult | null {
    const candidates = availableModels.filter((m) => m.enabled !== false);
    if (candidates.length === 0) return null;
 
    const selected = candidates[0]; // Your custom selection logic
    return {
      model: selected,
      confidence: 0.8,
      reason: `Selected ${selected.id} via custom logic`,
      alternatives: candidates.slice(1),
    };
  }
}
 
orchestrator.register(new MyCustomStrategy());

Related Packages

License

MIT

All packagesllm-router overview