@reaatech/llm-judge-providers

Status: Pre-1.0 — APIs may change in minor versions. Pin to a specific version in production.

LLM provider implementations with a factory pattern for OpenAI, Anthropic, and local (OpenAI-compatible) endpoints. All providers implement the shared LLMProvider interface with dynamic SDK loading.

Installation

npm install @reaatech/llm-judge-providers
# or
pnpm add @reaatech/llm-judge-providers

For provider SDKs (optional, lazily loaded):

npm install openai               # for OpenAIProvider
npm install @anthropic-ai/sdk    # for AnthropicProvider

Feature Overview

• OpenAI provider — GPT-4o, GPT-4o-mini, GPT-4 Turbo with per-model pricing tables
• Anthropic provider — Claude 3.5 Sonnet, Claude 3 Haiku with system/user message separation
• Local provider — raw fetch to any OpenAI-compatible endpoint (Ollama, vLLM, LM Studio)
• ProviderFactory — create() for explicit config, fromEnv() for environment variable resolution
• Dynamic SDK loading — SDKs only loaded when a provider is actually instantiated
• Built-in cost calculation — per-million-token pricing in USD for every model
• Health checks — checkHealth() returns latency and status per provider

Quick Start

import { ProviderFactory } from "@reaatech/llm-judge-providers";
 
// From environment variables (uses LLM_JUDGE_PROVIDER or defaults to 'openai')
const provider = ProviderFactory.fromEnv();
 
// Or create explicitly by name
const localProvider = ProviderFactory.create({ name: "local" });
 
const response = await provider.generateCompletion({
  model: "gpt-4o-mini",
  messages: [
    { role: "system", content: "You are an evaluator." },
    { role: "user", content: "Score this response: ..." },
  ],
  temperature: 0.1,
  maxTokens: 2000,
});
 
console.log(response.content);
console.log(provider.calculateCost(response.usage));

API Reference

OpenAIProvider

Constructor options:

Supported models:

Per-model pricing (per 1M tokens):

AnthropicProvider

Constructor options:

Supported models:

Per-model pricing (per 1M tokens):

LocalProvider

Constructor options:

ProviderFactory

Static methods:

LLMProvider Interface

All providers implement this shared interface from @reaatech/llm-judge-types:

Related Packages

• @reaatech/llm-judge-types — Core types including LLMProvider interface
• @reaatech/llm-judge-engine — Judgment engine that consumes providers

License

MIT