Observability & Cost

These packages give you a real-time budget enforcement layer for LLM-powered agents that checks every request against per-task, per-user, per-session, or per-organization spend limits before it executes. You'd adopt them to prevent runaway agent loops from exhausting your LLM budget in minutes, with graceful degradation like model downgrades and tool filtering before a hard stop. The system is built as a set of composable packages—a core engine with a state machine, a circular-buffer spend tracker, pricing tables, and optional integrations for Express/Fastify middleware, an LLM Router plugin, and an OpenTelemetry bridge that automatically records GenAI spans as spend entries.

These packages give you a semantic caching layer for LLM calls that returns cached responses for both exact prompt matches and semantically similar prompts above a configurable cosine similarity threshold. You'd adopt them to reduce API costs and latency by avoiding redundant LLM calls, especially when users ask the same question in different phrasings. The system is built as a modular engine with pluggable storage adapters (Redis, DynamoDB, Qdrant) and optional cost tracking, observability, and HTTP server packages that compose together through well-defined interfaces rather than a monolithic service.

These packages give you drop-in wrappers for OpenAI, Anthropic, and Google Generative AI SDKs that automatically capture token usage and cost from every API call, plus a cost calculation engine, multi-tenant aggregation and budget enforcement, OpenTelemetry observability, cloud exporters to CloudWatch/Cloud Monitoring/Loki, an MCP server, and a CLI. You'd adopt them to track, aggregate, and enforce budgets on LLM spend across providers and tenants without building your own cost telemetry pipeline. The packages are designed as independent modules that share common types and schemas, so you can use just the provider wrappers or combine them with the aggregation, budget checking, and export layers as needed.

These packages convert GenAI semantic convention spans into real-time USD cost metrics, exporting them via Prometheus, OTLP, or JSON. You would adopt them to track LLM spend per model and provider without manually maintaining pricing tables for OpenAI, Anthropic, Google, AWS Bedrock, and Azure. The packages compose as a layered pipeline—core types, pricing tables, a calculator with LRU caching, and an exporter that can run either as an in-process Node.js SpanProcessor or as a standalone OTLP collector.

These packages give you instrumented wrappers for OpenAI, Anthropic, Vertex AI, and AWS Bedrock that emit OpenTelemetry GenAI semantic convention spans, plus deployable dashboards for Phoenix, Langfuse, and Cloud Trace. You would adopt them to get spec-compliant observability across multiple LLM providers without writing instrumentation code yourself. The packages are designed as independent, installable modules—core types, instrumentation framework, provider wrappers, utilities, and exporters—so you can compose exactly what you need rather than pulling in a monolithic library.