llm-cache · packages

A caching engine for LLM calls that provides both exact-match (SHA-256 hash) and semantic (cosine similarity on embeddings) cache lookups, with model-aware fingerprinting, use-case segmentation, and adaptive TTL. It exports a `CacheEngine` class that requires `StorageAdapter` and `VectorStorageAdapter` implementations (e.g., in-memory, Redis, DynamoDB) and an `Embedder` for semantic matching.

A DynamoDB storage adapter for `@reaatech/llm-cache` that persists exact-match cache entries with native TTL, GSI-backed metadata queries, and batch operations chunked to AWS limits. Exports a `DynamoDBAdapter` class implementing the `StorageAdapter` interface from `@reaatech/llm-cache`.

A Qdrant vector database adapter for `@reaatech/llm-cache` that implements the `VectorStorageAdapter` interface, providing HNSW approximate nearest neighbor search with metadata filtering and deterministic UUID-based point IDs.

A Redis storage adapter for the `@reaatech/llm-cache` library that implements the `StorageAdapter` interface, providing exact-match cache operations with automatic TTL via `SETEX`, batch operations, and metadata queries using `SCAN`.

A cost calculator and pricing database for LLM API usage, providing a `CostCalculator` class that computes per-request costs from token counts and model pricing, and tracks savings from cache hits. It ships with reference pricing for 40+ models across OpenAI, Anthropic, and Google, and implements the `CostCalculatorLike` interface for drop-in integration with `@reaatech/llm-cache`.

A structured JSON logger and Prometheus-compatible metrics collector for LLM cache operations, providing automatic PII redaction on 17 sensitive field names, correlation ID propagation via `child()`, and cardinality-protected counters and histograms with zero runtime dependencies.

An HTTP server wrapper for llm-cache that exposes a REST API for cache operations, Prometheus metrics, and health endpoints, configurable via environment variables for storage (memory, Redis, DynamoDB) and vector search (memory, Qdrant) backends. Exports `createApp()` returning an `App` object with an `http.Server`, cache engine instance, and `shutdown()` method, plus a `main()` convenience function for direct CLI or programmatic use.