Skip to content
reaatechREAATECH
Book a conversation

@reaatech/llm-cache-adapters-dynamodb

pending npm

Provides a DynamoDB storage adapter class for the `llm-cache` library, enabling persistent caching with native TTL support and GSI-based metadata querying. It implements the `StorageAdapter` interface and requires a pre-configured DynamoDB table with specific partition and sort key schemas.

source →

@reaatech/llm-cache-adapters-dynamodb

npm version License: MIT CI

Status: Pre-1.0 — APIs may change in minor versions. Pin to a specific version in production.

DynamoDB storage adapter for llm-cache exact-match metadata. Provides serverless, scalable persistence with native DynamoDB TTL, GSI-based metadata queries, and batch operations chunked to AWS limits.

Installation

terminal
npm install @reaatech/llm-cache-adapters-dynamodb
# or
pnpm add @reaatech/llm-cache-adapters-dynamodb

Feature Overview

  • Native DynamoDB TTL — writes an epoch-second attribute (expiresAtEpoch by default) for automatic row expiration
  • GSI-backed queries — gsi1 indexes useCase, gsi2 indexes modelVersion for efficient metadata queries
  • Batch operations — getBatch, setBatch, and deleteBatch chunked to DynamoDB’s 25-item limit
  • Paginated invalidation — invalidateByCriteria uses GSI queries when possible, falling back to Scan for broad criteria
  • Stats — getStats() returns item count and table size from DescribeTable
  • Health check — healthCheck() runs a lightweight Scan (limit 1)

Quick Start

typescript
import { CacheEngine, OpenAIEmbedder } from "@reaatech/llm-cache";
import { DynamoDBAdapter } from "@reaatech/llm-cache-adapters-dynamodb";
 
const storage = new DynamoDBAdapter({
  region: "us-east-1",
  tableName: "llm-cache",
});
 
const cache = new CacheEngine({
  storage,
  vectorStorage: /* QdrantAdapter or InMemoryAdapter */,
  embedder: new OpenAIEmbedder({
    provider: "openai",
    model: "text-embedding-3-small",
    dimensions: 1536,
    apiKey: process.env.OPENAI_API_KEY,
  }),
  config: { /* ... */ },
});

Table Schema

Your DynamoDB table must have these attributes and indexes:

AttributeTypeKeyDescription
pkStringHASHExact-match key (promptHash:generationConfigHash)
gsi1pkStringGSI1 HASHUSECASE#<useCase> for useCase queries
gsi1skStringGSI1 RANGEmodelVersion#generationConfigHash
gsi2pkStringGSI2 HASHMODEL#<modelVersion> for model queries
gsi2skStringGSI2 RANGEuseCase#createdAt
expiresAtEpochNumber(TTL)Epoch seconds for native DynamoDB TTL

Enable TTL on the expiresAtEpoch attribute (or your custom ttlAttribute name) in the DynamoDB console.

API Reference

DynamoDBAdapter (class)

Implements StorageAdapter from @reaatech/llm-cache.

typescript
import { DynamoDBAdapter } from "@reaatech/llm-cache-adapters-dynamodb";
 
const adapter = new DynamoDBAdapter({
  region: "us-east-1",
  tableName: "llm-cache",
});

DynamoDBAdapterConfig

PropertyTypeDefaultDescription
regionstring(required)AWS region
tableNamestring(required)DynamoDB table name
endpointstring—Override endpoint (e.g., DynamoDB Local: http://localhost:8000)
ttlAttributestringexpiresAtEpochAttribute name for native DynamoDB TTL

Methods

MethodReturnsDescription
get(key)Promise<CacheEntry | null>Retrieve and deserialize a cache entry (auto-deletes if expired)
set(key, entry)Promise<void>Store an entry with native TTL epoch attribute
delete(key)Promise<boolean>Remove a key
exists(key)Promise<boolean>Check if a key exists (reads then checks expiry)
getBatch(keys)Promise<(CacheEntry | null)[]>Batch retrieve via BatchGetCommand
setBatch(items)Promise<void>Batch store chunked at 25 items per BatchWriteCommand
deleteBatch(keys)Promise<number>Batch delete chunked at 25 keys per BatchWriteCommand
findByUseCase(useCase, limit?)Promise<CacheEntry[]>Query GSI1 for entries by use case
findByModelVersion(modelVersion, limit?)Promise<CacheEntry[]>Query GSI2 for entries by model version
invalidateByCriteria(criteria)Promise<number>Delete matching entries — uses GSI when possible
getStats()Promise<StorageStats>Get ItemCount and TableSizeBytes from DescribeTable
healthCheck()Promise<HealthStatus>Run a limit-1 Scan and report status

Usage Patterns

Local Development with DynamoDB Local

typescript
const adapter = new DynamoDBAdapter({
  region: "us-east-1",
  tableName: "llm-cache",
  endpoint: "http://localhost:8000",
});

Custom TTL Attribute

typescript
const adapter = new DynamoDBAdapter({
  region: "us-east-1",
  tableName: "llm-cache",
  ttlAttribute: "ttl", // Match your table's TTL attribute name
});

IAM Credentials

The adapter uses the AWS SDK credential chain. Configure via environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY), IAM roles (EC2, ECS, Lambda), or ~/.aws/credentials.

Notes

  • Semantic search requires a separate vector database (e.g., Qdrant). DynamoDB does not support native vector similarity search.
  • Batch writes (setBatch, deleteBatch) are automatically chunked to DynamoDB’s 25-item limit across multiple requests.
  • invalidateByCriteria prefers GSI-indexed criteria (useCase, modelVersion). Broad invalidation (e.g., olderThan only) falls back to a full table Scan.

Related Packages

License

MIT

All packagesllm-cache overview