prompt-injection-bench · packages

A collection of pluggable defense adapters for prompt injection detection, providing a standard `DefenseAdapter` interface with `detect()` and `sanitize()` methods across 8 built-in implementations (Rebuff, Lakera Guard, LLM Guard, Garak, OpenAI/Azure/Anthropic/Cohere Moderation, and Custom HTTP). Each adapter extends `BaseAdapter` which handles input validation, SSRF protection, rate limiting, and injection pattern detection.

Canonical TypeScript types, Zod schemas, and attack taxonomy for the prompt-injection-bench benchmarking suite. Exports 18 domain types, 16 Zod schemas for runtime validation, and an 8-category attack taxonomy with weights and severity levels, with zero runtime dependencies beyond Zod.

A corpus builder, validator, and variant generation engine for prompt-injection-bench that produces balanced injection samples from YAML templates using synonym replacement, Unicode homoglyphs, and other obfuscation strategies. Exports factory functions (`createCorpusBuilder`, `createCorpusValidator`) and a `generateDefaultCorpus` function.

A leaderboard manager for prompt-injection-bench that ranks defenses by composite score into S/A/B/C/D tiers, with JSON file persistence and pairwise comparison. Exports a `createLeaderboardManager` factory function and standalone `loadLeaderboardEntries`/`saveLeaderboardEntries` utilities.

An MCP server that exposes four tools (`run_benchmark`, `compare_defenses`, `generate_report`, `submit_results`) for running prompt injection benchmarks against defenses, comparing results, and generating reports. It provides a `createMCPServer` factory function and a `normalizeReportData` utility for standardizing benchmark results across different formats.

A structured logging, OpenTelemetry-compatible metrics, and span-based tracing toolkit for benchmark operations, built on Pino v10 and the OpenTelemetry SDK. It provides `createLogger()`, `createMetricsCollector()`, and `createTracingManager()` factory functions, plus global singleton accessors (`getGlobalLogger()`, `getGlobalTracer()`, `getGlobalMetrics()`) for zero-config usage.

A benchmark execution engine that runs prompt injection attacks against defense adapters in parallel with configurable timeouts, progress reporting, and PII-safe result collection. Exports factory functions (`createBenchmarkEngine`, `createAttackExecutor`, `createDefenseEvaluator`) and requires a `DefenseAdapter` and attack corpus (e.g., from `@reaatech/pi-bench-adapters` and `@reaatech/pi-bench-corpus`).

Computes weighted defense scores, confidence intervals, effect sizes, and statistical comparisons (z-test, chi-square, ANOVA) for prompt-injection benchmark results, exporting functions like `calculateDefenseScore`, `compareMetrics`, and `createStatisticalTests`.

A CLI and library for benchmarking LLM prompt-injection defenses against standardized attack corpora. Exports a `createBenchmarkEngine` function and CLI with subcommands (`benchmark`, `attack`, `compare`, `corpus`, `leaderboard`, `report`) that work with defense adapters for services like Rebuff, Lakera, and OpenAI Moderation.