reaatech/prompt-injection-bench

★ 0Last commit: May 12, 2026GitHub →

These packages provide a standardized framework for benchmarking and evaluating the effectiveness of prompt-injection defenses in AI agent systems. They allow you to measure security posture by running a diverse corpus of adversarial attacks against pluggable defense adapters and calculating statistically significant performance scores. The system is designed as a modular pipeline where you can swap defense implementations, execute parallelized benchmarks, and generate reproducible reports through a unified CLI or MCP-compatible interface.

adversarial-testing agentic-ai ai-safety ai-security benchmark defense llm llm-security prompt-injection red-teaming sanitization security test-corpus typescript

Packages

Sort

9 packages

pi-bench-adapters

@reaatech/pi-bench-adapters

pending npm

Provides a collection of standardized adapter classes and a registry for integrating various prompt injection detection services and libraries. Each adapter implements a common interface with `detect` and `sanitize` methods, inheriting built-in SSRF protection, rate limiting, and input validation from a shared base class.

View package

status: awaiting publish

pi-bench-core

@reaatech/pi-bench-core

pending npm

Provides TypeScript types, Zod schemas, and a standardized attack taxonomy for validating and scoring prompt injection benchmarks. It exports utility functions and schema objects that require `zod` as a runtime dependency.

View package

status: awaiting publish

pi-bench-corpus

@reaatech/pi-bench-corpus

pending npm

Generates and validates datasets of prompt injection attacks using a template-based engine that applies obfuscation strategies like synonym replacement and character manipulation. It provides factory functions to create builders and validators that ensure balanced, versioned, and reproducible corpora for security testing.

View package

status: awaiting publish

pi-bench-leaderboard

@reaatech/pi-bench-leaderboard

pending npm

Manages and persists ranked leaderboard data for prompt injection defenses using a factory-provided manager object. It calculates composite scores and assigns performance tiers, with built-in support for JSON file I/O and pairwise entry comparisons.

View package

status: awaiting publish

pi-bench-mcp-server

@reaatech/pi-bench-mcp-server

pending npm

Exposes prompt-injection-bench operations as an MCP server, providing tools to execute benchmarks, compare defense results, and generate reports via stdio. It also includes utility functions for normalizing benchmark data and managing deterministic seeds for reproducible testing.

View package

status: awaiting publish

pi-bench-observability

@reaatech/pi-bench-observability

pending npm

Provides pre-configured observability utilities for prompt injection benchmarks, including a Pino-based structured logger with PII sanitization, an OpenTelemetry-compatible metrics collector, and a span-based tracing manager. It exposes factory functions for creating these instances or provides global singletons for zero-config access.

View package

status: awaiting publish

pi-bench-runner

@reaatech/pi-bench-runner

pending npm

Executes prompt injection benchmarks by running attack suites against defense adapters in parallel with configurable timeouts and progress tracking. It provides factory functions to create a benchmark engine, attack executor, and defense evaluator, requiring a compatible defense adapter implementation to function.

View package

status: awaiting publish

pi-bench-scoring

@reaatech/pi-bench-scoring

pending npm

Calculates weighted scores, statistical significance, and effect sizes for prompt injection defense benchmarks. It provides a collection of utility functions for computing metrics like Wilson score intervals, Cohen's h, and z-tests from benchmark result objects.

View package

status: awaiting publish

prompt-injection-bench

pending npm

Evaluates LLM prompt injection defenses using a CLI for running benchmarks and a library of re-exported APIs for programmatic execution. It provides a suite of attack corpora and defense adapters to measure and compare the security performance of different injection mitigation strategies.