Testing & Security

These packages give you a middleware-based fault injection engine that sits between your agent and its tools, injecting failures like latency spikes, rate limits, and malformed output to test whether your circuit breakers, confidence gates, and fallback trees actually work. You'd adopt them to validate agent reliability under realistic failure conditions before they hit production. The most distinctive thing is the transparent interceptor pattern — adapters for LangChain, LlamaIndex, and Vercel AI SDK wrap your existing tool calls without modifying agent code, while a declarative YAML/JSON scenario system with probability-based fault selection and hot reloading lets you change failure modes at runtime.

These packages give you a reproducible benchmark for evaluating prompt-injection defenses in AI agent systems, including an attack corpus with 300+ templates across 8 categories, pluggable defense adapters (Rebuff, Lakera Guard, LLM Guard, Garak, OpenAI/Azure/Anthropic/Cohere Moderation, Custom HTTP), a scoring engine with statistical analysis, a public leaderboard, and an MCP server. You would adopt them to objectively measure and compare how well different defenses detect and block prompt injection attacks, using a standardized methodology with deterministic seeds and SHA-256 proofs for reproducibility. The packages are designed as independent modules—core types, corpus builder, adapters, runner, scoring, leaderboard, observability, and MCP server—that share canonical Zod schemas and a common `DefenseAdapter` interface, so you can mix and match only the pieces you need or run the full benchmark via the umbrella CLI.

These packages give you a policy enforcement layer that sits between an AI agent and its MCP servers, intercepting every `tools/call` to validate, rate-limit, and audit tool invocations before they reach the upstream. You'd adopt them to prevent an agent from accidentally or maliciously executing destructive operations like `DROP TABLE` or `rm -rf /`, and to enforce budgets, approval workflows, and read-only modes across database, filesystem, or network MCP servers. The system is built as a pluggable middleware pipeline—rate limiter, cost tracker, secret scanner, argument validator, policy engine, anomaly detector, and approval workflow each implement the same `Middleware` interface and run in strict order, with stages registered only when enabled in the policy configuration.