@reaatech/pi-bench-core
Status: Pre-1.0 — APIs may change in minor versions. Pin to a specific version in production.
Canonical TypeScript types, Zod schemas, and attack taxonomy for the prompt-injection-bench suite. This package is the single source of truth for all domain types used throughout the @reaatech/pi-bench-* ecosystem.
Installation
terminal
npm install @reaatech/pi-bench-core
# or
pnpm add @reaatech/pi-bench-coreFeature Overview
- 18 domain types —
AttackCategory,InjectionSample,BenchmarkResult,DefenseScore, and more - 16 Zod schemas — parse and validate injection samples, benchmark results, scores, and leaderboard entries at the boundary
- Attack taxonomy — 8 categories with weights, descriptions, and severity levels
- Zero runtime dependencies beyond
zod— lightweight and tree-shakeable - Dual ESM/CJS output — works with
importandrequire
Quick Start
typescript
import {
AttackCategory,
InjectionSampleSchema,
type InjectionSample,
} from "@reaatech/pi-bench-core";
// Validate an injection sample at the boundary
const raw = JSON.parse(incomingJson);
const sample = InjectionSampleSchema.parse(raw);
// Use the attack taxonomy
import { getCategoryIds, getCategoryWeight, ATTACK_TAXONOMY } from "@reaatech/pi-bench-core";
const categories = getCategoryIds();
for (const cat of categories) {
console.log(`${cat}: weight=${getCategoryWeight(cat)}`);
}Exports
Domain Types
| Export | Description |
|---|---|
AttackCategory | Union type of all 8 attack categories |
SeverityLevel | Severity: low, medium, high, critical |
InjectionSample | An attack sample: sampleId, category, prompt, severity, tags |
BenchmarkResult | Full result of a benchmark run: runId, defense, attackResults, benignResults |
AttackResult | Single attack result: sampleId, attackCategory, defense, detected, confidence |
DefenseScore | Calculated score: overallScore, attackSuccessRate, falsePositiveRate, categoryScores |
CategoryScore | Per-category scoring: category, detectionRate, weightedScore |
AttackTemplate | Template definition: id, category, template, variables, severity |
CorpusManifest | Corpus snapshot: version, createdAt, categoryCounts, severityDistribution |
LeaderboardEntry | Leaderboard record: defense, version, score, submittedAt |
EvaluationResult | Defense evaluation: overall, categories, comparisons |
Zod Schemas
| Schema | Validates |
|---|---|
InjectionSampleSchema | Attack sample shape |
AttackResultSchema | Single attack execution result |
BenchmarkResultSchema | Full benchmark run output |
DefenseScoreSchema | Calculated defense score |
CategoryScoreSchema | Per-category score |
LeaderboardEntrySchema | Leaderboard submission |
CorpusManifestSchema | Corpus version manifest |
AttackTemplateSchema | Attack template definition |
Taxonomy
| Export | Description |
|---|---|
ATTACK_TAXONOMY | Full taxonomy with metadata, weights, and descriptions |
ATTACK_CATEGORIES | Array of category metadata objects |
getCategoryIds() | Returns all 8 category identifiers |
getCategoryWeight(category) | Returns the scoring weight for a category |
isValidCategory(v) | Type guard for category identity |
Attack Taxonomy
| Category | Weight | Description |
|---|---|---|
direct-injection | 1.0 | Obvious malicious instruction overrides |
prompt-leaking | 1.2 | Attempts to extract system prompts |
role-playing | 1.3 | DAN, developer mode, persona adoption |
encoding-attacks | 1.1 | Base64, ROT13, Unicode obfuscation |
multi-turn-jailbreaks | 1.4 | Gradual persuasion across turns |
payload-splitting | 1.2 | Attack split across multiple inputs |
translation-attacks | 1.1 | Low-resource language injection |
context-stuffing | 1.3 | Hiding injection in large context |
Usage Pattern
Every schema has a matching type. Use the schema for runtime validation and the type for compile-time checking:
typescript
import { InjectionSampleSchema, type InjectionSample } from "@reaatech/pi-bench-core";
function handleSample(raw: unknown): InjectionSample {
return InjectionSampleSchema.parse(raw);
}Related Packages
@reaatech/pi-bench-corpus— Attack corpus builder and validator@reaatech/pi-bench-adapters— Defense adapter implementations@reaatech/pi-bench-scoring— Scoring engine and statisticsprompt-injection-bench— CLI and umbrella package