@reaatech/pi-bench-observability
Status: Pre-1.0 — APIs may change in minor versions. Pin to a specific version in production.
Structured logging, OpenTelemetry-compatible metrics, and span-based tracing for benchmark operations. Built on Pino (v10) and the OpenTelemetry SDK.
Installation
terminal
npm install @reaatech/pi-bench-observability
# or
pnpm add @reaatech/pi-bench-observabilityFeature Overview
- Structured JSON logging — Pino-powered, fast and low-overhead, with built-in PII sanitization
- OpenTelemetry metrics — Counter, histogram, and gauge with periodic export
- Span-based tracing — Benchmark → Attack → Detection three-level span hierarchy
- Global singletons —
getGlobalLogger(),getGlobalTracer(),getGlobalMetrics()for zero-config usage - Dual ESM/CJS output — works with
importandrequire
Quick Start
typescript
import {
createLogger,
createTracingManager,
createMetricsCollector,
} from "@reaatech/pi-bench-observability";
const logger = createLogger();
logger.info("Benchmark started", { runId: "abc123" });
const tracer = createTracingManager();
const span = tracer.startSpan("benchmark.run");
span.addEvent("attack.executed", { sampleId: "att-001" });
span.end();
const metrics = createMetricsCollector();
metrics.incrementCounter("pi_bench.runs.total", 1);API Reference
StructuredLogger
All log methods accept an optional context object as the first argument:
typescript
export type LogLevel = "debug" | "info" | "warn" | "error";
logger.info("Simple message");
logger.warn({ sampleId: "att-001" }, "Attack sample triggered warning");
logger.error({ err: new Error("timeout") }, "Benchmark execution failed");createLogger(service?: string): StructuredLogger
Creates a logger with the given service name (default: "prompt-injection-bench").
MetricsCollector
| Method | Description |
|---|---|
incrementCounter(name, value?, labels?) | Increment a counter metric |
recordHistogram(name, value, labels?) | Record a histogram observation |
setGauge(name, value, labels?) | Set a gauge value |
export() | Export all metrics |
Built-in Metrics
| Metric | Type | Description |
|---|---|---|
pi_bench.runs.total | Counter | Total benchmark runs |
pi_bench.attacks.executed | Counter | Attacks executed |
pi_bench.defense.detection_rate | Gauge | Current detection rate |
pi_bench.latency.ms | Histogram | Execution latency |
pi_bench.cost.usd | Histogram | Cost per run |
TracingManager
| Method | Description |
|---|---|
startSpan(name, attrs?) | Start a new span, returns a Span |
Span
| Method | Description |
|---|---|
end() | End the span |
addEvent(name, attrs?) | Add a named event to the span |
setAttributes(attrs) | Set key-value attributes |
setStatus(code, message?) | Set span status (0 = OK) |
recordException(error) | Record an error on the span |
Usage Patterns
PII Sanitization
The logger automatically truncates input strings for safe logging:
typescript
import { sanitizeForLogging } from "@reaatech/pi-bench-observability";
const safe = sanitizeForLogging(longPromptText, 50);
// → "... [truncated, 2048 chars total]"Global Access
For convenience, get shared instances from anywhere:
typescript
import {
getGlobalLogger,
getGlobalTracer,
getGlobalMetrics,
} from "@reaatech/pi-bench-observability";
getGlobalLogger().info("Using global logger");
getGlobalMetrics().incrementCounter("custom.metric");Related Packages
@reaatech/pi-bench-runner— Uses tracing for benchmark lifecycleprompt-injection-bench— CLI and umbrella package