Skip to content
reaatech

Evals & Quality

Agent and RAG eval harnesses, LLM-as-judge, prompt versioning.

Filtering by

2 repos

reaatech/agent-replay

★ 0
These packages give you a deterministic recording and replay system for AI agent interactions. You'd adopt them to debug agent behavior without burning LLM tokens on every iteration — record a trace once, then replay it in stubbed, partial, or diff modes for zero-cost debugging and regression testing. The system is built around a trace-based data model with hierarchical spans and events, with interceptors that monkey-patch OpenAI and Anthropic SDKs transparently, so recording happens without modifying your agent code.
packages
7
updated
2 months ago

reaatech/prompt-version-control

★ 0
These packages give you Git-like version control for AI prompts — an API server, TypeScript SDK, CLI, and MCP server that let you track prompt changes, gate promotions on eval results, and serve A/B deployments. You'd adopt them to solve the problem of managing prompt iterations across development, staging, and production without manual copy-pasting or ad-hoc versioning. The most distinctive thing is that the entire lifecycle — from creating a draft to promoting it to production — is gated by evaluation harness results, with AI agents able to pull managed prompts at runtime via the MCP server.
packages
5
updated
4 months ago