A solo recruiter at a 5-person agency spends 10+ hours per role manually scoring resumes against client rubrics. With 50-200 resumes per role and 40 active roles, this becomes unsustainable. The recruiter needs a consistent, auditable scoring system to surface top candidates quickly without hiring more staff.
A complete, working implementation of this recipe — downloadable as a zip or browsable file by file. Generated by our build pipeline; tested with full coverage before publishing.
A solo recruiter at a 5-person agency spends 10+ hours per role manually scoring resumes against client rubrics. With 50-200 resumes per role and 40 active roles, that workload doesn’t scale. This tutorial walks you through building an Automated Resume Rubric Scorer — a Next.js App Router application that scores resumes against custom rubrics using an LLM, caches results for consistency, tracks costs, and ranks candidates automatically.
You’ll connect six REAA evaluation packages (golden trajectories, tool-use validation, cost tracking, observability, and semantic caching), build a BullMQ-backed scoring queue, and cover the pipeline with 90%+ test coverage.
Expected output: All four files compile cleanly with pnpm typecheck. The schemas enforce constraints like criteria must be non-empty, percentage 0-100, and weight 0-1.
Step 3: Create the in-memory store
Since this recipe has no database, data is stored in memory. The InMemoryStore wraps a Map with get, set, delete, and list:
Expected output:InMemoryStore is generic over any object with an id field. Each service creates its own singleton store instance.
Step 4: Build the LLM client
The LLM client wraps Vercel AI SDK’s generateText with the OpenAI provider, returning both the generated text and token usage:
ts
// src/lib/llm-client.tsimport { generateText } from "ai";import { openai } from "@ai-sdk/openai";export function getDefaultModel() { return openai(process.env.LLM_MODEL ?? "gpt-5.2");}export async function generateWithModel(systemPrompt: string, userPrompt: string) { const result = await generateText({ model: getDefaultModel(), system: systemPrompt, prompt: userPrompt, }); return { text: result.text, usage: { inputTokens: result.usage.inputTokens, outputTokens: result.usage.outputTokens, }, };}
Expected output:generateWithModel("You are a scorer", "Score this resume...") returns { text, usage: { inputTokens, outputTokens } }.
Step 5: Wire up the cache engine
The cache layer uses @reaatech/llm-cache with a RedisAdapter for primary storage and an InMemoryAdapter for vector embeddings. It supports semantic similarity lookups and TTL-based expiry.
Expected output: On first call with a key, getCachedOrGenerate runs the generator and caches the result. On subsequent calls with the same key, it returns the cached value with fromCache: true.
Step 6: Implement the rubric service
The rubric service provides full CRUD over rubrics. Every mutation validates through RubricSchema.parse().
ts
// src/services/rubric-service.tsimport { RubricSchema, type Rubric } from "../schemas/rubric";import { InMemoryStore } from "../lib/storage";const store = new InMemoryStore<Rubric>();export function createRubric(data: Omit<Rubric, "id" | "createdAt">): Rubric { const rubric: Rubric = { ...data, id: crypto.randomUUID(), createdAt: new Date().toISOString(), }; const parsed = RubricSchema.parse(rubric); store.set(parsed.id, parsed); return parsed;}export function getRubric(id: string): Rubric | undefined { return store.get(id);}export function listRubrics(): Rubric[] { return store.list();}export function updateRubric(id: string, changes: Partial<Omit<Rubric, "id" | "createdAt">>): Rubric | undefined { const existing = store.get(id); if (!existing) return undefined; const updated: Rubric = { ...existing, ...changes }; const parsed = RubricSchema.parse(updated); store.set(parsed.id, parsed); return parsed;}export function deleteRubric(id: string): boolean { return store.delete(id);}
Expected output:createRubric({ name: "SDE", roleName: "Software Engineer", criteria: [...] }) returns a rubric with a generated UUID and ISO timestamp. Deleting a non-existent id returns false without throwing.
Step 7: Build the resume parser
The resume parser handles PDF files via unpdf and Office documents (DOCX, XLSX, PPTX) via office-text-extractor.
ts
// src/services/resume-parser.tsimport { getTextExtractor } from "office-text-extractor";import { extractText, getDocumentProxy } from "unpdf";import { type Resume, ResumeSchema } from "../schemas/resume";import { InMemoryStore } from "../lib/storage";const store = new InMemoryStore<Resume>();export async function extractTextFromFile(buffer: Uint8Array, mimeType: string, _filename?: string): Promise<string> { void _filename; if (mimeType === "application/pdf") { const pdf = await getDocumentProxy(buffer); const result = await extractText(pdf, { mergePages: true }); return result.text; } if ( mimeType === "application/vnd.openxmlformats-officedocument.wordprocessingml.document" || mimeType === "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" || mimeType === "application/vnd.openxmlformats-officedocument.presentationml.presentation" ) { const extractor = getTextExtractor(); const text = await extractor.extractText({ input: buffer, type: "buffer" }); return text; } throw new Error(`Unsupported mime type: ${mimeType}`);}export async function storeResume(filename: string, mimeType: string, buffer: Uint8Array): Promise<Resume> { const extractedText = await extractTextFromFile(buffer, mimeType); const resume: Resume = { id: crypto.randomUUID(), filename, mimeType, extractedText, uploadedAt: new Date().toISOString(), }; const parsed = ResumeSchema.parse(resume); store.set(parsed.id, parsed); return parsed;}export function getResume(id: string): Resume | undefined { return store.get(id);}export function listResumes(): Resume[] { return store.list();}export function deleteResume(id: string): boolean { return store.delete(id);}
Expected output: Uploading resume.pdf extracts the full text. Uploading an unsupported format throws "Unsupported mime type: ...".
Step 8: Build the scorer service
The scorer service is where all six REAA packages come together. It constructs a prompt from the rubric criteria, calls the LLM, validates the structured JSON response, computes cost, checks budgets, creates golden trajectories, validates tool calls, and runs curation.
ts
// src/services/scorer-service.tsimport { createGolden, compareAgainstGolden, quickCreateGolden, GoldenCurator, createCurator, batchCompare, findBestGolden } from "@reaatech/agent-eval-harness-golden";import { validateToolCall, createToolSchema, verifyResult, validateTrajectory, validateTurn } from "@reaatech/agent-eval-harness-tool-use";import { calculateTrajectoryCost, checkBudget, createBudget, generateCostReport, CostTracker } from "@reaatech/agent-eval-harness-cost";import { getLogger, getTracingManager, getMetricsManager, getDashboardManager, withTracing } from "@reaatech/agent-eval-harness-observability";import { CacheEngine } from "@reaatech/llm-cache";const logger = getLogger();const dashboardManager = getDashboardManager();const tracingManager = getTracingManager();import { generateWithModel } from "../lib/llm-client";
Expected output:scoreResume(resumeId, rubricId) returns a validated ScoreResult with per-criterion scores, an LLM trajectory stored as a golden, and budget checks enforced. The CostTracker tracks cumulative spend across all scoring runs.
Step 9: Create the candidate service
The candidate service links resumes to people and tracks the best score across multiple scoring runs.
Expected output:enqueueScoringJob(resumeId, rubricId) adds a job to the queue. The worker processes it, calling scoreResumeWithCache, and logs completion or failure through the observability logger.
Step 12: Create the API route handlers
All routes use NextRequest / NextResponse as required by the App Router. Here are the key route handlers.
Health check (app/api/health/route.ts):
ts
import { NextResponse } from "next/server";export function GET() { return NextResponse.json({ status: "ok", queue: "connected", });}
Rubrics CRUD (app/api/rubrics/route.ts) — list and create:
ts
import { NextRequest, NextResponse } from "next/server";import { ZodError } from "zod";import type { Rubric } from "@/src/schemas/rubric";import { createRubric, listRubrics } from "@/src/services/rubric-service";export function GET(_req: NextRequest) { void _req; return NextResponse.json(listRubrics());}export async function POST(req: NextRequest) { try { const body = (await req.json()) as Omit<Rubric, "id" | "createdAt">; const rubric = createRubric(body); return NextResponse.json(rubric, { status: 201 }); } catch (error) { if (error instanceof ZodError) { return NextResponse.json({ error: error.message }, { status: 400 }); } return NextResponse.json({ error: "Internal server error" }, { status: 500 }); }}
Single rubric (app/api/rubrics/[id]/route.ts) — get, update, delete with Next 16 async params:
ts
import { NextRequest, NextResponse } from "next/server";import type { Rubric } from "@/src/schemas/rubric";import { getRubric, updateRubric, deleteRubric } from "@/src/services/rubric-service";export async function GET( _req: NextRequest, { params }: { params: Promise<{ id: string }> },) { const { id } = await params; const rubric = getRubric(id); if (!rubric) { return NextResponse.json({ error: "Rubric not found" }, { status: 404 }); } return NextResponse.json(rubric);}export async function PUT( req: NextRequest, { params }: { params: Promise<{ id: string }> },) { const { id } = await params; const body = (await req.json()) as Partial<Omit<Rubric, "id" | "createdAt">>; const rubric = updateRubric(id, body); if (!rubric) { return NextResponse.json({ error: "Rubric not found" }, { status: 404 }); } return NextResponse.json(rubric);}export async function DELETE( _req: NextRequest, { params }: { params: Promise<{ id: string }> },) { const { id } = await params; deleteRubric(id); return new NextResponse(null, { status: 204 });}
Expected output: All 12 API endpoints respond correctly:
GET /api/health returns { status: "ok", queue: "connected" }
POST /api/rubrics returns 201 with the created rubric
GET /api/rubrics/:id returns 404 for a non-existent id
POST /api/resumes with a file returns 201; without a file returns 400
POST /api/resumes/:id/score with a rubricId returns 202 with a jobId
Step 13: Set up instrumentation
The src/instrumentation.ts module initializes REAA observability singletons at startup (Node.js runtime only). The next.config.ts enables instrumentationHook so Next.js calls register() during server startup.
Expected output: Next.js calls register() on server boot. The observability singletons are available across all services without manual initialization.
Step 14: Run the tests
The test suite uses Vitest with MSW for HTTP mocking and vi.mock for package-level mocks. The tests/setup.ts mocks the six REAA packages and the OpenAI API endpoint so tests run without live HTTP calls:
API route handlers — all 12 endpoints with happy/error/boundary tests
E2E flows — full scoring pipeline from rubric creation to score result validation, multi-candidate ranking
Next steps
Add a database backend — Replace InMemoryStore with PostgreSQL or SQLite to persist rubrics, resumes, and score results across server restarts.
Extend scoring with multiple LLM providers — The LLM client’s getDefaultModel() can read an env var to switch between OpenAI, Anthropic, or Google providers via the Vercel AI SDK.
Build a candidate comparison UI — Add a page that shows two candidates side by side with their per-criterion score breakdowns, rationales, and original resume text.
import
{ getCachedOrGenerate }
from
"../lib/cache"
;
import { InMemoryStore } from "../lib/storage";
import { type Rubric, type RubricCriterion } from "../schemas/rubric";
import { type ScoreResult, ScoreResultSchema, type CriterionScore } from "../schemas/score-result";
const system = `You are an expert resume scorer. Score the candidate's resume against the rubric criteria for "${rubric.roleName}". Return ONLY valid JSON with no markdown formatting.`;
const user = `Rubric: ${rubric.name}
Role: ${rubric.roleName}
Criteria:
${criteriaText}
Resume text:
${resumeText}
Score each criterion and provide:
- criterionId, criterionName, score, maxScore, rationale for each criterion
- totalScore (sum of weighted scores)
- maxScore (sum of max scores)
- percentage (0-100)
- explanation of overall assessment
Respond with a JSON object matching this structure:
