Files · Automated Resume Rubric Scorer for Boutique Recruiters

# Automated Resume Rubric Scorer for Boutique Recruiters
 
> Score 200 resumes per role against custom rubrics in minutes, not hours.
 
A solo recruiter at a 5-person agency spends 10+ hours per role manually scoring resumes against client rubrics. With 50–200 resumes per role and 40 active roles, this becomes unsustainable. This recipe provides a consistent, auditable scoring system to surface top candidates quickly without hiring more staff.
 
## How it works
 
1. **Create a rubric** — Define scoring criteria with weights and max scores for a specific role.
2. **Upload resumes** — Upload PDF or DOCX resumes via the API or dashboard.
3. **Score automatically** — Each resume is scored against the rubric using an LLM (OpenAI GPT via Vercel AI SDK).
4. **Rank candidates** — View candidates sorted by their best score across multiple scoring runs.
5. **Review and iterate** — Score results include per-criterion scores, rationales, and cost tracking.
 
## Architecture
 
```
app/page.tsx          Dashboard UI (server component)
app/api/
  rubrics/            CRUD for rubrics
  resumes/            Resume upload and listing
  resumes/[id]/score/ Trigger and list scoring results
  candidates/         Candidate management and ranking
  candidates/[id]/    Candidate detail with score results
  score-results/[id]/ Score result detail
  health/             Health check endpoint
src/
  schemas/            Zod schemas for Rubric, Resume, Candidate, ScoreResult
  lib/                InMemoryStore, LLM client (Vercel AI SDK), LLM cache engine
  services/           Business logic (rubric, resume-parser, scorer, candidate, eval, scoring-queue)
  instrumentation.ts  Next.js instrumentation hook initializing observability
tests/                Vitest suite with 100+ tests
```
 
### Key packages
 
| Package | Purpose |
|---------|---------|
| `next@16` | App Router framework |
| `ai` + `@ai-sdk/openai` | Vercel AI SDK with OpenAI provider |
| `zod` | Runtime schema validation |
| `unpdf` | PDF text extraction |
| `office-text-extractor` | DOCX/XLSX/PPTX text extraction |
| `bullmq` | Redis-backed async scoring queue |
| `@reaatech/agent-eval-harness-golden` | Golden trajectory management for scoring consistency |
| `@reaatech/agent-eval-harness-tool-use` | Tool call validation for LLM scoring |
| `@reaatech/agent-eval-harness-cost` | Per-trajectory LLM cost tracking and budget enforcement |
| `@reaatech/agent-eval-harness-observability` | OTel tracing, metrics, structured logging, and dashboards |
| `@reaatech/llm-cache` | Semantic + exact-match LLM response caching |
| `@reaatech/llm-cache-adapters-redis` | Redis storage adapter for the cache |
 
## Setup
 
```bash
pnpm install
cp .env.example .env    # edit with your API keys
pnpm dev                # start Next.js dev server
pnpm test               # run vitest with coverage
pnpm typecheck          # TypeScript type checking
pnpm lint               # ESLint
```
 
## API Reference
 
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/rubrics` | List all rubrics |
| `POST` | `/api/rubrics` | Create a new rubric |
| `GET` | `/api/rubrics/:id` | Get rubric by ID |
| `PUT` | `/api/rubrics/:id` | Update rubric |
| `DELETE` | `/api/rubrics/:id` | Delete rubric |
| `GET` | `/api/resumes` | List all resumes |
| `POST` | `/api/resumes` | Upload a resume (multipart form) |
| `POST` | `/api/resumes/:id/score` | Enqueue scoring job for a resume |
| `GET` | `/api/resumes/:id/score` | List score results for a resume |
| `GET` | `/api/candidates` | List candidates sorted by best score |
| `POST` | `/api/candidates` | Create a candidate |
| `GET` | `/api/candidates/:id` | Get candidate with score result |
| `GET` | `/api/score-results/:id` | Get score result with criterion breakdown |
| `GET` | `/api/health` | Health check |
 
## Usage
 
1. Create a rubric: `POST /api/rubrics` with `{ name, roleName, criteria: [{ name, description, weight, maxScore }] }`.
2. Upload resumes: `POST /api/resumes` with multipart form data (field `file`).
3. Trigger scoring: `POST /api/resumes/:id/score` with `{ "rubricId": "..." }`.
4. View results: `GET /api/score-results/:id` for a detailed per-criterion breakdown.
5. Rank candidates: `GET /api/candidates` returns candidates sorted by `bestPercentage` descending.
 
## Configuration
 
| Env var | Default | Description |
|---------|---------|-------------|
| `OPENAI_API_KEY` | — | OpenAI API key (required) |
| `REDIS_URL` | `redis://localhost:6379` | Redis connection URL |
| `LLM_MODEL` | `gpt-5.2` | OpenAI model for scoring |
| `CACHE_SIMILARITY_THRESHOLD` | `0.85` | Cosine similarity threshold for cache |
| `CACHE_DEFAULT_TTL` | `3600` | Cache TTL in seconds |
| `BUDGET_PRESET` | `moderate` | Budget preset (strict/moderate/lenient) |
 
## Testing
 
```bash
pnpm test        # 104 tests, 90%+ coverage on all metrics
pnpm typecheck
pnpm lint
```
 
## License
 
MIT — see [LICENSE](./LICENSE).