Files · Anthropic Receipt-to-Expense Automation for SMB Bookkeeping

# Anthropic Receipt-to-Expense Automation for SMB Bookkeeping
 
> Extract line items from receipts and PDF invoices using Anthropic, repair malformed outputs, and log costs – all within a simple Express API an SMB can deploy in minutes.
 
A tutorialized reference solution from [reaatech.com](https://reaatech.com), demonstrating how to build production-grade AI systems with the `@reaatech/*` package family.
 
## Problem
 
Small businesses waste hours manually typing receipt data into accounting software. Off-the-shelf OCR misses varied layouts and handwritten notes, and poorly formatted LLM outputs break downstream automation. This recipe solves all three with a single upload-and-extract API.
 
## Architecture
 
Next.js API routes receive file uploads, chain document extraction (unpdf/tesseract.js/sharp) → Anthropic Claude vision → structured-repair-core (JSON repair) → session-continuity-storage-memory (session state) → llm-cost-telemetry (cost tracking), turning messy receipts into structured expense records.
 
### Package roles
 
| Package | Role |
|---|---|
| `@reaatech/media-pipeline-mcp-doc-extraction` | Foundation — document OCR + field extraction via LLM providers |
| `@reaatech/structured-repair-core` | JSON repair with 6 graduated strategies (strip-fences, fix-syntax, coerce-types, fuzzy-match, remove-extra, extract-json) |
| `@reaatech/session-continuity-storage-memory` | In-memory session adapter with TTL for multi-page upload state |
| `@reaatech/llm-cost-telemetry` | Cost span tracking, budget config, and token-usage calculation |
 
## Getting started
 
```bash
pnpm install
cp .env.example .env
# Edit .env with your ANTHROPIC_API_KEY
pnpm dev
```
 
## API Reference
 
### POST /api/extract
 
Upload a receipt or invoice image/PDF.
 
- **Method:** POST
- **Content-Type:** multipart/form-data
- **Fields:**
  - `receipt` (File, required) — Receipt or invoice image (JPEG/PNG) or PDF
  - `sessionId` (string, optional) — Existing session ID for multi-page uploads
- **Success (200):** `{ record: ExpenseRecord, cost: CostSpan, repaired: boolean, sessionId: string, jobId: string }`
- **Errors:** 400 (no file / too large), 500 (pipeline failure)
 
### GET /api/sessions
 
List extraction sessions for a user.
 
- **Method:** GET
- **Query params:** `userId` (string, required)
- **Success (200):** `{ sessions: SessionSummary[] }`
 
## Pipeline flow
 
1. File uploaded → detect file type (magic bytes)
2. Document processed: PDF → unpdf text extraction; image → sharp preprocessing + tesseract.js OCR
3. Anthropic Claude vision (`claude-sonnet-4-6`) extracts structured expense data
4. Cost recorded via llm-cost-telemetry (`CostSpan` with `calculateCostFromTokens`)
5. LLM output repaired via structured-repair-core (6 strategies)
6. Extraction event stored in session-continuity MemoryAdapter
7. Typed `ExpenseRecord` returned with cost and repair metadata
 
## Testing
 
```bash
pnpm test
```
 
## Project layout
 
```
app/api/extract/route.ts    POST /api/extract
app/api/sessions/route.ts   GET /api/sessions
src/types/                  Shared expense + session types
src/lib/                    Package adapters (Anthropic, repair, session, cost, observability, Instructor)
src/middleware/              Multer upload configuration
src/services/pipeline.ts    Extraction pipeline orchestrator
tests/                      Vitest suite (MSW-mocked, per-module)
DEV_PLAN.md                 Build checklist
packages/                   API references for every dependency
```
 
## License
 
MIT — see [LICENSE](./LICENSE).