SMB construction project managers spend hours manually entering payment application data from PDFs into accounting software, leading to errors, delayed payments, and cash flow issues.
A complete, working implementation of this recipe — downloadable as a zip or browsable file by file. Generated by our build pipeline; tested with full coverage before publishing.
Construction project managers at small and medium businesses spend hours manually keying payment application data from contractor PDFs into accounting software. This recipe builds a Next.js app that automates that process: you upload a PDF, it extracts line items, retainage, and change orders using Vertex AI’s Gemini model, repairs noisy OCR output with structured repair, flags low-confidence fields for human review, and tracks cost per extraction — all through a single POST endpoint.
Prerequisites
Node.js 22+ and pnpm 10 installed on your machine
A Google Cloud Project with the Vertex AI API enabled
A service account key (JSON) for Vertex AI authentication; set GOOGLE_APPLICATION_CREDENTIALS to its path
Basic familiarity with TypeScript and Next.js App Router conventions
Step 1: Scaffold the project and install dependencies
Create a Next.js 16 project with the App Router. The scaffold gives you package.json, tsconfig.json, next.config.ts, and the app/ directory. After scaffolding, add all the dependencies.
Expected output:pnpm install finishes without errors. pnpm typecheck runs cleanly on the scaffold.
Step 2: Define the payment application data schema
Create src/services/payment-app-schema.ts. This file defines the shape of a contractor payment application with Zod — line items, change orders, retainage, and status tracking. Every field gets runtime validation automatically.
The retainagePercentage constraint .min(0).max(100) catches impossible values like 150 at parse time. Later, the repair pipeline feeds this schema into @reaatech/structured-repair-core to coerce raw LLM output into valid data.
Expected output: Your editor shows no type errors. The types PaymentApplication, LineItem, and ChangeOrder are available for import.
Step 3: Build the PDF text extractor
Create src/lib/pdf-extractor.ts. This wraps pdfjs-dist to turn a PDF buffer into plain text. pdfjs-dist v6 uses an ESM entry at pdfjs-dist/legacy/build/pdf.mjs.
ts
import { getDocument } from "pdfjs-dist/legacy/build/pdf.mjs";export function extractItemText(item: { str?: string }): string { return item.str ?? "";}export async function extractTextFromPdf(buffer: Uint8Array): Promise<string> { if (buffer.length === 0) { throw new Error("PDF extraction failed: empty buffer"); } try { const pdf = await getDocument({ data: buffer }).promise; const pageTexts: string[] = []; for (let i = 1; i <= pdf.numPages; i++) { const page = await pdf.getPage(i); const content = await page.getTextContent(); const text = content.items.map((item) => extractItemText(item as { str?: string })).join(" "); pageTexts.push(text); } return pageTexts.join("\n"); } catch (cause) { throw new Error(`PDF extraction failed: ${String(cause)}`); }}
Key behaviors:
Empty buffers throw immediately with a clear error message.
Each page’s text items are joined with spaces; pages are separated by newlines.
A corrupt PDF produces a thrown error, never a silent empty string.
Expected output: The module exports extractTextFromPdf and extractItemText, ready for import by the extraction pipeline.
Step 4: Create the Vertex AI provider
Create src/lib/vertex-provider.ts. This class wraps @google-cloud/vertexai and conforms to the MediaProvider interface from @reaatech/media-pipeline-mcp-provider-core. It exposes execute(), extractFields(), and estimateCost() methods.
ts
import { VertexAI } from "@google-cloud/vertexai";import { MediaProvider, type ProviderInput, type ProviderOutput, type CostEstimate } from "@reaatech/media-pipeline-mcp-provider-core";export class VertexAIProvider extends MediaProvider { readonly name = "vertex"; readonly supportedOperations = ["ocr", "extract_fields", "summarize"]; protected retryConfig = { maxRetries: 3, baseDelay: 1000, maxDelay: 10000 }; protected cacheStore = new Map(); private vertexAI: VertexAI;
The provider tracks input and output token counts from Gemini’s usageMetadata — you’ll use those later to record per-extraction cost. The extractFields method builds a prompt from the field schema, sends it to Gemini, and returns metadata about which fields were extracted.
Step 5: Implement storage adapters for session continuity
Create src/lib/storage-adapters.ts. The extraction pipeline uses @reaatech/session-continuity to track multi-document project sessions. The IStorageAdapter interface requires about a dozen methods. This in-memory implementation uses Map objects and satisfies every method of the contract.
ts
import { type Session, type SessionId, type MessageId, type Message, type IStorageAdapter } from "@reaatech/session-continuity";type SessionData = Omit<Session, "id" | "createdAt" | "lastActivityAt">;function now(): Date { return new Date();}export class MemorySessionStore implements IStorageAdapter { private sessions = new Map<string, Session>(); private messages = new Map
Expected output: The file compiles. Each method is backed by a Map and returns immediately with a resolved promise.
Step 6: Create the token counter and session manager
Create src/lib/simple-token-counter.ts. This provides a rough token count estimate (4 characters per token) for the session manager’s token budget tracking.
ts
import { type Message } from "@reaatech/session-continuity";export class SimpleTokenCounter { readonly model = "gemini-2.5-flash"; readonly tokenizer = "simple-char-ratio"; count(text: string): number { return Math.ceil(text.length / 4); } countMessages(messages: Message[]): number { return messages.reduce((sum, m) => { if (typeof m.content === "string") { return sum + this.count(m.content); } return sum + this.count(JSON.stringify(m.content)); }, 0); }}
Create src/lib/session-manager.ts. This wires the MemorySessionStore and SimpleTokenCounter into the SessionManager from @reaatech/session-continuity, configured with a token budget and sliding-window compression.
ts
import { SessionManager } from "@reaatech/session-continuity";import { MemorySessionStore } from "./storage-adapters.js";import { SimpleTokenCounter } from "./simple-token-counter.js";export function createSessionManager() { return new SessionManager({ storage: new MemorySessionStore(), tokenCounter: new SimpleTokenCounter(), tokenBudget: { maxTokens: 8192, reserveTokens: 500, overflowStrategy: "compress", }, compression: { strategy: "sliding_window", targetTokens: 7000, }, });}
Expected output:createSessionManager() returns a SessionManager instance ready to create and manage extraction sessions.
Step 7: Build the cost tracker
Create src/lib/cost-tracker.ts. This uses @reaatech/llm-cost-telemetry to record cost spans for each Gemini model call, estimating cost at $15 per million tokens.
Expected output: Calling recordCall("gemini-2.5-flash", 500, 200) returns a CostSpan with a populated id, costUsd greater than 0, and timestamp. getTotalCost() returns the sum of all recorded spans.
Step 8: Set up the confidence router
Create src/services/confidence-checker.ts. The @reaatech/confidence-router package evaluates per-field confidence scores and decides whether a field should be routed automatically, flagged for human clarification, or sent to a fallback handler. Fields below the route threshold (85%) get marked for review.
Expected output: A field at 0.95 gets ROUTE; a field at 0.25 gets FALLBACK and is added to lowConfidenceFields. needsReview is true when any field falls below the route threshold.
Step 9: Wire the extraction pipeline
Create src/services/extraction-service.ts. This is the orchestrator — it connects every module you’ve built so far into a single processPaymentApp function.
ts
import { repairOutput } from "@reaatech/structured-repair-core";import { createDocumentExtractionOperations } from "@reaatech/media-pipeline-mcp-doc-extraction";import { ArtifactRegistry } from "@reaatech/media-pipeline-mcp-core";import { LocalStorage } from "@reaatech/media-pipeline-mcp-storage";import { PaymentApplicationSchema } from "./payment-app-schema.js";import { extractTextFromPdf } from "../lib/pdf-extractor.js";import { createVertexProvider } from "../lib/vertex-provider.js";import { createCostTracker } from "../lib/cost-tracker.js";import { createSessionManager } from "../lib/session-manager.js";import { createPaymentAppConfidenceRouter, evaluateFieldConfidence } from "./confidence-checker.js";
The pipeline follows seven stages: PDF extraction, artifact registration, field extraction via Gemini, schema repair, confidence evaluation, cost recording, and session persistence.
Expected output:createExtractionPipeline({ projectId, location }) returns an object with processPaymentApp(buffer) that produces { paymentApplication, confidence, routingDecision, costSpan, session }.
Step 10: Create the API route and upload form
Create app/api/upload/route.ts. This is the Next.js App Router route handler that accepts PDF uploads, runs the pipeline, and returns results. Use NextRequest/NextResponse (never bare Request/Response).
The route includes a health-check GET handler. When the extraction result contains low-confidence fields and REVIEW_QUEUE_URL is set, it fire-and-forgets a POST to that webhook (with a 5-second timeout) so a human reviewer can inspect the document.
Replace the scaffolded app/page.tsx with an upload form:
Create tests/api/upload/route.test.ts to test the route handler with mocked pipeline:
ts
import { describe, it, expect, vi, beforeEach, beforeAll, afterAll } from "vitest";import { NextRequest } from "next/server";import { POST, GET } from "../../../app/api/upload/route.js";import * as extractionService from "../../../src/services/extraction-service.js";beforeAll(() => { globalThis.fetch = vi.fn().mockResolvedValue(new Response("ok"));});afterAll(() => { delete (globalThis as { fetch?: unknown }).fetch;});vi.mock("../../../src/services/extraction-service.js"
Now run the tests:
terminal
pnpm vitest run --coverage
Expected output: All tests pass. numFailedTests: 0. Coverage for lines, branches, functions, and statements is each 90% or higher (matching the thresholds in vitest.config.ts).
Next steps
Replace in-memory storage with a database — swap MemorySessionStore and LocalStorage for Postgres or Redis adapters to persist extractions and sessions across server restarts.
Add batch upload — extend the route to accept multiple PDFs in a single request, processing each through the pipeline concurrently.
Integrate with accounting software — wire the extracted PaymentApplication into QuickBooks, Xero, or Procore via their APIs for one-click data entry.
