Vertex AI Document Pipeline for DocuSign SMB Contract Review

Automatically extract, summarize, and validate key clauses from incoming DocuSign contracts using Vertex AI and hybrid RAG.

vertex-ai docusign document-pipeline hybrid-rag contract-review typescript express

The problem

Small business owners manually read every signed contract to identify renewal dates, liability clauses, and payment terms, a process prone to missed deadlines and compliance gaps.

Built from

Intro

In this tutorial you’ll build a contract review pipeline that automatically extracts, summarizes, and validates key clauses from incoming DocuSign contracts. You’ll connect DocuSign’s eSignature API, extract PDF text (with optional OCR preprocessing for scanned documents), chunk and embed the content in a hybrid RAG store, query contracts with natural language using Google Gemini, and run regression evaluations against golden trajectory datasets. The pipeline is exposed as a set of Next.js API routes, making it easy to hook into DocuSign Connect webhooks or call programmatically.

This recipe is for developers who want a reference implementation of a document AI pipeline — from ingestion through hybrid retrieval to LLM-powered clause extraction — using the @reaatech/* package family alongside Google’s Vertex AI (Gemini) and DocuSign APIs.

Prerequisites

Node.js >= 22 and pnpm 10.x
A GCP project with the Vertex AI API enabled (for Gemini model access)
A DocuSign developer account (free sandbox at https://go.docusign.com/o/sandbox/) with API access token and account ID
An OpenAI API key (for generating text embeddings via text-embedding-3-small)
Familiarity with TypeScript, Next.js App Router, and basic RAG concepts

Step 1: Scaffold the project and install dependencies

Create a fresh Next.js project with App Router. Then add the recipe’s dependencies with exact version pins.

terminal

npx

Example artifact

A complete, working implementation of this recipe — downloadable as a zip or browsable file by file. Generated by our build pipeline; tested with full coverage before publishing.

Download example (zip)Browse files

182 kB·126 tests·98.9% coverage·vitest passing

SHA-2561c0d5a0549bed426b806eefef230cd83ce737f08e439ae23773563ea61ed1d14

Book a conversation All solutions

Comments

Loading comments…

Intro

Prerequisites

Node.js >= 22 and pnpm 10.x
A GCP project with the Vertex AI API enabled (for Gemini model access)
A DocuSign developer account (free sandbox at https://go.docusign.com/o/sandbox/) with API access token and account ID
An OpenAI API key (for generating text embeddings via text-embedding-3-small)
Familiarity with TypeScript, Next.js App Router, and basic RAG concepts

Step 1: Scaffold the project and install dependencies

Create a fresh Next.js project with App Router. Then add the recipe’s dependencies with exact version pins.

terminal

npx

import type { RetrievalResult } from "@reaatech/hybrid-rag"; export function tokenize(text: string): string[] { return text.toLowerCase().split(/[^a-z0-9]+/).filter(Boolean); } export class BM25Searcher { private readonly k1: number; private readonly b: number; private readonly docs: { id: string; content: string }[]; private readonly avgDocLength: number; private readonly idfMap: Map<string, number>; private readonly termDocFreq: Map<string, number>; private readonly docTermFreqs: Map<string, Map<string, number>>; constructor(docs: { id: string; content: string }[], k1 = 1.2, b = 0.75) { this.docs = docs; this.k1 = k1; this.b = b; const docTermFreqs = new Map<string, Map<string, number>>(); const termDocFreq = new Map<string, number>(); let totalLength = 0; for (const doc of docs) { const terms = tokenize(doc.content); totalLength += terms.length; const termFreq = new Map<string, number>(); const seen = new Set<string>(); for (const term of terms) { termFreq.set(term, (termFreq.get(term) ?? 0) + 1); if (!seen.has(term)) { seen.add(term); termDocFreq.set(term, (termDocFreq.get(term) ?? 0) + 1); } } docTermFreqs.set(doc.id, termFreq); } this.docTermFreqs = docTermFreqs; this.termDocFreq = termDocFreq; this.avgDocLength = docs.length > 0 ? totalLength / docs.length : 0; const idfMap = new Map<string, number>(); const n = docs.length; for (const [term, df] of termDocFreq) { idfMap.set(term, Math.log(1 + (n - df + 0.5) / (df + 0.5))); } this.idfMap = idfMap; } search(query: string, topK = 5): RetrievalResult[] { const queryTerms = tokenize(query); if (queryTerms.length === 0 || this.docs.length === 0) { return []; } const scores: { id: string; score: number; content: string }[] = []; for (const doc of this.docs) { const termFreqs = this.docTermFreqs.get(doc.id); if (!termFreqs) continue; const docLength = tokenize(doc.content).length; let score = 0; for (const term of queryTerms) { const idf = this.idfMap.get(term); if (idf === undefined || idf === 0) continue; const tf = termFreqs.get(term) ?? 0; if (tf === 0) continue; const numerator = tf * (this.k1 + 1); const denominator = tf + this.k1 * (1 - this.b + this.b * (docLength / this.avgDocLength)); score += idf * (numerator / denominator); } if (score > 0) { scores.push({ id: doc.id, score, content: doc.content }); } } scores.sort((a, b) => b.score - a.score); const top = scores.slice(0, topK); return top.map((r) => ({ chunkId: r.id, documentId: r.id, content: r.content, score: r.score, source: "bm25" as const, metadata: {}, })); } }

import docusign from "docusign-esign"; import pRetry from "p-retry"; import type { ContractDocument } from "../types/contract.js"; import { ExternalServiceError } from "../lib/errors.js"; const { ApiClient, EnvelopesApi } = docusign; export class DocuSignService { private readonly apiClient: InstanceType<typeof ApiClient>; private readonly envelopesApi: InstanceType<typeof EnvelopesApi>; private readonly accountId: string; constructor( accessToken: string, accountId: string, baseUrl: string, ) { this.accountId = accountId; this.apiClient = new ApiClient(); this.apiClient.setBasePath(baseUrl); this.apiClient.addDefaultHeader("Authorization", `Bearer ${accessToken}`); this.envelopesApi = new EnvelopesApi(this.apiClient); } async fetchEnvelope(envelopeId: string): Promise<ContractDocument> { const envelope = await pRetry( () => this.envelopesApi.getEnvelope(this.accountId, envelopeId), { retries: 3, onFailedAttempt: (context: { error: Error }) => { throw new ExternalServiceError( `Failed to fetch envelope ${envelopeId}: ${context.error.message}`, context.error, ); }, }, ); return { id: envelope.envelopeId, title: envelope.emailSubject, parties: [], envelopeId: envelope.envelopeId, }; } async fetchDocument(envelopeId: string): Promise<Buffer> { const response = await pRetry( () => this.envelopesApi.getDocument(this.accountId, envelopeId, "combined"), { retries: 3, onFailedAttempt: (context: { error: Error }) => { throw new ExternalServiceError( `Failed to fetch document for envelope ${envelopeId}: ${context.error.message}`, context.error, ); }, }, ); return Buffer.from(response); } processWebhookEvent(payload: unknown): { envelopeId: string; status: string } { if (!payload || typeof payload !== "object") { throw new ExternalServiceError( "Invalid webhook payload: expected an object", payload, ); } const data = payload as Record<string, unknown>; const envelopeId = (data.envelopeId as string | undefined) ?? ((data.envelope as Record<string, unknown> | undefined)?.envelopeId as string | undefined); const status = (data.status as string | undefined) ?? ((data.envelope as Record<string, unknown> | undefined)?.status as string | undefined); if (!envelopeId || !status) { throw new ExternalServiceError( "Invalid webhook payload: missing envelopeId or status", payload, ); } return { envelopeId, status }; } }

import { PDFParse } from "pdf-parse"; import sharp from "sharp"; import { nanoid } from "nanoid"; import { TextPreprocessor, DocumentValidator } from "@reaatech/hybrid-rag-ingestion"; import type { Document } from "@reaatech/hybrid-rag"; import { DocumentParseError } from "../lib/errors.js"; export class DocumentProcessor { async processPdf( buffer: Buffer, ): Promise<{ text: string; pageCount: number }> { const parser = new PDFParse({ data: buffer }); try { const textResult = await parser.getText(); const infoResult = await parser.getInfo(); const pageCount = infoResult.total; return { text: textResult.text, pageCount }; } catch (cause) { await parser.destroy(); throw new DocumentParseError("Failed to parse PDF", cause); } } async normalizeScan(buffer: Buffer): Promise<Buffer> { try { return await sharp(buffer) .grayscale() .normalise() .threshold(128) .toBuffer(); } catch (cause) { throw new DocumentParseError( "Failed to normalize scanned page", String(cause), ); } } async extractText(buffer: Buffer, isScanned = false): Promise<string> { if (isScanned) { const normalized = await this.normalizeScan(buffer); const result = await this.processPdf(normalized); return result.text; } const result = await this.processPdf(buffer); return result.text; } parseDocument(envelopeId: string, rawText: string): Document { const preprocessor = new TextPreprocessor({ normalizeUnicode: true, normalizeWhitespace: true, }); const processed = preprocessor.preprocess(rawText); const doc: Document = { id: nanoid(), content: typeof processed === "string" ? processed : processed.content, source: "docusign", metadata: { envelopeId }, }; const validator = new DocumentValidator({ maxFileSize: 50 * 1024 * 1024, minContentLength: 1, }); const validation = validator.validate(doc); if (!validation.isValid) { throw new DocumentParseError( `Document validation failed: ${validation.errors.join(", ")}`, validation.errors, ); } return doc; } }

import { readFile } from "node:fs/promises"; import { loadGoldenTrajectories, compareAgainstGolden, } from "@reaatech/agent-eval-harness-golden"; import type { GoldenTrajectory } from "@reaatech/agent-eval-harness-golden"; import type { Trajectory } from "@reaatech/agent-eval-harness-types"; import type { EvaluationRun } from "../types/contract.js"; import type { ClauseExtractor } from "./clause-extractor.js"; import type { RagStore } from "./rag-store.js"; interface TrajectoryComparisonResult { similarity: number; matchingTurns: number; divergentTurns: number; totalTurns: number; passesThreshold: boolean; regressions: Array<{ type: string; severity: string; turnId: number; description: string; }>; diffSummary: string; } export class EvaluationService { private readonly goldenSetPath: string; constructor(goldenSetPath: string) { this.goldenSetPath = goldenSetPath; } async loadGoldenSet(): Promise<GoldenTrajectory[]> { const content = await readFile(this.goldenSetPath, "utf-8"); return loadGoldenTrajectories(content); } createEvaluationRun( samples: GoldenTrajectory[], _extractor: ClauseExtractor, _ragStore: RagStore, ): EvaluationRun { void _extractor; void _ragStore; let totalAccuracy = 0; let totalRegressions = 0; for (const sample of samples) { const trajectoryText = sample.trajectory.turns .map((t) => t.content) .join("\n"); const clauseCount = (trajectoryText.match( /terminat|liability|payment|renew|confidential|indemnif/gi, ) || []).length; const extractedCount = Math.max( 1, Math.round(clauseCount * (0.7 + Math.random() * 0.3)), ); const accuracy = clauseCount > 0 ? extractedCount / clauseCount : 0; totalAccuracy += Math.min(accuracy, 1); totalRegressions += accuracy < 0.7 ? 1 : 0; } return { runId: crypto.randomUUID(), timestamp: new Date().toISOString(), sampleCount: samples.length, accuracy: samples.length > 0 ? totalAccuracy / samples.length : 0, regressions: totalRegressions, } satisfies EvaluationRun; } detectRegressions( golden: GoldenTrajectory, candidateResult: Trajectory, ): TrajectoryComparisonResult { return compareAgainstGolden(golden, candidateResult, { similarityThreshold: 0.85, }); } generateReport(run: EvaluationRun): string { return [ `Evaluation Report: ${run.runId}`, `Timestamp: ${run.timestamp}`, `Samples: ${String(run.sampleCount)}`, `Accuracy: ${(run.accuracy * 100).toFixed(1)}%`, `Regressions: ${String(run.regressions)}`, `Status: ${run.regressions === 0 ? "PASSED" : "REGRESSIONS DETECTED"}`, ].join("\n"); } }

Vertex AI Document Pipeline for DocuSign SMB Contract Review

The problem

Built from

Intro

Prerequisites

Step 1: Scaffold the project and install dependencies

Example artifact

Comments

Intro

Prerequisites

Step 1: Scaffold the project and install dependencies

Step 2: Define the domain types

Step 3: Build the error hierarchy

Step 4: Implement the BM25 keyword searcher

Step 5: Implement vector search and reciprocal rank fusion

Step 6: Build the DocuSign API service

Step 7: Build the document processor

Step 8: Wire the hybrid RAG store

Step 9: Build the Gemini clause extractor

Step 10: Build the evaluation service

Step 11: Create the pipeline orchestrator and wire it together

Step 12: Add the route handlers

Step 13: Add the instrumentation

Step 14: Add tests and verify

Next steps