A complete, working implementation of this recipe — downloadable as a zip or browsable file by file. Generated by our build pipeline; tested with full coverage before publishing.
This tutorial builds a Perplexity-powered competitive intelligence system for small e-commerce businesses. You’ll set up a Next.js app with a cron-scheduled agent that scrapes competitor websites via Browserbase, researches pricing and product changes via Perplexity’s online LLM, stores findings with embeddings in ChromaDB, and posts daily briefs to Slack. The system uses agent-memory for retrieval, a confidence router to prioritize competitors, and structured-repair-core to fix unreliable LLM JSON output.
Expected output: A package.json with all exact-pinned dependencies and a working pnpm-lock.yaml. The scripts block should include dev, build, typecheck, lint, and test.
Step 2: Configure environment variables and Next.js settings
Create your .env file with all the API keys the system will use. The COMPETITOR_LIST is a JSON array that defines the competitors to track.
env
# Env vars used by perplexity-competitive-intelligence-for-e-commerce-smbs.# The builder adds entries here as it wires up each integration.# Keep placeholders only — never commit real values.NODE_ENV=developmentPERPLEXITY_API_KEY=<your-perplexity-api-key>BROWSERBASE_API_KEY=<your-browserbase-api-key>BROWSERBASE_PROJECT_ID=<your-browserbase-project-id>OPENAI_API_KEY=<your-openai-api-key>SLACK_WEBHOOK_URL=https://hooks.slack.com/services/<your-webhook-path>LANGFUSE_PUBLIC_KEY=<your-langfuse-public-key>LANGFUSE_SECRET_KEY=<your-langfuse-secret-key>CRON_SCHEDULE=0 8 * * *COMPETITOR_LIST=[{"id":"c1","name":"Example Store","websiteUrl":"https://example.com","priority":1}]
Enable the Next.js instrumentation hook so the cron scheduler starts at boot.
Expected output:.env.example lists every env var with placeholder values, and next.config.ts has experimental.instrumentationHook: true (exact spelling — this is required for src/instrumentation.ts to work).
Step 3: Define the shared data schemas with Zod
Create the TypeScript types and Zod validation schemas that the entire system uses. These define what a competitor, a change event, an analysis result, and a daily brief look like.
Expected output: A src/lib/schemas.ts file that exports five Zod schemas and their inferred TypeScript types. All other modules import these types.
Step 4: Create the configuration loader
Build a config module that reads environment variables and the COMPETITOR_LIST JSON, returning a typed Config object. This keeps env-var access centralized in one file.
Expected output:src/lib/config.ts exports loadConfig() and getTrackedCompetitors(). When COMPETITOR_LIST env var is missing or invalid, three example competitors are returned as defaults.
Step 5: Build the Perplexity research client
Create a module that wraps the perplexity-sdk. The researchCompetitor function builds a prompt that asks Perplexity to analyze a competitor’s page and return structured JSON. The raw response is then piped through the repair module to fix any JSON issues.
Expected output:src/lib/perplexity.ts exports createPerplexityClient(), PerplexityApiError, and researchCompetitor(). The researchCompetitor function uses the pplx-70b-online model and passes the LLM’s raw output through a repair function before returning.
Step 6: Build the Browserbase scraper
Create a scraper module that uses Browserbase to fetch competitor pages as markdown. This gives you clean text content to feed into Perplexity’s analysis.
Expected output:src/lib/scraper.ts exports scrapeCompetitorPage() and BrowserbaseApiError. The function creates a Browserbase session, fetches the URL as markdown, then releases the session in the finally block.
Step 7: Create the embedding and vector store modules
This step creates two modules: one that wraps fastembed for local embedding generation, and one that wraps chromadb for storing and querying change-event embeddings.
ts
// src/lib/embedding.tsimport { EmbeddingModel, FlagEmbedding } from "fastembed";let model: FlagEmbedding | null = null;export async function getEmbeddingModel(): Promise<FlagEmbedding> { if (!model) { model = await FlagEmbedding.init({ model: EmbeddingModel.BGEBaseEN }); } return model;}export async function embedQuery(text: string): Promise<number[]> { const m = await getEmbeddingModel(); return m.queryEmbed(text);}export async function* embedDocuments(texts: string[], batchSize: number = 256): AsyncGenerator<number[][]> { const m = await getEmbeddingModel(); for await (const batch of m.passageEmbed(texts, batchSize)) { yield batch; }}
Expected output:src/lib/embedding.ts initializes a singleton FlagEmbedding model using BGEBaseEN and exports getEmbeddingModel(), embedQuery(), and embedDocuments(). src/lib/chromadb-store.ts exports createChromaClient(), ensureCollection(), storeCompetitorEmbedding(), and querySimilarChanges().
Step 8: Build the memory and confidence router modules
The memory module wraps @reaatech/agent-memory with an OpenAI embedding provider to store and retrieve past competitor findings. The router module uses @reaatech/confidence-router to prioritize which competitors to investigate next.
Expected output:src/lib/memory.ts creates an AgentMemory instance with in-memory storage, OpenAI embeddings, and fact extraction. src/lib/router.ts creates a ConfidenceRouter and uses its decideBatch() method to score and sort competitors by priority and context relevance.
Step 9: Create the LLM output repair module
Perplexity sometimes returns JSON wrapped in markdown fences, with trailing commas, or with string values where numbers are expected. This module uses @reaatech/structured-repair-core to fix these issues automatically.
ts
// src/lib/repair.tsimport { repair, repairOutput, UnrepairableError } from "@reaatech/structured-repair-core";import { CompetitorAnalysisSchema } from "./schemas.js";interface RepairOutputWithFieldErrors { data: unknown; errors: unknown[]; fieldErrors?: unknown[];}export class RepairFailedError extends Error { constructor(message: string, cause?: unknown) { super(message); this.cause = cause; } cause?: unknown;}export async function repairAnalysis(raw: string): Promise<import("./schemas.js").CompetitorAnalysis> { try { const data = await repair(CompetitorAnalysisSchema, raw); return data; } catch (error) { if (error instanceof UnrepairableError) { throw new RepairFailedError("Failed to repair LLM output", error); } throw error; }}export function safeRepairAnalysis(raw: string): { data: import("./schemas.js").CompetitorAnalysis | null; errors: unknown[]; fieldErrors: unknown[];} { const result = repairOutput({ schema: CompetitorAnalysisSchema, input: raw, debug: true, }); const r = result as RepairOutputWithFieldErrors; return { data: r.data as import("./schemas.js").CompetitorAnalysis | null, errors: r.errors, fieldErrors: r.fieldErrors ?? [], };}
Expected output:src/lib/repair.ts exports repairAnalysis() (throws on failure) and safeRepairAnalysis() (returns errors + fieldErrors instead of throwing). Both use the CompetitorAnalysisSchema to validate and repair the LLM’s raw JSON output.
Step 10: Build the observability and Slack notification modules
The observability module wraps Langfuse for distributed tracing. The Slack module formats daily briefs as Slack Block Kit messages and posts them via webhook.
Expected output:src/lib/observability.ts exports initObservability() (gracefully returns null when keys are missing) and withTrace() (wraps any async function with Langfuse tracing). src/lib/slack.ts exports postDailyBriefToSlack() which formats the brief as Slack Block Kit and POSTs it to the webhook.
Step 11: Build the analysis orchestrator
The analyzer module ties together Perplexity research, memory retrieval, and structured repair. It retrieves prior findings for context, calls Perplexity, repairs the output, and stores the result.
Expected output:src/lib/analyzer.ts exports analyzeCompetitorChanges() which retrieves prior findings from memory, includes them as context in the Perplexity prompt, calls the LLM, repairs the JSON, and stores the new findings.
Step 12: Build the CompetitorIntelService
This is the main service class that wires everything together. It scrapes each competitor’s page, runs the analysis, stores embeddings in ChromaDB, generates the daily brief, and posts it to Slack.
ts
// src/lib/competitor-intel.tsimport pRetry from "p-retry";import type { Config } from "./config.js";import { createPerplexityClient } from "./perplexity.js";import { scrapeCompetitorPage } from "./scraper.js";import { getEmbeddingModel, embedDocuments } from "./embedding.js";import { createChromaClient, ensureCollection, storeCompetitorEmbedding } from "./chromadb-store.js";import { postDailyBriefToSlack } from "./slack.js";import { createMemoryStore, runMemoryMaintenance } from "./memory.js";import { createPrioritizationRouter, prioritizeCompetitors } from "./router.js";import { initObservability, withTrace } from "./observability.js"
Expected output:src/lib/competitor-intel.ts exports the CompetitorIntelService class with initialize(), runDailyIntel(), runDailyIntelWithRetry(), getChangeHistory(), getLatestBrief(), and close() methods. The service uses retry logic from p-retry, traces through Langfuse, prioritizes via the confidence router, scrapes pages, writes embeddings to ChromaDB, and posts briefs to Slack.
Step 13: Create the API route handler
The API exposes two endpoints: GET /api/report returns the latest brief, and POST /api/report triggers an immediate intelligence run.
Expected output:app/api/report/route.ts exports GET and POST handlers using NextRequest / NextResponse. Both use Langfuse tracing via withTrace(). GET returns 404 when no brief exists yet. POST triggers a full intel run and returns the generated brief.
Step 14: Create the cron scheduler via instrumentation
The instrumentation module starts a node-cron scheduler when the Next.js server boots in the Node.js runtime. This runs the daily intelligence pipeline on the cron schedule from your config.
ts
// src/instrumentation.tsimport { loadConfig } from "./lib/config.js";import { CompetitorIntelService } from "./lib/competitor-intel.js";import { initObservability } from "./lib/observability.js";export async function register(): Promise<void> { if (process.env.NEXT_RUNTIME !== "nodejs") { return; } try { const cron = await import("node-cron"); const config = loadConfig(); const langfuse = initObservability({ publicKey: config.langfusePublicKey, secretKey: config.langfuseSecretKey, }); const service = new CompetitorIntelService(config); await service.initialize(); if (langfuse) { const span = langfuse.span({ name: "instrumentation-boot" }); span.update({ output: { schedule: config.scheduleCron } }); span.end(); } console.log(`Cron scheduler started with schedule ${config.scheduleCron}`); const task = cron.default.schedule(config.scheduleCron, async () => { try { await service.runDailyIntel(); } catch (error) { console.error("Scheduled intel run failed:", error); } }); void task.start(); } catch (error) { console.error("Failed to initialize instrumentation:", error); }}
Expected output:src/instrumentation.ts exports register() which guards on NEXT_RUNTIME === "nodejs", initializes the service, and schedules cron.default.schedule() to run runDailyIntel() on the configured cron expression. The scheduler always logs "Cron scheduler started with schedule ..." on successful boot.
Step 15: Create the dashboard page
Build a simple client-side dashboard that displays the latest brief and lets you trigger a manual intelligence run with a button.
Expected output:app/page.tsx is a "use client" component that fetches the latest brief on mount and provides a “Run Now” button to trigger an immediate intelligence run. It renders the date, summary, top threat, competitor count, and a list of change events.
Step 16: Write and run the tests
Create tests for the API route handler and the main service. These mock all external dependencies so they run without live API calls.
ts
// tests/api-report.test.tsimport { describe, it, expect, vi, beforeEach } from "vitest";import { GET, POST } from "../app/api/report/route.js";import type { NextRequest } from "next/server";vi.mock("next/server", () => ({ NextResponse: { json: vi.fn((data: unknown, init?: { status?: number }) => ({ data, status: init?.status ?? 200, json: () => data, })), },}));const createMockRequest
Run the tests:
terminal
pnpm vitest run --coverage --reporter=json --outputFile=vitest-report.json
Expected output: All tests pass with numFailedTests=0. Coverage should be at or above 90% on source files in src/ and app/api/. The test output shows a coverage summary with lines, branches, functions, and statements.
Run the full quality gate:
terminal
pnpm typecheckpnpm lintpnpm vitest run --coverage --reporter=json --outputFile=vitest-report.json
Expected output: All three commands exit 0. TypeScript compiles without errors, ESLint finds no violations, and all tests pass with coverage above the threshold.
Next steps
Add a web dashboard with historical trend charts showing competitor change frequency and severity over time
Integrate email notifications via SendGrid or Resend in addition to Slack
Implement a feedback loop that lets users mark change events as relevant or irrelevant, improving the confidence router’s priority scores
Extend the scraper to handle JavaScript-rendered pages with more complex selectors and pagination
Add a Claude or GPT-4 evaluator that assigns a competitive-threat score to each change event for better prioritization
;
import type { AgentMemory } from "@reaatech/agent-memory";
import type { ChromaClient, Collection } from "chromadb";
import type { ConfidenceRouter } from "@reaatech/confidence-router";
import type { FlagEmbedding } from "fastembed";
import type { Langfuse } from "langfuse";
import type { LangfuseSpanClient } from "langfuse";
import type Perplexity from "perplexity-sdk";
import type { DailyBrief, ChangeEvent, CompetitorAnalysis } from "./schemas.js";
