Files · Anthropic RAG Knowledge Base for Construction Project Specs

# Anthropic RAG Knowledge Base for Construction Project Specs
 
> Let field crews instantly query building specifications, codes, and submittals via a Claude‑powered retrieval pipeline backed by hybrid search.
 
A production-grade RAG system for construction SMBs. PDF specs stored in S3 are ingested into a Qdrant vector store via Voyage AI embeddings, then queried through a hybrid (vector + BM25) retriever with Claude Haiku generating specification‑accurate answers.
 
```
  S3 ──► IngestionService ──► pdf-parse ──► splitIntoChunks
                                              │
                                              ▼
                                        Voyage Embedding
                                              │
                                              ▼
   ┌──────────────────────────────────────────────────┐
   │                  Qdrant (vector DB)              │
   │     + in-process BM25 index (HybridRetriever)    │
   └──────────────────────────────────────────────────┘
                         │
                         ▼
              HybridRetrieval + Reranker
                         │
                         ▼
               Claude Haiku (Anthropic)
                         │
                         ▼
                  { answer, sources }
```
 
## Setup
 
```bash
# Install dependencies
pnpm install
 
# Copy and populate environment variables
cp .env.example .env
# Edit .env with your API keys
 
# Ingest PDF specs from S3
pnpm ingest                    # uses S3_BUCKET_NAME and optional prefix arg
pnpm ingest "specs/2024/"      # process only a specific prefix
 
# Start dev server
pnpm dev
 
# Run tests
pnpm test
 
# Type check
pnpm typecheck
 
# Lint
pnpm lint
```
 
## API Usage
 
### Query endpoint
 
```bash
curl -X POST http://localhost:3000/api/chat \
  -H "Content-Type: application/json" \
  -d '{"query": "What is the concrete compressive strength requirement?"}'
```
 
Response:
```json
{
  "answer": "The concrete compressive strength is 3000 psi at 28 days (per Section 3.2 of spec doc-001).",
  "sources": [
    { "content": "...", "documentId": "doc-001", "score": 0.95, "source": "vector" }
  ]
}
 
### Ingest endpoint
 
curl -X POST http://localhost:3000/api/ingest \
  -H "Authorization: Bearer $INGEST_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"prefix": "specs/2024/"}'
```
 
## Environment Variables
 
| Variable | Description |
|----------|-------------|
| `QDRANT_URL` | Qdrant server URL |
| `QDRANT_API_KEY` | Qdrant API key |
| `ANTHROPIC_API_KEY` | Anthropic Claude API key |
| `VOYAGE_API_KEY` | Voyage AI API key |
| `COHERE_API_KEY` | Cohere reranker API key |
| `AWS_REGION` | AWS region |
| `AWS_ACCESS_KEY_ID` | AWS access key |
| `AWS_SECRET_ACCESS_KEY` | AWS secret key |
| `S3_BUCKET_NAME` | S3 bucket with PDF specs |
| `INGEST_API_KEY` | API key to protect the ingest endpoint |
| `LANGFUSE_PUBLIC_KEY` | Langfuse public key |
| `LANGFUSE_SECRET_KEY` | Langfuse secret key |
 
## Project Layout
 
```
app/api/chat/route.ts      Query endpoint (hybrid RAG → Claude)
app/api/ingest/route.ts    Ingestion trigger
src/lib/embedding.ts       Voyage AI embedding adapter
src/lib/ingestion.ts       S3 → PDF → chunk → Qdrant pipeline
src/lib/retrieval.ts       Hybrid retriever (vector + BM25 + fusion)
src/lib/generation.ts      Claude answer generation (sync + stream)
src/lib/observability.ts   Langfuse tracing
src/scripts/ingest.ts      CLI ingestion script
tests/                     Vitest suite (unit + integration)
```
 
## License
 
MIT — see [LICENSE](./LICENSE).