Files · Databricks Code Sandbox for Secure SMB Data Analysis

# Databricks Code Sandbox for Secure SMB Data Analysis
 
An AI agent that translates natural language into safe SQL and Python queries, runs them on Databricks, and returns results with cost tracking and guardrails.
 
Built with Next.js 16+ (App Router), TypeScript, and the `@reaatech/*` package ecosystem.
 
## Problem
 
Small businesses with data in Databricks need ad-hoc reports and analyses, but hiring a data engineer for every query isn't feasible. Non-technical staff often write inefficient or unsafe code, risking runaway costs. This solution layers AI guardrails — intent classification, code repair, sandboxed dry-runs, a destructive-operation firewall, and per-session cost tracking — to make Databricks analytics safe and accessible for non-technical users.
 
## Architecture
 
```
User Input → ConfidenceRouter (classify intent) → OpenAI SDK (generate code)
→ StructuredRepairCore (fix output) → E2B Sandbox (dry-run) → ToolUseFirewall (block destructive ops)
→ Databricks SDK (execute) → LLMCostTelemetry (track spend) → Langfuse (observe) → SessionContinuity (context)
```
 
Pipeline steps: classify → generate → repair → sandbox → firewall → execute → track → session.
 
## Prerequisites
 
- **OpenAI** API key (or any OpenAI-compatible provider via `baseURL`)
- **Databricks** workspace host and token
- **E2B** sandbox API key
- **Langfuse** public/secret keys (optional — falls back to no-op)
 
## Quick Start
 
```bash
pnpm install
cp .env.example .env       # fill in your keys
pnpm dev                   # http://localhost:3000
pnpm test                  # 97 tests, 96%+ coverage
```
 
## API Endpoints
 
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/analyze` | Natural language → analyzed result |
| POST | `/api/session` | Create a new analysis session |
| GET | `/api/session?sessionId=...` | Get session context |
| GET | `/api/budget?sessionId=...` | Get session budget status |
 
## REAA Packages
 
| Package | Role |
|---------|------|
| `@reaatech/confidence-router` | Classify SQL vs Python intent |
| `@reaatech/structured-repair-core` | Fix malformed LLM JSON output |
| `@reaatech/tool-use-firewall-core` | Block destructive DROP/DELETE/os.system |
| `@reaatech/llm-cost-telemetry` | Track token spend and budget caps |
| `@reaatech/session-continuity` | Multi-turn conversation context |
 
## Environment Variables
 
| Variable | Required | Description |
|----------|----------|-------------|
| `DATABRICKS_HOST` | Yes | Databricks workspace URL |
| `DATABRICKS_TOKEN` | Yes | Databricks personal access token |
| `E2B_API_KEY` | Yes | E2B sandbox API key |
| `OPENAI_API_KEY` | Yes | LLM API key (OpenAI SDK, also works with DeepSeek via custom baseURL) |
| `OPENAI_MODEL` | No | Model for code generation (default: `gpt-5.2-mini`) |
| `LANGFUSE_PUBLIC_KEY` | No | Langfuse public key for observability |
| `LANGFUSE_SECRET_KEY` | No | Langfuse secret key for observability |
| `LANGFUSE_HOST` | No | Langfuse host URL (for self-hosted) |
| `DATABRICKS_WAREHOUSE_ID` | No | SQL warehouse ID for query execution |
| `ROUTE_THRESHOLD` | No | Confidence-router threshold (default: `0.8`) |
| `FALLBACK_THRESHOLD` | No | Confidence-router fallback threshold (default: `0.3`) |
| `MAX_BUDGET_USD` | No | Per-session cost cap for budget enforcement |
| `SESSION_TTL_SECONDS` | No | Session expiry in seconds (default: `3600`) |
| `SANDBOX_TIMEOUT_MS` | No | E2B sandbox execution timeout (default: `30000`)
 
## License
 
MIT — see [LICENSE](./LICENSE).