Tech Stack — Backend

Parent: Tech Stack Overview

Covers: apps/api, apps/servers/*, and all packages/ libraries.

Runtime & Language

All server processes run with tsx directly (no separate build step in development). Production builds transpile to JS via tsc.

API Server (apps/api)

Key patterns:

• Every route schema uses { type: "object", additionalProperties: true } to prevent silent field stripping
• New routers registered in both app.ts (test entry) and index.ts (production entry)
• SSE: reply.raw.writeHead() bypasses @fastify/cors — origin mirrored manually in startSse()
• Schemas live in apps/api/src/schemas/ — shared.ts + domain files

Worker Servers (apps/servers/)

Seven standalone Node processes, each started by tsx src/index.ts. All consume BullMQ queues and share packages via the monorepo.

Databases

ORM — Prisma v6.19:

MongoDB (@leadmetrics/nosqldb): Used exclusively for the audit log writer. connectMongo() is called automatically inside writeAuditLog() — no manual connection call needed.

S3 storage key convention:

webpages/{tenantId}/{channelId}/screenshots/{webPageId}.jpg
webmedia/{tenantId}/images/{contentHash}.{ext}
webmedia/{tenantId}/videos/{contentHash}.{ext}
webmedia/{tenantId}/documents/{contentHash}.{ext}
{tenantId}/rag/{datasetId}/{documentId}.{ext}

Task Queue (BullMQ v5)

One shared queue per agent role (agent__{role}), used by all tenants. tenantId is in the job payload for isolation. Concurrency controlled per worker, not per queue.

Critical rules:

• dedupeKey must be cleared in both completed and failed handlers — silent drop otherwise
• Never create a separate IORedis instance inside workers — workers manage their own connections via getRedisConnection()
• Dynamic job IDs (timestamp-based) for revision chains — never static IDs that silently drop re-enqueues

Redis client: ioredis v5.3 — used directly in BullMQ workers and the API’s Socket.IO Redis adapter. 4 connections per API instance (Socket.IO emitter, BullMQ, rate-limit, session).

Authentication

JWT flow: 15-min access tokens + 7-day refresh tokens, both HS256. Fastify API is the single issuer (/auth/v1). Portals verify locally via middleware; expired access tokens are silently refreshed via POST /auth/v1/refresh.

Cookie names:

• Dashboard: dashboard_access_token / dashboard_refresh_token
• DM: dm_access_token / dm_refresh_token
• Manage: manage_access_token / manage_refresh_token

LLM Providers

All adapters implement a common AgentAdapter interface. Model selection is per AgentConfig.adapter in the DB. ANTHROPIC_API_KEY must be in apps/servers/agents/.env.

RAG embedding / reranking: Handled in apps/servers/ragengine via @leadmetrics/provider-qdrant.

Real-Time

Search

Email & Notifications

All notification sends go through enqueueNotification() → BullMQ → apps/servers/notifications worker. Dev filter: DEV_ALLOWED_EMAIL_DOMAINS=leadmetrics.ai.

Payments

Webhook handler: POST /admin/v1/billing/webhook — HMAC-verified.

File Processing

Website Crawler (agent__tenant-web-crawler)

Triggered at signup when the tenant provides a website URL. Runs in apps/servers/agents.

1. Fetches robots.txt + sitemap.xml to seed BFS queue
2. Playwright BFS up to maxPages (default 100), same-origin only
3. Per page: stores WebPage (title, text, contentHash), uploads screenshot, queues text for RAG
4. Per page: downloads images (compressed via sharp → DO Spaces), videos, documents → WebMedia
5. Homepage only (pageUrl === normalizedStart): runs extractBrandAssets() before DOM mutation — extracts CSS custom property colors, CTA/nav colors, Google Fonts links → writes to brand_assets if primaryColor is currently null
6. On completion: sets ConnectedChannel.isConnected = true

Brand extractor: packages/agents/src/utils/brand-extractor.ts

Observability & Logging

Production log aggregation via Grafana Loki. Error monitoring for mobile via Sentry.

Testing

Key patterns:

• vi.stubEnv for env vars; vi.hoisted() for factories; hookTimeout: 30_000
• Agent worker tests: export pure functions + standard 8-mock boilerplate
• Dashboard auth mock: mock @/lib/server-auth, not @/lib/auth; requireSession returns {payload, user}
• Server component tests: page.tsx queries Prisma directly — never fetch to Fastify

Key Environment Variables

# LLM
ANTHROPIC_API_KEY=          # required in apps/servers/agents/.env
OPENAI_API_KEY=
 
# Databases
DATABASE_URL=postgresql://user:pass@postgres:5432/leadmetrics
MONGO_URL=mongodb://mongo:27017/leadmetrics
REDIS_URL=redis://redis:6379
QDRANT_URL=http://qdrant:6333
 
# Auth
JWT_SECRET=                 # shared across API + all portals
REFRESH_TOKEN_SECRET=
 
# Storage (DigitalOcean Spaces — all 6 required)
DO_SPACES_ENDPOINT=https://{region}.digitaloceanspaces.com
DO_SPACES_REGION=
DO_SPACES_BUCKET=
DO_SPACES_ACCESS_KEY_ID=
DO_SPACES_SECRET_ACCESS_KEY=
DO_SPACES_CDN_URL=
 
# Search
TYPESENSE_API_KEY=
TYPESENSE_HOST=
 
# Email
SENDGRID_API_KEY=
 
# Payments
RAZORPAY_KEY_ID=
RAZORPAY_KEY_SECRET=
RAZORPAY_WEBHOOK_SECRET=
 
# Internal
INTERNAL_API_SECRET=        # API-to-API secret for server-to-server calls
DEV_ALLOWED_EMAIL_DOMAINS=leadmetrics.ai
DEV_CC_EMAIL=moble@leadmetrics.ai