servers/search-indexer — `@leadmetrics/server-search-indexer`

A dedicated Node.js background service that consumes search__sync BullMQ jobs and keeps the Typesense search index in sync with the PostgreSQL database. It bootstraps all 13 Typesense collections on startup and upserts or deletes documents as records change.

Source: apps/servers/search-indexer/

Why a Separate Service

Concern	Reason
Isolation	Typesense failures never affect API response times
Async indexing	API routes enqueue jobs and return immediately; indexing happens in background
Independent scaling	Index throughput scales by replica count
Single bootstrap	`bootstrapCollections()` runs once at startup rather than on every API request

Architecture


API route / server action
  └─ enqueueSearchSync({ collection, operation, recordId, tenantId })
        │
        ▼
  Redis (BullMQ) — queue: search__sync
        │
        ▼
  apps/servers/search-indexer
    └─ sync.worker.ts
         ├─ upsert: db.<model>.findUnique (scoped select) → Typesense upsert
         └─ delete: Typesense delete (404 swallowed — idempotent)

Supported Collections (13)

Collection	Prisma model	Indexed fields
`blogs`	`BlogPost`	`title`, `metaDescription`, `status`, `tenantId`
`social_posts`	`SocialPost`	`bodyText`, `engagementHook`, `platform`, `status`, `tenantId`
`landing_pages`	`LandingPage`	`title`, `metaDescription`, `status`, `tenantId`
`newsletters`	`EmailNewsletter`	`subject`, `previewText`, `status`, `tenantId`
`activities`	`Activity`	`label`, `notes`, `type`, `status`, `tenantId`
`campaigns`	`Campaign`	`name`, `status`, `tenantId`
`content_briefs`	`ContentBrief`	`title`, `topic`, `angle`, `status`, `tenantId`
`contacts`	`Contact`	`name`, `email`, `company`, `stage`, `tenantId`
`leads`	`Lead`	`name`, `company`, `jobTitle`, `status`, `tenantId`
`keywords`	`Keyword`	`keyword`, `source`, `tenantId`
`reports`	`Report`	`label`, `tenantId`
`backlinks`	`Backlink`	`sourceDomain`, `anchorText`, `status`, `tenantId`
`tenants`	`Tenant`	`name`, `website`, `pocName`, `industry`, `status`

Each findUnique uses a scoped select containing only the indexed fields plus id, createdAt, updatedAt — large text fields (e.g. BlogPost.htmlContent) are never fetched.

Sync Worker (`src/workers/sync.worker.ts`)

Queue: search__sync | Concurrency: SYNC_WORKER_CONCURRENCY (default 10)

Upsert flow:


1. Fetch record from Prisma with scoped select
2. If not found: log warn + return (soft skip — record may have been deleted)
3. Build document: { id, updatedAt (ms), ...FIELD_MAP fields }
4. client.collections(collection).documents().upsert(document)

Delete flow:


1. client.collections(collection).documents(recordId).delete()
2. 404 → swallow (already deleted — idempotent)
3. Other errors → rethrow (BullMQ will retry)

Config (`src/config.ts`)

Variable	Required	Default	Description
`DATABASE_URL`	yes	—	Prisma connection string
`REDIS_URL`	no	`redis://localhost:6379`	BullMQ connection
`TYPESENSE_URL`	no	`http://localhost:8108`	Typesense server URL
`TYPESENSE_ADMIN_API_KEY`	yes	—	Typesense admin key
`SYNC_WORKER_CONCURRENCY`	no	`10`	Worker concurrency

Startup Flow

Load config via Zod safeParse (fails fast with formatted error on missing vars).
bootstrapCollections() — creates all 13 Typesense collections if they don’t exist.
Connect IORedis with maxRetriesPerRequest: null.
Start sync worker.
Graceful shutdown on SIGTERM/SIGINT: drain worker (worker.close()) then redis.quit().

File Structure


apps/servers/search-indexer/
|-- src/
|   |-- index.ts                 Entry point — bootstrap, Redis, worker, shutdown
|   |-- config.ts                Zod env validation
|   +-- workers/
|       +-- sync.worker.ts       BullMQ worker: upsert/delete documents in Typesense
|
|-- .env
|-- package.json
+-- tsconfig.json

Global Search feature — Ctrl+K modal, 13 collections, API routes, enqueueSearchSync wiring

servers/search-indexer — @leadmetrics/server-search-indexer