Documentation Index
This is the master index for the DM Agency AI Platform documentation.
Start Here
| Doc | What it answers |
|---|---|
| Introduction | What Leadmetrics is, the problem it solves, key concepts, and where to go next |
| Architecture | How the system is structured — control plane, adapter pattern, phone-home callbacks |
| Architecture Diagrams | Network diagrams — full system map, control plane internals, agent flow, data layer, deployment |
| Tech Stack | Monorepo structure, all packages and apps, technology choices |
| Workflow Model | The operational flow: onboarding → strategy → goals → deliverables → activities |
Developer Setup
| Doc | What it covers |
|---|---|
| Local Setup | Prerequisites, Docker, DB init, seed accounts, starting all 10 services, common issues |
| Environment Variables | Every env var for every app and server — required vs optional, per-service grouping |
| CI/CD Pipeline | GitHub Actions jobs (lint, typecheck, unit tests), triggers, branch strategy, Coolify deployment |
Core System
| Doc | What it covers |
|---|---|
| Database — Index | Database folder overview, split pattern, quick decision guide |
| Database Overview | Which data goes where and why — decision guide, key relationships |
| PostgreSQL Schema | Full PostgreSQL schema: all tables, indexes, useful queries |
| MongoDB Collections | All MongoDB collections with schemas, indexes, TTLs, and rationale |
| SQL Entity Relations | ER diagrams for all 31 PostgreSQL tables and their relationships |
| NoSQL Entity Relations | MongoDB collections and their cross-references to PostgreSQL |
| Multi-Tenancy | Tenant isolation, plans, subdomain routing, enterprise on-prem |
| Infrastructure | Docker Compose, Coolify deployment, enterprise packaging |
| API Gateway | Gateway layer — request lifecycle, auth, audit logging, rate limiting, throttling, Option B future path |
| Adapters | Adapter pattern: Claude Code CLI, OpenAI, Ollama, Webhook — I/O mechanics per backend |
| Adapter — Claude | Claude Code CLI subprocess: NDJSON stream, session resumption, --add-dir skills |
| Adapter — OpenAI | OpenAI REST: SSE streaming, manual session history, Codex/o1 support |
| Adapter — Ollama | Local Ollama REST: NDJSON streaming, zero cost, data-local tasks |
| Adapter — Webhook | Async phone-home: enterprise custom runtimes, JWT callback auth |
| Scalability | Queue design, partitioning, target: millions of activities/day |
| Testing | Three-tier strategy: unit, integration (real DBs), E2E (Playwright) |
Chat & Online Presence (docs/chat/)
| Doc | What it covers |
|---|---|
| Overview & Framework Decision | Why Socket.IO, scope by portal, feature summary |
| Architecture | Component diagram, auth flow, Redis layout, tenant isolation, multi-instance scaling |
| Data Schema | ChatMessage Prisma model, Redis key conventions, retention |
| Event Catalog | Full Socket.IO event reference — presence, chat, typing, read receipts |
| Implementation Plan | Four-phase rollout: server setup, presence UI, chat panel, unread/notifications |
Feature Docs (docs/features/)
| Doc | What it covers |
|---|---|
| Agent Hierarchy | All agents, roles, deliverable mapping, plan availability |
| Agent Execution Engine | Adapter implementations, streaming, session, cost calculation |
| Task Queue & Orchestration | BullMQ, tenant-namespaced queues, activity pipeline progression |
| Skills System | Skill injection, client context file, MongoDB storage |
| Session / Memory Management | Claude session resumption, Ollama history management |
| Tool / Integration Layer | External API integrations (SEMrush, GA4, WordPress, etc.) |
| Cost & Usage Tracking | LLM cost tracking, budget caps, billing events |
| Governance & Guardrails | Permission whitelist, output validators, HITL enforcement |
| Observability | Audit logging, OpenTelemetry tracing, Grafana + Loki |
| RAG Integration | How RAG fits into Leadmetrics — two-layer context strategy, datasets, agent tool, onboarding flow |
| RAG Architecture | Technical design — Prisma schema, Qdrant collections, BullMQ ingestion pipeline, hybrid search |
Insights (docs/insights/)
| Doc | What it covers |
|---|---|
| Overview | Feature summary, supported channels, how it works, key design decisions |
| Architecture | ChannelInsight DB model, job data shapes, worker pattern, API routes, data flow |
| Agents | Per-channel agent specs — data fetched, system prompt focus, output sections |
| Implementation Plan | 6-phase build guide: DB → API → workers → cron → UI → prompt tuning |
| UI Screens | Dashboard screens I1 (list), I2 (detail), I3 (channel tab), real-time update spec |
Newsletters (docs/newsletters.md)
| Doc | What it covers |
|---|---|
| Newsletters | EmailNewsletter model, status machine (draft→dm_review→approved→sending→sent), API endpoints, Resend send flow, unsubscribe |
Campaigns (docs/campaigns/)
| Doc | What it covers |
|---|---|
| Overview & Campaign Types | Five campaign types, approval chain, existing foundations, open questions |
| Data Model | Campaign, CampaignEmail, CampaignMetrics, CampaignSequence models |
| AI Agents | Which agents handle each campaign type |
| API Routes | All campaign endpoints — tenant + DM portal |
| Dashboard UI | Campaigns hub, detail tabs, creation wizard |
| Workflow & Permissions | Status machine, approval chain, notifications |
CRM & Pipeline (docs/crm/)
| Doc | What it covers |
|---|---|
| Overview | Lead tracking, CRM vs Contacts distinction, contents |
| Leads | Lead data model, capture sources, schema |
| Pipeline | Lead lifecycle, statuses, qualification workflow |
| Activities | Lead activity timeline, touchpoints |
| Lead Scoring | Scoring model, score bands, ranking logic |
| Nurture Sequences | Automated follow-up sequence model |
Contacts (docs/contacts/)
| Doc | What it covers |
|---|---|
| Overview | Contacts vs CRM leads distinction |
| Contacts | Contact model, stages, sources, subscription management |
| Contact Lists | Static and dynamic lists, campaign audience targeting |
Content Toolkit (docs/content/)
The full content creation, optimization, and distribution pipeline — from topic discovery to multi-channel publishing. See content/index.md for the feature map, gap analysis vs SEMrush, and build priority order.
| Doc | Status | What it covers |
|---|---|---|
| Overview | — | Feature map, gap analysis, build priority, new credit rates |
| Content Optimizer | To Build · P1 | SEO, readability, brand voice, AI search scoring + in-editor guided fixes |
| Social Publishing | To Build · P1 | Scheduled + on-demand publishing to Facebook, Instagram, LinkedIn, TikTok, X |
| Repurposing Pipeline | ✅ Implemented | Blog → social posts, email newsletter, ads copy via content-repurposer agent |
| SEO Brief Export | To Build · P2 | Shareable brief links, Google Docs export, brief compliance check |
| Mailchimp Integration | To Build · P2 | OAuth connect, draft campaign push, Mailchimp audience selector |
| Content Audit | To Build · P2 | Post freshness audit, health score, refresh workflow, cannibalisation detection |
| Multi-Language | To Build · P3 | 7-language support across all writer agents |
| Location Targeting | To Build · P3 | Geo-specific topics, keywords, and content for local service businesses |
| Performance Feedback | To Build · P3 | GSC ingestion, ranking signals, performance-enriched refresh briefs |
| Blog Image Generation | To Build · P3 | Hero + inline images via Azure GPT Image, brand-matched styles |
Code Reviews (docs/code-reviews/)
| Doc | What it covers |
|---|---|
| API Code Review v1 | Security & rate limiting audit — gaps between docs and implementation, fix priority order |
New Features (docs/new-features/)
Comprehensive documentation for recently implemented features.
Content Repurposing Engine ✅ Implemented
Transform approved content into multiple platform-optimized derivatives with a single click. Includes template management portal and analytics dashboard.
| Doc | What it covers |
|---|---|
| Complete Feature Doc | Full feature overview, architecture, implementation status, getting started |
| Implementation Summary | Detailed implementation notes, design decisions, component list |
| Migration Guide | Step-by-step migration, database setup, worker configuration |
| Quick Reference | Handy reference card, credit costs, user workflows, monitoring |
| Original Feature Spec | Original specification and requirements |
Key Features:
- 🔄 Core Engine: 14 repurposing paths (blog→social/email/ads, etc.), 0.5 credits per derivative
- 🎨 Template Portal: Customize prompts, manage templates, configure credit costs (
/templates/repurposing) - 📊 Analytics Dashboard: Track ROI, usage patterns, credit savings (
/analytics/repurposing) - ✅ Quality Control: 2-gate approval maintained for all derivatives
- 💰 Cost Efficiency: 50-75% cheaper than creating content from scratch
Agent Docs (docs/agents/)
Design principle: one agent = one function. See agents/README.md for the master index with quick-reference tables (queue names, models, costs, plan availability). See agent-hierarchy.md for pipeline architecture, trigger types, and plan availability details.
Setup Chain — [To Build]
| Doc | Input → Output |
|---|---|
| Client Researcher | Domain URL → client research notes |
| Competitor Researcher | Competitor domains → competitor analysis |
| Context File Writer | Research notes → Client Context File (skill) |
Strategy Chain — [To Build]
| Doc | Input → Output |
|---|---|
| Strategy Writer | Context File → marketing strategy document |
| Deliverable Planner | Strategy → goals + monthly deliverable volumes |
Orchestration — [To Build]
| Doc | Input → Output |
|---|---|
| Activity Planner | Deliverable plan → activity pipeline (dispatches to workers) |
SEO Agents — [To Build]
| Doc | Input → Output |
|---|---|
| Keyword Researcher | Domain/industry → keyword clusters |
| Content Brief Writer | Keyword cluster → SEO content brief |
| Site Auditor | Domain URL → technical SEO audit report |
| Backlink Researcher | Niche + existing links → prospect list |
| Backlink Outreach Writer | Single prospect → personalized outreach email |
Content Agents — [To Build]
| Doc | Input → Output |
|---|---|
| Blog Writer | SEO brief → full blog post |
| GBP Post Writer | Business info + topic → GBP post |
| Social Post Writer | Platform + calendar item → single social post |
| Email Writer | Campaign brief → email newsletter copy |
| Landing Page Writer | Product brief → landing page copy |
| Google Ads Writer | Campaign brief + keywords → RSA copy set |
| Meta Ads Writer | Campaign brief + audience → Meta ad copy variants |
Planning + Analysis + Reporting — [To Build]
| Doc | Input → Output |
|---|---|
| Social Calendar Planner | Strategy + platforms → monthly content calendar |
| Ads Analyst | Ads performance data → analysis + recommendations |
| Report Writer | All channel data + goals → monthly performance report |
| Anomaly Detector | Metrics vs prior period → anomaly alerts |
Research Agents (Ollama/local) — [To Build]
| Doc | Input → Output |
|---|---|
| Topic Researcher | Industry + recent topics → 10–15 topic ideas |
| Research Note Writer | Single topic → stats + citations + reference notes |
Reactive Agents — [To Build]
| Doc | Input → Output |
|---|---|
| Review Response Writer | Single review → draft response + classification |
System Agents (live background automation)
| Agent | What it does |
|---|---|
| Blog AI Agent [Live] | Auto-drafts blogs from Request Queue items; shows as assignee in M12/M13 |
| Billing Agent [Live] | Auto-creates invoices from billing events; shows in invoice audit trail |
Superseded docs deleted. The old multi-function agent files have been removed. See agents/README.md for the replacement mapping.
Apps (docs/apps/)
Each app has its own folder with screens, navigation flows, and app-specific technical docs.
| App | Folder | What it covers |
|---|---|---|
| Apps — Index | docs/apps/ | Master index of all apps, API surface map |
| Dashboard | docs/apps/dashboard/ | Tenant-facing web app (D1–D13 Live, P1–P16 To Build) |
| DM Portal | docs/apps/dm-portal/ | Internal reviewer interface (P1–P8) |
| Manage | docs/apps/manage/ | Super admin platform control (M1–M15) + Knowledge Base management |
| Auth | docs/apps/auth/ | JWT auth system (HS256), sessions, social providers [To Build], OAuth server [To Build] |
| Knowledge Base | apps/knowledgebase | Internal docs site (Nextra v4, port 3004, junction to docs/) |
| Mobile | docs/apps/mobile/ | React Native Dashboard — tech, screen availability, push notifications |
CLI (lm) | docs/apps/cli/ | Interactive REPL for the internal DM team — agents, approvals, queues, KB |
Per-app docs quick reference
| Doc | App | What it covers |
|---|---|---|
| Dashboard — Screens | Dashboard | All screen specs (D1–D13 + P1–P16) |
| Dashboard — Screen Flows | Dashboard | Navigation flows |
| Dashboard — Workflow Screens | Dashboard | Strategy, Activity Detail, Inbox, Recurring Tasks |
| DM Portal — Screens | DM Portal | All screen specs (P1–P8) |
| DM Portal — Screen Flows | DM Portal | Navigation flows |
| Manage — Screens | Manage | All screen specs (M1–M15) |
| Manage — Screen Flows | Manage | Navigation flows |
| Auth — Screens | Auth | Login, Forgot PW, Reset PW, Register wizard |
| Auth — Screen Flows | Auth | Auth & registration navigation flows |
| Auth — Sessions | Auth | JWT, refresh tokens, 2FA, biometrics |
| Auth — OAuth Server | Auth | Leadmetrics as OAuth 2.0 / OIDC provider |
| Auth — Social Providers | Auth | Google, LinkedIn, Microsoft, Apple |
| Manage — Knowledge Base Screens | Manage | Full KB screen specs — dataset overview, file management, upload, sandbox, config (KB1–KB6) |
| Manage — Knowledge Base Flows | Manage | KB navigation flows |
| Dashboard — Knowledge Base | Dashboard | What tenants can do — upload/remove docs, sandbox, retrieved context |
| DM Portal — Knowledge Base | DM Portal | What reviewers can do — upload/remove docs per tenant, retrieved context in activity detail |
| Mobile — Screen Flows | Mobile | React Native navigation, push deep links, offline behaviour |
| CLI — Commands | CLI | Full command reference — slash commands, flags, output formats |
| CLI — Workflows | CLI | Common team workflows — queue review, onboarding, investigations, scripting |
Shared UI (docs/ui/)
Design decisions shared across all apps.
| Doc | What it covers |
|---|---|
| UI Layer | Three web apps + mobile, Next.js setup, auth, SSE, SendGrid integration |
| Design System | Color palette, typography, spacing, component patterns, motion |
| Responsive Design | Breakpoints, shell layout, per-app nav items, responsive patterns for tables/charts/forms |
| Theming | Dark/light mode — CSS variables, next-themes setup, color palette, no-flash strategy |
HITL — Human-in-the-Loop (docs/hitl/)
| Doc | What it covers |
|---|---|
| HITL Index | All HITL touchpoints across 7 categories — onboarding, content, SEO, ads, reporting, mid-run decisions, safety gates |
| Approval Flow | Full approval lifecycle — creation → notifications → review → decision → downstream effects → expiry → audit trail |
| UI Patterns | How approvals are displayed and actioned — DM Portal, Dashboard, Mobile, CLI — ASCII wireframes + notification preferences |
Servers (docs/servers/)
Agents (docs/servers/agents/)
| Doc | What it covers |
|---|---|
| Overview | Service overview, worker list, scaling, required config |
Notifications (docs/servers/notifications/)
| Doc | What it covers |
|---|---|
| Overview | Service overview, queue names, job schema, notification types reference |
| Design & Implementation | File structure, startup flow, per-handler dispatch logic, templates, dev-mode filtering, config |
| Channels | Per-channel handler details — Email, WhatsApp, Telegram, SMS (stub), Web (in-app) |
| Providers & Resolution | Per-tenant vs platform-default provider resolution, provider packages, config shapes |
| Provider Packages | provider-sendgrid, provider-smtp, provider-ses, provider-whatsapp, provider-telegram — interfaces, tests, adding new providers |
Billing (docs/servers/billing/)
| Doc | What it covers |
|---|---|
| Overview | 3 BullMQ workers (invoice, overdue, credits-reset), 4 cron schedules, queue names, env vars, shutdown |
Reporting (docs/servers/reporting/)
| Doc | What it covers |
|---|---|
| Overview | Service overview, three daily reports, scheduler, timezone handling, deduplication, trigger script |
Credits (docs/credits/)
| Doc | What it covers |
|---|---|
| Credits Overview | What credits are, plan allocations, credit rates per deliverable type, top-ups |
| Credit Consumption | Reservation vs consumption, retry handling, HITL rejection charges, overage policy |
| Credit Tracking | DB schema (credit_balances, credit_ledger), API endpoints, monthly reset, reconciliation |
External Providers (docs/providers/)
One doc per external service — purpose, config structure, integration pattern, test cases.
| Doc | Provider | Category |
|---|---|---|
| Providers — Index | All providers | Master index |
| Claude | Anthropic Claude Code CLI | AI / LLM |
| OpenAI | OpenAI Chat Completions | AI / LLM |
| Ollama | Local open-source LLMs | AI / LLM |
| SEMrush | Keyword research & competitor analysis | SEO |
| DataForSEO | SERP data, backlinks, keyword difficulty | SEO |
| Ahrefs | Backlinks & domain authority | SEO |
| Google Search Console | Organic search performance | SEO / Analytics |
| Bing Webmaster Tools | Bing/DuckDuckGo search performance | SEO |
| Google Analytics 4 | Website traffic & conversions | Analytics |
| Google My Business | GBP posts + review responses | Local SEO |
| Google Ads | Campaign performance + RSA push | Paid Ads |
| Meta Ads | Facebook/Instagram campaigns | Paid Ads |
| LinkedIn Ads | B2B LinkedIn campaigns | Paid Ads |
| Organic Page posts + Insights | Social Publishing | |
| Organic feed posts + carousels | Social Publishing | |
| Organic Company Page posts | Social Publishing | |
| Community posts + trend research | Social Publishing | |
| WordPress | Blog post publishing | Publishing |
| Webflow | CMS item publishing with field mapping | Publishing |
| Mailchimp | Email campaigns | Email Marketing |
| Klaviyo | E-commerce email flows | Email Marketing |
| SendGrid | Platform default email + tenant BYOP | Notification |
| SMTP | Generic SMTP for tenant email | Notification |
| AWS SES | Tenant AWS SES email | Notification |
| MSG91 | Platform default SMS (India) | Notification |
| Twilio | International SMS | Notification |
| WhatsApp Business API | Template WhatsApp messages | Notification |
| Telegram Bot | Ops/internal alerts | Notification |
| Razorpay | Subscription billing, credit top-ups | Payments |
| AWS S3 | File storage (attachments, PDFs, media) | Storage |
| Qdrant | Vector store for RAG | Vector Search |
| Redis | BullMQ backend, caching, dedup | Infrastructure |
| PostgreSQL | Primary relational store | Infrastructure |
| MongoDB | Document store for agent outputs & blobs | Infrastructure |
Other
| Doc | What it covers |
|---|---|
| Introduction | Platform overview, key concepts, “where to go next” navigation guide |
| Onboarding | Self-registration, Razorpay, onboarding wizard, context file generation |
| Self-Signup | 4-step customer wizard — session management, tax logic, payment flow, abandonment emails |
| MCP | Model Context Protocol — Playwright dev tooling, planned Leadmetrics MCP server |
| Open Questions | Decisions pending, ambiguities, areas needing more design |
Key Concepts (Quick Reference)
| Concept | Where Defined |
|---|---|
| Control plane + adapter pattern | Architecture + Adapters |
| Onboarding → Context File → Strategy → Goals → Deliverables → Activities | Workflow Model |
| Channels — OAuth-connected publishing platforms | Workflow Model + PostgreSQL Schema |
| Requests → Content Entities (BlogRequest → BlogPost, SocialMediaRequest → SocialPost) | Workflow Model + PostgreSQL Schema |
| Paid Campaigns — Google Ads/Meta sync + classify + recommend + HITL push | Workflow Model + PostgreSQL Schema |
| Recurring Task Templates (predefined periodic tasks) | Workflow Model |
| Pipeline Activity Templates (per-deliverable pipelines) | Workflow Model |
| HITL approval flow | HITL Index + Approval Flow + UI Patterns |
| Client Context File | Onboarding Agent |
| Tenant isolation | Multi-Tenancy |
| Base record fields (id, refId, tenantId, audit fields) | PostgreSQL Schema |
| Backlink tracking + DataForSEO status checks | PostgreSQL Schema + Tool Integration Layer |
| Audit logging + OpenTelemetry + Grafana Loki | Observability |
providers/ vs features/ directory convention | Tech Stack |
| Contracts — tenant acceptance gate on first login | PostgreSQL Schema + MongoDB |
| Which data goes to PostgreSQL vs MongoDB | Database Overview |
| All PostgreSQL table relationships (ER diagram) | SQL Entity Relations |
| MongoDB collections and cross-references | NoSQL Entity Relations |
Deleted / Archived
The following files were removed or repurposed:
docs/agents.md→ superseded bydocs/agents/directorydocs/database.md→ moved todocs/database/overview.md(redirect stub left at old path)docs/database-postgres.md→ moved todocs/database/postgres.md(redirect stub left at old path)docs/database-mongo.md→ moved todocs/database/mongo.md(redirect stub left at old path)