Backlink Email Outreach Pipeline

Status: [Live]

The email outreach pipeline is a 3-agent BullMQ chain: researcher → crawler → email writer. Each stage enqueues the next on completion. A Campaign wraps the entire run; a CampaignEmail is created per prospect once the outreach email is drafted.

Pipeline Stages

backlink-researcher
    └── [for each prospect] → website-crawler
                                    └── [on crawl success] → backlink-outreach-writer
                                                                    └── [all emails drafted] → Campaign: dm_review

Stage 1 — backlink-researcher

Trigger: Manual or from a deliverable run (DM initiates the campaign).

Input: Agent prompt with client context, target niche, and any seed domains or competitor URLs.

Process:

1. LLM generates a JSON list of link-building prospects.
2. Prospects are parsed and domain-normalised via backlink-researcher.utils.ts.
3. One Campaign (type: email_outreach, status: draft) is created.
4. For each prospect: a Backlink row is created with outreachStatus: "prospecting", then a website-crawler job is enqueued.

Output fields written to Backlink:

• sourceDomain, sourceUrl, prospectType, relevanceScore, pitchRationale

Credit type: backlink_research

Stage 2 — website-crawler

Trigger: Enqueued by backlink-researcher per prospect.

Input: { backlinkId, tenantId, campaignId, domain }

Process:

1. Check CrawlCache for a valid (<14 days old) entry on this domain.
2. On cache miss: spawn a Claude sub-process to crawl the domain’s contact page.
3. Upsert CrawlCache with: contactPageUrl, recipientEmail, recipientName, rawExtract, isReachable, hasCaptcha, isBlocked.
4. Update Backlink.sourceUrl with the contact page URL.
5. Auto-create a Contact record if recipientEmail was found.
6. Enqueue backlink-outreach-writer.

On failure or unreachable site: Decrement Campaign.totalEmails; skip the prospect (no email generated).

Timeout: 300 s (5 min per domain)

Stage 3 — backlink-outreach-writer

Trigger: Enqueued by website-crawler on success.

Input: { backlinkId, tenantId, campaignId }

Process:

1. Load Backlink, CrawlCache, AgentConfig, ClientContext, Campaign, Tenant.
2. Build prompt using: agent system prompt, client name, prospect domain, prospectType, pitchRationale, contactPageUrl, crawl extract, and client context.
3. Reserve credits, execute Claude, parse JSON output { subject, body }.
4. Create/update CampaignEmail (status: draft) with recipient info from the crawl cache.
5. Consume credits, publish agent:completed event.
6. Check campaign completion: if all emails are drafted → set Campaign.status = "dm_review".

On failure: Decrement Campaign.totalEmails, release credits, log error.

Concurrency: 3 jobs
Lock duration: 180 s
Credit type: backlink_outreach

Status Lifecycles

Backlink outreachStatus

prospecting → outreach_sent → responded → agreed → published
                                        ↘ rejected
               ↘ no_response

Campaign status

draft → dm_review → dm_approved → sent

CampaignEmail status

draft → approved → sent

Approval Gates

Clients are not in the approval chain for outreach emails — DMs own the full send workflow.

Prospect Types

Crawl Cache

CrawlCache is tenant-agnostic — the same crawl result is reused across all tenants for 14 days. This avoids hammering the same domains repeatedly when multiple clients target overlapping prospect lists.

Fields stored: domain (unique), contactPageUrl, recipientEmail, recipientName, rawExtract, isReachable, hasCaptcha, isBlocked, crawledAt.