Testing Strategy

Overview

Three tiers of test coverage. The guiding principle: no mocking of databases or queues in integration tests. Mocked infrastructure tests give false confidence — real integration tests catch schema mismatches, migration failures, and queue race conditions that mocks cannot.


Unit tests          → fast, no I/O, pure logic
Integration tests   → real PostgreSQL + MongoDB + Redis in Docker
E2E tests           → full Next.js + Fastify stack in Docker Compose

Tier 1 — Unit Tests (Vitest)

What: Pure functions with no side effects and no I/O. Where: Co-located next to source files: pricing.test.ts beside pricing.ts. Speed: < 1 second per file. Full suite < 30 seconds. Command: pnpm test:unit

What gets unit tested

Module	Tests
`packages/agent-engine/src/pricing.ts`	`calculateCost()` — all models, edge cases (0 tokens, unknown model)
`packages/agent-engine/src/validators/`	Character limit checker, banned words filter, brand voice score thresholds
`packages/agent-engine/src/budget.ts`	Budget cap logic: under/over cap, exactly at cap
`packages/skills/src/resolver.ts`	Skill resolution: default + client override merge, deduplication
`packages/control-plane/src/routing.ts`	Task routing rules: which agent type handles which task type
`packages/control-plane/src/prompt-builder.ts`	Prompt assembly: skill injection, session context, task input merging
`packages/shared/src/utils.ts`	Date formatting, slug generation, truncation helpers

Example


// packages/agent-engine/src/pricing.test.ts
import { describe, it, expect } from 'vitest';
import { calculateCost, MODEL_PRICING } from './pricing';
 
describe('calculateCost', () => {
  it('calculates Claude Sonnet cost correctly', () => {
    // $3.00 / 1M input + $15.00 / 1M output
    const cost = calculateCost('claude-sonnet-4-6', 1_000_000, 1_000_000);
    expect(cost).toBe(18.00);
  });
 
  it('returns $0 for local Ollama models', () => {
    expect(calculateCost('gemma3:4b', 50_000, 20_000)).toBe(0);
  });
 
  it('returns $0 for unknown model (safe default)', () => {
    expect(calculateCost('unknown-model', 1_000, 1_000)).toBe(0);
  });
 
  it('handles zero tokens', () => {
    expect(calculateCost('claude-sonnet-4-6', 0, 0)).toBe(0);
  });
});

Tier 2 — Integration Tests (Vitest + Docker)

What: Tests that cross package boundaries and touch real infrastructure. Where: tests/integration/ directory. Infrastructure: Docker Compose spins up PostgreSQL + MongoDB + Redis before the suite runs. Containers are torn down after. Speed: 30s–3 minutes per suite depending on DB seed size. Command: pnpm test:integration

CI Setup


# .github/workflows/integration.yml
jobs:
  integration:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env: { POSTGRES_DB: dmagency_test, POSTGRES_USER: dmagency, POSTGRES_PASSWORD: dmagency }
        options: --health-cmd pg_isready --health-interval 5s
      mongo:
        image: mongo:7
        env: { MONGO_INITDB_ROOT_USERNAME: dmagency, MONGO_INITDB_ROOT_PASSWORD: dmagency }
        options: --health-cmd "mongosh --eval db.adminCommand('ping')"
      redis:
        image: redis:7-alpine
        options: --health-cmd "redis-cli ping"
    steps:
      - uses: actions/checkout@v4
      - run: pnpm install
      - run: pnpm db:migrate:test
      - run: pnpm test:integration

What gets integration tested

Database layer


// tests/integration/db/campaigns.test.ts
describe('campaign repository', () => {
  it('creates a campaign scoped to the correct tenant', async () => {
    const tenant = await seedTenant('test-tenant');
    const campaign = await createCampaign({ tenantId: tenant.id, name: 'Test' });
 
    // Querying with a different tenantId must return nothing
    const result = await getCampaign(campaign.id, 'different-tenant-id');
    expect(result).toBeNull();  // tenant isolation enforced
  });
 
  it('increments totalCostUsd atomically', async () => {
    // Concurrent updates should not lose increments
    await Promise.all([
      incrementCampaignCost(campaignId, 0.10),
      incrementCampaignCost(campaignId, 0.15),
    ]);
    const campaign = await getCampaign(campaignId, tenantId);
    expect(campaign.totalCostUsd).toBe(0.25);
  });
});

BullMQ queue lifecycle


// tests/integration/queue/task-lifecycle.test.ts
describe('task queue', () => {
  it('routes task to correct agent queue', async () => {
    await enqueueTask({ agentType: 'copywriter', tenantId, taskId });
    const jobs = await copywriterQueue.getWaiting();
    expect(jobs[0].data.taskId).toBe(taskId);
  });
 
  it('moves to dead letter after max retries', async () => {
    const task = await enqueueTask({ agentType: 'copywriter', maxRetries: 2 });
    // Simulate failures
    await failJob(task.jobId, 'LLM timeout');
    await failJob(task.jobId, 'LLM timeout');
    await failJob(task.jobId, 'LLM timeout');
 
    const dlq = await deadLetterQueue.getCompleted();
    expect(dlq).toHaveLength(1);
  });
});

API contract tests


// tests/integration/api/campaigns.test.ts
describe('POST /campaigns', () => {
  it('returns 401 without auth token', async () => {
    const res = await fetch('http://localhost:3001/campaigns', { method: 'POST' });
    expect(res.status).toBe(401);
  });
 
  it('creates campaign and returns 201 with tenant scope', async () => {
    const res = await fetch('http://localhost:3001/campaigns', {
      method: 'POST',
      headers: { Authorization: `Bearer ${tenantToken}` },
      body: JSON.stringify({ name: 'Test', brief: '...', types: ['blog'] }),
    });
    const data = await res.json();
    expect(res.status).toBe(201);
    expect(data.tenantId).toBe(tenantId);
  });
 
  it('cannot access another tenants campaigns', async () => {
    const res = await fetch(`http://localhost:3001/campaigns/${otherTenantCampaignId}`, {
      headers: { Authorization: `Bearer ${tenantToken}` },
    });
    expect(res.status).toBe(404);  // not 403 — do not reveal existence
  });
});

Approval gate enforcement


// tests/integration/governance/approval-gate.test.ts
describe('approval gate', () => {
  it('blocks tool dispatch without an approved approval record', async () => {
    await expect(
      dispatchToolCall('wordpress', 'createPost', payload, taskRun)
    ).rejects.toThrow('ApprovalRequired');
  });
 
  it('allows tool dispatch after approval', async () => {
    await createApproval({ taskId, status: 'approved' });
    await expect(
      dispatchToolCall('wordpress', 'createPost', payload, taskRun)
    ).resolves.toBeDefined();
  });
});

Tier 3 — E2E Tests (Playwright)

What: Full user journeys through the UI against a running stack. Where: tests/e2e/ directory. Infrastructure: Full Docker Compose stack (Next.js + Fastify + DBs + Redis). LLM calls are intercepted and return fixture responses — we do not call real LLM APIs in E2E. Speed: 2–10 minutes for the full suite. Command: pnpm test:e2e

LLM Mocking in E2E

Real LLM calls are replaced by a deterministic mock adapter in the E2E environment:


// packages/agent-engine/src/adapters/mock.ts
export class MockAdapter implements AgentAdapter {
  async dispatch(task: AdapterTask): Promise<DispatchResult> {
    // Return fixture response based on task type
    const fixture = E2E_FIXTURES[task.agentRole]?.[task.type];
    if (!fixture) throw new Error(`No E2E fixture for ${task.agentRole}/${task.type}`);
 
    // Simulate the phone-home callback
    await fetch(task.callbackUrl, {
      method: 'POST',
      body: JSON.stringify({ status: 'completed', output: fixture.text, usage: fixture.usage }),
    });
 
    return { jobId: crypto.randomUUID() };
  }
}

The mock adapter is activated when NODE_ENV=test — no code change required in the main flow.

Coverage Areas

Authentication & tenant isolation


// tests/e2e/auth.spec.ts
test('tenant A cannot see tenant B campaigns', async ({ page, context }) => {
  await loginAs(page, 'tenant-a-user');
  const tenantBCampaignUrl = `/campaigns/${TENANT_B_CAMPAIGN_ID}`;
  await page.goto(tenantBCampaignUrl);
  await expect(page).toHaveURL('/campaigns');  // redirected away
  await expect(page.locator('[data-testid="error-toast"]')).toBeVisible();
});

Campaign submission flow


// tests/e2e/campaigns/submit.spec.ts
test('submit campaign → agents run → deliverable appears', async ({ page }) => {
  await loginAs(page, 'acme-user');
  await page.goto('/campaigns/new');
 
  await page.selectOption('[data-testid="client-select"]', 'acme-corp');
  await page.fill('[data-testid="campaign-name"]', 'E2E Test Campaign');
  await page.fill('[data-testid="brief-textarea"]', 'Write a blog post about...');
  await page.check('[data-testid="type-blog"]');
  await page.click('[data-testid="submit-campaign"]');
 
  // Should redirect to campaign detail
  await expect(page).toHaveURL(/\/campaigns\/[a-z0-9-]+/);
 
  // Tasks tab should show running agent
  await page.click('[data-testid="tasks-tab"]');
  await expect(page.locator('[data-testid="task-status-running"]')).toBeVisible({ timeout: 10_000 });
 
  // Wait for completion (mock adapter is fast)
  await expect(page.locator('[data-testid="task-status-completed"]')).toBeVisible({ timeout: 15_000 });
 
  // Deliverable should appear
  await page.click('[data-testid="deliverables-tab"]');
  await expect(page.locator('[data-testid="deliverable-item"]')).toHaveCount(1);
});

Approval gate flow


// tests/e2e/approvals/gate.spec.ts
test('deliverable requires approval before appearing as published', async ({ page }) => {
  // ... setup: campaign runs, deliverable created ...
 
  await page.goto('/approvals');
  await expect(page.locator('[data-testid="approval-pending"]')).toHaveCount(1);
 
  // Approve
  await page.click('[data-testid="approval-approve-btn"]');
  await page.click('[data-testid="confirm-approve"]');
 
  await expect(page.locator('[data-testid="approval-pending"]')).toHaveCount(0);
  await expect(page.locator('[data-testid="approval-approved"]')).toHaveCount(1);
});

Org chart visibility


// tests/e2e/team/org-chart.spec.ts
test('org chart shows both human users and agents', async ({ page }) => {
  await page.goto('/team/org-chart');
  await expect(page.locator('[data-testid="principal-human"]')).toHaveCount(greaterThan(0));
  await expect(page.locator('[data-testid="principal-agent"]')).toHaveCount(greaterThan(0));
});

Test Data & Fixtures

Database seeds


packages/db/src/seeds/
├── test-tenant.ts       # Minimal tenant with 1 user
├── full-tenant.ts       # Tenant with campaigns, agents, deliverables, costs
├── multi-tenant.ts      # Two tenants for isolation tests
└── e2e-tenant.ts        # E2E-specific data with known IDs

Seed is run before integration and E2E test suites:


pnpm db:seed:test

E2E fixtures (LLM mock responses)


tests/e2e/fixtures/
├── activity-planner-decompose.json
├── copywriter-blog-post.json
├── seo-specialist-brief.json
├── social-media-manager-calendar.json
└── data-analyst-report.json

Coverage Targets

Tier	Coverage target	Enforced
Unit	90% line coverage on `packages/*`	Yes — CI fails below 90%
Integration	All DB repositories, all API routes, approval gate	Yes — required in CI
E2E	Critical user journeys (submit, approve, logs)	Yes — required before deploy

Next.js App Unit Testing (apps/dm, apps/manage)

Each Next.js app has its own Vitest setup independent of the monorepo root.

vitest.config.ts


import { defineConfig } from "vitest/config";
import path from "path";
 
export default defineConfig({
  test: {
    environment: "node",
    globals: true,
    setupFiles: ["./src/__tests__/setup.ts"],
    include: ["src/__tests__/**/*.test.{ts,tsx}"],
    testTimeout: 15_000,
  },
  resolve: {
    alias: { "@": path.resolve(__dirname, "./src") },
  },
});

setup.ts — always mock these Next.js modules


import { vi } from "vitest";
 
vi.mock("next/headers", () => ({
  headers: vi.fn(() => new Headers()),
  cookies: vi.fn(() => ({ get: vi.fn() })),
}));
 
vi.mock("next/navigation", () => ({
  redirect: vi.fn((url: string) => {
    throw Object.assign(new Error("NEXT_REDIRECT"), { digest: `NEXT_REDIRECT;${url}` });
  }),
  notFound: vi.fn(),
}));
 
vi.mock("next/cache", () => ({
  revalidatePath: vi.fn(),
  revalidateTag: vi.fn(),
}));

Without these, tests that import Server Actions or route handlers will throw “next/headers is not available” at import time.

vi.hoisted() — required for mocks referenced inside vi.mock() factories

vi.mock() factories are hoisted before const declarations. Any mock object shared between the factory and test assertions must use vi.hoisted():


// ✗ WRONG — "Cannot access 'mockDb' before initialization"
const mockDb = { invoice: { count: vi.fn() } };
vi.mock("@leadmetrics/db", () => ({ db: mockDb }));
 
// ✓ CORRECT
const mockDb = vi.hoisted(() => ({
  invoice: { count: vi.fn(), findUnique: vi.fn() },
  $transaction: vi.fn(async (cb) => cb(mockDb)),
  $executeRaw: vi.fn().mockResolvedValue(undefined),
}));
vi.mock("@leadmetrics/db", () => ({ db: mockDb }));

vi.stubEnv() — for controlling environment variables

Never use Object.defineProperty(process.env, "NODE_ENV", ...) in Vitest — it throws because Vitest already owns those property descriptors. Use vi.stubEnv() instead:


afterEach(() => { vi.unstubAllEnvs(); });  // always restore in afterEach
 
it("sets secure:true in production", () => {
  vi.stubEnv("NODE_ENV", "production");
  // ... test
});

Test scripts


"test:unit": "vitest run",
"test:unit:watch": "vitest"


pnpm --filter @leadmetrics/dm test:unit
pnpm --filter @leadmetrics/manage test:unit

Running Tests Locally


# Unit tests (no Docker needed)
pnpm test:unit
 
# Integration tests (Docker must be running)
pnpm test:integration
 
# E2E tests (full stack in Docker Compose)
pnpm test:e2e
 
# All tests
pnpm test
 
# Watch mode (unit only)
pnpm test:unit --watch
 
# Run a specific E2E spec
pnpm test:e2e -- tests/e2e/campaigns/submit.spec.ts