Redis

Category: Infrastructure
Integration type: Platform-level (sidecar service in Docker Compose)
External SDK: ioredis (via BullMQ)

Purpose

Redis is the backbone of the async task system. Every queue, cache, and ephemeral key in the platform flows through Redis:

Use	Description
BullMQ queue backend	All job queues store jobs, results, and delays in Redis
Notification dedup	`notifyOnce()` keys with 24h TTL prevent duplicate sends
Rate limiting	Per-tenant API call rate limiters
Session cache	Short-lived data within active user sessions
`--resume` session IDs	Claude session IDs cached for warm retrieval
Activity locking	Distributed lock while an activity is being processed

Config Structure

Platform config (env vars)


REDIS_URL=redis://redis:6379           # Docker service name in Compose
REDIS_PASSWORD=                        # Optional; set in production
REDIS_DB=0                             # Database index (0–15)

For production with Redis Sentinel or Cluster:


REDIS_SENTINEL_HOSTS=sentinel1:26379,sentinel2:26379
REDIS_SENTINEL_NAME=mymaster
REDIS_PASSWORD=your_redis_password

Integration Pattern

BullMQ connection (`packages/queue/src/connection.ts`)


import IORedis from 'ioredis';
 
export const redisConnection = new IORedis(config.REDIS_URL, {
  password:          config.REDIS_PASSWORD || undefined,
  maxRetriesPerRequest: null,   // Required by BullMQ
  enableReadyCheck:  false,     // Required by BullMQ
  lazyConnect:       true,
});

The maxRetriesPerRequest: null and enableReadyCheck: false settings are required by BullMQ and must not be changed.

Queue naming convention

All queues use a {namespace}__{name} double-underscore separator (BullMQ v5 does not allow colons in queue names). Queues are shared across all tenants — tenant isolation is via tenantId in the job payload, not separate queues.


agent__{agentRole}     One queue per agent role, all tenants share it
rag__ingestion         RAG document processing (all tenants)
notifications__{channel}  Notification delivery by channel (all tenants)

Current queues:


agent__activity-planner
agent__deliverable-planner
agent__blog-writer
agent__content-brief-writer
agent__keyword-researcher
agent__social-post-writer
...all other agent roles...
rag__ingestion
notifications__email
notifications__sms
notifications__whatsapp
notifications__telegram
notifications__web

Notification deduplication keys


// Pattern: notify:{tenantId}:{type}:{dedupeKey}
const key = `notify:${tenantId}:${type}:${dedupeKey}`;
const exists = await redis.exists(key);
if (exists) return;  // Already sent
 
await redis.setex(key, 86400, '1');  // 24h TTL
// Then enqueue the notification

Rate limiting


// Per-tenant API call rate limiting for external integrations
const key = `ratelimit:${tenantId}:semrush`;
const count = await redis.incr(key);
if (count === 1) await redis.expire(key, 60);  // 1-minute window
if (count > 10) throw new RateLimitError('SEMrush rate limit exceeded (10/min)');

Redis Data TTLs

Key pattern	TTL	Purpose
`notify:{tenantId}:{type}:{key}`	24 hours	Notification dedup
`ratelimit:{tenantId}:{provider}`	60 seconds	API rate limit window
`session:{sessionId}`	2 hours	User session data
`claude_session:{activityId}`	7 days	Claude session ID cache
BullMQ completed jobs	24 hours	BullMQ default for completed jobs
BullMQ failed jobs	7 days	Retained for debugging

Docker Compose Setup


# docker-compose.yml
services:
  redis:
    image: redis:7.2-alpine
    command: redis-server --save 20 1 --loglevel warning
    volumes:
      - redis_data:/data
    ports:
      - "6379:6379"
 
volumes:
  redis_data:

The --save 20 1 flag enables RDB persistence (save if ≥1 key changed in 20 seconds). For production, also enable AOF (--appendonly yes) for durability.

Test Cases

Unit tests

Redis is a shared-state dependency. Unit tests that involve queues or dedup should use ioredis-mock:


import RedisMock from 'ioredis-mock';
jest.mock('ioredis', () => RedisMock);

Integration tests

Test	Approach
Notification dedup key set with 24h TTL	Set key; assert `redis.ttl(key)` between 86340 and 86400
Rate limit triggers on 11th call	Call 11 times; assert `RateLimitError` on 11th
BullMQ job enqueue and consume	Start worker; enqueue job; assert worker processes it within timeout

Task Queue & Orchestration — BullMQ design patterns
Notifications Service — notification dedup and queues
Infrastructure — Docker Compose, Redis in production