AWS S3

Category: Storage
Integration type: Platform-level (single S3 bucket; enterprise tenants may bring their own)
External SDK: @aws-sdk/client-s3, @aws-sdk/s3-request-presigner

Purpose

AWS S3 is the platform’s file storage layer. All binary assets that are too large or ephemeral for the database go to S3:

Asset type	S3 key pattern	Retention
Email attachments (reports, invoices)	`attachments/{tenantId}/{type}/{date}/{filename}`	90 days
Contract PDFs	`contracts/{tenantId}/{contractId}.pdf`	Permanent
Blog featured images	`media/{tenantId}/blog/{postId}/{filename}`	Permanent
Knowledge base file uploads	`rag/{tenantId}/{datasetId}/{filename}`	Until deleted
Agent output exports	`exports/{tenantId}/{activityId}/{filename}`	30 days
Tenant logo / brand assets	`brand/{tenantId}/{filename}`	Permanent

Config Structure

Platform config (env vars)


AWS_ACCESS_KEY_ID=AKIAxxxxxxxxxxxxxxxx
AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
AWS_REGION=ap-south-1
S3_BUCKET=leadmetrics-platform
S3_CDN_BASE_URL=https://cdn.leadmetrics.io   # CloudFront distribution in front of S3

Enterprise tenant bring-your-own S3

Enterprise on-premise deployments can configure their own S3 bucket (or compatible object storage like MinIO):


interface S3Config {
  accessKeyId:     string;
  secretAccessKey: string;
  region:          string;
  bucket:          string;
  endpoint?:       string;   // Override for MinIO or other S3-compatible stores
  cdnBaseUrl?:     string;   // Optional CDN in front of bucket
}

Integration Pattern

S3 utility (`packages/storage/src/s3.ts`)


import {
  S3Client,
  PutObjectCommand,
  GetObjectCommand,
  DeleteObjectCommand,
} from '@aws-sdk/client-s3';
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';
 
class S3Storage {
  private client: S3Client;
 
  constructor(private cfg: S3Config) {
    this.client = new S3Client({
      region:      cfg.region,
      credentials: {
        accessKeyId:     cfg.accessKeyId,
        secretAccessKey: cfg.secretAccessKey,
      },
      endpoint: cfg.endpoint,   // undefined = use default AWS endpoint
    });
  }
 
  async upload(options: {
    key:         string;
    body:        Buffer | ReadableStream;
    contentType: string;
    metadata?:   Record<string, string>;
  }): Promise<{ key: string; url: string }> {
    await this.client.send(new PutObjectCommand({
      Bucket:      this.cfg.bucket,
      Key:         options.key,
      Body:        options.body,
      ContentType: options.contentType,
      Metadata:    options.metadata,
    }));
    return {
      key: options.key,
      url: `${this.cfg.cdnBaseUrl ?? `https://${this.cfg.bucket}.s3.${this.cfg.region}.amazonaws.com`}/${options.key}`,
    };
  }
 
  async download(key: string): Promise<Buffer> {
    const response = await this.client.send(new GetObjectCommand({
      Bucket: this.cfg.bucket,
      Key:    key,
    }));
    const stream = response.Body as NodeJS.ReadableStream;
    const chunks: Buffer[] = [];
    for await (const chunk of stream) chunks.push(Buffer.from(chunk));
    return Buffer.concat(chunks);
  }
 
  async presignedUrl(key: string, expiresIn = 3600): Promise<string> {
    return getSignedUrl(
      this.client,
      new GetObjectCommand({ Bucket: this.cfg.bucket, Key: key }),
      { expiresIn },
    );
  }
 
  async delete(key: string): Promise<void> {
    await this.client.send(new DeleteObjectCommand({
      Bucket: this.cfg.bucket,
      Key:    key,
    }));
  }
}

Pre-signed URL usage

For time-limited download links (contract PDFs, report exports), the platform uses pre-signed URLs:


// In report delivery handler
const reportKey = `exports/${tenantId}/${activityId}/report-march-2026.pdf`;
const downloadUrl = await s3.presignedUrl(reportKey, 60 * 60 * 24 * 7); // 7 days
 
await notifyTenant(tenantId, 'monthly_report_ready', {
  reportUrl: downloadUrl,
  month:     'March 2026',
});

Knowledge base uploads

Files uploaded to the Knowledge Base (CSV, DOCX, PDF) are stored in S3 before being queued for RAG ingestion:


// POST /api/dashboard/knowledge-base/datasets/:id/files
const key = `rag/${tenantId}/${datasetId}/${uuid()}-${filename}`;
await s3.upload({ key, body: req.file.buffer, contentType: req.file.mimetype });
 
// Enqueue for ingestion
await ragIngestionQueue.add('ingest-file', { tenantId, datasetId, key });

Lifecycle Rules

S3 lifecycle rules (configured in AWS Console or IaC) handle automatic cleanup:

Prefix	Action	After
`attachments/`	Delete	90 days
`exports/`	Delete	30 days
All	Transition to Glacier	1 year

Permanent assets (contracts/, rag/, media/, brand/) have no expiry rule.

Test Cases

Unit tests (`packages/storage/src/s3.test.ts`)

Test	Approach
`upload()` calls `PutObjectCommand` with correct params	Mock `S3Client.send`; assert `Bucket`, `Key`, `ContentType`
`download()` reads stream and returns Buffer	Mock `GetObjectCommand` returns readable stream; assert Buffer
`presignedUrl()` returns URL with expiry	Mock `getSignedUrl`; assert URL returned
`delete()` calls `DeleteObjectCommand`	Assert correct `Key`
Throws on access denied (403)	Mock `S3Client.send` throws `AccessDenied`; assert propagated
Uses custom endpoint for MinIO	Assert `endpoint` passed to `S3Client` constructor

Integration tests

Test	Approach
Upload and download round-trip	Upload buffer; download by key; assert content matches
Pre-signed URL is accessible	Generate URL; fetch via `axios.get`; assert 200
Delete removes object	Upload then delete; try download; assert 404

Local development with MinIO


# docker-compose.dev.yml
minio:
  image: minio/minio
  command: server /data --console-address ":9001"
  ports:
    - "9000:9000"   # S3 API
    - "9001:9001"   # Web console
  environment:
    MINIO_ROOT_USER:     minioadmin
    MINIO_ROOT_PASSWORD: minioadmin

Set S3_ENDPOINT=http://minio:9000 and S3_BUCKET=leadmetrics-dev in .env.dev. Create the bucket via MinIO console on first run.

SendGrid Provider — uses S3 for email attachment storage
RAG Architecture — S3 for knowledge base file uploads
Infrastructure — S3 in Docker Compose / Coolify setup