Loki + Grafana Setup Verification Results

✅ Completed Setup

1. Infrastructure Running

• Loki: ✅ Running on port 3100, healthy (http://localhost:3100/ready  returns “ready”)
• Grafana: ✅ Running on port 3200 (http://localhost:3200 )
• API: ✅ Running on port 3003

2. Dependencies Installed

• pino-loki: ✅ Installed in @leadmetrics/logger package

3. Configuration Updated

• Logger code: ✅ Updated to support lokiEndpoint parameter
• .env file: ✅ Created with LOKI_ENDPOINT=http://localhost:3100
• loki-config.yml: ✅ Fixed volume permissions (changed from /tmp/loki to /loki)
• grafana-datasources.yml: ✅ Pre-configured Loki as datasource

4. Grafana UI

• ✅ Accessible at http://localhost:3200/explore 
• ✅ Loki datasource pre-configured and working
• ✅ Anonymous auth enabled (no login required)

⚠️ Current Limitation

Logs are NOT yet flowing to Loki because:

The API currently uses two different logging approaches:

1. Fastify’s built-in logger (in apps/api/src/index.ts):

   const fastify = Fastify({ logger: { level: "info" } });

   • This logs HTTP requests/responses
   • Does NOT send to Loki (only stdout)

2. Custom logger (from @leadmetrics/logger):

   const log = createLogger({ 
     service: "api",
     lokiEndpoint: process.env.LOKI_ENDPOINT 
   });

   • DOES send to Loki
   • Currently only used for startup errors

🔧 To Fix: Integrate Custom Logger with Fastify

Option 1: Replace Fastify Logger (Recommended)

Update apps/api/src/index.ts to use the custom logger with Fastify:

import { createLogger } from "@leadmetrics/logger";
 
const pinoLogger = createLogger({ 
  service: "api",
  lokiEndpoint: process.env.LOKI_ENDPOINT,
});
 
const fastify = Fastify({ 
  logger: pinoLogger,  // Use custom logger instead of { level: "info" }
  // ... other options
});

Option 2: Add Request Logging Hook

Keep Fastify’s logger for HTTP and add custom logging:

import { createLogger } from "@leadmetrics/logger";
 
const customLog = createLogger({ 
  service: "api",
  lokiEndpoint: process.env.LOKI_ENDPOINT,
});
 
// Log all requests to Loki
fastify.addHook('onRequest', async (request) => {
  customLog.info({ 
    method: request.method,
    url: request.url,
  }, 'HTTP request');
});
 
fastify.addHook('onResponse', async (request, reply) => {
  customLog.info({
    method: request.method,
    url: request.url,
    statusCode: reply.statusCode,
  }, 'HTTP response');
});

📊 Testing After Fix

Once integrated, test by:

1. Making API requests:

   curl http://localhost:3003/health

2. Wait 5-10 seconds for log batching

3. Query Loki:

   curl "http://localhost:3100/loki/api/v1/query?query={service=\"api\"}"

4. View in Grafana:

   • Open http://localhost:3200/explore 
   • Enter query: {service="api"}
   • Click “Run query”

🎯 Verified Components

📸 Screenshots Captured

• grafana-home.png - Grafana homepage
• grafana-explore.png - Grafana Explore view with Loki datasource

Next Steps

1. Integrate custom logger with Fastify (see options above)
2. Restart API
3. Generate logs by making requests
4. Verify in Grafana that logs appear
5. Create dashboards for monitoring

Setup Date: April 26, 2026
Test Environment: Local development