GitHub Channel — Architecture & Roadmap
Full design for the GitHub source code channel: data model, OAuth flow, provider package, API routes, workers, coding agent, and dashboard UI — delivered in five phases.
Architecture Overview
The GitHub integration has three layers:
Layer 1 — GitHub channel (ConnectedChannel of type "GitHub") stores the OAuth token and selected repository. This is a standard channel — it appears on the Channels page and has a detail view like any other channel.
Layer 2 — Source map (ChannelSourceMap) links a specific GitHub channel + file path(s) to an existing Website or LandingPage channel. This is the mapping that makes the integration meaningful: “this repo file is the source code for this landing page the platform tracks.”
Layer 3 — Code change (AgentCodeChange) records a DM-approved set of recommendations that a coding agent will apply to the source files. Each record tracks status from pending_review through to a live PR URL.
One GitHub channel can map to multiple website/landing page channels. One website channel can have at most one GitHub source map at a time.
Data Model
ChannelSourceMap
model ChannelSourceMap {
id String @id @default(cuid())
tenantId String
githubChannelId String // ConnectedChannel of type "GitHub"
targetChannelId String // ConnectedChannel of type Website or LandingPage
repoOwner String // GitHub repo owner (user or org)
repoName String // Repository name
defaultBranch String @default("main")
filePaths String[] // Specific file paths to track (e.g. src/pages/index.tsx)
framework String? // Detected on first sync: "nextjs-app" | "nextjs-pages" | "nuxt" | "vue" | "gatsby" | "html"
webhookId Int? // GitHub webhook ID (set in Phase 5)
webhookSecret String? // HMAC secret for webhook verification (encrypted, set in Phase 5)
extractedAt DateTime? // When content was last successfully extracted
lastSyncedAt DateTime? // When the last sync job completed
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
tenant Tenant @relation(fields: [tenantId], references: [id])
githubChannel ConnectedChannel @relation("GitHubSourceChannel", fields: [githubChannelId], references: [id])
targetChannel ConnectedChannel @relation("GitHubTargetChannel", fields: [targetChannelId], references: [id])
codeChanges AgentCodeChange[]
@@unique([targetChannelId]) // One source map per target channel
@@index([tenantId])
@@index([githubChannelId])
}AgentCodeChange
model AgentCodeChange {
id String @id @default(cuid())
tenantId String
sourceMappingId String
status String // "pending_review" | "approved" | "running" | "pr_opened" | "merged" | "failed" | "cancelled"
recommendations Json // RecommendationItem[] — see types below
targetFiles String[] // File paths the agent will read and potentially modify
prUrl String? // Set when PR is created
prNumber Int?
branchName String? // e.g. "leadmetrics/seo-opt-20260423-1430-abc123"
agentLog String? @db.Text // Streamed log from the coding agent run
requestedByUserId String
approvedByUserId String?
approvedAt DateTime?
completedAt DateTime?
errorMessage String?
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
tenant Tenant @relation(fields: [tenantId], references: [id])
sourceMap ChannelSourceMap @relation(fields: [sourceMappingId], references: [id])
requestedBy User @relation("CodeChangeRequester", fields: [requestedByUserId], references: [id])
approvedBy User? @relation("CodeChangeApprover", fields: [approvedByUserId], references: [id])
@@index([tenantId])
@@index([sourceMappingId])
@@index([tenantId, status])
}Recommendation types
type RecommendationItem = {
text: string // The recommendation instruction for the coding agent
source: RecommendationSource
targetFilePath?: string // Which file this applies to (if known at push time)
priority: "high" | "medium" | "low"
}
type RecommendationSource = {
type: "github-insights" // From the GitHub AI Insights worker
| "site-auditor" // From the Site Auditor deliverable
| "content-brief" // From a Content Brief
| "keyword-researcher" // From a keyword researcher output
| "manual" // Typed by the DM directly
insightId?: string // ChannelInsight.id if source is github-insights
activityId?: string // Activity.id if source is a deliverable
contentBriefId?: string // ContentBrief.id if source is content-brief
}ChannelMaster seed row
// packages/db/prisma/seed.ts — add to CHANNEL_CATALOGUE array
{
type: "GitHub",
name: "GitHub",
description: "Map repository source files to landing pages and websites",
iconKey: "github",
categories: ["Source Control"],
authenticationType: "OAuth",
requiresUrl: false,
isActive: true,
}OAuth App
Leadmetrics registers one GitHub OAuth App (not a GitHub App). All tenants authenticate through it.
| Setting | Value |
|---|---|
| App type | GitHub OAuth App (oauth_app) |
| Authorization callback URL | {API_BASE_URL}/tenant/v1/channel-connect/github/callback |
| Scopes requested | repo + admin:repo_hook |
| Env vars | GITHUB_CLIENT_ID, GITHUB_CLIENT_SECRET |
repo scope grants read and write access to all repos the user authorizes. admin:repo_hook is needed only for Phase 5 webhook registration — it can be dropped from the initial scope request if Phase 5 is deferred (the user would need to re-authorize when webhooks are added).
Token storage follows the same pattern as Google OAuth channels: tokenInfo on ConnectedChannel stores { accessToken, refreshToken, expireOn, scope } encrypted via @leadmetrics/crypto.
Provider Package — packages/providers/github/
File structure
packages/providers/github/
package.json
tsconfig.json
src/
index.ts ← exports GitHubService + all types
types.ts ← OAuthInfo, GitHubRepo, GitHubTreeEntry, GitHubCommit, etc.
github.ts ← GitHubService classGitHubService API
All methods use plain fetch with Authorization: Bearer {accessToken}. No Octokit dependency.
class GitHubService {
constructor(
private config: { clientId: string; clientSecret: string },
private accessToken?: string
) {}
// ── OAuth ──────────────────────────────────────────────────────────────────
// Returns the GitHub authorization URL — user is redirected here
authorize(callbackUrl: string, state: string): string
// Exchanges authorization code for access + refresh tokens
authorizeCallback(code: string, callbackUrl: string): Promise<OAuthInfo>
// Refreshes an expired access token (GitHub OAuth tokens don't expire by default,
// but fine-grained tokens can — kept for forward compatibility)
refreshToken(refreshToken: string): Promise<OAuthInfo>
// Returns the authenticated user's login and display name
getAuthenticatedUser(): Promise<{ login: string; name: string }>
// ── Repository access ─────────────────────────────────────────────────────
// List all repos accessible to the token (paginated, returns up to 100)
listRepos(): Promise<GitHubRepo[]>
// Get the default branch name for a repo
getDefaultBranch(owner: string, repo: string): Promise<string>
// Recursively list all files in a repo tree (flattened — no directories)
getFileTree(owner: string, repo: string, branch: string): Promise<GitHubTreeEntry[]>
// Fetch decoded content of a single file
getFileContent(owner: string, repo: string, path: string, branch: string): Promise<string>
// Get recent commits for a file path (for change tracking)
getCommitHistory(owner: string, repo: string, path: string, limit?: number): Promise<GitHubCommit[]>
// Get repo-level stats returned in the analytics endpoint
getRepoStats(owner: string, repo: string): Promise<GitHubRepoStats>
// ── Code write-back (used by coding agent) ────────────────────────────────
// Create a new branch from the tip of baseBranch
createBranch(owner: string, repo: string, branchName: string, baseBranch: string): Promise<void>
// Create or update a file on a branch (handles base64 encoding + SHA for updates)
putFile(owner: string, repo: string, path: string, branch: string, content: string, message: string): Promise<void>
// Open a pull request
createPR(owner: string, repo: string, params: CreatePRParams): Promise<{ number: number; url: string }>
// ── Webhooks (Phase 5) ────────────────────────────────────────────────────
createWebhook(owner: string, repo: string, callbackUrl: string, secret: string): Promise<{ id: number }>
deleteWebhook(owner: string, repo: string, webhookId: number): Promise<void>
verifyWebhookSignature(payload: string, signature: string, secret: string): boolean
}Types
type OAuthInfo = {
accessToken: string
refreshToken?: string
expireOn?: string
scope: string
}
type GitHubRepo = {
owner: string
name: string
fullName: string // "owner/repo"
defaultBranch: string
isPrivate: boolean
pushedAt: string // ISO date
}
type GitHubTreeEntry = {
path: string
type: "blob" | "tree"
size?: number
}
type GitHubCommit = {
sha: string
message: string
authorName: string
committedAt: string
}
type GitHubRepoStats = {
openPRs: number
commitCount30d: number
lastPushedAt: string
}
type CreatePRParams = {
head: string // branch name
base: string // target branch (usually "main")
title: string
body: string
}API Routes
OAuth flow — apps/api/src/routers/channel-connect.ts
New entries added to the existing channel-connect router:
| Method | Path | Auth | What it does |
|---|---|---|---|
GET | /github/connect | JWT required | Returns GitHub OAuth authorization URL with state={channelId} |
GET | /github/callback | None (OAuth callback) | Receives code, exchanges for token, saves encrypted tokenInfo + userInfo on channel, redirects to repo select |
GET | /github/repo/select?state={channelId} | None | Returns HTML page listing accessible repos (same popup pattern as GSC/GA4 site selection) |
POST | /github/repo/select | None | Saves selected subChannelInfo{owner, repo, defaultBranch}, sets isConnected = true, closes popup |
After POST /github/repo/select completes, the channel enters the connected state. No post-connect workers are enqueued here — first sync is triggered after the user creates a source map.
Source maps router — apps/api/src/routers/source-maps.ts
Registered at /tenant/v1/source-maps in both app.ts and index.ts.
Source map CRUD:
| Method | Path | What it does |
|---|---|---|
GET | / | List all ChannelSourceMap records for the tenant |
GET | /:id | Single source map with last sync status and code change count |
POST | / | Create — validates both channels belong to tenant; validates token can access the repo |
PATCH | /:id | Update filePaths or defaultBranch |
DELETE | /:id | Delete source map and deregister webhook if set; does not delete either channel |
POST | /:id/sync | Trigger manual re-sync — enqueues github-source-sync job |
Code change lifecycle:
| Method | Path | What it does |
|---|---|---|
GET | /:id/code-changes | List all AgentCodeChange records for a source map |
GET | /:id/code-changes/:changeId | Single change with full recommendation list and agent log |
POST | /:id/code-changes | Create a new code change request (status: pending_review) |
POST | /:id/code-changes/:changeId/approve | DM approves — sets status to approved, enqueues github-code-agent job |
POST | /:id/code-changes/:changeId/cancel | Cancel a pending or approved change before the agent runs |
Repository browser — apps/api/src/routers/channels.ts
Added for GitHub channel type only:
| Method | Path | What it does |
|---|---|---|
GET | /:id/repos | List all repos accessible to the channel’s OAuth token |
GET | /:id/repos/:owner/:repo/tree | File tree for a repo + branch (used by file picker in UI) |
Analytics case
In GET /tenant/v1/channels/:id/analytics — add "GitHub" dispatch:
case "GitHub": {
const token = decrypt(channel.tokenInfo).accessToken
const svc = new GitHubService(config, token)
const stats = await svc.getRepoStats(subChannelInfo.owner, subChannelInfo.repo)
const maps = await db.channelSourceMap.findMany({ where: { githubChannelId: id } })
const pendingChanges = await db.agentCodeChange.count({
where: { sourceMappingId: { in: maps.map(m => m.id) }, status: "pending_review" }
})
return { stats, sourceMaps: maps, pendingChanges }
}OAuth Connection Flow
1. User opens Channels page → clicks "Connect GitHub"
2. ConnectDialog: user enters display name → clicks "Connect"
→ ConnectDialog calls GET /tenant/v1/channel-connect/github/connect
→ Opens OAuth popup window pointing to GitHub authorization URL
3. GitHub authorization page:
→ User reviews requested scopes (repo + admin:repo_hook)
→ User clicks "Authorize Leadmetrics"
4. GitHub redirects to /github/callback with code
→ Route exchanges code for access token via GitHub token endpoint
→ Saves encrypted tokenInfo + userInfo on ConnectedChannel
→ Redirects popup to /github/repo/select?state={channelId}
5. Repo selection page (inside popup):
→ Lists all repos the token can access
→ User selects one repo
→ POST /github/repo/select saves subChannelInfo{owner, repo, defaultBranch}
→ isConnected = true, connectionHistory entry written
→ Popup posts message to parent window → popup closes
6. ConnectDialog detects popup closed → navigates to GitHub channel detail page
7. User opens "Source Maps" tab → clicks "Add Source Map"
8. Source map modal:
a. "Target page" — dropdown of all Website/LandingPage channels for the tenant
b. "Repository" — pre-filled from subChannelInfo (or dropdown if user wants a different repo)
c. "Branch" — pre-filled with defaultBranch
d. "Files" — tree browser loaded from GET /channels/:id/repos/:owner/:repo/tree
User selects one or more files (e.g. src/pages/index.tsx, src/pages/pricing.tsx)
9. User confirms → POST /tenant/v1/source-maps
→ ChannelSourceMap created
→ github-source-sync job enqueued immediately (first sync, framework detection)
10. Sync completes → framework stored on ChannelSourceMap → extracted content in RAGWorkers
github-source-sync.worker.ts
Queue: agent__github-source-sync
Job payload: { tenantId: string, sourceMappingId: string }
Job ID pattern: github-sync__${sourceMappingId}__${Date.now()}
Concurrency: 3
Worker flow:
1. Load ChannelSourceMap by sourceMappingId
2. Load ConnectedChannel (githubChannelId) — decrypt tokenInfo → accessToken
→ Refresh token if expireOn is within 5 minutes
3. Instantiate GitHubService with refreshed token
4. If ChannelSourceMap.framework is null (first sync):
→ Fetch root file tree
→ Detect framework by checking for config files (next.config.*, nuxt.config.*,
gatsby-config.*, vite.config.*) and directory structure (app/ vs pages/)
→ Set ChannelSourceMap.framework
5. For each path in sourceMap.filePaths:
a. Fetch file content via getFileContent(owner, repo, path, branch)
b. Determine extraction strategy from file extension + detected framework
c. Extract semantic content (see Content Extraction section below)
d. Build ExtractedPageContent record
6. Ingest each ExtractedPageContent into the RAG pipeline
→ dataset type: "github-source", document ID: `${sourceMappingId}:${filePath}` (upsert)
7. Update ChannelSourceMap.lastSyncedAt = now(), extractedAt = now()
8. enqueueNotification() → DM: "Source sync complete for [channel title]"Error handling:
| Error | Action |
|---|---|
| 401 from GitHub | Refresh token; if refresh also 401, set isConnected = false on channel, notify DM “GitHub token expired — please reconnect” |
| 404 repo not found | Fail job, notify DM with error detail |
| 403 with rate limit header | BullMQ retry with delay set to X-RateLimit-Reset epoch value |
| File not found (404 on specific path) | Skip that file, log warning, continue with others |
github-insights.worker.ts
Queue: agent__github-insights
Extends: insight-worker-base.ts
Trigger: Manual — “Generate Insights” button on the AI Insights tab
Worker flow:
1. Load ConnectedChannel — decrypt tokenInfo
2. Load all ChannelSourceMap records for this channel
3. For each source map → load ExtractedPageContent from RAG store
4. Load tenant's active keywords (keyword-researcher output)
5. Build Claude prompt:
System: "You are an SEO analyst reviewing a client's actual page source code.
Identify gaps between the current content and the target keywords.
Return structured JSON only."
User: { pages: ExtractedPageContent[], keywords: string[] }
6. Parse Claude JSON → SWOT structure:
{
strengths: string[],
weaknesses: string[],
opportunities: string[],
recommendations: RecommendationItem[], // includes priority + targetFilePath
summary: string
}
7. Store as ChannelInsight record (same model as all other insight workers)The recommendations array in the insight output uses the same RecommendationItem type as AgentCodeChange.recommendations. This means the “Push to Coding Agent” button on the Insights tab can pass them directly into POST /source-maps/:id/code-changes without transformation.
github-code-agent.worker.ts
Queue: agent__github-code-agent
Job payload: { tenantId: string, codeChangeId: string }
Job ID pattern: github-code__${codeChangeId}__${Date.now()}
Concurrency: 1 (sequential — each code change is a stateful operation on a repo)
This worker uses Claude tool use (Anthropic SDK tools parameter), not the subprocess adapter system used by content workers. Claude drives the edit loop — it reads files, plans changes, writes updated files, and calls create_pr when done.
Tools exposed to Claude
const tools = [
{
name: "read_file",
description: "Read the current content of a source file from the repository",
input_schema: { type: "object", properties: { path: { type: "string" } } }
},
{
name: "list_directory",
description: "List files in a directory of the repository",
input_schema: { type: "object", properties: { path: { type: "string" } } }
},
{
name: "write_file",
description: "Stage an updated version of a file. Call this once per file with the complete new content.",
input_schema: {
type: "object",
properties: {
path: { type: "string" },
content: { type: "string" },
reason: { type: "string" } // brief explanation of what changed and why
}
}
},
{
name: "create_pr",
description: "Create a pull request with all staged file changes. Call this once when done.",
input_schema: {
type: "object",
properties: {
title: { type: "string" },
body: { type: "string" }
}
}
}
]Worker flow
1. Load AgentCodeChange (codeChangeId) — verify status is "approved"
2. Load ChannelSourceMap → githubChannelId → ConnectedChannel → decrypt tokenInfo
3. Refresh token if needed
4. Set AgentCodeChange.status = "running", log start timestamp
5. Instantiate GitHubService with fresh token
6. Create branch: "leadmetrics/seo-opt-{YYYYMMDD-HHmm}-{codeChangeId.slice(0,6)}"
→ Store branchName on AgentCodeChange
7. Build system prompt for Claude:
- Framework: {ChannelSourceMap.framework}
- Repo: {owner}/{repo}, branch: {branchName}
- Target files: {codeChange.targetFiles}
- The following SEO and content optimizations have been approved by the DM.
Apply all of them. Make deep structural changes where needed — you may add
new sections, new components, new JSON-LD blocks, rewrite headings and paragraphs.
Do not change functionality, routing, or styling.
- Recommendations (numbered list): {codeChange.recommendations}
8. Start Claude tool use loop:
a. Send messages to Claude with tools defined above
b. For each tool call Claude makes:
- read_file → call GitHubService.getFileContent() → return content to Claude
- list_directory → call GitHubService.getFileTree() filtered to path → return to Claude
- write_file → call GitHubService.putFile() on the new branch → append to agentLog
- create_pr → call GitHubService.createPR() → store prUrl + prNumber → break loop
c. Append each tool call + result to AgentCodeChange.agentLog (streaming update)
d. Stop when Claude calls create_pr or when message count exceeds 40 (safety limit)
9. Update AgentCodeChange:
- status = "pr_opened"
- prUrl, prNumber set
- completedAt = now()
10. enqueueNotification() → DM: "PR ready for review: {prTitle}" with prUrlPR body template
## SEO & Content Optimization — Generated by Leadmetrics
This PR was generated by the Leadmetrics coding agent based on DM-approved recommendations.
### Changes Applied
{numbered list of recommendations with source attribution}
### Review Checklist
- [ ] Review each file diff and confirm copy aligns with brand voice
- [ ] Check that no functionality, routing, or styling was altered
- [ ] Verify all new structured data (JSON-LD) is valid
- [ ] Merge when satisfied — Leadmetrics will auto-sync on merge
---
*Generated {date} · Requested by {requestedByName} · Approved by {approvedByName}*
*[View in Leadmetrics]({dashboardUrl}/channels/{githubChannelId})*Framework-aware instructions injected into system prompt
The system prompt includes framework-specific guidance so Claude writes syntactically correct code:
| Framework | Injected guidance |
|---|---|
nextjs-app | Use export const metadata = { title, description } for metadata. New sections as server components. JSON-LD via <script type="application/ld+json"> in layout.tsx or page. |
nextjs-pages | Use <Head> from next/head for metadata. <NextSeo> if already present. New sections as React components in the same file or in components/. |
html | Edit <title>, <meta> tags directly. New sections as semantic HTML. JSON-LD as <script type="application/ld+json"> before </body>. |
vue | Use useHead() from @vueuse/head or <Head> component if Nuxt. New sections as .vue SFCs or inline template blocks. |
gatsby | Use <Seo> component (standard Gatsby pattern) or react-helmet. |
”Push to Coding Agent” — Sources
Any SEO or content recommendation in the platform can be pushed to the coding agent, as long as a ChannelSourceMap exists for the relevant website or landing page channel. The source type is stored on each RecommendationItem for attribution in the PR body.
| Source | Where the button appears | What gets pushed |
|---|---|---|
| GitHub Insights | AI Insights tab → each item in the recommendations[] array | The recommendation text + targetFilePath from the insight |
| Site Auditor | Site auditor deliverable detail page | Each actionable finding (missing meta, broken heading structure, missing JSON-LD, etc.) |
| Content Brief | Content brief detail page → SEO suggestions section | Keyword placement instructions, heading structure suggestions |
| Keyword Researcher | Keyword researcher output | On-page keyword placement suggestions |
| Manual | ”New Code Change” button on the Source Maps tab | DM types free-form instructions directly |
Multi-source batching: The DM can select recommendations from multiple sources before creating a single AgentCodeChange. The coding agent receives them all in one job and handles them in a single PR — one PR per approved change request, not one PR per recommendation.
Which source map? The “Push to Coding Agent” button is only shown when:
- A
ChannelSourceMapexists linking the relevant Website/LandingPage channel to a GitHub channel - The
ChannelSourceMap.frameworkis set (i.e., at least one sync has completed)
If no source map exists, the button is replaced by a “Connect GitHub” prompt that links to the Channels page.
Dashboard UI — GitHubChannelDetail.tsx
Location: apps/dashboard/src/app/(dashboard)/channels/[id]/GitHubChannelDetail.tsx
Dispatched from [id]/page.tsx when channel.type === "GitHub".
Tab 1 — Overview
- Repo card: owner/repo, default branch, detected framework badge, last push date, open PRs, 30-day commit count
- Connection status banner (if
isConnected = false): “GitHub token expired — click Reconnect” - Linked pages table: one row per
ChannelSourceMap— target channel name, file count, framework, last sync time, sync status badge
Tab 2 — Source Maps
- Table: target page | files | last sync | framework | actions (sync now, edit, delete)
- “Add Source Map” button → modal:
- Target page dropdown (Website + LandingPage channels)
- Repository — pre-filled from
subChannelInfo.repo - Branch field — pre-filled with
defaultBranch - Files — tree browser loaded from
/channels/:id/repos/:owner/:repo/tree; multi-select
- “Sync Now” action →
POST /source-maps/:id/sync→ spinner until complete
Tab 3 — Code Changes
- Table of all
AgentCodeChangerecords for all source maps on this channel:- Target page | recommendations count | status badge | branch | PR link | requested by | created date
- Status badges:
Pending Review(amber) ·Approved(blue) ·Running(spinning blue) ·PR Open(green) ·Merged(purple) ·Failed(red) ·Cancelled(grey) - Row click → expands to show full recommendation list + agent log (streaming if status is “running”)
- “Approve & Run” button (on
pending_reviewrows) →POST /source-maps/:id/code-changes/:changeId/approve - “Cancel” button (on
pending_reviewandapprovedrows) - “View PR” link (on
pr_openedandmergedrows) → opens GitHub PR in new tab
Tab 4 — AI Insights
Standard ChannelInsightPanel component (identical to all other channel detail pages). Each recommendation in the insight output has a “Push to Code Agent” checkbox. Selecting one or more and clicking “Create Code Change” pre-fills the code change modal with the selected items.
Implementation Phases
Phase 1 — OAuth Connect + Source Map + Sync [To Build]
Goal: User can authorize GitHub via OAuth, select a repository, map it to a landing page channel, and trigger a source sync.
Files to create/modify:
| File | Change |
|---|---|
packages/db/prisma/schema.prisma | Add ChannelSourceMap model |
packages/db/prisma/seed.ts | Add GitHub ChannelMaster seed row |
.env.example | Add GITHUB_CLIENT_ID, GITHUB_CLIENT_SECRET |
packages/providers/github/ | New package — GitHubService (OAuth + repo read methods) |
apps/api/src/routers/channel-connect.ts | Add GitHub OAuth routes (connect/callback/repo-select) |
apps/api/src/routers/channels.ts | Add repo browser endpoints + GitHub analytics case |
apps/api/src/routers/source-maps.ts | New — source map CRUD + sync trigger |
apps/api/src/app.ts | Register source-maps router |
apps/api/src/index.ts | Register source-maps router |
packages/queue/src/queues.ts | Register agent__github-source-sync queue |
packages/queue/src/types.ts | Add GitHubSyncJobData type |
packages/agents/src/workers/github-source-sync.worker.ts | New — framework detection + extract + RAG ingest |
apps/servers/agents/src/index.ts | Register github-source-sync worker |
apps/dashboard/src/app/(dashboard)/channels/[id]/page.tsx | Add GitHub dispatch case |
apps/dashboard/src/app/(dashboard)/channels/[id]/GitHubChannelDetail.tsx | New — Overview + Source Maps tabs |
Phase 2 — AI Insights [To Build]
Goal: “Generate Insights” on the GitHub channel detail produces an SEO gap analysis comparing extracted page content against target keywords. Each recommendation includes a targetFilePath and priority so it can be pushed to the coding agent in Phase 3.
Files to create/modify:
| File | Change |
|---|---|
packages/agents/src/workers/insights/github-insights.worker.ts | New — insight worker |
packages/queue/src/queues.ts | Register agent__github-insights queue |
apps/servers/agents/src/index.ts | Register github-insights worker |
apps/dashboard/src/app/(dashboard)/channels/[id]/GitHubChannelDetail.tsx | Add AI Insights tab with “Push to Code Agent” checkboxes |
Phase 3 — Coding Agent [To Build]
Goal: DM selects recommendations (from any source), reviews them, approves, and the coding agent opens a PR with deep structural changes applied.
Files to create/modify:
| File | Change |
|---|---|
packages/db/prisma/schema.prisma | Add AgentCodeChange model |
packages/providers/github/src/github.ts | Add write-back methods: createBranch, putFile, createPR |
apps/api/src/routers/source-maps.ts | Add code change CRUD + approve + cancel endpoints |
packages/queue/src/queues.ts | Register agent__github-code-agent queue |
packages/queue/src/types.ts | Add GitHubCodeAgentJobData type |
packages/agents/src/workers/github-code-agent.worker.ts | New — Claude tool use loop + PR creation |
apps/servers/agents/src/index.ts | Register github-code-agent worker |
apps/dashboard/src/app/(dashboard)/channels/[id]/GitHubChannelDetail.tsx | Add Code Changes tab with approval UI |
apps/dashboard/src/app/(dashboard)/channels/[id]/GitHubChannelDetail.tsx | Add “Push to Code Agent” on Insights tab |
| Site Auditor deliverable detail page | Add “Push to GitHub” action per finding |
| Content Brief detail page | Add “Push to GitHub” on SEO suggestions |
Phase 4 — DM Portal Parity [To Build]
Goal: DM portal shows the GitHub channel, source maps, and code changes (read-only per DM parity rules). DMs can trigger sync and approve code changes from the DM portal.
DM exception: Code change approval is allowed in the DM portal because DMs are the primary actor in the approval flow — this is analogous to DMs being able to approve blog posts in the DM portal.
Files to create/modify:
| File | Change |
|---|---|
apps/dm/src/app/(dm)/channels/[id]/GitHubChannelDetailDMClient.tsx | New — DM view with approval actions |
apps/dm/src/app/(dm)/channels/[id]/page.tsx | Add GitHub dispatch case |
apps/api/src/routers/source-maps.ts | Expose read + approve endpoints under /dm/v1/ prefix |
Phase 5 — Push Webhook Auto-Sync [To Build]
Goal: Every push to the tracked branch automatically triggers a source re-sync — the platform stays current with every deploy without any manual action.
Design:
- On source map creation, register a GitHub push webhook via
GitHubService.createWebhook() - Webhook points to
POST /webhooks/githubon the API server - Payload verified via HMAC-SHA256 signature against per-map
webhookSecret - Handler finds the
ChannelSourceMapbyrepoOwner/repoName, enqueues sync job only if the push was todefaultBranch webhookIdandwebhookSecretstored onChannelSourceMap; webhook deleted on source map deletion
Files to create/modify:
| File | Change |
|---|---|
packages/providers/github/src/github.ts | Add createWebhook, deleteWebhook, verifyWebhookSignature |
apps/api/src/routers/webhooks.ts | New — POST /webhooks/github (no JWT — HMAC verified) |
apps/api/src/app.ts + index.ts | Register webhooks router |
apps/api/src/routers/source-maps.ts | Auto-register webhook on create; delete on map delete |
Content Extraction Detail
Framework detection (first sync only)
The sync worker fetches the root file tree and checks for the following in order:
| Check | Detected as |
|---|---|
next.config.js or next.config.ts exists AND app/ directory exists | nextjs-app |
next.config.js or next.config.ts exists AND pages/ directory exists | nextjs-pages |
nuxt.config.js or nuxt.config.ts exists | nuxt |
gatsby-config.js exists | gatsby |
vite.config.js/ts exists AND src/App.vue exists | vue |
| None of the above | html |
Extraction per file type
HTML / HTM
Use htmlparser2:
<title>content<meta name="description">,<meta name="keywords">content attributes<meta property="og:title">,<meta property="og:description">content attributes- All
<h1>–<h6>text content <script type="application/ld+json">blocks parsed as JSON- Visible body text: strip
<script>,<style>,<noscript>,<svg>→ collect all remaining text nodes → collapse whitespace
TSX / JSX
Regex approach (avoids a full TypeScript AST parse):
export const metadata = { title: "...", description: "..." }(Next.js App Router)- JSX props:
title="...",description="...",content="..."inside<Head>,<NextSeo>,<Helmet>,<SEO>elements - JSX element children:
<h1>...</h1>through<h6>...</h6>string literals <p>and<span>string literal children (skip{expression}interpolations)
Vue SFC
- Extract
<template>block → treat as HTML for heading + text extraction - Extract
useHead({...})oruseSeoMeta({...})call arguments for metadata
Markdown / MDX
- Strip YAML frontmatter → extract
title,description - First
# Heading= title if no frontmatter title - All
##and###headings for structure - Strip MDX component syntax (
<Component>) → remaining text = body prose
Output schema per file
type ExtractedPageContent = {
filePath: string
fileType: "html" | "tsx" | "jsx" | "vue" | "md" | "mdx"
title: string | null
metaDescription: string | null
ogTitle: string | null
ogDescription: string | null
headings: { level: 1 | 2 | 3 | 4 | 5 | 6; text: string }[]
structuredData: object[] // JSON-LD blocks parsed from the file
bodyText: string // stripped prose, whitespace-collapsed
wordCount: number
extractedAt: string // ISO date
}Key Files Reference
| File | Type | Phase |
|---|---|---|
packages/db/prisma/schema.prisma | Schema | 1 (ChannelSourceMap), 3 (AgentCodeChange) |
packages/db/prisma/seed.ts | Seed | 1 |
.env.example | Config | 1 |
packages/providers/github/src/github.ts | Provider | 1 (OAuth + read), 3 (write-back), 5 (webhooks) |
packages/providers/github/src/types.ts | Types | 1 |
apps/api/src/routers/channel-connect.ts | Router (modified) | 1 |
apps/api/src/routers/channels.ts | Router (modified) | 1 |
apps/api/src/routers/source-maps.ts | Router | 1 (CRUD + sync), 3 (code changes) |
apps/api/src/routers/webhooks.ts | Router | 5 |
packages/queue/src/queues.ts | Queue | 1, 2, 3 |
packages/agents/src/workers/github-source-sync.worker.ts | Worker | 1 |
packages/agents/src/workers/insights/github-insights.worker.ts | Worker | 2 |
packages/agents/src/workers/github-code-agent.worker.ts | Worker | 3 |
apps/dashboard/src/app/(dashboard)/channels/[id]/GitHubChannelDetail.tsx | UI | 1–3 |
apps/dm/src/app/(dm)/channels/[id]/GitHubChannelDetailDMClient.tsx | UI | 4 |