Conformance Milestone iv — CI drift gate

Goal: Ship pnpm conformance — a cross-feature drift check that aggregates ALL feature manifests and fails on an orphan event consumer (a manifest declares consumes: ["X"] but no other manifest declares publishes: ["X"]). Wired into turbo.json and GitHub Actions CI.

Architecture: A small Node.js script (scripts/conformance.mjs) walks packages/*/src/feature.manifest.ts files, reuses the iii.b AST parser (parseManifestUseCases), builds global publish/consume sets across all features, exits non-zero on orphan consumers. Turbo task + CI step wire it into the normal validate pipeline.

Tasks

Task 1: Story 04 scaffold

Create docs/work/conformance-system-v1/04-ci-drift-gate/_story.md:

---
id: 04-ci-drift-gate
epic: conformance-system-v1
title: CI drift gate — pnpm conformance with cross-feature event closure
type: technical-story
status: in-progress
feature: scripts
depends-on: [03-b-ast-eslint-rules]
blocks: [05-generator-updates]
---

## Goal
`pnpm conformance` aggregates cross-feature checks that no single-file
ESLint rule can perform — most importantly, event closure: every event
declared in any manifest's `consumes` must have at least one matching
`publishes` somewhere in the repo.

## Why
Per-file lint can't see cross-feature contracts. Without this gate, a
feature can declare it consumes `X` while no feature publishes `X` —
silent until the broken handler is exercised in prod.

## Done when
- `pnpm conformance` exits 0 when manifests are consistent; non-zero
  with a clear error message on orphan consumers
- Wired into `turbo.json` as the `conformance` task
- Wired into `.github/workflows/ci.yml` after `pnpm lint`

## In scope
- `scripts/conformance.mjs` — orphan-consumer check
- Tests via vitest
- turbo.json + CI wiring

## Out of scope
- Scaffold drift check (regenerate via `turbo gen feature`, diff against
  on-disk state) — depends on generator updates landing
- Repository write outside use-cases check — separate concern
- Reverse check (orphan publishers — events nothing consumes) — many
  events are intentionally "fire and forget"; not a closure violation

## Tasks
- [ ] Story scaffold
- [ ] `scripts/conformance.mjs` implementation
- [ ] Tests for the script
- [ ] Wire into root package.json + turbo.json
- [ ] Wire into ci.yml
- [ ] Final verification + closeout

Commit: docs(work): story 04 — CI drift gate.

Task 2: `scripts/conformance.mjs` + tests

Create the script and its test alongside.

scripts/conformance.mjs:

#!/usr/bin/env node
/**
 * pnpm conformance — cross-feature drift gate.
 *
 * Walks every `packages/*\/src/feature.manifest.ts`, reuses the AST parser
 * from `@repo/core-eslint` to extract per-use-case publishes/consumes,
 * builds global publish + consume sets across all features, and fails on:
 *
 *   - Orphan consumer: a feature declares `consumes: ["X"]` but no
 *     feature publishes "X".
 *
 * Exits 0 on success, 1 on any violation. Prints a tabular summary of
 * the event graph for transparency.
 */
import fs from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { parseManifestUseCases } from "../packages/core-eslint/rules/_manifest-ast.js";

const __dirname = path.dirname(fileURLToPath(import.meta.url));
const REPO_ROOT = path.resolve(__dirname, "..");

export function findAllManifests(repoRoot = REPO_ROOT) {
  const packagesDir = path.join(repoRoot, "packages");
  if (!fs.existsSync(packagesDir)) return [];
  const out = [];
  for (const entry of fs.readdirSync(packagesDir)) {
    const manifestPath = path.join(packagesDir, entry, "src", "feature.manifest.ts");
    if (fs.existsSync(manifestPath)) {
      out.push({ feature: entry, path: manifestPath });
    }
  }
  return out;
}

export function buildEventGraph(manifests) {
  // event -> { publishers: [{ feature, useCase }], consumers: [{ feature, useCase }] }
  const graph = new Map();
  for (const { feature, path: manifestPath } of manifests) {
    const useCases = parseManifestUseCases(manifestPath);
    if (!useCases) continue;
    for (const [useCase, entry] of Object.entries(useCases)) {
      for (const event of entry.publishes) {
        if (!graph.has(event)) graph.set(event, { publishers: [], consumers: [] });
        graph.get(event).publishers.push({ feature, useCase });
      }
      for (const event of entry.consumes) {
        if (!graph.has(event)) graph.set(event, { publishers: [], consumers: [] });
        graph.get(event).consumers.push({ feature, useCase });
      }
    }
  }
  return graph;
}

export function findOrphanConsumers(graph) {
  const orphans = [];
  for (const [event, { publishers, consumers }] of graph.entries()) {
    if (consumers.length > 0 && publishers.length === 0) {
      orphans.push({ event, consumers });
    }
  }
  return orphans;
}

function main() {
  const manifests = findAllManifests();
  console.log(`Found ${manifests.length} feature manifest(s):`);
  for (const { feature } of manifests) console.log(`  - ${feature}`);
  console.log();

  const graph = buildEventGraph(manifests);
  if (graph.size === 0) {
    console.log("No cross-feature events declared yet — nothing to check.");
    process.exit(0);
  }

  console.log(`Event graph (${graph.size} event(s)):`);
  for (const [event, { publishers, consumers }] of graph.entries()) {
    console.log(`  ${event}`);
    console.log(`    publishers: ${publishers.length === 0 ? "(none)" : publishers.map((p) => `${p.feature}.${p.useCase}`).join(", ")}`);
    console.log(`    consumers:  ${consumers.length === 0 ? "(none)" : consumers.map((c) => `${c.feature}.${c.useCase}`).join(", ")}`);
  }
  console.log();

  const orphans = findOrphanConsumers(graph);
  if (orphans.length === 0) {
    console.log("✓ pnpm conformance — passed");
    process.exit(0);
  }
  console.error(`✗ pnpm conformance — ${orphans.length} orphan consumer(s):`);
  for (const { event, consumers } of orphans) {
    console.error(`  ${event}`);
    for (const c of consumers) {
      console.error(`    consumed by ${c.feature}.${c.useCase}, but no feature publishes it`);
    }
  }
  process.exit(1);
}

// Only run main when invoked as a script (not when imported by tests).
if (import.meta.url === `file://${process.argv[1]}`) {
  main();
}

scripts/conformance.test.mjs:

import { describe, it, expect } from "vitest";
import path from "node:path";
import os from "node:os";
import fs from "node:fs";
import { findAllManifests, buildEventGraph, findOrphanConsumers } from "./conformance.mjs";

function makeRepo({ features }) {
  const root = fs.mkdtempSync(path.join(os.tmpdir(), "conformance-"));
  for (const [name, useCases] of Object.entries(features)) {
    const dir = path.join(root, "packages", name, "src");
    fs.mkdirSync(dir, { recursive: true });
    const useCasesStr = Object.entries(useCases)
      .map(([ucName, uc]) =>
        `    ${ucName}: { mutates: ${uc.mutates ?? false}, audits: [], publishes: [${(uc.publishes ?? []).map((p) => `"${p}"`).join(", ")}], consumes: [${(uc.consumes ?? []).map((c) => `"${c}"`).join(", ")}] },`,
      )
      .join("\n");
    fs.writeFileSync(
      path.join(dir, "feature.manifest.ts"),
      `export const ${name}Manifest = defineFeature({
  name: "${name}",
  requiredCores: [],
  useCases: {
${useCasesStr}
  },
  realtimeChannels: [],
  jobs: [],
} as const);`,
    );
  }
  return root;
}

describe("conformance script", () => {
  describe("findAllManifests", () => {
    it("returns one entry per feature with a manifest", () => {
      const root = makeRepo({
        auth: { signIn: {} },
        blog: { getArticles: {} },
      });
      const ms = findAllManifests(root);
      expect(ms.map((m) => m.feature).sort()).toEqual(["auth", "blog"]);
    });

    it("skips packages without a manifest", () => {
      const root = fs.mkdtempSync(path.join(os.tmpdir(), "conformance-empty-"));
      fs.mkdirSync(path.join(root, "packages", "no-manifest", "src"), { recursive: true });
      expect(findAllManifests(root)).toEqual([]);
    });
  });

  describe("buildEventGraph + findOrphanConsumers", () => {
    it("finds zero orphans when consumers and publishers line up", () => {
      const root = makeRepo({
        auth: { signUp: { mutates: true, publishes: ["auth.signed-up"] } },
        marketing: { onAuthSignedUp: { consumes: ["auth.signed-up"] } },
      });
      const manifests = findAllManifests(root);
      const graph = buildEventGraph(manifests);
      expect(findOrphanConsumers(graph)).toEqual([]);
    });

    it("flags orphan consumers", () => {
      const root = makeRepo({
        marketing: { onAuthSignedUp: { consumes: ["auth.signed-up"] } },
      });
      const manifests = findAllManifests(root);
      const graph = buildEventGraph(manifests);
      const orphans = findOrphanConsumers(graph);
      expect(orphans).toHaveLength(1);
      expect(orphans[0].event).toBe("auth.signed-up");
      expect(orphans[0].consumers).toEqual([{ feature: "marketing", useCase: "onAuthSignedUp" }]);
    });

    it("treats publish-only events as fine (no consumers is not an orphan)", () => {
      const root = makeRepo({
        auth: { signUp: { mutates: true, publishes: ["auth.signed-up"] } },
      });
      const manifests = findAllManifests(root);
      const graph = buildEventGraph(manifests);
      expect(findOrphanConsumers(graph)).toEqual([]);
    });
  });
});

Commits:

git add scripts/conformance.mjs scripts/conformance.test.mjs && git commit -m "feat(scripts): conformance drift gate + tests"

(One commit for both because they're trivially related and the test file is part of the script's interface.)

Task 3: Wire `pnpm conformance` + turbo task

Modify root package.json. Add this script entry (alongside existing scripts):

"conformance": "node scripts/conformance.mjs"

Modify turbo.json. Add this task (alongside existing lint, test, etc.):

"conformance": {
  "inputs": [
    "packages/*/src/feature.manifest.ts",
    "scripts/conformance.mjs",
    "packages/core-eslint/rules/_manifest-ast.js"
  ],
  "outputs": []
}

Verify by running pnpm conformance. With auth as the only manifest (no publishes, no consumes), output should report "No cross-feature events declared yet — nothing to check." and exit 0.

Also run the vitest:

pnpm vitest run scripts/conformance.test.mjs

5 tests should pass.

Commit: feat: wire pnpm conformance script + turbo task.

Task 4: Wire conformance into CI

Modify .github/workflows/ci.yml. Find the step - run: pnpm lint. Add immediately after it:

      - run: pnpm conformance

Commit: ci: add conformance step after lint.

Task 5: Final verification + closeout

Run:

pnpm typecheck
pnpm test
pnpm lint
pnpm conformance
pnpm turbo boundaries

All pass.

Update docs/work/conformance-system-v1/04-ci-drift-gate/_story.md:

frontmatter status: in-progress → done
6 task checkboxes → all - [x]

Update docs/work/conformance-system-v1/_epic.md:

Find - [ ] 04 — CI drift gate (later plan)
Replace with - [x] [04 — CI drift gate](04-ci-drift-gate/_story.md)

Commit: docs(work): close story 04 — CI drift gate.

11 KiB Raw Blame History