plan(conformance): milestone iv — CI drift gate
This commit is contained in:
@@ -0,0 +1,336 @@
|
||||
# Conformance Milestone iv — CI drift gate
|
||||
|
||||
**Goal:** Ship `pnpm conformance` — a cross-feature drift check that aggregates ALL feature manifests and fails on an orphan event consumer (a manifest declares `consumes: ["X"]` but no other manifest declares `publishes: ["X"]`). Wired into turbo.json and GitHub Actions CI.
|
||||
|
||||
**Architecture:** A small Node.js script (`scripts/conformance.mjs`) walks `packages/*/src/feature.manifest.ts` files, reuses the iii.b AST parser (`parseManifestUseCases`), builds global publish/consume sets across all features, exits non-zero on orphan consumers. Turbo task + CI step wire it into the normal validate pipeline.
|
||||
|
||||
---
|
||||
|
||||
## Tasks
|
||||
|
||||
### Task 1: Story 04 scaffold
|
||||
|
||||
Create `docs/work/conformance-system-v1/04-ci-drift-gate/_story.md`:
|
||||
|
||||
```markdown
|
||||
---
|
||||
id: 04-ci-drift-gate
|
||||
epic: conformance-system-v1
|
||||
title: CI drift gate — pnpm conformance with cross-feature event closure
|
||||
type: technical-story
|
||||
status: in-progress
|
||||
feature: scripts
|
||||
depends-on: [03-b-ast-eslint-rules]
|
||||
blocks: [05-generator-updates]
|
||||
---
|
||||
|
||||
## Goal
|
||||
`pnpm conformance` aggregates cross-feature checks that no single-file
|
||||
ESLint rule can perform — most importantly, event closure: every event
|
||||
declared in any manifest's `consumes` must have at least one matching
|
||||
`publishes` somewhere in the repo.
|
||||
|
||||
## Why
|
||||
Per-file lint can't see cross-feature contracts. Without this gate, a
|
||||
feature can declare it consumes `X` while no feature publishes `X` —
|
||||
silent until the broken handler is exercised in prod.
|
||||
|
||||
## Done when
|
||||
- `pnpm conformance` exits 0 when manifests are consistent; non-zero
|
||||
with a clear error message on orphan consumers
|
||||
- Wired into `turbo.json` as the `conformance` task
|
||||
- Wired into `.github/workflows/ci.yml` after `pnpm lint`
|
||||
|
||||
## In scope
|
||||
- `scripts/conformance.mjs` — orphan-consumer check
|
||||
- Tests via vitest
|
||||
- turbo.json + CI wiring
|
||||
|
||||
## Out of scope
|
||||
- Scaffold drift check (regenerate via `turbo gen feature`, diff against
|
||||
on-disk state) — depends on generator updates landing
|
||||
- Repository write outside use-cases check — separate concern
|
||||
- Reverse check (orphan publishers — events nothing consumes) — many
|
||||
events are intentionally "fire and forget"; not a closure violation
|
||||
|
||||
## Tasks
|
||||
- [ ] Story scaffold
|
||||
- [ ] `scripts/conformance.mjs` implementation
|
||||
- [ ] Tests for the script
|
||||
- [ ] Wire into root package.json + turbo.json
|
||||
- [ ] Wire into ci.yml
|
||||
- [ ] Final verification + closeout
|
||||
```
|
||||
|
||||
Commit: `docs(work): story 04 — CI drift gate`.
|
||||
|
||||
### Task 2: `scripts/conformance.mjs` + tests
|
||||
|
||||
Create the script and its test alongside.
|
||||
|
||||
`scripts/conformance.mjs`:
|
||||
|
||||
```js
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* pnpm conformance — cross-feature drift gate.
|
||||
*
|
||||
* Walks every `packages/*\/src/feature.manifest.ts`, reuses the AST parser
|
||||
* from `@repo/core-eslint` to extract per-use-case publishes/consumes,
|
||||
* builds global publish + consume sets across all features, and fails on:
|
||||
*
|
||||
* - Orphan consumer: a feature declares `consumes: ["X"]` but no
|
||||
* feature publishes "X".
|
||||
*
|
||||
* Exits 0 on success, 1 on any violation. Prints a tabular summary of
|
||||
* the event graph for transparency.
|
||||
*/
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { parseManifestUseCases } from "../packages/core-eslint/rules/_manifest-ast.js";
|
||||
|
||||
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||
const REPO_ROOT = path.resolve(__dirname, "..");
|
||||
|
||||
export function findAllManifests(repoRoot = REPO_ROOT) {
|
||||
const packagesDir = path.join(repoRoot, "packages");
|
||||
if (!fs.existsSync(packagesDir)) return [];
|
||||
const out = [];
|
||||
for (const entry of fs.readdirSync(packagesDir)) {
|
||||
const manifestPath = path.join(packagesDir, entry, "src", "feature.manifest.ts");
|
||||
if (fs.existsSync(manifestPath)) {
|
||||
out.push({ feature: entry, path: manifestPath });
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
export function buildEventGraph(manifests) {
|
||||
// event -> { publishers: [{ feature, useCase }], consumers: [{ feature, useCase }] }
|
||||
const graph = new Map();
|
||||
for (const { feature, path: manifestPath } of manifests) {
|
||||
const useCases = parseManifestUseCases(manifestPath);
|
||||
if (!useCases) continue;
|
||||
for (const [useCase, entry] of Object.entries(useCases)) {
|
||||
for (const event of entry.publishes) {
|
||||
if (!graph.has(event)) graph.set(event, { publishers: [], consumers: [] });
|
||||
graph.get(event).publishers.push({ feature, useCase });
|
||||
}
|
||||
for (const event of entry.consumes) {
|
||||
if (!graph.has(event)) graph.set(event, { publishers: [], consumers: [] });
|
||||
graph.get(event).consumers.push({ feature, useCase });
|
||||
}
|
||||
}
|
||||
}
|
||||
return graph;
|
||||
}
|
||||
|
||||
export function findOrphanConsumers(graph) {
|
||||
const orphans = [];
|
||||
for (const [event, { publishers, consumers }] of graph.entries()) {
|
||||
if (consumers.length > 0 && publishers.length === 0) {
|
||||
orphans.push({ event, consumers });
|
||||
}
|
||||
}
|
||||
return orphans;
|
||||
}
|
||||
|
||||
function main() {
|
||||
const manifests = findAllManifests();
|
||||
console.log(`Found ${manifests.length} feature manifest(s):`);
|
||||
for (const { feature } of manifests) console.log(` - ${feature}`);
|
||||
console.log();
|
||||
|
||||
const graph = buildEventGraph(manifests);
|
||||
if (graph.size === 0) {
|
||||
console.log("No cross-feature events declared yet — nothing to check.");
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
console.log(`Event graph (${graph.size} event(s)):`);
|
||||
for (const [event, { publishers, consumers }] of graph.entries()) {
|
||||
console.log(` ${event}`);
|
||||
console.log(` publishers: ${publishers.length === 0 ? "(none)" : publishers.map((p) => `${p.feature}.${p.useCase}`).join(", ")}`);
|
||||
console.log(` consumers: ${consumers.length === 0 ? "(none)" : consumers.map((c) => `${c.feature}.${c.useCase}`).join(", ")}`);
|
||||
}
|
||||
console.log();
|
||||
|
||||
const orphans = findOrphanConsumers(graph);
|
||||
if (orphans.length === 0) {
|
||||
console.log("✓ pnpm conformance — passed");
|
||||
process.exit(0);
|
||||
}
|
||||
console.error(`✗ pnpm conformance — ${orphans.length} orphan consumer(s):`);
|
||||
for (const { event, consumers } of orphans) {
|
||||
console.error(` ${event}`);
|
||||
for (const c of consumers) {
|
||||
console.error(` consumed by ${c.feature}.${c.useCase}, but no feature publishes it`);
|
||||
}
|
||||
}
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Only run main when invoked as a script (not when imported by tests).
|
||||
if (import.meta.url === `file://${process.argv[1]}`) {
|
||||
main();
|
||||
}
|
||||
```
|
||||
|
||||
`scripts/conformance.test.mjs`:
|
||||
|
||||
```js
|
||||
import { describe, it, expect } from "vitest";
|
||||
import path from "node:path";
|
||||
import os from "node:os";
|
||||
import fs from "node:fs";
|
||||
import { findAllManifests, buildEventGraph, findOrphanConsumers } from "./conformance.mjs";
|
||||
|
||||
function makeRepo({ features }) {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), "conformance-"));
|
||||
for (const [name, useCases] of Object.entries(features)) {
|
||||
const dir = path.join(root, "packages", name, "src");
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
const useCasesStr = Object.entries(useCases)
|
||||
.map(([ucName, uc]) =>
|
||||
` ${ucName}: { mutates: ${uc.mutates ?? false}, audits: [], publishes: [${(uc.publishes ?? []).map((p) => `"${p}"`).join(", ")}], consumes: [${(uc.consumes ?? []).map((c) => `"${c}"`).join(", ")}] },`,
|
||||
)
|
||||
.join("\n");
|
||||
fs.writeFileSync(
|
||||
path.join(dir, "feature.manifest.ts"),
|
||||
`export const ${name}Manifest = defineFeature({
|
||||
name: "${name}",
|
||||
requiredCores: [],
|
||||
useCases: {
|
||||
${useCasesStr}
|
||||
},
|
||||
realtimeChannels: [],
|
||||
jobs: [],
|
||||
} as const);`,
|
||||
);
|
||||
}
|
||||
return root;
|
||||
}
|
||||
|
||||
describe("conformance script", () => {
|
||||
describe("findAllManifests", () => {
|
||||
it("returns one entry per feature with a manifest", () => {
|
||||
const root = makeRepo({
|
||||
auth: { signIn: {} },
|
||||
blog: { getArticles: {} },
|
||||
});
|
||||
const ms = findAllManifests(root);
|
||||
expect(ms.map((m) => m.feature).sort()).toEqual(["auth", "blog"]);
|
||||
});
|
||||
|
||||
it("skips packages without a manifest", () => {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), "conformance-empty-"));
|
||||
fs.mkdirSync(path.join(root, "packages", "no-manifest", "src"), { recursive: true });
|
||||
expect(findAllManifests(root)).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("buildEventGraph + findOrphanConsumers", () => {
|
||||
it("finds zero orphans when consumers and publishers line up", () => {
|
||||
const root = makeRepo({
|
||||
auth: { signUp: { mutates: true, publishes: ["auth.signed-up"] } },
|
||||
marketing: { onAuthSignedUp: { consumes: ["auth.signed-up"] } },
|
||||
});
|
||||
const manifests = findAllManifests(root);
|
||||
const graph = buildEventGraph(manifests);
|
||||
expect(findOrphanConsumers(graph)).toEqual([]);
|
||||
});
|
||||
|
||||
it("flags orphan consumers", () => {
|
||||
const root = makeRepo({
|
||||
marketing: { onAuthSignedUp: { consumes: ["auth.signed-up"] } },
|
||||
});
|
||||
const manifests = findAllManifests(root);
|
||||
const graph = buildEventGraph(manifests);
|
||||
const orphans = findOrphanConsumers(graph);
|
||||
expect(orphans).toHaveLength(1);
|
||||
expect(orphans[0].event).toBe("auth.signed-up");
|
||||
expect(orphans[0].consumers).toEqual([{ feature: "marketing", useCase: "onAuthSignedUp" }]);
|
||||
});
|
||||
|
||||
it("treats publish-only events as fine (no consumers is not an orphan)", () => {
|
||||
const root = makeRepo({
|
||||
auth: { signUp: { mutates: true, publishes: ["auth.signed-up"] } },
|
||||
});
|
||||
const manifests = findAllManifests(root);
|
||||
const graph = buildEventGraph(manifests);
|
||||
expect(findOrphanConsumers(graph)).toEqual([]);
|
||||
});
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
Commits:
|
||||
1. `git add scripts/conformance.mjs scripts/conformance.test.mjs && git commit -m "feat(scripts): conformance drift gate + tests"`
|
||||
|
||||
(One commit for both because they're trivially related and the test file is part of the script's interface.)
|
||||
|
||||
### Task 3: Wire `pnpm conformance` + turbo task
|
||||
|
||||
Modify root `package.json`. Add this script entry (alongside existing scripts):
|
||||
|
||||
```json
|
||||
"conformance": "node scripts/conformance.mjs"
|
||||
```
|
||||
|
||||
Modify `turbo.json`. Add this task (alongside existing `lint`, `test`, etc.):
|
||||
|
||||
```json
|
||||
"conformance": {
|
||||
"inputs": [
|
||||
"packages/*/src/feature.manifest.ts",
|
||||
"scripts/conformance.mjs",
|
||||
"packages/core-eslint/rules/_manifest-ast.js"
|
||||
],
|
||||
"outputs": []
|
||||
}
|
||||
```
|
||||
|
||||
Verify by running `pnpm conformance`. With auth as the only manifest (no publishes, no consumes), output should report "No cross-feature events declared yet — nothing to check." and exit 0.
|
||||
|
||||
Also run the vitest:
|
||||
```
|
||||
pnpm vitest run scripts/conformance.test.mjs
|
||||
```
|
||||
|
||||
5 tests should pass.
|
||||
|
||||
Commit: `feat: wire pnpm conformance script + turbo task`.
|
||||
|
||||
### Task 4: Wire conformance into CI
|
||||
|
||||
Modify `.github/workflows/ci.yml`. Find the step `- run: pnpm lint`. Add immediately after it:
|
||||
|
||||
```yaml
|
||||
- run: pnpm conformance
|
||||
```
|
||||
|
||||
Commit: `ci: add conformance step after lint`.
|
||||
|
||||
### Task 5: Final verification + closeout
|
||||
|
||||
Run:
|
||||
```
|
||||
pnpm typecheck
|
||||
pnpm test
|
||||
pnpm lint
|
||||
pnpm conformance
|
||||
pnpm turbo boundaries
|
||||
```
|
||||
|
||||
All pass.
|
||||
|
||||
Update `docs/work/conformance-system-v1/04-ci-drift-gate/_story.md`:
|
||||
- frontmatter `status: in-progress` → `done`
|
||||
- 6 task checkboxes → all `- [x]`
|
||||
|
||||
Update `docs/work/conformance-system-v1/_epic.md`:
|
||||
- Find `- [ ] 04 — CI drift gate (later plan)`
|
||||
- Replace with `- [x] [04 — CI drift gate](04-ci-drift-gate/_story.md)`
|
||||
|
||||
Commit: `docs(work): close story 04 — CI drift gate`.
|
||||
Reference in New Issue
Block a user