diff --git a/docs/architecture/agent-first-workflow-and-conformance.md b/docs/architecture/agent-first-workflow-and-conformance.md new file mode 100644 index 0000000..af426bf --- /dev/null +++ b/docs/architecture/agent-first-workflow-and-conformance.md @@ -0,0 +1,650 @@ +--- +title: Agent-first development workflow + feature conformance +status: design +created: 2026-05-12 +authors: [danijel, claude] +related: + - docs/architecture/feature-conformance-explainer.html + - docs/architecture/vertical-feature-spec.md + - docs/guides/tdd-workflow.md + - docs/guides/scaffolding-a-feature.md + - CLAUDE.md +--- + +# Agent-first development workflow + feature conformance + +## Why this document exists + +`template-vertical` is being shaped around the assumption that **AI coding agents will author most feature work**. Humans set direction, write PRDs (with agent help), review diffs, and intervene when escalation is needed; agents do the bulk of the coding. This document defines: + +1. The **feature-conformance enforcement system** that gives agents tight, layered, machine-readable feedback on architectural drift. +2. The **agent-first workflow** — manifest → contracts → tests → code — that the conformance system enforces. +3. The **local task system** at `docs/work/` that holds PRDs, epics, stories, and tasks as markdown, parseable by both humans and agents. +4. The **sandcastle orchestrator** that dispatches implementer and reviewer agents per task, respecting a dependency DAG. + +These four pillars are co-designed. The conformance system is the enforcement substrate; the workflow is the shape of work; the task system is the address space; sandcastle is the dispatch loop. + +The conformance design is illustrated separately at [`docs/architecture/feature-conformance-explainer.html`](./feature-conformance-explainer.html). This document complements that with the surrounding workflow + tooling. + +## Mental model + +| Pillar | What it is | Primary artifact | +|---|---|---| +| **Conformance engine** | manifest + TS brands + ESLint + boot-time assertion + CI gate | `feature.manifest.ts` per feature; `_state.json` snapshot of compliance | +| **Agent workflow** | PRD → Epic → Story → Task; manifest-first ordering; TDD-per-slice | `docs/work/**/*.md` | +| **Local task system** | filesystem markdown, single state file, dispatchable | `docs/work/` | +| **Sandcastle orchestrator** | implementer + reviewer agents per task, DAG-respecting, retry-capped | `.sandcastle/` config + `scripts/work-*.ts` | + +The same artifacts are read by humans, AI implementers, AI reviewers, the orchestrator, and the pre-commit hooks. There is no separate "process layer." + +## Hierarchy + +``` +PRD (initiative — one .prd.md) +└── Epic (large body of work — one folder + _epic.md) + └── Story (one use case OR one technical capability) + └── Task (one vertical slice = one PR = one commit) + └── Subtask (rare; only for unexpectedly complex slices) +``` + +**Feature is metadata, not a hierarchy level.** The use-case identifier (`auth.signUp`) already encodes the feature. ClickUp/Linear/etc. tags can carry it; in this system it sits in frontmatter. + +**Story type is metadata** — `user-story` or `technical-story` — same hierarchy slot, different body template: + +- **User story:** `As a , I want , so that ` +- **Technical story:** `Goal / Why / Done when` + +## File system + +``` +docs/work/ +├── README.md # how this folder is used +├── _state.json # derived, committed, orchestrator-managed +├── _templates/ +│ ├── prd.template.md +│ ├── epic.template.md +│ ├── user-story.template.md +│ ├── technical-story.template.md +│ └── task.template.md +├── prds/ +│ ├── 2026-05-12-conformance-system.prd.md +│ └── ... +├── conformance-system-v1/ # one folder per epic +│ ├── _epic.md +│ ├── 01-define-feature-helper/ # one folder per story +│ │ ├── _story.md +│ │ ├── 01-define-feature-helper-exists.task.md +│ │ ├── 02-instrumented-brand-attached.task.md +│ │ └── ... +│ └── ... +└── work-system-v1/ + └── ... + +.sandcastle/ +├── Dockerfile # extends existing CI image +├── implementer.prompt.md +├── reviewer.prompt.md +├── decomposer.prompt.md +├── prd-eliciter.prompt.md +└── .env.example + +scripts/ +├── work-prd-new.ts # invokes PRD elicitation skill +├── work-decompose.ts # PRD → epic + stories +├── work-decompose-tasks.ts # story → tasks +├── work-dispatch.ts # orchestrator loop +├── work-status.ts # human-readable status tree +└── work-rebuild-state.ts # regen _state.json from markdown +``` + +### Naming conventions + +- **Underscored system files** (`_epic.md`, `_story.md`, `_state.json`, `_templates/`) — orchestrator-managed indexes or templates +- **Numeric prefix on filenames** (`01-`, `02-`) — execution order; doubles as sort key +- **`.task.md`** — individual tasks +- **`-.prd.md`** — PRDs date-prefixed for chronological sort + +## File formats + +### PRD — `docs/work/prds/-.prd.md` + +```markdown +--- +id: 2026-05-12-conformance-system +title: Feature Conformance System +type: prd +status: draft | in-review | approved | superseded +author: danijel +elicitation-session: +created: 2026-05-12 +--- + +## Problem +What's broken or missing today? Who hurts because of it? + +## Goal +What state are we trying to reach? + +## In scope +- ... + +## Out of scope +- ... + +## Constraints +- ... + +## Success criteria +- ... + +## Requirements +- R1: ... +- R2: ... + +## Open questions +- Q1: ... +``` + +**Authoring flow:** +1. Human runs `pnpm work prd-new ""` +2. Agent invokes the **PRD elicitation skill** — runs a question-driven interview with the human (similar in shape to `superpowers:brainstorming`) until it has enough context across Problem / Goal / Scope / Constraints / Success / Requirements +3. Agent drafts PRD with `status: draft` +4. Human reviews, edits, flips to `status: approved` +5. **Decomposer refuses to run on `draft` PRDs.** + +### Epic — `/_epic.md` + +```markdown +--- +id: conformance-system-v1 +prd: 2026-05-12-conformance-system +title: Conformance system v1 +type: epic +status: todo | in-progress | done | cancelled +features: [cross-cutting] +created: 2026-05-12 +target: 2026-Q3 +--- + +## Goal +Build the feature-conformance enforcement system so AI agents get +layered, sub-second feedback on drift between manifest and code. + +## Why +(brief — link to PRD for detail) + +## In scope +- ... + +## Out of scope +- ... + +## Stories +- [ ] [01 — defineFeature helper + Instrumented brand](01-define-feature-helper/_story.md) +- [ ] [02 — Boot assertions](02-boot-assertions/_story.md) +- ... +``` + +### Story (technical) — `//_story.md` + +```markdown +--- +id: 01-define-feature-helper +epic: conformance-system-v1 +title: defineFeature helper + Instrumented brand +type: technical-story +status: todo | in-progress | done +feature: core-shared +depends-on: [] +blocks: [02-boot-assertions, 05-generator-updates] +--- + +## Goal +Manifest helper + brand types enable type-level enforcement that every +use-case binding is wrapped with `withSpan` + `withCapture` +(and `withAudit` when mutating). + +## Why +Compile-time feedback is the cheapest layer and the foundation every other +milestone reads. + +## Done when +Compile-time TS2322 fires at the IDE when an unwrapped factory is bound +through `ProductionUseCase<...>`. + +## In scope +- `defineFeature` helper signature + tests +- Brand types: `Instrumented`, `Captured`, `Audited` +- Wiring brands into existing wrappers (no API changes) +- `auth` as the reference feature using the new pattern + +## Out of scope +- Migration of other features (each is its own story) +- Boot-time `assertConformance` (story 02) +- ESLint rules consuming the brands (story 03) + +## Tasks +- [ ] [01 — defineFeature helper exists](01-define-feature-helper-exists.task.md) +- [ ] [02 — Instrumented brand attached via withSpan](02-instrumented-brand-attached.task.md) +- ... +``` + +### Story (user) — same skeleton, body uses As a / I want / So that + +```markdown +--- +id: 01-sign-up +epic: auth-v1 +title: Sign up with email and password +type: user-story +status: todo +feature: auth +depends-on: [] +--- + +## As a / I want / So that +**As a** visitor +**I want** to create an account with email and password +**So that** I can access member-only content + +## In scope +- Email/password sign-up flow +- Password hashing +- Audit + event emission on success +- tRPC procedure exposure + +## Out of scope +- OAuth sign-up (separate story) +- Email verification (separate story) + +## Tasks +- [ ] [01 — reject invalid email format + scaffold](01-reject-invalid-email-format.task.md) +- [ ] [02 — reject duplicate email](02-reject-duplicate-email.task.md) +- ... +``` + +### Task — `//.task.md` + +```markdown +--- +id: 02-instrumented-brand-attached +story: 01-define-feature-helper +epic: conformance-system-v1 +title: Attach Instrumented brand via withSpan +type: task +status: todo | ready | in-progress | done | escalated +depends-on: [01-define-feature-helper-exists] +blocks: [06-signin-rebound-via-branded-slot] +sandbox: default +max-attempts: 3 # default; override per task +attempts: { implementer: 0, reviewer: 0 } +--- + +## Goal +Attach the `Instrumented` brand to functions returned by `withSpan`. + +## Why this matters +The brand is the type-level seam the binding signature checks. Without it, +the compiler can't tell a wrapped factory from an unwrapped one. + +## Acceptance criteria +- [ ] `Instrumented` = `F & { readonly __instrumented: true }` +- [ ] `withSpan` return type is `Instrumented` +- [ ] Brand re-exported from `@repo/core-shared/conformance` +- [ ] Test asserts wrapped function carries brand at the type level +- [ ] All existing tests still pass + +## Out of scope +- Updating `with-capture` and `with-audit` (separate tasks) +- Refactoring `withSpan`'s existing signature beyond adding the brand +- Adding runtime brand markers (type-only) +- Renaming existing types or symbols + +## Files likely touched +- `packages/core-shared/src/instrumentation/with-span.ts` +- `packages/core-shared/src/instrumentation/with-span.test.ts` +- `packages/core-shared/src/conformance/index.ts` + +## Reviewer notes +Reject if brand is implemented with runtime tag rather than pure type. +``` + +## State file — `docs/work/_state.json` + +A derived, committed, orchestrator-written index. Markdown is source of truth; `_state.json` is a fast-to-query mirror. + +```json +{ + "updated_at": "2026-05-12T16:42:00Z", + "ready": ["02-instrumented-brand-attached"], + "in_progress": [], + "blocked": [], + "escalated": [], + "epics": { + "conformance-system-v1": { + "status": "in-progress", + "ac_total": 47, + "ac_completed": 8, + "stories": { + "01-define-feature-helper": { + "status": "in-progress", + "ac_total": 9, + "ac_completed": 4, + "tasks": { + "01-define-feature-helper-exists": { + "status": "done", + "depends_on": [], + "blocks": ["02-instrumented-brand-attached"], + "ac_total": 4, + "ac_completed": 4, + "attempts": { "implementer": 1, "reviewer": 1 }, + "branch": "task/01-define-feature-helper-exists", + "completed_at": "2026-05-12T14:23:00Z" + } + } + } + } + } + } +} +``` + +### Rules for `_state.json` + +1. **Committed to git.** Audit trail visible in PRs. +2. **Single writer:** orchestrator + pre-commit hook only. No agent writes it directly. +3. **Derived from markdown.** Regenerable any time via `pnpm work rebuild-state`. +4. **Canonical formatting.** Sorted keys, stable indentation, no trailing whitespace. Pre-commit normalizes via Prettier. +5. **Merges serialized.** Orchestrator merges PRs one at a time. Parallel implementation in sandboxes is fine; only merge step is sequential. +6. **Pre-commit regen.** When any `.task.md` / `.story.md` / `_epic.md` is staged, the hook regenerates `_state.json` from the markdown, re-stages it, and lets the commit proceed. The hook only blocks the commit if regeneration itself fails (e.g. malformed frontmatter, broken `depends-on` reference). This makes the markdown the unambiguous source of truth: if humans edit checkboxes directly, the JSON quietly catches up. + +## Scope guards + +| Level | In scope | Out of scope | +|---|---|---| +| PRD | **required** | **required** | +| Story | **required** | **required** | +| Task | implicit (= AC list) | **optional but encouraged** | + +The reviewer agent **explicitly checks the task's `Out of scope` section against the diff**. Rejects if the diff touches anything declared out of scope. This is the cheapest possible enforcement of "don't over-engineer" — pure text match, no AST needed. + +## Conformance system integration + +The four enforcement layers (detailed in [`feature-conformance-explainer.html`](./feature-conformance-explainer.html)): + +| Layer | Latency | Catches | +|---|---|---| +| TypeScript brands | 0s | forgotten `withSpan` / `withAudit`; manifest ↔ binding-slot type mismatch | +| AST-aware ESLint | <1s | manifest ↔ code drift; undeclared `bus.publish` / `auditLogger.log`; required cores not installed | +| Boot assertion (`assertConformance`) | ~3s | binding type-casts that hid unwrapped factories; manifests edited without rebinding | +| CI drift gate (`pnpm conformance`) | ~120s | orphan event consumers; scaffold drift from generator; required-cores ↔ workspace mismatch | + +### How conformance interacts with tasks + +When a task adds an audit emission (e.g. `audits: ["user.created"]`): + +1. Agent edits `feature.manifest.ts` +2. The binding's branded slot type *now* demands `Audited` — TS2322 if the wrapper is missing +3. Agent adds `withAudit(...)` in `bind-production.ts` → TS goes quiet +4. Agent adds `auditLogger.log(...)` in the use-case factory → ESLint goes quiet +5. Pre-commit `pnpm conformance` confirms all four layers pass +6. PR submitted + +Each step gives sub-second feedback. The agent's iteration loop is dominated by think + write, not by waiting for feedback. + +## Workflow ordering (per task) + +For any new use case or new behavior: + +1. **Manifest** — declare the use case (or update audits/publishes/consumes if the task adds them). Pure declaration. +2. **Contracts** — `xInputSchema`, `xOutputSchema`, `IXUseCase` type alias in the use-case file. Factory body throws `"not implemented"` if not yet written. +3. **Tests (red)** — import contracts; write failing assertions that match the AC bullet. +4. **Implementation (green)** — fill factory body, repository, binding, until tests pass. + +For incremental work on an existing use case, step 1 is often a no-op (manifest already declared). For the first slice of a new use case, all four steps happen in one commit. + +## Work shapes + +The Epic / Story / Task hierarchy holds for everything; the **inner workflow shape varies** with the kind of work. Three shapes are recognised: + +| Shape | Default home | Manifest involvement | Test gates | +|---|---|---|---| +| **Backend** | feature packages | full (use cases, audits, publishes, consumes, jobs, realtime) | type-check + lint + conformance + unit/integration | +| **Frontend** | `@repo/core-ui` and `features//src/ui/` | partial — pages consume use cases via controllers | type-check + lint + component tests + Playwright screenshot (CI) | +| **Infrastructure** | core packages, `apps/*/server/`, `docker-compose.yml`, `.github/`, ADRs | declarative — `requiredCores`, bind context | type-check + lint + conformance + ADR review | + +The default shape is **backend** — what the rest of this doc describes. The two adapted shapes are summarised below; operational detail lives in their guides. + +### Frontend (see `docs/guides/frontend-work-shape.md`) + +- **Atomic design tiers** — atoms / molecules / organisms / templates / pages, generated via `pnpm turbo gen core-ui-component`. Tier-direction enforced by a new ESLint rule (`atomic-tier-import-direction`). +- **Storybook is the spec.** Each AC bullet on a UI task maps to a story variant or a Storybook `play` function. Story files become the shared visual contract between human, implementer, and reviewer. +- **Two test layers:** + - **Component tests** (Vitest + Testing Library, or `play` on stories) — pre-commit gate, <5s + - **Visual regression** via **Playwright screenshot tests** — CI gate, 30–120s, blocks PR merge on unapproved visual diffs +- **Adapted four-step ordering for a pure UI slice:** + 1. **Story file** (the visual spec — analogous to the manifest entry for backend work) + 2. **Contracts** — props interface, variant types + 3. **Tests (red)** — component test + a default story + 4. **Implementation (green)** — make the component render and pass tests +- **Reviewer agent** uses the existing Storybook MCP at `http://localhost:6006/mcp` to read existing components, list variants, and verify story coverage against the task's AC. Visual diff verdicts come from the Playwright screenshot CI step. +- **Story split for page-level features:** + ``` + Epic: auth-v1 + ├── Story (user): auth.signUp use case ← backend slices + ├── Story (technical): SignUpForm component(s) ← depends-on: signUp use case + └── Story (technical): /sign-up page composition + E2E ← depends-on: SignUpForm + ``` + Each `depends-on` edge enforces sequential dispatch by the orchestrator. + +### Infrastructure (see `docs/guides/infrastructure-work-shape.md`) + +- **ADRs precede infrastructure work** the same way PRDs precede feature work — decision first, code second. ADRs live at `docs/adr/NNN-.md`. +- **Two categories:** + - **New optional core package** — `pnpm turbo gen core-package `. Generator + conformance already accommodate (via `requiredCores` in manifests). No conformance extensions required. + - **New infrastructure layer** (Redis, CDN, alternative CMS, additional message bus, …) — ADR + integration PRD + multiple stories. +- **ADR authoring flow:** + 1. Human runs `pnpm work adr-new ""` + 2. **Dedicated ADR elicitation skill** interviews the human on Context / Drivers / Considered options / Trade-offs / Decision / Consequences (similar shape to the PRD elicitation skill but distinct template + heuristics) + 3. Agent drafts ADR at `docs/adr/NNN-.md` with `status: proposed` + 4. Human reviews, flips to `status: accepted` (or `rejected` / `superseded`) + 5. Accepted ADR(s) trigger integration PRD(s); the PRD flow proceeds normally +- **Conformance extensions for infra:** + - `core-package-shape-conforms-to-generator` — extends milestone iv's scaffold-drift check to core packages + - `required-cores-in-workspace` — manifest declarations must match `pnpm-workspace.yaml` (already in milestone iv) + +Two elicitation skills now sit at the funnel mouth — one for PRDs, one for ADRs. Same interview-style intake; different templates and decision frameworks. + +## Pre-commit gates + +When a commit lands in the sandbox or locally: + +1. **Type-check** — brand satisfaction, manifest typing +2. **Lint** — manifest ↔ code rules, in-file shape rules, pattern restrictions +3. **Conformance script** — `pnpm conformance` (boot-style assertion at static scope) +4. **Tests for changed feature** — `pnpm test --filter @repo/` passes +5. **`_state.json` ↔ markdown sync** — pre-commit regen verifies consistency + +Tests for *all* features are NOT required to pass at pre-commit (that's CI's job). The gate enforces local soundness without blocking work in unaffected areas. + +## Agent roles + +### PRD eliciter agent + +- Skill: dedicated PRD elicitation (interview-style, similar shape to `superpowers:brainstorming`) +- Inputs: short brief from human (`pnpm work prd-new ""`) +- Behavior: asks questions one at a time, builds shared understanding across Problem / Goal / Scope / Constraints / Success / Requirements +- Output: `-.prd.md` with `status: draft` +- Hand-off: human reviews, flips to `status: approved` + +### ADR eliciter agent + +- Skill: dedicated ADR elicitation (interview-style; distinct from PRD elicitation) +- Inputs: short brief from human (`pnpm work adr-new ""`) +- Behavior: drives the conversation across Context / Drivers / Considered options / Trade-offs / Decision / Consequences. Pushes the human to articulate alternatives explicitly before settling on a decision. +- Output: `docs/adr/NNN-.md` with `status: proposed` +- Hand-off: human reviews, flips to `status: accepted` (or `rejected` / `superseded`) +- Accepted ADRs are the trigger for downstream integration PRDs + +### Decomposer agent + +- Skill: structured PRD-to-epic-and-stories decomposition +- Inputs: a PRD file with `status: approved` +- Behavior: produces `_epic.md` and one `_story.md` per requirement, with story-level AC bullets that hint at task decomposition +- **Default scope: stories only.** Task-level decomposition is a second pass: `pnpm work decompose-tasks ` +- Does NOT write `_state.json` directly (orchestrator does that on next dispatch tick) + +### Implementer agent + +- Sandcastle dispatch with `implementer.prompt.md` +- Inputs: a single task markdown file (full context) +- Behavior: writes code + tests to satisfy the AC; runs `pnpm test --filter` and `pnpm conformance` locally; commits; pushes the sandbox branch +- **Read-only on task markdown.** Returns structured output via sandcastle: + ```json + { + "status": "complete" | "blocked" | "needs-clarification", + "ac_satisfied": [0, 1, 2, 3], + "files_changed": ["packages/.../with-span.ts", "..."], + "commit_sha": "abc123", + "notes": "..." + } + ``` +- Does NOT edit `.task.md`, `.story.md`, `_epic.md`, or `_state.json`. The orchestrator translates the structured output into markdown checkbox flips and JSON state updates in a single post-merge commit. + +### Reviewer agent + +- Sandcastle dispatch with `reviewer.prompt.md` +- Inputs: task markdown + diff from implementer +- Behavior: verifies each AC bullet against the diff; checks `Out of scope` is respected; verifies tests cover AC bullets; runs `pnpm conformance` and `pnpm test --filter` +- Returns structured output: + ```json + { + "decision": "approve" | "reject", + "ac_verified": [0, 1, 2, 3], + "scope_violations": [], + "notes": "..." + } + ``` +- Does NOT edit anything in the repo. +- **For frontend tasks**, the reviewer additionally: + - Queries the Storybook MCP (`http://localhost:6006/mcp`) to verify story coverage and inspect rendered output + - Treats the Playwright screenshot CI step's verdict as a required input — unapproved visual diffs trigger `reject` + +### Orchestrator + +- Plain TypeScript script (`scripts/work-dispatch.ts`) +- Reads `_state.json` to find ready tasks (all deps `done`) +- For each ready task: + 1. Marks task `status: in-progress`, regenerates `_state.json`, commits the marker + 2. Dispatches sandcastle implementer + 3. On implementer return: dispatches sandcastle reviewer with task + diff + 4. On reviewer `approve`: serial-merges to `main`, in one commit flips task checkbox, increments parent story checkbox if all tasks done, increments parent epic checkbox if all stories done, regenerates `_state.json` + 5. On reviewer `reject`: appends reviewer notes to the task's "Reviewer notes" section, increments `attempts.implementer`, re-dispatches (subject to `max-attempts`) + 6. On `attempts.implementer >= max-attempts`: marks `status: escalated`, posts a summary, stops dispatching +- Continues until no ready tasks remain + +## Sandcastle config + +`.sandcastle/Dockerfile` **extends the existing CI image**. To be identified during the work-system-v1 epic. Must include: + +- Node + pnpm at repo's pinned versions +- `pnpm install --frozen-lockfile` baked in +- Access to `pnpm conformance`, `pnpm test`, `pnpm lint`, `pnpm typecheck` +- Git config for agent commits + +Prompt templates use sandcastle's `{{VAR}}` substitution + `` !`cmd` `` injection: + +- `implementer.prompt.md` uses `{{TASK_FILE_CONTENT}}` and `` !`git log -1 --oneline` `` for context +- `reviewer.prompt.md` uses `{{TASK_FILE_CONTENT}}` + `{{DIFF}}` +- `decomposer.prompt.md` uses `{{PRD_FILE_CONTENT}}` +- `prd-eliciter.prompt.md` uses `{{INITIAL_BRIEF}}` and runs the interview loop + +Branch strategy: per-task feature branch (`task/`), merged sequentially to `main` by the orchestrator. + +## Phasing — conformance-first bootstrap + +### Phase 1 — Conformance system (human-driven) + +Build the conformance system manually. `docs/work/conformance-system-v1/` markdown files capture the work (practising the task format on real work) but no `_state.json`, no sandcastle dispatch, no orchestrator. + +Stories in order (each itself a vertical slice — system code + generator update + doc update + applied to one feature): + +1. **defineFeature helper + Instrumented brand** — applied to `auth.signIn` +2. **Captured + Audited brands** — wrappers updated +3. **`assertConformance` + boot wiring** — rolled to all three apps +4. **AST-aware ESLint rules** — 5–6 new type-aware rules +5. **CI drift gate** (`pnpm conformance`) +6. **Generator emits manifest + contracts + test stubs** +7. **Documentation rewrite** — agent-workflow.md, CLAUDE.md, AGENTS.md, tdd-workflow.md +8. **Migrate auth feature** to the new pattern (reference) + +### Phase 2 — Work system (human-driven, bootstraps automation) + +Once conformance is in place, build the dispatch substrate: + +1. `docs/work/` skeleton, README, templates +2. `_state.json` schema + `pnpm work rebuild-state` +3. Orchestrator script + DAG + retry-cap logic +4. `.sandcastle/` config (extends existing CI image) +5. PRD elicitation skill +6. ADR elicitation skill (separate skill, similar interview shape) +7. Decomposer agent + prompts +8. Implementer + reviewer prompts (with Storybook MCP wiring for frontend reviewer) +9. `pnpm work` CLI surface (including `work adr-new`) +10. Pre-commit hooks (state regen, conformance gate) +11. Playwright screenshot test infrastructure (CI gate for frontend tasks) + +### Phase 3 — Migration + future work (dispatch-driven) + +With both systems in place, remaining feature migrations and all future work flows through sandcastle: + +- Migrate `blog`, `media`, `navigation`, `marketing-pages` (one story each) +- All new features authored via PRD → decompose → dispatch loop + +## Deferred decisions + +These are explicitly deferred until the phase that needs them: + +1. **Existing `task-create` skill (ClickUp mirror)** — coexist or retire. Decide during Phase 2 or after. Frontmatter is open-ended so `clickup-id` can be added later if needed. +2. **Existing CI Docker image identity** — identify and document during the `.sandcastle/Dockerfile` story in Phase 2. +3. **Per-epic state files vs. one global** — start with one global `_state.json`. Move to per-epic only if serialized merges become a throughput bottleneck. +4. **Custom git merge driver for `_state.json`** — start without; serialized merges should suffice. Add if needed. + +## Open questions (to revisit during implementation) + +- **Q1: Single manifest registry per app, or per-feature?** (From the explainer §10.) Lean: per-feature, with a tiny app-side aggregator. +- **Q2: How much of the manifest is generated vs hand-written?** Lean: humans edit; ESLint flags mismatches without auto-fixing. +- **Q3: Inline symbol declaration in manifest, or registry mapping?** Lean: registry holds the mapping, manifest stays content-only. +- **Q4: What happens when an optional core is absent?** Lean: typed surface gates the field — `audits: readonly never[]` when `core-audit` is unbound. +- **Q5: Escape hatch for legitimate exceptions?** Lean: `// @conformance-skip: — ` comment honoured by ESLint + boot assertion, with allowlist growth gated in CI. + +## Acceptance criteria (for this whole design) + +This design is "done" when: + +- [ ] `docs/work/` exists with templates, `_state.json` schema, README +- [ ] Conformance system v1 is implemented through all four enforcement layers +- [ ] All five feature packages have manifests +- [ ] All three apps run `assertConformance` at boot +- [ ] `pnpm conformance` is a CI gate +- [ ] `turbo gen feature` emits manifest + contracts + test stubs +- [ ] `.sandcastle/` config exists with implementer/reviewer/decomposer/eliciter prompts +- [ ] PRD elicitation skill exists and is invocable +- [ ] ADR elicitation skill exists and is invocable +- [ ] Orchestrator (`pnpm work dispatch`) runs end-to-end on a real task +- [ ] Frontend ESLint rules cover atomic-tier direction and story/test sibling presence +- [ ] Playwright screenshot tests run as a CI gate for frontend work +- [ ] Documentation reflects the manifest-first workflow (CLAUDE.md, AGENTS.md, `docs/guides/`) +- [ ] Frontend + infrastructure work-shape guides exist at `docs/guides/` + +## References + +- [Feature conformance explainer (interactive)](./feature-conformance-explainer.html) +- [Vertical feature spec](./vertical-feature-spec.md) +- [Template tiers](./template-tiers.md) +- [Frontend work shape guide](../guides/frontend-work-shape.md) +- [Infrastructure work shape guide](../guides/infrastructure-work-shape.md) +- [TDD workflow guide](../guides/tdd-workflow.md) +- [Scaffolding a feature guide](../guides/scaffolding-a-feature.md) +- [Adding a feature guide](../guides/adding-a-feature.md) +- [Sandcastle (orchestration library)](https://github.com/mattpocock/sandcastle) diff --git a/docs/guides/frontend-work-shape.md b/docs/guides/frontend-work-shape.md new file mode 100644 index 0000000..4def475 --- /dev/null +++ b/docs/guides/frontend-work-shape.md @@ -0,0 +1,129 @@ +# Frontend work shape + +How frontend work — components, pages, media UI, anything visual — flows through the agent-first workflow. For the broader workflow context see [`docs/architecture/agent-first-workflow-and-conformance.md`](../architecture/agent-first-workflow-and-conformance.md). + +## When this guide applies + +You're working on: + +- A new or existing component (atom / molecule / organism / template / page) +- A page in `apps/web-next` or `apps/web-tanstack` +- A visual or interactive change in `@repo/core-ui` or `features//src/ui/` +- Storybook stories or visual regression tests + +If you're working on a backend use case (factory, repository, manifest entry, event handler, job, realtime channel), use the default backend shape documented in the architecture doc — not this guide. + +## Where frontend code lives + +| Surface | Location | When to use | +|---|---|---| +| Atomic-design primitives | `@repo/core-ui/src/{atoms,molecules,organisms,templates}/` | Cross-feature reusables. Generated via `pnpm turbo gen core-ui-component`. | +| Feature-scoped UI | `features//src/ui/` | Components and queries specific to a feature. Exported behind the feature's `./ui` subpath, never the root barrel (see CLAUDE.md). | +| Pages | `apps//src/...` (Next.js: `app/`, TanStack: `routes/`) | Page-level composition. Calls use cases via tRPC controllers; composes components. | + +## Atomic design conventions + +Tier rules (enforced by the `atomic-tier-import-direction` ESLint rule): + +| Tier | May import from | May NOT import from | +|---|---|---| +| `atoms` | nothing else in `core-ui` | molecules / organisms / templates / pages / features | +| `molecules` | atoms | organisms / templates / pages / features | +| `organisms` | atoms, molecules | templates / pages / features | +| `templates` | atoms, molecules, organisms | pages / features | +| `pages` | all of the above | other pages | + +Feature-scoped UI (`features//src/ui/`) follows the same tier order internally, and may import from `@repo/core-ui` at any tier. + +## Storybook is the spec + +Every component has a sibling `.stories.tsx`. The story file: + +- Declares one story per AC bullet (or one variant per AC bullet) +- Uses Storybook's `play` function for interaction tests where applicable +- Becomes the shared visual contract between human, implementer agent, reviewer agent + +**Acceptance criteria for a frontend task map directly to story variants:** + +```markdown +## Acceptance criteria +- [ ] Renders default variant +- [ ] Renders loading state +- [ ] Renders error state with `message` slot +- [ ] Renders disabled state +- [ ] Click handler fires with event payload +``` + +Each bullet becomes one `Story` export (or one `play` step on a story). + +## Test gates + +| Gate | What it covers | Tool | Latency | When it runs | +|---|---|---|---|---| +| Component test | behavior, props, interactions | Vitest + Testing Library, or `play` on stories | <5s | pre-commit, CI | +| Visual regression | rendered appearance vs. baseline | Playwright screenshot tests | 30–120s | CI only; blocks merge on unapproved diffs | + +The visual regression infrastructure ships as part of the `work-system-v1` epic. Until it's in place, frontend work relies on component tests + manual visual review. + +## Adapted four-step ordering + +For a pure UI slice (no backend coupling): + +1. **Story file** — write or extend `.stories.tsx`; declare the variant for this slice's AC bullet. *This is the visual spec — analogous to the manifest entry for backend work.* +2. **Contracts** — props interface in the component file (factory body / render body may still throw `not implemented`) +3. **Tests (red)** — component test + the new story (the rendered output is asserted as part of the visual regression baseline, or via Testing Library) +4. **Implementation (green)** — render JSX, wire interactions, satisfy AC + +For a UI slice that consumes a backend use case (e.g. a form component calling `auth.signUp`), the use case lives in a separate story with its own four steps; the UI story `depends-on` the use-case story. + +## Reviewer integration + +The reviewer agent for frontend tasks uses two extra inputs beyond the standard diff: + +1. **Storybook MCP** at `http://localhost:6006/mcp` — query existing components, list variants, fetch the metadata for stories that already exist. The reviewer can `list-all-documentation` to verify no duplicate component was created. +2. **Playwright screenshot CI verdict** — the visual diff status from CI. Unapproved diffs trigger `reject` regardless of code quality. + +Reviewer checklist for frontend tasks: + +- [ ] Every AC bullet has a matching story or `play` step +- [ ] No accidental cross-tier imports (lint already catches this, but verify) +- [ ] Component test exercises each AC bullet +- [ ] Visual regression diff is either zero or human-approved +- [ ] No new component created without checking for an existing equivalent via Storybook MCP + +## Story split for page-level features + +A user-facing feature like "sign up" decomposes into multiple stories under one epic: + +``` +Epic: auth-v1 +├── Story (user-story): auth.signUp use case +├── Story (technical-story): SignUpForm component +│ depends-on: signUp use case +└── Story (technical-story): /sign-up page + depends-on: SignUpForm +``` + +Each story has its own AC checklist; the `depends-on` edges enforce sequential dispatch by the orchestrator. The page story typically includes the Playwright E2E test for the full flow. + +## Conformance for frontend + +New ESLint rules added under the frontend work-shape: + +- `component-must-have-story` — every `.tsx` file exporting a default OR named React component must have a sibling `.stories.tsx` +- `component-must-have-test` — every component must have a sibling `.test.tsx` (or be covered by a `play` function on its story) +- `atomic-tier-import-direction` — tier imports must respect the table above +- `story-must-cover-all-prop-variants` (advisory) — discriminated-union props should have a story per variant + +## Common pitfalls + +- **Stale stories.** A component changes prop shape; stories don't get updated. The `component-must-have-story` rule catches missing stories but not stale ones. Reviewer should flag this; visual regression catches the rendered consequence. +- **Visual-only changes without story update.** A CSS tweak that doesn't change behavior still needs the story to be re-snapshotted. Treat the visual regression diff approval as part of the slice. +- **Cross-tier creep.** Tempting to import an organism from inside an atom for "convenience". The lint rule blocks it; refactor the shared bit down to an atom or molecule. +- **Duplicate components.** An agent creates a `Button` in a feature's `ui/atoms/` when one already exists in `core-ui`. Reviewer queries Storybook MCP for `Button` before approval; if duplicated, rejects. + +## Open items (filled in as we ship) + +- Playwright screenshot infrastructure — exact setup, baseline storage, approval flow +- Visual regression PR check — GitHub workflow + diff hosting +- Storybook test integration (`test:stories`) — current status and what we extend diff --git a/docs/guides/infrastructure-work-shape.md b/docs/guides/infrastructure-work-shape.md new file mode 100644 index 0000000..d3bc11a --- /dev/null +++ b/docs/guides/infrastructure-work-shape.md @@ -0,0 +1,154 @@ +# Infrastructure work shape + +How infrastructure work — new core packages, new external services, new database backends, new build/deploy primitives — flows through the agent-first workflow. For the broader workflow context see [`docs/architecture/agent-first-workflow-and-conformance.md`](../architecture/agent-first-workflow-and-conformance.md). + +## When this guide applies + +You're proposing or implementing: + +- A new optional core package (`core-cache`, `core-email`, `core-feature-flags`, …) +- A new external service or layer (Redis, CDN, alternative CMS, additional message bus, …) +- A change to bootstrap, build, or CI infrastructure +- A swap of an existing infrastructure component + +If you're working on backend feature code or frontend, this is not your guide. + +## Two categories + +| Category | Example | Path | +|---|---|---| +| **A. New optional core package** | `core-cache`, `core-email` | `pnpm turbo gen core-package ` | +| **B. New infrastructure layer** | Redis, CDN, deploy target, CI image swap | ADR → integration PRD → epic + stories | + +Category A is well-trodden: the generator emits a canonical core-package shape; the conformance system already handles optional cores via `requiredCores` in manifests. No new conformance rules required. + +Category B is the one that needs the ADR-first dance described below. + +## ADRs precede infrastructure work + +ADRs (Architecture Decision Records) live at `docs/adr/NNN-.md`. The pattern is established in this repo (ADR-018 captured the audit-and-compliance system, for example). + +ADRs are to infrastructure what PRDs are to features: decision documents, written first, providing the rationale that downstream PRDs and stories implement. An ADR documents *what was decided and why*; a PRD documents *what to build*. + +## ADR authoring flow + +``` +1. Human runs `pnpm work adr-new ""` +2. Dedicated ADR elicitation skill interviews the human: + - Context (what's the situation today?) + - Drivers (what's forcing a decision?) + - Considered options (what alternatives are on the table?) + - Trade-offs (what does each option cost?) + - Decision (which option, and why?) + - Consequences (what changes downstream?) +3. Agent drafts ADR at docs/adr/NNN-.md with status: proposed +4. Human reviews, edits, flips to status: accepted (or rejected / superseded) +5. Accepted ADR(s) trigger one or more integration PRDs +6. PRDs flow through the normal decompose → dispatch loop +``` + +The ADR elicitation skill is distinct from the PRD elicitation skill. Same interview shape, different template and pushiness: + +- PRD eliciter focuses on **problem framing** and **success criteria** +- ADR eliciter focuses on **alternatives** and **trade-offs** — it actively pushes back if only one option is articulated + +## ADR template + +```markdown +--- +id: NNN +title: +status: proposed | accepted | rejected | superseded +date: YYYY-MM-DD +supersedes: [] +superseded-by: null +related-prds: [] +--- + +## Context +What's the situation? What's broken or about to break? Who is affected? + +## Drivers +What's forcing this decision now? + +## Considered options +- Option 1: ... + - Pros / Cons +- Option 2: ... + - Pros / Cons +- Option 3: ... + - Pros / Cons + +## Decision +We chose Option X because ... + +## Consequences + +### Positive +- ... + +### Negative / accepted trade-offs +- ... + +### Follow-up work +- PRD: ... +- PRD: ... +``` + +## Path A: new optional core package + +When the new infrastructure is *small enough to be a single package* (cache, email, feature flags, etc.): + +1. ADR (if the decision is non-trivial — for "add core-cache because we need response caching", a short ADR is still worthwhile) +2. `pnpm turbo gen core-package ` — generator emits canonical shape +3. Implement package contents (interface, impls, tests) — flows through normal story-and-task decomposition +4. Add `` to manifests' `requiredCores` in features that adopt it +5. Wire into `BindContext` in `core-shared/di/` +6. Wire into each app's `bindAll()` aggregator +7. Update generator templates if features need to scaffold differently when this core is present (e.g., `core-events` causes `turbo gen feature` to emit event-handler stubs) + +The conformance system catches forgotten wiring automatically: + +- `required-cores-installed` ESLint rule flags manifests declaring `requiredCores: [""]` when the package isn't in `pnpm-workspace.yaml` +- Boot assertion `assertConformance(ctx)` fails on missing bind-context entries +- CI's `pnpm conformance` aggregates the cross-feature view + +## Path B: new infrastructure layer + +When the new infrastructure is bigger than one package — a new external service, a swap of an existing component, a deploy-target change: + +1. **ADR** captures the decision and trade-offs +2. **Integration PRD(s)** specify what to build/wire/migrate; one PRD per integration concern is common +3. PRDs decompose into the normal Epic → Story → Task hierarchy +4. Stories typically include: a new core package (if one fits), configuration plumbing, bootstrap wiring, feature adoption, documentation updates, and (often) a deprecation/removal of the prior approach + +Examples of category B work: + +- "Add Redis for response caching" — ADR + core-cache package + bind-context wiring + adoption stories per feature +- "Replace Payload with Sanity" — ADR + repository implementation swap + migration scripts + docs rewrite +- "Move from Vercel to Cloudflare Workers" — ADR + new deploy target stories per app + CI workflow updates + +## Conformance for infrastructure + +Two rules extend the conformance system for infra concerns: + +| Rule | Layer | What it catches | +|---|---|---| +| `required-cores-in-workspace` | ESLint + CI | manifest declares a core that isn't in `pnpm-workspace.yaml` | +| `core-package-shape-conforms-to-generator` | CI | a core package's structure has drifted from what `pnpm turbo gen core-package ` would produce today | + +These extend the existing milestone iv work; no new tooling required. + +## Common pitfalls + +- **Skipping the ADR for "small" infra changes.** A swap of a single dependency is still a decision worth recording. The ADR doesn't have to be long — but it should exist. +- **Implementing before the ADR is accepted.** Don't decompose an integration PRD into stories until the ADR is `accepted`. The work-system orchestrator can refuse to dispatch tasks whose epic's ADR is still `proposed`. +- **Manifest drift after infra adoption.** A new core is added but features that use it don't declare it in `requiredCores`. The ESLint rule catches the inverse (declared-but-not-installed); CI's full conformance run also checks the other direction. +- **Bootstrap wiring forgotten.** A new core package is created but never bound in `apps/*/server/bind-production.ts`. Boot assertion catches this on first `pnpm dev`. +- **Documentation lag.** New infra is shipped but `docs/guides/` and `CLAUDE.md` aren't updated. Make documentation a required story under any integration epic. + +## Open items (filled in as we ship) + +- ADR elicitation skill — exact prompt shape and heuristics +- ADR ↔ PRD linkage — how an accepted ADR fans out into one or more PRDs (frontmatter `related-prds`?) +- Refusal logic — should the work-system refuse to decompose a PRD whose triggering ADR is still proposed?