docs: introduce library evaluation policy (ADR-022 + PRD)
- ADR-022 codifies the tiered library-evaluation policy: 8 hard auto-reject filters (license, types, maintenance, boundary-fit, shadow-check, EU residency, CVE scan, named consumer), 3 discussion prompts, per-decision trace artifact at docs/library-decisions/, and a 4-layer enforcement stack (Claude PreToolUse/PostToolUse hook -> evaluate-library skill -> pre-commit hook -> sandcastle reviewer prompt). Mirrors the conformance-system latency pattern from ADR-012. - PRD at docs/work/prds/2026-05-14-library-evaluation-policy.prd.md seeds the implementation epic; status: approved, ready for \`pnpm work decompose\`. - Glossary gains "Library trace" + "Pre-shipped trace" entries referenced by both artifacts. Catalyst: the 2026-05-14 grill session nearly adopted trpc-to-openapi + zod-to-json-schema before someone asked who the HTTP consumer was. Honest answer: none -- all callers are TS via createCaller. This policy makes that question structurally unavoidable for any future feature- or core-tier dep.
This commit is contained in:
272
docs/decisions/adr-022-library-evaluation-policy.md
Normal file
272
docs/decisions/adr-022-library-evaluation-policy.md
Normal file
@@ -0,0 +1,272 @@
|
||||
# ADR-022 — Library evaluation policy
|
||||
|
||||
**Status:** Accepted
|
||||
**Date:** 2026-05-14
|
||||
**Related:** ADR-006 (vertical feature packages), ADR-010 (turbo boundaries), ADR-014 (Sentry observability), ADR-017 (OpenTelemetry + vendor isolation), ADR-019 (sandcastle agent orchestration), ADR-021 (release-please versioning)
|
||||
**Companion guide:** `docs/guides/adding-a-library.md` (human reading-room)
|
||||
**Companion skill:** `.claude/skills/evaluate-library/SKILL.md` (authoritative agent runbook)
|
||||
|
||||
## Context
|
||||
|
||||
This template ships with a deliberately narrow third-party surface. Every feature
|
||||
package today carries the same six runtime dependencies — `@repo/core-shared`,
|
||||
`@trpc/server`, `inversify`, `payload`, `reflect-metadata`, `zod` — and nothing
|
||||
else. That uniformity isn't an accident; it's the result of unstated discipline
|
||||
that the boundary-tag system (ADR-006, ADR-010) and the manifest-first ordering
|
||||
(ADR-012) silently reward.
|
||||
|
||||
The discipline is not codified. New dependencies get added by anyone — human or
|
||||
agent — running `pnpm add <pkg>`, with no checkpoint between intent and lockfile.
|
||||
Three recent signals show the gap:
|
||||
|
||||
1. An exploratory grill session on **2026-05-14** nearly added `trpc-to-openapi`
|
||||
plus `zod-to-json-schema` plus a build-time generator to the repo before
|
||||
stopping to ask "who calls this code path?" The honest answer was "nobody —
|
||||
all callers are TypeScript via `createCaller`." The library would have shipped
|
||||
~30 lines of `.meta({...})` annotations per router and a `superjson`-incompatible
|
||||
HTTP handler in exchange for zero downstream consumers. Pure carrying cost,
|
||||
caught by a chance question, not by a system.
|
||||
2. **Three existing ADRs already record post-hoc library decisions** —
|
||||
ADR-002 (Inversify), ADR-014 (Sentry), ADR-017 (OpenTelemetry). Each notes
|
||||
"we picked X over Y" but the records were written after adoption. By the time
|
||||
the ADR existed the lockfile already held the dep. No mechanism existed to
|
||||
catch a _bad_ choice before it became a migration project.
|
||||
3. The repo's automation depends on the lockfile staying small and predictable.
|
||||
`pnpm fallow` audits for dead code; `pnpm conformance` audits manifest drift;
|
||||
`pnpm coverage:diff` audits change coverage. There is no equivalent audit for
|
||||
"did we just adopt a library nobody asked for?"
|
||||
|
||||
A fourth pressure comes from this template being EU-resident and GDPR-bound.
|
||||
A library that defaults to a US-only SaaS endpoint (telemetry, analytics, AI,
|
||||
log aggregation) silently moves user data out of the EU as soon as it's imported
|
||||
and configured with defaults. The current process has no point where that gets
|
||||
caught.
|
||||
|
||||
The decision below codifies the de-facto discipline, formalizes the agent-loop
|
||||
hook that prevents drift, and makes rejection records first-class so future
|
||||
agents don't re-evaluate libraries that were already rejected for known reasons.
|
||||
|
||||
## Decision
|
||||
|
||||
Adopt a **tiered library-evaluation policy** with eight hard auto-reject filters,
|
||||
three discussion prompts, a per-decision trace artifact, and a four-layer
|
||||
enforcement stack.
|
||||
|
||||
### 1. Tiered trigger (by boundary tag)
|
||||
|
||||
| Tier the dep lands in | Process | Companion record |
|
||||
| ------------------------------------ | ------------------------ | ---------------- |
|
||||
| `apps/<x>` | Author's call; no policy | – |
|
||||
| `feature` (e.g. `packages/auth`) | Trace required | – |
|
||||
| `core` (e.g. `packages/core-shared`) | Trace required | ADR required |
|
||||
| New optional-core category | Trace required | ADR required |
|
||||
|
||||
The trigger maps onto the workspace-tag boundary already enforced by ESLint
|
||||
(`eslint-plugin-boundaries`) and Turborepo (`turbo boundaries`). No new mental
|
||||
model — the policy is a corollary of an existing one.
|
||||
|
||||
### 2. Eight hard auto-reject filters
|
||||
|
||||
Failing any single filter is an automatic reject. Trace records the failure.
|
||||
|
||||
1. **License allowlist.** Only `MIT`, `Apache-2.0`, `BSD-*`, `ISC`, `MPL-2.0`.
|
||||
Anything else (GPL family, AGPL, CC-BY-NC, custom EULAs) is a no.
|
||||
2. **TypeScript-native or `@types/*` available.** The repo is strict TS;
|
||||
un-typed JS libraries shift maintenance cost to the integrating feature.
|
||||
3. **Not abandoned.** Last release < 18 months **AND** PR/issue activity
|
||||
< 12 months. The triple-AND avoids killing finished-but-stable libraries
|
||||
(`reflect-metadata`-style).
|
||||
4. **Boundary-tag fit.** A dep added to a feature package cannot require
|
||||
imports the boundary rules forbid (e.g. a Sentry SDK in a feature —
|
||||
ADR-017 §4 forbids this).
|
||||
5. **Doesn't shadow an existing must-have.** Proposing `valibot` when `zod`
|
||||
is locked, or `tsyringe` when Inversify is locked by ADR-002, is an
|
||||
auto-reject; the replacement must be a separate ADR with consequences
|
||||
analysis, not a parallel adoption.
|
||||
6. **EU data residency for hosted/SaaS components.** If the package transmits
|
||||
user data, telemetry, or business state to a vendor-controlled endpoint
|
||||
by default, that vendor must offer an EU data region, and the integration
|
||||
must be configured to use it. Self-hostable packages, on-device libraries,
|
||||
and build-time-only tools are exempt.
|
||||
7. **CVE scan clean.** `pnpm audit --audit-level=moderate` clean at adoption
|
||||
time. Documented allowlist mechanism for accepted-risk advisories.
|
||||
8. **Named consumer exists now, not hypothetically.** The integration must
|
||||
answer "who calls this code path today, or who is blocked waiting for it?"
|
||||
"Future code that might exist" is not a consumer. This filter is the
|
||||
direct response to the 2026-05-14 OpenAPI near-miss.
|
||||
|
||||
### 3. Three discussion prompts
|
||||
|
||||
Filters that don't auto-reject but must be answered in the trace, either
|
||||
direction acceptable with justification.
|
||||
|
||||
- **What does it replace?** New-and-old running in parallel is a smell.
|
||||
- **Migration cost out.** What does ripping this back out look like 18 months
|
||||
from now? Mechanical, hard, or impossible?
|
||||
- **Alternatives considered.** Two named alternatives at minimum (or "none
|
||||
with explanation"). Required for `feature`-tier; required _and_ duplicated
|
||||
into the ADR for `core`-tier.
|
||||
|
||||
### 4. Trace artifact
|
||||
|
||||
Every decision — approved or rejected — emits a trace file at
|
||||
`docs/library-decisions/<YYYY-MM-DD>-<package-name>.md`. Always written,
|
||||
regardless of whether an accompanying ADR exists. Shape:
|
||||
|
||||
```markdown
|
||||
---
|
||||
package: <name>
|
||||
version: "<semver range>"
|
||||
tier: app | feature | core
|
||||
decision: approved | rejected
|
||||
date: <YYYY-MM-DD>
|
||||
deciders: [<author>, ...]
|
||||
adr: adr-NNN | null
|
||||
filter-results:
|
||||
license: <SPDX id>
|
||||
types: native | "@types/<x>" | none
|
||||
maintenance: active | dormant | abandoned
|
||||
boundary-fit: pass | fail
|
||||
shadow-check: pass | fail | "shadows <x>"
|
||||
eu-residency: ok | n/a | self-hostable | fail
|
||||
cve-scan: clean | "<advisory-id>" | fail
|
||||
named-consumer: pass | fail
|
||||
verification-commands:
|
||||
- <literal command that produced each filter result>
|
||||
---
|
||||
|
||||
## Filter: <name>
|
||||
|
||||
<prose>
|
||||
|
||||
## Prompt: <name>
|
||||
|
||||
<prose>
|
||||
```
|
||||
|
||||
Frontmatter is the machine surface (greppable, schema-stable). Headings are
|
||||
the human surface. Rejection traces are first-class — the OpenAPI scenario,
|
||||
had this policy existed, would have produced a permanent record so the next
|
||||
agent considering `trpc-to-openapi` finds the prior reasoning in <1s of
|
||||
`ls docs/library-decisions/`.
|
||||
|
||||
### 5. Four-layer enforcement stack
|
||||
|
||||
Mirrors the latency-tiered shape of the conformance system (ADR-012).
|
||||
|
||||
| Layer | Latency | Catches |
|
||||
| -------------------------------------- | ---------- | ---------------------------------------------------------------- |
|
||||
| Claude `PreToolUse`/`PostToolUse` hook | inline | Agent skipping the skill before `pnpm add` / `package.json` edit |
|
||||
| `evaluate-library` skill | seconds | The decision itself + writes the trace |
|
||||
| Git pre-commit hook | pre-commit | Humans or agents bypassing the skill |
|
||||
| Sandcastle reviewer prompt | per-slice | Bypasses that slipped past pre-commit |
|
||||
|
||||
The Claude hook injects a `<system-reminder>` directing the agent to the skill
|
||||
but does **not** auto-deny (false-positive paths like dev-deps and app-tier
|
||||
additions are common). The pre-commit hook is the deterministic gate.
|
||||
|
||||
### 6. Composition with `pnpm turbo gen core-package`
|
||||
|
||||
The optional-cores generator emits **pre-shipped traces** — one per direct
|
||||
runtime dep of the new core — pre-marked `decision: approved` and cited
|
||||
against the relevant ADR (ADR-015 for events, ADR-016 for realtime,
|
||||
ADR-018 for audit). Same frozen-snapshot discipline that the optional cores
|
||||
already follow (`turbo/generators/__snapshots__/core-package/`). New optional
|
||||
cores added later inherit this requirement.
|
||||
|
||||
### 7. Skill invocation
|
||||
|
||||
```
|
||||
/evaluate-library <package-name> --tier <feature|core|app> --target <package-path>
|
||||
```
|
||||
|
||||
The skill walks the eight filters in **collect-cheap-skip-expensive** order:
|
||||
cheap structural filters (license, types, shadow-check, boundary-fit) run to
|
||||
completion regardless of failure; expensive filters (CVE scan, EU residency
|
||||
probe, maintenance signals) short-circuit after the first reject. The trace
|
||||
records which filters ran and which were skipped, so a partial trace is still
|
||||
useful evidence.
|
||||
|
||||
### 8. Backfill at policy adoption
|
||||
|
||||
Every existing runtime dependency in feature- and core-tier packages
|
||||
(~10 deps at this writing — `payload`, `inversify`, `zod`, `@trpc/server`,
|
||||
`reflect-metadata`, `superjson`, `@sentry/*`, `@opentelemetry/*` family,
|
||||
`socket.io`, etc.) gets a backfilled trace dated 2026-05-14 (adoption day).
|
||||
ADR-002, ADR-014, ADR-017 are cited via the `adr:` frontmatter field;
|
||||
verification-command output is captured at backfill time.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **No policy, keep relying on instinct.** Rejected. The 2026-05-14 OpenAPI
|
||||
near-miss demonstrated the gap. Instinct catches some adds, misses others;
|
||||
the failure mode is silent.
|
||||
- **Every `package.json` change requires a trace, no tier distinction.**
|
||||
Rejected. Devdeps in tooling packages and ESLint plugin bumps in apps would
|
||||
drown the trace directory in noise. The boundary-tag system already
|
||||
partitions blast radius — the policy reuses that partition.
|
||||
- **Only "category" decisions require process** (new auth provider, new ORM,
|
||||
new queue). Rejected. The OpenAPI scenario was a sub-tool inside an existing
|
||||
category, not a category swap. Category-only triggers miss it.
|
||||
- **A central library-evaluation service / Linear queue / Slack bot.**
|
||||
Rejected. The repo is agent-first and currently single-developer (+ agents).
|
||||
Human-in-the-loop services don't fit the dispatch loop; the policy must be
|
||||
agent-runnable end-to-end.
|
||||
- **No CVE filter — rely on `npm audit` ambient noise.** Rejected. CVE
|
||||
status is point-in-time; pinning the result to the trace at adoption is the
|
||||
whole value. Re-running the verification commands later detects drift.
|
||||
- **Drop the named-consumer filter to a discussion prompt.** Rejected. The
|
||||
filter is the one that would have stopped the OpenAPI flirtation; demoting
|
||||
it back to a prompt is the same as not adding the filter.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive**
|
||||
|
||||
- New dependencies require deliberate intent. The trace artifact + four-layer
|
||||
enforcement stack make accidental adds detectable at four latencies, mirroring
|
||||
the conformance-system pattern that already works.
|
||||
- Rejection records are permanent. Future agents considering a previously
|
||||
rejected library find the trace in `docs/library-decisions/` and don't
|
||||
re-litigate.
|
||||
- EU data residency becomes a binary, machine-readable filter result, not
|
||||
an afterthought.
|
||||
- Per-decision verification commands give future agents a single source of
|
||||
truth they can re-run to verify the trace is still valid.
|
||||
- The policy composes with `pnpm turbo gen core-package`: optional cores
|
||||
remain a one-command scaffold without bypassing the rule.
|
||||
|
||||
**Negative**
|
||||
|
||||
- New feature-tier deps take longer to land. Walking eight filters + writing
|
||||
the trace is ~5 minutes of agent work per addition.
|
||||
- Backfilling ~10 existing deps is one-time work; expected ~half a day of
|
||||
agent dispatch.
|
||||
- The Claude `PreToolUse` hook adds latency to every `pnpm add` invocation.
|
||||
Mitigated by the hook being a reminder-injector, not a blocker.
|
||||
- The pre-commit hook adds a new failure mode ("you added a dep but forgot
|
||||
the trace"). Mitigated by the skill being the natural path the hook
|
||||
reminders point to.
|
||||
- The CVE filter creates a maintenance obligation: when `pnpm audit` finds
|
||||
a new advisory in an already-approved dep, the trace becomes stale. Acceptable
|
||||
trade-off — staleness detection is exactly what `pnpm audit` already does;
|
||||
the trace adds a per-dep anchor for the conversation that follows.
|
||||
|
||||
**Neutral**
|
||||
|
||||
- ADR-002, ADR-014, and ADR-017 remain authoritative for their respective
|
||||
libraries. Backfilled traces cite them rather than duplicating their reasoning.
|
||||
- The policy doesn't constrain transitive dependencies; `pnpm audit` and license
|
||||
scanning already handle those recursively. Only direct deps require a trace.
|
||||
|
||||
## Related
|
||||
|
||||
- ADR-006 — Vertical feature packages (the tag system the trigger maps to)
|
||||
- ADR-010 — Turbo boundaries (the enforcement substrate)
|
||||
- ADR-012 — Feature conventions (the conformance-system shape this policy mirrors)
|
||||
- ADR-017 — OpenTelemetry migration (vendor-isolation pattern the policy extends)
|
||||
- ADR-019 — Sandcastle agent orchestration (reviewer-prompt layer of enforcement)
|
||||
- ADR-021 — release-please versioning (where dep additions show up in release notes)
|
||||
- Companion guide: `docs/guides/adding-a-library.md`
|
||||
- Companion skill: `.claude/skills/evaluate-library/SKILL.md`
|
||||
- Glossary: `docs/glossary.md` entries for **Library trace** and **Pre-shipped trace**
|
||||
@@ -296,6 +296,12 @@ A `pnpm turbo gen <kind>` invocation. Generator-first is **non-negotiable** —
|
||||
**Slice = task = PR = commit**:
|
||||
The shipping rhythm. One vertical slice closes one task, becomes one PR, lands as one commit.
|
||||
|
||||
**Library trace**:
|
||||
A per-decision artifact at `docs/library-decisions/<YYYY-MM-DD>-<package-name>.md` recording the outcome of the `evaluate-library` skill against a candidate third-party dependency. Frontmatter is the machine surface (tier, decision, filter-results enum); headings are the human surface (one per filter + one per prompt). Both `decision: approved` and `decision: rejected` traces are first-class — the rejection record is the value, since it stops future agents from re-evaluating the same library. Required for any new runtime dep in a `feature`- or `core`-tagged package; the pre-commit hook blocks the commit if the trace is missing.
|
||||
|
||||
**Pre-shipped trace**:
|
||||
A library trace emitted by `pnpm turbo gen core-package <name>` for each direct runtime dep of a pre-curated optional core. Pre-approved by the template's ADRs (015 events / 016 realtime / 018 audit). Generated alongside the core's frozen snapshot so optional cores satisfy the library-evaluation policy by default.
|
||||
|
||||
## Modes
|
||||
|
||||
**Production mode**:
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"updated_at": "2026-05-13T18:41:19.463Z",
|
||||
"updated_at": "2026-05-14T04:41:30.276Z",
|
||||
"epics": {
|
||||
"2026-05-13-binder-wrap-helper": {
|
||||
"status": "done",
|
||||
|
||||
333
docs/work/prds/2026-05-14-library-evaluation-policy.prd.md
Normal file
333
docs/work/prds/2026-05-14-library-evaluation-policy.prd.md
Normal file
@@ -0,0 +1,333 @@
|
||||
---
|
||||
id: 2026-05-14-library-evaluation-policy
|
||||
title: Library evaluation policy — skill, traces, enforcement stack
|
||||
type: prd
|
||||
status: approved
|
||||
author: danijel
|
||||
created: 2026-05-14
|
||||
adr: adr-022
|
||||
---
|
||||
|
||||
## Problem
|
||||
|
||||
This template ships with a deliberately narrow third-party surface — every
|
||||
feature package today holds the same 6 runtime deps and nothing else. That
|
||||
discipline is uncodified. New dependencies enter via `pnpm add <pkg>` with no
|
||||
checkpoint between intent and lockfile, and three recent signals show the gap:
|
||||
|
||||
1. The 2026-05-14 grill session nearly added `trpc-to-openapi` + `zod-to-json-schema`
|
||||
- a build-time generator before someone asked "who calls this code path?"
|
||||
The honest answer was "nobody — all callers are TypeScript via `createCaller`."
|
||||
2. ADR-002 (Inversify), ADR-014 (Sentry), ADR-017 (OpenTelemetry) each record
|
||||
library decisions, but every record was written _after_ adoption. No mechanism
|
||||
exists to catch a bad choice before it becomes a migration project.
|
||||
3. The repo is EU-resident and GDPR-bound. A library that defaults to a US-only
|
||||
SaaS endpoint silently moves user data out of the EU the moment it's
|
||||
imported with defaults. Nothing currently flags this.
|
||||
|
||||
ADR-022 codifies the policy. This PRD implements it.
|
||||
|
||||
## Goal
|
||||
|
||||
A four-layer enforcement stack — Claude hook, skill, pre-commit hook, sandcastle
|
||||
reviewer prompt — that makes every new runtime dependency in a feature- or
|
||||
core-tier package produce a permanent **library trace** at
|
||||
`docs/library-decisions/<YYYY-MM-DD>-<package-name>.md`, with rejection traces
|
||||
treated as first-class records.
|
||||
|
||||
## In scope
|
||||
|
||||
- The `evaluate-library` skill at `.claude/skills/evaluate-library/SKILL.md` —
|
||||
authoritative agent runbook; walks 8 hard filters + 3 prompts; writes the trace.
|
||||
- The human reading-room guide at `docs/guides/adding-a-library.md` with worked
|
||||
examples (approved + rejected).
|
||||
- The `docs/library-decisions/` directory + `_template.md` schema reference.
|
||||
- A Zod-validated trace-schema module (`scripts/library-decisions/schema.mjs`)
|
||||
shared by the skill, the pre-commit checker, and the generator.
|
||||
- Claude `PreToolUse` hook (`.claude/hooks/library-policy-nudge.sh`) — matches
|
||||
`pnpm add` / `pnpm i <pkg>` in Bash invocations; emits skill reminder.
|
||||
- Claude `PostToolUse` hook (also in `library-policy-nudge.sh`, dispatching by
|
||||
event type) — matches `Edit`/`Write` against any `**/package.json`.
|
||||
- Pre-commit hook check script (`scripts/library-decisions/check.mjs`) wired
|
||||
into `.husky/pre-commit`. Blocks the commit when a new runtime dep is staged
|
||||
in a feature/core package and no sibling trace file is staged.
|
||||
- Sandcastle reviewer prompt update (`.sandcastle/reviewer.prompt.md`) — the
|
||||
reviewer agent runs the same check before issuing approve/reject.
|
||||
- Optional-cores generator templates (`turbo/generators/templates/core-package/`)
|
||||
emit pre-shipped traces per direct dep, dated at generation time, marked
|
||||
`decision: approved`, citing the relevant ADR (015/016/018). Five generators
|
||||
updated: `events`, `realtime`, `audit`, `trpc`, `ui`.
|
||||
- Backfill traces for every existing runtime dependency in feature- and core-
|
||||
tier packages, dated 2026-05-14, citing existing ADRs (002/014/017) where
|
||||
applicable. Approx 10 traces.
|
||||
- `CLAUDE.md` "Key Conventions" gets a one-line bullet pointing to ADR-022 +
|
||||
the guide.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Transitive dependency tracing — `pnpm audit` already handles recursive scanning.
|
||||
- Bundle-size analysis — Vercel / Vite build output already reports this.
|
||||
- Auto-removal of approved-then-unused deps — `pnpm fallow` territory.
|
||||
- License auto-enforcement at the lockfile layer (license-checker plugins) —
|
||||
defer until the policy has run for some time and we know where it leaks.
|
||||
- Anything app-tier. Deps in `apps/*` are author's call per the tier model.
|
||||
- Devdeps in any tier. Only `dependencies` (runtime) require traces.
|
||||
|
||||
## Constraints
|
||||
|
||||
- **ADR-022** is the source of truth. This PRD implements but does not extend it.
|
||||
- **ADR-006 + ADR-010** — the tier trigger maps onto the existing boundary-tag
|
||||
system. No new mental model; ESLint already partitions blast radius.
|
||||
- **ADR-019** — the sandcastle reviewer prompt is one of four enforcement
|
||||
layers. Whatever the agent loop does must compose with the existing prompt
|
||||
shape at `.sandcastle/reviewer.prompt.md`.
|
||||
- **ADR-021** — release-please picks up dependency changes from commit history.
|
||||
The trace file landing in the **same commit** as the `package.json` change
|
||||
is required so release notes correlate cleanly with policy records.
|
||||
- **Conformance system parity** — the enforcement stack mirrors the 5-gate
|
||||
latency pattern from ADR-012. Same vocabulary, same agent feedback loop.
|
||||
- **Conventional Commits** — every commit produced by the implementation
|
||||
follows `<type>(<scope>): <subject>`.
|
||||
- **`--no-verify` is forbidden** — the bash-guard hook already enforces this;
|
||||
the new pre-commit check inherits that protection.
|
||||
- **Skill must be deterministic from explicit args** — `/evaluate-library <name>
|
||||
--tier <feature|core|app> --target <package-path>`. The Claude hook produces
|
||||
exactly this invocation from a `pnpm add` command line.
|
||||
|
||||
## Success criteria
|
||||
|
||||
- `pnpm typecheck && pnpm test && pnpm lint && pnpm conformance && pnpm fallow:audit`
|
||||
pass green at the end of the epic.
|
||||
- `pnpm coverage:diff` covers every changed executable line introduced by
|
||||
the implementation slices.
|
||||
- Attempting to commit a new feature-tier runtime dep without a sibling trace
|
||||
file is blocked by the pre-commit hook with a clear error pointing to the skill.
|
||||
- Running the `evaluate-library` skill against `trpc-to-openapi` (the rejected
|
||||
library from the grill session) produces a `decision: rejected` trace with
|
||||
`named-consumer: fail` and prose citing the conversation, in <90 seconds of
|
||||
agent work.
|
||||
- Running `pnpm turbo gen core-package events` (or any other optional core)
|
||||
emits pre-shipped traces for every direct dep of that core into
|
||||
`docs/library-decisions/`, all `decision: approved` and ADR-cited.
|
||||
- The Claude `PreToolUse` hook fires on `pnpm add <anything>` and emits the
|
||||
skill-reminder system-reminder; does **not** auto-deny.
|
||||
- All existing runtime deps in feature- and core-tier packages have backfilled
|
||||
trace files dated 2026-05-14 in `docs/library-decisions/`.
|
||||
- `CLAUDE.md` Key Conventions includes the one-line policy bullet.
|
||||
- `docs/glossary.md` already includes **Library trace** and **Pre-shipped
|
||||
trace** entries (landed inline during the grill session).
|
||||
|
||||
## User stories
|
||||
|
||||
1. **As a developer adding a new feature dependency**, I want a deterministic
|
||||
skill that walks me through the 8 filters and 3 prompts in collect-cheap-
|
||||
skip-expensive order, so I don't forget any check and the trace file is
|
||||
written automatically with my answers.
|
||||
2. **As an agent dispatched against a slice that needs a new dep**, I want the
|
||||
Claude `PreToolUse` hook to inject a system-reminder pointing me at the
|
||||
skill the moment I'm about to run `pnpm add`, so I don't bypass the policy
|
||||
by reflex.
|
||||
3. **As an agent editing a `package.json` by hand**, I want the Claude
|
||||
`PostToolUse` hook to inject the same reminder, so the policy isn't
|
||||
sidestepped by paste-then-install.
|
||||
4. **As a reviewer (human or agent)**, I want the pre-commit hook to refuse a
|
||||
commit that adds a runtime dep to a feature/core package without a sibling
|
||||
trace file, so I don't have to remember to check during review.
|
||||
5. **As a future agent considering a previously-rejected library**, I want
|
||||
to find the rejection trace in `docs/library-decisions/` in <1s of
|
||||
`ls`/`grep`, so I don't re-litigate a decision that has already been made.
|
||||
6. **As an EU-resident maintainer**, I want the EU-data-residency filter to
|
||||
reject US-only SaaS components by default and force a `self-hostable` or
|
||||
`EU-region-configured` justification in the trace, so user data doesn't
|
||||
leave the EU silently.
|
||||
7. **As a maintainer scaffolding an optional core via
|
||||
`pnpm turbo gen core-package <name>`**, I want pre-shipped traces emitted
|
||||
for every direct dep of the new core, so the policy is satisfied by
|
||||
construction.
|
||||
8. **As a security-conscious maintainer**, I want the CVE-scan filter to run
|
||||
`pnpm audit --audit-level=moderate` at evaluation time and snapshot the
|
||||
result + commands into the trace, so I can re-run them later to detect drift.
|
||||
9. **As an agent reviewing a slice in sandcastle**, I want the reviewer prompt
|
||||
to check for trace presence + correctness, so I can reject incompliant
|
||||
slices without needing a separate workflow.
|
||||
10. **As a maintainer reading the repo for the first time**, I want
|
||||
`docs/guides/adding-a-library.md` to explain the policy with worked
|
||||
examples (one approved, one rejected), so I understand the why and how
|
||||
before I face the gate myself.
|
||||
|
||||
## Implementation decisions
|
||||
|
||||
**Module sketch** — what lands where, by package and concern (no file paths
|
||||
where prose suffices):
|
||||
|
||||
- **The skill itself** — `.claude/skills/evaluate-library/` follows the same
|
||||
shape as `to-prd`, `grill-with-docs`, `improve-codebase-architecture`.
|
||||
SKILL.md is authoritative; supporting files (`POLICY.md` mirroring ADR-022,
|
||||
`TRACE-TEMPLATE.md` showing the YAML+headings shape, `EXAMPLES/` worked
|
||||
cases) flesh it out. The skill is invocable via slash command
|
||||
`/evaluate-library`.
|
||||
- **Trace schema module** — a small shared module at
|
||||
`scripts/library-decisions/schema.mjs` exporting (1) a Zod schema for the
|
||||
trace's frontmatter, (2) a parser that reads a trace file and returns the
|
||||
validated frontmatter, (3) a serializer that takes filter results + prose
|
||||
blocks and emits a trace file. Both the skill and the pre-commit checker
|
||||
import this module. Deep module — small interface (parse/serialize/validate),
|
||||
high leverage across the four enforcement layers.
|
||||
- **Pre-commit check script** — `scripts/library-decisions/check.mjs`. Walks
|
||||
`git diff --cached --name-only -- '**/package.json'`, for each file extracts
|
||||
newly-added dep lines via `git diff --cached <file>`, derives the tier from
|
||||
the path, and for each new runtime dep checks that
|
||||
`docs/library-decisions/*-<name>.md` is also staged with `decision: approved`.
|
||||
Exits non-zero with a pointer to the skill if any check fails. Invoked from
|
||||
`.husky/pre-commit` as step 4 (after the existing state-sync guard).
|
||||
- **Claude hooks** — a single `.claude/hooks/library-policy-nudge.sh` that
|
||||
dispatches on `tool_use_type` to handle both `PreToolUse` (Bash with
|
||||
`pnpm add` / `pnpm i <pkg>` pattern) and `PostToolUse` (Edit/Write on
|
||||
`**/package.json`). Same style as the existing `generator-first-nudge.sh`.
|
||||
Emits a non-blocking system-reminder to stdout that the harness threads
|
||||
into the agent's next turn.
|
||||
- **Sandcastle reviewer prompt** — append a "Library-trace check" section
|
||||
to `.sandcastle/reviewer.prompt.md`. The reviewer runs
|
||||
`node scripts/library-decisions/check.mjs --staged-against <base>` in the
|
||||
sandbox before issuing its verdict.
|
||||
- **Generator templates** — each `turbo/generators/templates/core-package/<name>/`
|
||||
gets a `docs/library-decisions/` subtree with one `.md` per direct dep of
|
||||
that core. The generator copies these alongside the core's package.json
|
||||
into the workspace. Frozen via the existing
|
||||
`turbo/generators/__snapshots__/core-package/<name>.snapshot.json` mechanism.
|
||||
- **Backfill traces** — write one trace per existing runtime dep in feature/
|
||||
core tier. The deps cluster naturally by ADR provenance: ADR-002 cluster
|
||||
(Inversify + reflect-metadata), ADR-014 cluster (Sentry family), ADR-017
|
||||
cluster (OpenTelemetry family), and the un-cited cluster (`payload`,
|
||||
`@trpc/server`, `zod`, `superjson`, plus any others surfaced by inventory).
|
||||
All traces dated 2026-05-14, `decision: approved`, ADR citation where the
|
||||
cluster maps to one.
|
||||
- **`CLAUDE.md` update** — one bullet in Key Conventions: _"New runtime
|
||||
dependencies in feature- or core-tier packages require a trace at
|
||||
`docs/library-decisions/<date>-<name>.md` produced by the `evaluate-library`
|
||||
skill — see ADR-022."_
|
||||
|
||||
**Trace schema (frontmatter)** — Zod schema (lifted from ADR-022 §4):
|
||||
|
||||
```
|
||||
package: string
|
||||
version: string // semver range as written in package.json
|
||||
tier: "app" | "feature" | "core"
|
||||
decision: "approved" | "rejected"
|
||||
date: ISO date string
|
||||
deciders: string[]
|
||||
adr: string | null // "adr-NNN" slug or null
|
||||
filter-results: {
|
||||
license: SPDX-id-string
|
||||
types: "native" | `@types/${string}` | "none"
|
||||
maintenance: "active" | "dormant" | "abandoned"
|
||||
boundary-fit: "pass" | "fail"
|
||||
shadow-check: "pass" | "fail" | `shadows ${string}`
|
||||
eu-residency: "ok" | "n/a" | "self-hostable" | "fail"
|
||||
cve-scan: "clean" | `${cve-id}` | "fail"
|
||||
named-consumer: "pass" | "fail"
|
||||
}
|
||||
verification-commands: string[]
|
||||
accepted-cves?: string[] // optional per-trace allowlist
|
||||
```
|
||||
|
||||
Headings (machine-checkable order): one `## Filter: <name>` per filter +
|
||||
one `## Prompt: <name>` per prompt, in the order listed in ADR-022.
|
||||
|
||||
**Skill fail behavior** — collect-cheap-skip-expensive. The four cheap
|
||||
structural filters (license, types, shadow-check, boundary-fit) always run
|
||||
to completion. The four expensive filters (maintenance, CVE scan, EU residency,
|
||||
named-consumer) short-circuit after the first reject. The trace records which
|
||||
filters ran and which were skipped, with a "skipped because earlier filter
|
||||
already rejected" sentinel value.
|
||||
|
||||
**Pre-commit hook decision-state check** — beyond presence, the script also
|
||||
verifies that the trace's `decision` matches the dep status: a dep listed in
|
||||
`package.json` requires `decision: approved`; a trace with `decision: rejected`
|
||||
that names a dep that's also in the package.json is a hard fail (rejected
|
||||
libraries cannot ship).
|
||||
|
||||
**Conformance system composition** — no new use cases, controllers, manifest
|
||||
entries, audits, events, or jobs. This PRD is a workflow/policy implementation,
|
||||
not a feature-domain change. The conformance gates apply only to the new
|
||||
TypeScript/JS modules (Zod schema module + check script) — they get standard
|
||||
vitest coverage.
|
||||
|
||||
## Testing decisions
|
||||
|
||||
- **`scripts/library-decisions/schema.mjs`** — unit tests covering: valid
|
||||
trace parses round-trip; missing required field rejected; unknown filter
|
||||
rejected; invalid enum value rejected.
|
||||
- **`scripts/library-decisions/check.mjs`** — integration tests covering: new
|
||||
feature-tier dep without trace → fail with exit 1; new feature-tier dep with
|
||||
approved trace → pass; new feature-tier dep with rejected trace listed in
|
||||
package.json → fail; new app-tier dep (no trace required) → pass; new devdep
|
||||
→ pass (devdeps exempt); multi-file staged diff with mixed pass/fail → fail
|
||||
with per-package report; non-runtime dep (`peerDependencies` only) → pass.
|
||||
Use a temp git repo as the test fixture.
|
||||
- **The skill** — no automated test in the conformance sense (it's a prose
|
||||
runbook for an agent). The success criterion is that running it against
|
||||
`trpc-to-openapi` produces the documented rejection trace; verified manually
|
||||
during the epic.
|
||||
- **Generator pre-shipped traces** — the existing
|
||||
`turbo/generators/__snapshots__/core-package/<name>.snapshot.json` snapshot
|
||||
test extends to cover the new trace files. A failing snapshot is the gate.
|
||||
- **Claude hook scripts** — bash smoke tests that pipe a mocked Claude
|
||||
hook payload (`{ "tool_input": { "command": "pnpm add foo" } }`) into the
|
||||
script and assert stderr contains the skill-reminder marker. Same style as
|
||||
`generator-first-nudge.sh`'s existing tests if any (check during impl).
|
||||
- **Prior art** — mirror the test patterns from `scripts/work/state-sync-guard.mjs`
|
||||
(the pre-commit `_state.json` check) for the new check script; the
|
||||
fixture/assertion shape carries over directly.
|
||||
- **Coverage bands** — the new scripts under `scripts/library-decisions/` are
|
||||
not feature packages, so they don't have a `feature.manifest.ts` and aren't
|
||||
bound by per-layer coverage thresholds. Add them to the diff-coverage
|
||||
exception list **only if** the diff-coverage gate is too strict on script
|
||||
files; default is they should hit 100% statement coverage because they're
|
||||
small.
|
||||
|
||||
## Open questions
|
||||
|
||||
- **Q1:** CVE-accepted-risk mechanism — per-trace `accepted-cves: ["CVE-XXXX-YYYY"]`
|
||||
frontmatter array vs central `docs/library-decisions/_cve-allowlist.md`? —
|
||||
**Recommended:** per-trace. Acceptance is library-scoped, not global; a
|
||||
central file becomes a god-object that no agent reads in full.
|
||||
- **Q2:** Should the policy also gate `peerDependencies` additions, or only
|
||||
`dependencies`? — **Recommended:** only `dependencies`. Peer deps are a
|
||||
contract, not a runtime addition; if a feature declares a peer, the actual
|
||||
runtime adopter (an app or another package) is the one whose `dependencies`
|
||||
the policy catches.
|
||||
- **Q3:** Should the backfill be one commit per trace or one batch commit per
|
||||
ADR cluster? — **Recommended:** one commit per cluster (4 commits total),
|
||||
each with conventional `chore(deps): backfill library traces for <cluster>`.
|
||||
Avoids both extremes (1 mega-commit and 10 noisy single-trace commits).
|
||||
- **Q4:** Should the skill be permitted to **write the trace** before the
|
||||
user/agent approves the final decision, or must the final write be a
|
||||
separate explicit step? — **Recommended:** skill writes the trace
|
||||
unconditionally at the end of evaluation; the trace IS the record, including
|
||||
for rejections. No "draft" state.
|
||||
- **Q5:** Where do the Claude hooks register themselves? — **Investigate:**
|
||||
the existing `.claude/hooks/*.sh` are referenced by some kind of settings
|
||||
file or auto-discovered. Confirm during the first slice that adds the new
|
||||
hook script and matches the existing registration pattern.
|
||||
|
||||
## Out of scope (deferred)
|
||||
|
||||
- **Periodic re-verification.** Running each trace's `verification-commands`
|
||||
on a schedule (nightly?) to detect drift — new CVEs, license changes,
|
||||
upstream abandonment. Deserves its own PRD; would compose with `pnpm fallow`
|
||||
as a sixth gate.
|
||||
- **Auto-generated PR comments** that summarize the trace for human reviewers.
|
||||
Nice-to-have once the policy has lived for a quarter.
|
||||
- **`pnpm libs <subcommand>` ergonomic CLI** — wrapping `check.mjs` as
|
||||
`pnpm libs check`, plus `pnpm libs list`, `pnpm libs orphans`, etc. Defer
|
||||
until the raw script proves the workflow.
|
||||
|
||||
## Further notes
|
||||
|
||||
- **Anchored by ADR-022** — Library evaluation policy. Read that first.
|
||||
- **Glossary entries** for **Library trace** and **Pre-shipped trace** landed
|
||||
during the 2026-05-14 grill session that produced ADR-022.
|
||||
- **Conversation provenance** — the 2026-05-14 grill-with-docs session that
|
||||
produced this PRD is captured in the session transcript; ADR-022 cites the
|
||||
OpenAPI near-miss as concrete catalyst.
|
||||
Reference in New Issue
Block a user