Files
agentic-dev/docs/decisions/adr-022-library-evaluation-policy.md
Danijel Martinek f77e6ea881 chore(template): clean-slate template snapshot from bb4a0c7
Curated, product-agnostic snapshot of the post-story-04 tree: demo
content deleted, auth-only reference feature, web-next shell, all gates
green. Product-specific docs, ADRs 027-029, PRDs/epics/archive, editor
library traces, and product naming are curated out; generic template
repairs (coverage provider devDeps, root test:coverage script, live
lint fixes, root-only release-please) are kept. See TEMPLATE.md for
provenance, curation list, and usage.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016j8z4VHjedXDTjEDNg7qHK
2026-07-12 20:40:54 +02:00

319 lines
16 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# ADR-022 — Library evaluation policy
**Status:** Accepted
**Date:** 2026-05-14
**Related:** ADR-006 (vertical feature packages), ADR-010 (turbo boundaries), ADR-014 (Sentry observability), ADR-017 (OpenTelemetry + vendor isolation), ADR-019 (sandcastle agent orchestration), ADR-021 (release-please versioning)
**Companion guide:** `docs/guides/adding-a-library.md` (human reading-room)
**Companion skill:** `.claude/skills/evaluate-library/SKILL.md` (authoritative agent runbook)
## Context
This template ships with a deliberately narrow third-party surface. Every feature
package today carries the same six runtime dependencies — `@repo/core-shared`,
`@trpc/server`, `inversify`, `payload`, `reflect-metadata`, `zod` — and nothing
else. That uniformity isn't an accident; it's the result of unstated discipline
that the boundary-tag system (ADR-006, ADR-010) and the manifest-first ordering
(ADR-012) silently reward.
The discipline is not codified. New dependencies get added by anyone — human or
agent — running `pnpm add <pkg>`, with no checkpoint between intent and lockfile.
Three recent signals show the gap:
1. An exploratory grill session on **2026-05-14** nearly added `trpc-to-openapi`
plus `zod-to-json-schema` plus a build-time generator to the repo before
stopping to ask "who calls this code path?" The honest answer was "nobody —
all callers are TypeScript via `createCaller`." The library would have shipped
~30 lines of `.meta({...})` annotations per router and a `superjson`-incompatible
HTTP handler in exchange for zero downstream consumers. Pure carrying cost,
caught by a chance question, not by a system.
2. **Three existing ADRs already record post-hoc library decisions**
ADR-002 (Inversify), ADR-014 (Sentry), ADR-017 (OpenTelemetry). Each notes
"we picked X over Y" but the records were written after adoption. By the time
the ADR existed the lockfile already held the dep. No mechanism existed to
catch a _bad_ choice before it became a migration project.
3. The repo's automation depends on the lockfile staying small and predictable.
`pnpm fallow` audits for dead code; `pnpm conformance` audits manifest drift;
`pnpm coverage:diff` audits change coverage. There is no equivalent audit for
"did we just adopt a library nobody asked for?"
A fourth pressure comes from this template being EU-resident and GDPR-bound.
A library that defaults to a US-only SaaS endpoint (telemetry, analytics, AI,
log aggregation) silently moves user data out of the EU as soon as it's imported
and configured with defaults. The current process has no point where that gets
caught.
The decision below codifies the de-facto discipline, formalizes the agent-loop
hook that prevents drift, and makes rejection records first-class so future
agents don't re-evaluate libraries that were already rejected for known reasons.
## Decision
Adopt a **tiered library-evaluation policy** with eight hard auto-reject filters,
three discussion prompts, a per-decision trace artifact, and a four-layer
enforcement stack.
### 1. Tiered trigger (by boundary tag)
| Tier the dep lands in | Process | Companion record |
| ------------------------------------ | ------------------------ | ---------------- |
| `apps/<x>` | Author's call; no policy | |
| `feature` (e.g. `packages/auth`) | Trace required | |
| `core` (e.g. `packages/core-shared`) | Trace required | ADR required |
| New optional-core category | Trace required | ADR required |
The trigger maps onto the workspace-tag boundary already enforced by ESLint
(`eslint-plugin-boundaries`) and Turborepo (`turbo boundaries`). No new mental
model — the policy is a corollary of an existing one.
### 2. Eight hard auto-reject filters
Failing any single filter is an automatic reject. Trace records the failure.
1. **License allowlist.** Only `MIT`, `Apache-2.0`, `BSD-*`, `ISC`, `MPL-2.0`.
Anything else (GPL family, AGPL, CC-BY-NC, custom EULAs) is a no.
2. **TypeScript-native or `@types/*` available.** The repo is strict TS;
un-typed JS libraries shift maintenance cost to the integrating feature.
3. **Not abandoned.** Last release < 18 months **AND** PR/issue activity
< 12 months. The triple-AND avoids killing finished-but-stable libraries
(`reflect-metadata`-style).
4. **Boundary-tag fit.** A dep added to a feature package cannot require
imports the boundary rules forbid (e.g. a Sentry SDK in a feature
ADR-017 §4 forbids this).
5. **Doesn't shadow an existing must-have.** Proposing `valibot` when `zod`
is locked, or `tsyringe` when Inversify is locked by ADR-002, is an
auto-reject; the replacement must be a separate ADR with consequences
analysis, not a parallel adoption.
6. **EU data residency for hosted/SaaS components.** If the package transmits
user data, telemetry, or business state to a vendor-controlled endpoint
by default, that vendor must offer an EU data region, and the integration
must be configured to use it. Self-hostable packages, on-device libraries,
and build-time-only tools are exempt.
7. **CVE scan clean.** `pnpm audit --audit-level=moderate` clean at adoption
time. Documented allowlist mechanism for accepted-risk advisories.
8. **Named consumer exists now, not hypothetically.** The integration must
answer "who calls this code path today, or who is blocked waiting for it?"
"Future code that might exist" is not a consumer. This filter is the
direct response to the 2026-05-14 OpenAPI near-miss.
### 3. Three discussion prompts
Filters that don't auto-reject but must be answered in the trace, either
direction acceptable with justification.
- **What does it replace?** New-and-old running in parallel is a smell.
- **Migration cost out.** What does ripping this back out look like 18 months
from now? Mechanical, hard, or impossible?
- **Alternatives considered.** Two named alternatives at minimum (or "none
with explanation"). Required for `feature`-tier; required _and_ duplicated
into the ADR for `core`-tier.
### 4. Trace artifact
Every decision approved or rejected emits a trace file at
`docs/library-decisions/<YYYY-MM-DD>-<package-name>.md`. Always written,
regardless of whether an accompanying ADR exists. Shape:
```markdown
---
package: <name>
version: "<semver range>"
tier: app | feature | core
decision: approved | rejected
date: <YYYY-MM-DD>
deciders: [<author>, ...]
adr: adr-NNN | null
filter-results:
license: <SPDX id>
types: native | "@types/<x>" | none
maintenance: active | dormant | abandoned
boundary-fit: pass | fail
shadow-check: pass | fail | "shadows <x>"
eu-residency: ok | n/a | self-hostable | fail
cve-scan: clean | "<advisory-id>" | fail
named-consumer: pass | fail
verification-commands:
- <literal command that produced each filter result>
---
## Filter: <name>
<prose>
## Prompt: <name>
<prose>
```
Frontmatter is the machine surface (greppable, schema-stable). Headings are
the human surface. Rejection traces are first-class the OpenAPI scenario,
had this policy existed, would have produced a permanent record so the next
agent considering `trpc-to-openapi` finds the prior reasoning in <1s of
`ls docs/library-decisions/`.
### 5. Four-layer enforcement stack
Mirrors the latency-tiered shape of the conformance system (ADR-012).
| Layer | Latency | Catches |
| -------------------------------------- | ---------- | ---------------------------------------------------------------- |
| Claude `PreToolUse`/`PostToolUse` hook | inline | Agent skipping the skill before `pnpm add` / `package.json` edit |
| `evaluate-library` skill | seconds | The decision itself + writes the trace |
| Git pre-commit hook | pre-commit | Humans or agents bypassing the skill |
| Sandcastle reviewer prompt | per-slice | Bypasses that slipped past pre-commit |
The Claude hook injects a `<system-reminder>` directing the agent to the skill
but does **not** auto-deny (false-positive paths like dev-deps and app-tier
additions are common). The pre-commit hook is the deterministic gate.
### 6. Composition with `pnpm turbo gen core-package`
The optional-cores generator emits **pre-shipped traces** one per direct
runtime dep of the new core pre-marked `decision: approved` and cited
against the relevant ADR (ADR-015 for events, ADR-016 for realtime,
ADR-018 for audit). Same frozen-snapshot discipline that the optional cores
already follow (`turbo/generators/__snapshots__/core-package/`). New optional
cores added later inherit this requirement.
### 7. Skill invocation
```
/evaluate-library <package-name> --tier <feature|core|app> --target <package-path>
```
The skill walks the eight filters in **collect-cheap-skip-expensive** order:
cheap structural filters (license, types, shadow-check, boundary-fit) run to
completion regardless of failure; expensive filters (CVE scan, EU residency
probe, maintenance signals) short-circuit after the first reject. The trace
records which filters ran and which were skipped, so a partial trace is still
useful evidence.
### 8. Backfill at policy adoption
Every existing runtime dependency in feature- and core-tier packages
(~10 deps at this writing `payload`, `inversify`, `zod`, `@trpc/server`,
`reflect-metadata`, `superjson`, `@sentry/*`, `@opentelemetry/*` family,
`socket.io`, etc.) gets a backfilled trace dated 2026-05-14 (adoption day).
ADR-002, ADR-014, ADR-017 are cited via the `adr:` frontmatter field;
verification-command output is captured at backfill time.
### 9. Sub-processor discriminated union (amendment: 2026-05-18)
Every trace carries two top-level frontmatter fields classifying the library
from a GDPR sub-processor perspective:
```yaml
is-sub-processor: false # boolean — true when the vendor receives personal data on the operator's behalf
processes-pii: false # boolean — true when the library processes PII in-process (even without transmitting it)
```
When `is-sub-processor: true`, five additional fields are **required**:
```yaml
data-sent: "<what personal data the library transmits to the vendor>"
region: "<vendor data region, e.g. eu-west-1>"
dpa-signed: true | false
sccs-required: true | false
contact: "<vendor DPO or privacy contact email/URL>"
```
**Discriminated-union rules:**
| `is-sub-processor` | `processes-pii` | Conditional fields required? |
| ------------------ | --------------- | --------------------------------------------------------------------------------- |
| `false` | `false` | No pure library, no data involvement |
| `false` | `true` | No in-process only, no vendor data flow |
| `true` | `true` | Yes all five conditional fields required |
| `true` | `false` | Technically possible but very unusual; still requires all five conditional fields |
**Baseline for backfill:** pure in-process libraries (no network calls to
vendor-controlled endpoints) get `is-sub-processor: false` + `processes-pii: false`.
Self-hosted software that stores PII but transmits nothing to the vendor (e.g.
`payload`) gets `is-sub-processor: false` + `processes-pii: true`.
These fields are the machine surface consumed by `scripts/emit-sub-processors.mjs`
(see Story 06 of the compliance-manifests-pii-retention-subprocessors epic). A
trace missing `is-sub-processor` is treated as `false` by the generator for
backward-compatibility; all new traces authored after this amendment must include
both fields. The `evaluate-library` skill 7) prompts for these fields
unconditionally and writes the conditional block only when `is-sub-processor: true`.
The weekly `dpa-signed` staleness check in CI (ADR-023 cross-reference) should
flag any `dpa-signed: true` traces where the DPA has not been revalidated within
the prior 365 days. Implementation of that cron is deferred to the CI security
hardening work.
## Alternatives considered
- **No policy, keep relying on instinct.** Rejected. The 2026-05-14 OpenAPI
near-miss demonstrated the gap. Instinct catches some adds, misses others;
the failure mode is silent.
- **Every `package.json` change requires a trace, no tier distinction.**
Rejected. Devdeps in tooling packages and ESLint plugin bumps in apps would
drown the trace directory in noise. The boundary-tag system already
partitions blast radius the policy reuses that partition.
- **Only "category" decisions require process** (new auth provider, new ORM,
new queue). Rejected. The OpenAPI scenario was a sub-tool inside an existing
category, not a category swap. Category-only triggers miss it.
- **A central library-evaluation service / Linear queue / Slack bot.**
Rejected. The repo is agent-first and currently single-developer (+ agents).
Human-in-the-loop services don't fit the dispatch loop; the policy must be
agent-runnable end-to-end.
- **No CVE filter rely on `npm audit` ambient noise.** Rejected. CVE
status is point-in-time; pinning the result to the trace at adoption is the
whole value. Re-running the verification commands later detects drift.
- **Drop the named-consumer filter to a discussion prompt.** Rejected. The
filter is the one that would have stopped the OpenAPI flirtation; demoting
it back to a prompt is the same as not adding the filter.
## Consequences
**Positive**
- New dependencies require deliberate intent. The trace artifact + four-layer
enforcement stack make accidental adds detectable at four latencies, mirroring
the conformance-system pattern that already works.
- Rejection records are permanent. Future agents considering a previously
rejected library find the trace in `docs/library-decisions/` and don't
re-litigate.
- EU data residency becomes a binary, machine-readable filter result, not
an afterthought.
- Per-decision verification commands give future agents a single source of
truth they can re-run to verify the trace is still valid.
- The policy composes with `pnpm turbo gen core-package`: optional cores
remain a one-command scaffold without bypassing the rule.
**Negative**
- New feature-tier deps take longer to land. Walking eight filters + writing
the trace is ~5 minutes of agent work per addition.
- Backfilling ~10 existing deps is one-time work; expected ~half a day of
agent dispatch.
- The Claude `PreToolUse` hook adds latency to every `pnpm add` invocation.
Mitigated by the hook being a reminder-injector, not a blocker.
- The pre-commit hook adds a new failure mode ("you added a dep but forgot
the trace"). Mitigated by the skill being the natural path the hook
reminders point to.
- The CVE filter creates a maintenance obligation: when `pnpm audit` finds
a new advisory in an already-approved dep, the trace becomes stale. Acceptable
trade-off staleness detection is exactly what `pnpm audit` already does;
the trace adds a per-dep anchor for the conversation that follows.
**Neutral**
- ADR-002, ADR-014, and ADR-017 remain authoritative for their respective
libraries. Backfilled traces cite them rather than duplicating their reasoning.
- The policy doesn't constrain transitive dependencies; `pnpm audit` and license
scanning already handle those recursively. Only direct deps require a trace.
## Related
- ADR-006 Vertical feature packages (the tag system the trigger maps to)
- ADR-010 Turbo boundaries (the enforcement substrate)
- ADR-012 Feature conventions (the conformance-system shape this policy mirrors)
- ADR-017 OpenTelemetry migration (vendor-isolation pattern the policy extends)
- ADR-019 Sandcastle agent orchestration (reviewer-prompt layer of enforcement)
- ADR-021 release-please versioning (where dep additions show up in release notes)
- Companion guide: `docs/guides/adding-a-library.md`
- Companion skill: `.claude/skills/evaluate-library/SKILL.md`
- Glossary: `docs/glossary.md` entries for **Library trace** and **Pre-shipped trace**