Files
agentic-dev/.claude/skills/evaluate-library/POLICY.md
Danijel Martinek b10ccba927 feat(scripts): add evaluate-library skill + supporting files
Adds the /evaluate-library skill runbook at .claude/skills/evaluate-library/
with SKILL.md (8-filter + 3-prompt protocol, collect-cheap-skip-expensive
ordering, trace-write step, skip sentinel), POLICY.md (ADR-022 summary
≤2 pages), TRACE-TEMPLATE.md (complete YAML frontmatter + 11 headings in
order), and EXAMPLES/ with one approved (clsx) and one rejected
(trpc-to-openapi, named-consumer: fail) worked trace.

Updates session-start.sh to surface the skill in session pointers.
The skill is auto-registered by the harness on SKILL.md creation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-14 05:45:43 +00:00

7.3 KiB

Library Evaluation Policy — Quick Reference

Authoritative source: docs/decisions/adr-022-library-evaluation-policy.md Authoritative runbook: .claude/skills/evaluate-library/SKILL.md


Why this policy exists

The repo ships with a deliberately narrow runtime surface (six deps per feature package). That discipline is uncodified. Three signals exposed the gap: a near-miss adding trpc-to-openapi for hypothetical REST consumers; three ADRs recording library choices after adoption; and no EU-residency gate before a library could silently transmit user data to a US-only SaaS endpoint. ADR-022 codifies the discipline and makes it agent-runnable.


Tier trigger

The policy applies to direct runtime dependencies in feature- and core-tier packages. Devdeps and app-tier deps are exempt.

Where the dep lands Process required Companion record
apps/<x> Author's call — no policy
packages/<feature> Trace required
packages/core-* Trace required ADR required
New optional-core category Trace required ADR required

The trigger maps onto the existing ESLint boundaries tag system (ADR-006, ADR-010) — no new mental model.


Eight hard auto-reject filters

Phase 1 — cheap (always run to completion)

# Filter Auto-reject condition
1 license Outside MIT, Apache-2.0, BSD-*, ISC, MPL-2.0
2 types No .d.ts and no @types/<pkg>
3 shadow-check Functional parallel to a locked must-have (zod, inversify, payload, @trpc/server, superjson, reflect-metadata)
4 boundary-fit Dep would violate ESLint boundary rules for the target tier (e.g., @sentry/node in a feature package — ADR-017 §4)

Phase 2 — expensive (short-circuit after first reject)

# Filter Auto-reject condition
5 maintenance Last release ≥ 18 months OR activity gap ≥ 12 months (abandoned)
6 cve-scan Open advisory at moderate severity or above (via pnpm audit)
7 eu-residency Library transmits user data/telemetry to a vendor endpoint with no EU data region available or not configured
8 named-consumer No concrete call site exists today and no feature is blocked waiting for it — hypothetical future use does not qualify

A single failure in any filter → decision: rejected. Cheap filters always run; expensive filters stop at the first fail.


Three discussion prompts

Not auto-reject filters — any answer is acceptable with justification. Required in every trace.

  1. replaces — What existing approach does this replace? Parallel adoption of the same capability is a smell.
  2. migration-cost-out — Rate the removal cost 18 months from now: mechanical / hard / impossible.
  3. alternatives-considered — Two named alternatives minimum. For core-tier, also duplicated into the companion ADR.

Trace artifact

Every decision — approved or rejected — produces a file at docs/library-decisions/<YYYY-MM-DD>-<package-name>.md.

Required frontmatter fields:

Field Values
package npm package name
version semver range
tier app | feature | core
decision approved | rejected
date YYYY-MM-DD
deciders list of authors (human and/or agent)
adr adr-NNN or null
filter-results.license SPDX id
filter-results.types native | @types/<x> | none
filter-results.maintenance active | dormant | abandoned
filter-results.boundary-fit pass | fail
filter-results.shadow-check pass | fail | "shadows <x>"
filter-results.eu-residency ok | n/a | self-hostable | fail
filter-results.cve-scan clean | advisory ID | fail
filter-results.named-consumer pass | fail
verification-commands list of literal commands run
accepted-cves list of accepted advisory IDs (optional)

Skipped expensive filters (short-circuited by an earlier reject) → write skip as the frontmatter value and note "Not evaluated" in the prose section.

The trace lands in the same commit as the package.json change. The pre-commit hook validates this for approved traces.


Four-layer enforcement stack

Layer Latency Catches
Claude PreToolUse/PostToolUse hook inline Agent skipping the skill before pnpm add or package.json edit
/evaluate-library skill seconds The decision itself + writes the trace
Git pre-commit hook (scripts/library-decisions/check.mjs) pre-commit Humans or agents bypassing the skill
Sandcastle reviewer prompt per-slice Bypasses that slipped past pre-commit

The Claude hook injects a <system-reminder> pointing to this skill. It is non-blocking — devdep additions and app-tier changes trigger the reminder but do not require a trace. The pre-commit hook is the deterministic gate.


Composition with generators

pnpm turbo gen core-package <name> emits pre-shipped traces — one per direct runtime dep of the new core package — pre-marked decision: approved and citing the relevant ADR (ADR-015 for events, ADR-016 for realtime, ADR-018 for audit). No separate evaluation needed for scaffolded optional cores.