diff --git a/.claude/skills/evaluate-library/SKILL.md b/.claude/skills/evaluate-library/SKILL.md index 5f123f7..be0b24b 100644 --- a/.claude/skills/evaluate-library/SKILL.md +++ b/.claude/skills/evaluate-library/SKILL.md @@ -1,6 +1,6 @@ --- name: evaluate-library -description: Walk the 8-filter + 3-prompt library evaluation protocol for a named package, write the decision trace to docs/library-decisions/, and return pass/fail. Use when adding a runtime dependency to a feature or core package, or when the library-policy-nudge hook fires. +description: Walk the 9-filter + 3-prompt library evaluation protocol for a named package, write the decision trace to docs/library-decisions/, and return pass/fail. Use when adding a runtime dependency to a feature or core package, or when the library-policy-nudge hook fires. --- @@ -17,7 +17,7 @@ All three arguments are required. The `library-policy-nudge` hook emits this exa ## Overview -Walk eight hard auto-reject filters in **collect-cheap-skip-expensive** order, then answer three discussion prompts. Write the trace unconditionally at the end — including for rejections. A rejection trace is a permanent record that prevents future agents from re-litigating the same decision. +Walk nine hard auto-reject filters in **collect-cheap-skip-expensive** order, then answer three discussion prompts. Write the trace unconditionally at the end — including for rejections. A rejection trace is a permanent record that prevents future agents from re-litigating the same decision. ## Phase 1 — Cheap filters (always run to completion, even if one fails) @@ -86,7 +86,7 @@ Result values: - `dormant` — stable, not actively developed (acceptable for finished libraries like `reflect-metadata`) - `abandoned` — last release ≥ 18 months **or** no activity in ≥ 12 months → auto-reject; short-circuit remaining expensive filters -On `abandoned` → set `cve-scan`, `eu-residency`, `named-consumer` to `skip` → write trace. +On `abandoned` → set `cve-scan`, `eu-residency`, `named-consumer`, `socketRisk` to `skip` → write trace. ### Filter 6: cve-scan @@ -96,7 +96,7 @@ pnpm audit --audit-level=moderate 2>&1 | head -40 Result values: `clean` (no advisories), an advisory ID like `GHSA-xxxx-xxxx-xxxx` (accepted risk — document in `accepted-cves` frontmatter), or `fail` (open advisory not accepted → auto-reject; short-circuit remaining expensive filters). -On `fail` → set `eu-residency`, `named-consumer` to `skip` → write trace. +On `fail` → set `eu-residency`, `named-consumer`, `socketRisk` to `skip` → write trace. ### Filter 7: eu-residency @@ -108,7 +108,7 @@ For non-exempt libraries: verify the vendor offers an EU data region AND that th Result values: `ok` (vendor offers EU region, integration configured), `n/a` (no data transmission), `self-hostable` (operator-controlled endpoint), `fail` → auto-reject; short-circuit `named-consumer`. -On `fail` → set `named-consumer` to `skip` → write trace. +On `fail` → set `named-consumer`, `socketRisk` to `skip` → write trace. ### Filter 8: named-consumer @@ -116,10 +116,44 @@ Answer: **Who calls this code path today, or who is blocked waiting for it?** A named consumer is a concrete call site that exists now or a feature blocked on this capability today. "We might want this later", "external clients could use this", and "it would be nice to have" are not named consumers. -If the only possible callers are hypothetical or future → `fail` → auto-reject. +If the only possible callers are hypothetical or future → `fail` → set `socketRisk` to `skip` → auto-reject. Result value: `pass` or `fail`. +### Filter 9: supply-chain behavior (Socket) + +**Expensive — network call. Run last in Phase 2. Short-circuit: if any earlier Phase 2 filter already rejected the library, set `socketRisk` to `skip` and proceed to the [Trace write step](#trace-write-step).** + +Verify the package's supply-chain health via `socket-cli`: + +``` +npx socket-cli@latest scan . --json 2>&1 +``` + +This scans the current directory's lockfile for packages installed from the target under evaluation. For a targeted single-package check before installing: + +``` +npx socket-cli@latest info @ --json 2>&1 +``` + +The JSON output contains an array of findings, each with a `severity` field. Cross-reference with the repo-root `.socket.json` `issueRules` to determine the classification: + +| Finding severity | `.socket.json` rule | `socketRisk` value | +| ----------------------------------- | ------------------- | ------------------- | +| No findings, or only `medium`/`low` | `ignore` | `clean` | +| `high`-severity finding present | `warn` | `flagged` | +| `critical`-severity finding present | `error` | `` | + +Where `` is a concise label for the critical finding (e.g. `"new-author-on-publish"`, `"install-scripts-added"`, `"exfiltrates-env"`). + +Set `filter-results.socketRisk` in the trace frontmatter to one of these three values. + +Result values: + +- `clean` — no meaningful supply-chain signals; proceed to Phase 3 prompts. +- `flagged` — `high`-severity finding; document the specific signal in the trace body and decide whether to accept with justification. Not an auto-reject. +- `` — `critical`-severity finding; auto-reject. This is the last filter — no further filters to skip. + --- ## Skip sentinel diff --git a/.claude/skills/evaluate-library/TRACE-TEMPLATE.md b/.claude/skills/evaluate-library/TRACE-TEMPLATE.md index 4037081..fcee28a 100644 --- a/.claude/skills/evaluate-library/TRACE-TEMPLATE.md +++ b/.claude/skills/evaluate-library/TRACE-TEMPLATE.md @@ -26,6 +26,7 @@ filter-results: eu-residency: ok | n/a | self-hostable | fail cve-scan: clean | "" | fail named-consumer: pass | fail + socketRisk: clean | flagged | verification-commands: - - @@ -59,6 +60,7 @@ filter-results: eu-residency: skip cve-scan: skip named-consumer: skip + socketRisk: skip verification-commands: - npm view time.modified accepted-cves: [] diff --git a/.socket.json b/.socket.json new file mode 100644 index 0000000..0e3dd7e --- /dev/null +++ b/.socket.json @@ -0,0 +1,8 @@ +{ + "issueRules": { + "critical": "error", + "high": "warn", + "medium": "ignore", + "low": "ignore" + } +}