feat(tooling): add .socket.json and Filter 9 to evaluate-library skill

Add repo-root .socket.json (critical=error, high=warn, medium/low=ignore)
and extend the evaluate-library skill's 8-filter protocol to 9 filters by
adding Filter 9 — Supply-chain behavior (Socket). Positions Socket last in
Phase 2 (most expensive: network call), documents socket-cli as the
verification command, and maps clean/flagged/<finding-summary> to the
trace's socketRisk field. Updates short-circuit skip lists for all earlier
Phase 2 filters and adds socketRisk to TRACE-TEMPLATE.md frontmatter.
This commit is contained in:
2026-05-14 17:10:15 +00:00
parent f704cc5a01
commit 432a606411
3 changed files with 50 additions and 6 deletions

View File

@@ -1,6 +1,6 @@
---
name: evaluate-library
description: Walk the 8-filter + 3-prompt library evaluation protocol for a named package, write the decision trace to docs/library-decisions/, and return pass/fail. Use when adding a runtime dependency to a feature or core package, or when the library-policy-nudge hook fires.
description: Walk the 9-filter + 3-prompt library evaluation protocol for a named package, write the decision trace to docs/library-decisions/, and return pass/fail. Use when adding a runtime dependency to a feature or core package, or when the library-policy-nudge hook fires.
---
<invocation>
@@ -17,7 +17,7 @@ All three arguments are required. The `library-policy-nudge` hook emits this exa
## Overview
Walk eight hard auto-reject filters in **collect-cheap-skip-expensive** order, then answer three discussion prompts. Write the trace unconditionally at the end — including for rejections. A rejection trace is a permanent record that prevents future agents from re-litigating the same decision.
Walk nine hard auto-reject filters in **collect-cheap-skip-expensive** order, then answer three discussion prompts. Write the trace unconditionally at the end — including for rejections. A rejection trace is a permanent record that prevents future agents from re-litigating the same decision.
## Phase 1 — Cheap filters (always run to completion, even if one fails)
@@ -86,7 +86,7 @@ Result values:
- `dormant` stable, not actively developed (acceptable for finished libraries like `reflect-metadata`)
- `abandoned` last release 18 months **or** no activity in 12 months auto-reject; short-circuit remaining expensive filters
On `abandoned` set `cve-scan`, `eu-residency`, `named-consumer` to `skip` write trace.
On `abandoned` set `cve-scan`, `eu-residency`, `named-consumer`, `socketRisk` to `skip` write trace.
### Filter 6: cve-scan
@@ -96,7 +96,7 @@ pnpm audit --audit-level=moderate 2>&1 | head -40
Result values: `clean` (no advisories), an advisory ID like `GHSA-xxxx-xxxx-xxxx` (accepted risk document in `accepted-cves` frontmatter), or `fail` (open advisory not accepted auto-reject; short-circuit remaining expensive filters).
On `fail` set `eu-residency`, `named-consumer` to `skip` write trace.
On `fail` set `eu-residency`, `named-consumer`, `socketRisk` to `skip` write trace.
### Filter 7: eu-residency
@@ -108,7 +108,7 @@ For non-exempt libraries: verify the vendor offers an EU data region AND that th
Result values: `ok` (vendor offers EU region, integration configured), `n/a` (no data transmission), `self-hostable` (operator-controlled endpoint), `fail` auto-reject; short-circuit `named-consumer`.
On `fail` set `named-consumer` to `skip` write trace.
On `fail` set `named-consumer`, `socketRisk` to `skip` write trace.
### Filter 8: named-consumer
@@ -116,10 +116,44 @@ Answer: **Who calls this code path today, or who is blocked waiting for it?**
A named consumer is a concrete call site that exists now or a feature blocked on this capability today. "We might want this later", "external clients could use this", and "it would be nice to have" are not named consumers.
If the only possible callers are hypothetical or future `fail` auto-reject.
If the only possible callers are hypothetical or future `fail` set `socketRisk` to `skip` auto-reject.
Result value: `pass` or `fail`.
### Filter 9: supply-chain behavior (Socket)
**Expensive — network call. Run last in Phase 2. Short-circuit: if any earlier Phase 2 filter already rejected the library, set `socketRisk` to `skip` and proceed to the [Trace write step](#trace-write-step).**
Verify the package's supply-chain health via `socket-cli`:
```
npx socket-cli@latest scan . --json 2>&1
```
This scans the current directory's lockfile for packages installed from the target under evaluation. For a targeted single-package check before installing:
```
npx socket-cli@latest info <pkg>@<version> --json 2>&1
```
The JSON output contains an array of findings, each with a `severity` field. Cross-reference with the repo-root `.socket.json` `issueRules` to determine the classification:
| Finding severity | `.socket.json` rule | `socketRisk` value |
| ----------------------------------- | ------------------- | ------------------- |
| No findings, or only `medium`/`low` | `ignore` | `clean` |
| `high`-severity finding present | `warn` | `flagged` |
| `critical`-severity finding present | `error` | `<finding-summary>` |
Where `<finding-summary>` is a concise label for the critical finding (e.g. `"new-author-on-publish"`, `"install-scripts-added"`, `"exfiltrates-env"`).
Set `filter-results.socketRisk` in the trace frontmatter to one of these three values.
Result values:
- `clean` no meaningful supply-chain signals; proceed to Phase 3 prompts.
- `flagged` `high`-severity finding; document the specific signal in the trace body and decide whether to accept with justification. Not an auto-reject.
- `<finding-summary>` `critical`-severity finding; auto-reject. This is the last filter no further filters to skip.
---
## Skip sentinel

View File

@@ -26,6 +26,7 @@ filter-results:
eu-residency: ok | n/a | self-hostable | fail
cve-scan: clean | "<advisory-id>" | fail
named-consumer: pass | fail
socketRisk: clean | flagged | <finding-summary>
verification-commands:
- <literal command that produced the license result>
- <literal command that confirmed types>
@@ -59,6 +60,7 @@ filter-results:
eu-residency: skip
cve-scan: skip
named-consumer: skip
socketRisk: skip
verification-commands:
- npm view <pkg> time.modified
accepted-cves: []

8
.socket.json Normal file
View File

@@ -0,0 +1,8 @@
{
"issueRules": {
"critical": "error",
"high": "warn",
"medium": "ignore",
"low": "ignore"
}
}