feat(tooling): add .socket.json and Filter 9 to evaluate-library skill
Add repo-root .socket.json (critical=error, high=warn, medium/low=ignore) and extend the evaluate-library skill's 8-filter protocol to 9 filters by adding Filter 9 — Supply-chain behavior (Socket). Positions Socket last in Phase 2 (most expensive: network call), documents socket-cli as the verification command, and maps clean/flagged/<finding-summary> to the trace's socketRisk field. Updates short-circuit skip lists for all earlier Phase 2 filters and adds socketRisk to TRACE-TEMPLATE.md frontmatter.
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: evaluate-library
|
||||
description: Walk the 8-filter + 3-prompt library evaluation protocol for a named package, write the decision trace to docs/library-decisions/, and return pass/fail. Use when adding a runtime dependency to a feature or core package, or when the library-policy-nudge hook fires.
|
||||
description: Walk the 9-filter + 3-prompt library evaluation protocol for a named package, write the decision trace to docs/library-decisions/, and return pass/fail. Use when adding a runtime dependency to a feature or core package, or when the library-policy-nudge hook fires.
|
||||
---
|
||||
|
||||
<invocation>
|
||||
@@ -17,7 +17,7 @@ All three arguments are required. The `library-policy-nudge` hook emits this exa
|
||||
|
||||
## Overview
|
||||
|
||||
Walk eight hard auto-reject filters in **collect-cheap-skip-expensive** order, then answer three discussion prompts. Write the trace unconditionally at the end — including for rejections. A rejection trace is a permanent record that prevents future agents from re-litigating the same decision.
|
||||
Walk nine hard auto-reject filters in **collect-cheap-skip-expensive** order, then answer three discussion prompts. Write the trace unconditionally at the end — including for rejections. A rejection trace is a permanent record that prevents future agents from re-litigating the same decision.
|
||||
|
||||
## Phase 1 — Cheap filters (always run to completion, even if one fails)
|
||||
|
||||
@@ -86,7 +86,7 @@ Result values:
|
||||
- `dormant` — stable, not actively developed (acceptable for finished libraries like `reflect-metadata`)
|
||||
- `abandoned` — last release ≥ 18 months **or** no activity in ≥ 12 months → auto-reject; short-circuit remaining expensive filters
|
||||
|
||||
On `abandoned` → set `cve-scan`, `eu-residency`, `named-consumer` to `skip` → write trace.
|
||||
On `abandoned` → set `cve-scan`, `eu-residency`, `named-consumer`, `socketRisk` to `skip` → write trace.
|
||||
|
||||
### Filter 6: cve-scan
|
||||
|
||||
@@ -96,7 +96,7 @@ pnpm audit --audit-level=moderate 2>&1 | head -40
|
||||
|
||||
Result values: `clean` (no advisories), an advisory ID like `GHSA-xxxx-xxxx-xxxx` (accepted risk — document in `accepted-cves` frontmatter), or `fail` (open advisory not accepted → auto-reject; short-circuit remaining expensive filters).
|
||||
|
||||
On `fail` → set `eu-residency`, `named-consumer` to `skip` → write trace.
|
||||
On `fail` → set `eu-residency`, `named-consumer`, `socketRisk` to `skip` → write trace.
|
||||
|
||||
### Filter 7: eu-residency
|
||||
|
||||
@@ -108,7 +108,7 @@ For non-exempt libraries: verify the vendor offers an EU data region AND that th
|
||||
|
||||
Result values: `ok` (vendor offers EU region, integration configured), `n/a` (no data transmission), `self-hostable` (operator-controlled endpoint), `fail` → auto-reject; short-circuit `named-consumer`.
|
||||
|
||||
On `fail` → set `named-consumer` to `skip` → write trace.
|
||||
On `fail` → set `named-consumer`, `socketRisk` to `skip` → write trace.
|
||||
|
||||
### Filter 8: named-consumer
|
||||
|
||||
@@ -116,10 +116,44 @@ Answer: **Who calls this code path today, or who is blocked waiting for it?**
|
||||
|
||||
A named consumer is a concrete call site that exists now or a feature blocked on this capability today. "We might want this later", "external clients could use this", and "it would be nice to have" are not named consumers.
|
||||
|
||||
If the only possible callers are hypothetical or future → `fail` → auto-reject.
|
||||
If the only possible callers are hypothetical or future → `fail` → set `socketRisk` to `skip` → auto-reject.
|
||||
|
||||
Result value: `pass` or `fail`.
|
||||
|
||||
### Filter 9: supply-chain behavior (Socket)
|
||||
|
||||
**Expensive — network call. Run last in Phase 2. Short-circuit: if any earlier Phase 2 filter already rejected the library, set `socketRisk` to `skip` and proceed to the [Trace write step](#trace-write-step).**
|
||||
|
||||
Verify the package's supply-chain health via `socket-cli`:
|
||||
|
||||
```
|
||||
npx socket-cli@latest scan . --json 2>&1
|
||||
```
|
||||
|
||||
This scans the current directory's lockfile for packages installed from the target under evaluation. For a targeted single-package check before installing:
|
||||
|
||||
```
|
||||
npx socket-cli@latest info <pkg>@<version> --json 2>&1
|
||||
```
|
||||
|
||||
The JSON output contains an array of findings, each with a `severity` field. Cross-reference with the repo-root `.socket.json` `issueRules` to determine the classification:
|
||||
|
||||
| Finding severity | `.socket.json` rule | `socketRisk` value |
|
||||
| ----------------------------------- | ------------------- | ------------------- |
|
||||
| No findings, or only `medium`/`low` | `ignore` | `clean` |
|
||||
| `high`-severity finding present | `warn` | `flagged` |
|
||||
| `critical`-severity finding present | `error` | `<finding-summary>` |
|
||||
|
||||
Where `<finding-summary>` is a concise label for the critical finding (e.g. `"new-author-on-publish"`, `"install-scripts-added"`, `"exfiltrates-env"`).
|
||||
|
||||
Set `filter-results.socketRisk` in the trace frontmatter to one of these three values.
|
||||
|
||||
Result values:
|
||||
|
||||
- `clean` — no meaningful supply-chain signals; proceed to Phase 3 prompts.
|
||||
- `flagged` — `high`-severity finding; document the specific signal in the trace body and decide whether to accept with justification. Not an auto-reject.
|
||||
- `<finding-summary>` — `critical`-severity finding; auto-reject. This is the last filter — no further filters to skip.
|
||||
|
||||
---
|
||||
|
||||
## Skip sentinel
|
||||
|
||||
@@ -26,6 +26,7 @@ filter-results:
|
||||
eu-residency: ok | n/a | self-hostable | fail
|
||||
cve-scan: clean | "<advisory-id>" | fail
|
||||
named-consumer: pass | fail
|
||||
socketRisk: clean | flagged | <finding-summary>
|
||||
verification-commands:
|
||||
- <literal command that produced the license result>
|
||||
- <literal command that confirmed types>
|
||||
@@ -59,6 +60,7 @@ filter-results:
|
||||
eu-residency: skip
|
||||
cve-scan: skip
|
||||
named-consumer: skip
|
||||
socketRisk: skip
|
||||
verification-commands:
|
||||
- npm view <pkg> time.modified
|
||||
accepted-cves: []
|
||||
|
||||
8
.socket.json
Normal file
8
.socket.json
Normal file
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"issueRules": {
|
||||
"critical": "error",
|
||||
"high": "warn",
|
||||
"medium": "ignore",
|
||||
"low": "ignore"
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user