agentic-dev/docs/work/prds/ci-security-and-supply-chain.prd.md

id, title, type, status, author, created, updated, adr, builds-on

id	title	type	status	author	created	updated	adr	builds-on
ci-security-and-supply-chain	CI security + supply-chain enforcement stack	prd	approved	danijel	2026-05-14T00:00:00Z	2026-05-14T19:16:52.691Z	adr-023	library-evaluation-policy

Problem

The repo's current security posture, audited 2026-05-14: zero security tooling. No Dependabot, no Renovate, no CodeQL, no Snyk, no Trivy, no OSV-Scanner, no Socket, no gitleaks, no pnpm audit signatures step. GitHub Actions are pinned to major-version tags (actions/checkout@v4, pnpm/action-setup@v4, googleapis/release-please-action@v4), which the 2025 tj-actions/changed-files incident proved unsafe.

ADR-022 + the in-flight library-evaluation epic close the adoption-time gate for new dependencies. They do not close the drift gate. Six post-adoption threats remain uncovered:

1. CVE disclosures against pinned versions. The trace's verification-commands snapshot goes stale silently when new advisories drop.
2. Supply-chain behavior compromise — event-stream, ua-parser-js, tj-actions/changed-files, xz-utils. CVE scanning is a lagging indicator; these shipped malware that no CVE database had seen at the moment of compromise.
3. Maintainer-account compromise. A trusted upstream maintainer's npm account gets phished; the next patch publishes a malicious post-install script; everyone on ^1.2.0 inherits it.
4. GitHub Actions supply chain. Major-tag pinning is documented insecure.
5. License drift (e.g. Sentry going BSL on a major; Elasticsearch going SSPL).
6. EU-residency drift when a vendor announces US-only changes mid-flight.

ADR-023 codifies the four-pillar enforcement stack that closes these gaps. This PRD implements it.

Goal

A four-pillar CI security stack — Renovate-managed bumps + Action SHA pinning, Socket-based supply-chain-behavior detection, continuous trace revalidation extending ADR-022, and baseline GitHub-native gates — composed via a single failure-mode hierarchy that the sandcastle reviewer prompt enforces machine-readably for agent-driven PRs.

In scope

• Renovate adoption — .github/renovate.json configuring per-workspace npm bumps (grouped by ecosystem cluster), Dockerfile bumps for .sandcastle/Dockerfile, GitHub Actions SHA pinning via pinGitHubActionDigests, major/minor split with automerge for green minor+patch PRs.
• One-time Action SHA-pin sweep. Renovate's first run rewrites the 6 existing uses: references in .github/workflows/*.yml from major-version tags to full 40-char SHAs.
• Socket.dev integration as the 9th hard filter in evaluate-library. Trace schema gains socket-risk: clean | flagged | "<finding-summary>" in filter-results:. Verification-commands gains the Socket scan command. .socket.json configures issue-rules with named severity thresholds (default: critical → error).
• Socket CI step in ci.yml's validate job — runs socket-cli scan against the lockfile, fails on critical.
• Socket GitHub App install instructions in the human guide for consumers.
• Trace revalidation workflow at .github/workflows/trace-revalidation-weekly.yml — weekly cron + workflow_dispatch. Scope: every approved + pre-shipped trace. Two-tier divergence action: soft → rolling dashboard issue (library-policy/dashboard label); hard → per-dep issue (library-policy/re-evaluation label). No auto-edit of traces, no auto-dispatch, no main-CI gating.
• Trace schema extensions in scripts/library-decisions/schema.mjs — last-revalidated: <ISO date> frontmatter field; socket-risk filter result; verification-commands array gains the Socket entry.
• Major-bump re-evaluation flow — scripts/library-decisions/check.mjs detects when a Renovate PR bumps a runtime dep across a semver-major boundary in a feature/core package and requires the trace's last-revalidated to be refreshed. Minor + patch bumps do NOT require re-evaluation.
• CodeQL workflow at .github/workflows/codeql.yml for javascript-typescript; runs on push to main + PRs + weekly schedule.
• pnpm audit signatures --audit-level=high added as one step in ci.yml's validate job.
• gitleaks pre-commit hook in .husky/pre-commit as a step alongside the existing state-sync guard. Custom-pattern allowlist via .gitleaks.toml.
• Sandcastle reviewer prompt update — extend .sandcastle/reviewer.prompt.md to read Socket CI output (via gh run view) and reject on critical, and to read CodeQL findings and reject on error severity.
• Failure-mode hierarchy table ships in docs/guides/ci-security.md and is referenced from the reviewer prompt.
• docs/guides/ci-security.md human reading-room — covers each gate, consumer-toggleable settings (GitHub native push protection, Socket App install, branch protection for library-policy/* labels), the failure-mode hierarchy table, and worked examples (a passing Renovate minor-bump PR, a blocked major-bump PR, a hard-divergence revalidation issue).
• CLAUDE.md "Key Conventions" gains a one-line bullet pointing to ADR-023 + the guide.
• Glossary already includes Trace revalidation and Major-bump re-evaluation (landed inline during the grill session that produced ADR-023).

Out of scope

• Paid Socket Team plan / server-side PR-block enforcement. The free App + self-hosted CLI achieves equivalent enforcement at $0; paid upgrade is a per-consumer decision.
• Snyk, Trivy, OSV-Scanner. Free GitHub-native (CodeQL + Dependabot alerts + push protection) + Socket + Renovate cover the surface at $0.
• Container scanning for the sandcastle Dockerfile. Renovate handles base-image bumps; the sandbox is short-lived and host-isolated.
• Auto-removal of approved-then-unused deps — pnpm fallow territory.
• License auto-enforcement at the lockfile layer (license-checker plugins). Defer until the policy has run for some time.
• Anything app-tier. ADR-022 exempts app tier from traces; this PRD inherits that exemption.
• Devdeps in any tier. Only dependencies (runtime) participate in the policy.
• Splitting ADR-023 from ADR-022 amendments into two ADRs. Decided in the grill (Q7d): one ADR, ADR-022 unedited but cited.
• Auto-dispatch on library-policy/re-evaluation issues. Human triage required; the dispatch loop drains the queue on demand.
• CI gating on library-policy/re-evaluation (block main). Main keeps deploying; trace re-walks happen in parallel.

Constraints

• ADR-023 is the source of truth. This PRD implements but does not extend it.
• ADR-022 stays unedited. The amendments in ADR-023 §6 are what the implementation honors. Both ADRs must be readable as a composed policy.
• ADR-019 — the sandcastle reviewer prompt is one of four enforcement layers. Reviewer-prompt extensions compose with the existing prompt shape and with the library-evaluation epic's story 06 (whose reviewer-prompt updates land first).
• ADR-021 — release-please picks up dep changes from commit history. Renovate's bump commits must use Conventional Commits (chore(deps):, chore(deps-major):) so release-please's per-package bump rules apply cleanly.
• Template-vs-consumer framing — every artifact ships as a consumer-inheritable default. Plan-gated tools (CodeQL on private repos) include clear error messages when the consumer's GitHub plan doesn't cover them.
• Conformance system parity — the failure-mode hierarchy mirrors ADR-012's latency-tiered shape. Same vocabulary, same agent feedback loop.
• Conventional Commits — every commit produced by the implementation follows <type>(<scope>): <subject>.
• --no-verify is forbidden — the bash-guard hook enforces this; new pre-commit checks inherit the protection.
• Reviewer prompt is the single composable gate for agent PRs — the sandcastle reviewer must be able to derive "approve/reject" from CI outputs (Socket findings, CodeQL severity) without needing a separate judgment surface.

Success criteria

• pnpm typecheck && pnpm test && pnpm lint && pnpm conformance && pnpm fallow:audit pass green at the end of the epic.
• pnpm coverage:diff covers every changed executable line introduced by the implementation slices.
• Renovate's first run (on this repo and on any consumer's downstream fork) opens a PR that SHA-pins every uses: reference in .github/workflows/*.yml. After merge, no @v<N>-style tag pin remains in any workflow.
• Running evaluate-library against any new package now collects and records a socket-risk filter result and includes the socket-cli invocation in verification-commands. Existing backfilled traces (from the library-evaluation epic) get socket-risk added via the Renovate-bump flow or via an explicit backfill task.
• A Renovate PR that bumps @sentry/node from 7.x → 8.x against packages/marketing-pages is blocked from auto-merge until the trace's last-revalidated is refreshed by running the evaluate-library skill.
• A Renovate PR that bumps @sentry/node from 7.5.0 → 7.6.0 against the same package auto-merges if all gates pass — no trace refresh required.
• .github/workflows/trace-revalidation-weekly.yml runs successfully on its first weekly cron, against the existing ~10 backfilled traces, and produces either zero divergence or a library-policy/dashboard issue with a comparison diff.
• A simulated hard-divergence trigger (manually mutating a trace's expected cve-scan value vs. what pnpm audit returns) opens a library-policy/re-evaluation issue with the correct title format and body.
• A simulated critical Socket finding in CI causes the sandcastle reviewer to reject the slice with notes referencing the Socket finding.
• A simulated error-severity CodeQL finding causes the same reviewer rejection.
• pnpm audit signatures runs as a step in CI and fails the job when a deliberately-tampered package signature is staged.
• gitleaks pre-commit hook blocks a commit that adds a known token pattern (Stripe-style test key) to any tracked file.
• docs/guides/ci-security.md includes the failure-mode hierarchy table, the consumer-toggleable settings list, and at least two worked examples (one approved flow, one blocked flow).
• CLAUDE.md Key Conventions includes the ADR-023 bullet.
• All existing backfilled traces (from the library-evaluation epic) carry a last-revalidated field after the first weekly cron run.

User stories

1. As a developer running pnpm add against a feature package, I want Socket's risk score to be one of the filter results the evaluate-library skill collects and records, so I get a single composed answer instead of having to remember to check Socket separately.
2. As an agent dispatched against a slice that bumps a runtime dep, I want Renovate's bump PR to either auto-merge (minor/patch) or require me to walk evaluate-library (major) so the policy gate is automatic, not remembered.
3. As a reviewer (human or agent) of a Renovate major-bump PR, I want the trace's last-revalidated field refreshed by the evaluate-library re-run so I can see at a glance "this dep was re-validated today" before approving.
4. As an agent reviewing a slice in sandcastle, I want the reviewer prompt to read Socket CI output + CodeQL findings + library-trace presence in one composed check and reject on any critical / error, so I have a single composable gate.
5. As a maintainer who hasn't touched the repo for a week, I want the weekly trace revalidation cron to produce at most one rolling dashboard issue (soft divergence) and zero per-dep issues (hard divergence) unless something actually drifted, so my notification surface stays clean.
6. As a future agent considering a previously-approved library that's now Socket-flagged, I want the trace revalidation cron to open a library-policy/re-evaluation issue with the trace path + the Socket finding + a clean re-walk handoff, so I can drive the re-evaluation without re-discovering the prior context.
7. As a maintainer reviewing CI output for a PR that touches package.json, I want Socket's comment + the socket-cli scan step's result + the library-trace presence check to all be visible in one place (the PR's checks panel), so the decision is one glance, not three.
8. As a security-conscious maintainer, I want every uses: reference in every workflow pinned to a 40-char SHA + a trailing # v<N> comment, so the tj-actions/changed-files class of attack is closed and Renovate keeps the SHAs current.
9. As a maintainer who accidentally pastes a token into a commit, I want the gitleaks pre-commit hook to refuse the commit + GitHub native push protection to be a second line of defense, so a leaked secret never reaches the remote.
10. As a code reviewer looking at a PR with a CodeQL error finding, I want the finding to appear in PR checks as a hard-block, so the pattern doesn't merge.
11. As a maintainer reading the repo for the first time, I want docs/guides/ci-security.md to walk me through the four pillars
    • the failure-mode hierarchy + the consumer-toggleable settings, so I understand what to enable in a downstream repo without spelunking workflows.

Implementation decisions

Module sketch — what lands where, by concern (no specific file paths where prose suffices):

• Renovate config — single .github/renovate.json extending a small set of presets: config:base, helpers:pinGitHubActionDigests, :separateMajorReleases, :automergeMinor, :automergePatch. Custom packageRules: group @sentry/*, @opentelemetry/*, @trpc/*, payload*, and inversify* into per-cluster weekly PRs. Dockerfile manager enabled for .sandcastle/Dockerfile. dependencyDashboard: true opens a single issue that summarizes open + queued PRs.
• Socket integration — schema layer. scripts/library-decisions/schema.mjs gains socketRisk: z.union([z.literal("clean"), z.literal("flagged"), z.string()]) in the filter-results Zod schema. The verification-commands array gains the Socket entry. The trace template (_template.md) mirrors the new field.
• Socket integration — skill layer. .claude/skills/evaluate-library/SKILL.md gains a "9 — Supply-chain behavior (Socket)" section. The skill's fail-fast logic (collect-cheap-skip-expensive) treats Socket as expensive (network call) and runs it after the cheap structural filters. socket-cli is the verification command; output parsing follows Socket's JSON schema.
• Socket integration — CI layer. One step in ci.yml's validate job: socket-cli scan --json | jq <severity-filter>. Fail on critical. .socket.json lives at repo root: { "issueRules": { "critical": "error", "high": "warn", "medium": "ignore", "low": "ignore" } }.
• Trace revalidation workflow. New file .github/workflows/trace-revalidation-weekly.yml. Triggers: schedule: - cron: "30 6 * * 1" (Monday 06:30 UTC, avoiding the Sunday→Monday CI peak), plus workflow_dispatch. Job: checkout, install, run a new script scripts/library-decisions/revalidate.mjs that walks every approved + pre-shipped trace, re-runs each trace's verification-commands, classifies divergence, opens or updates issues via gh CLI. Permissions: issues: write, contents: read (NO contents: write — no auto-edit).
• Major-bump re-evaluation flow. scripts/library-decisions/check.mjs gains a new mode: when invoked on a Renovate-generated PR (detected via branch prefix renovate/), it parses the lockfile diff to extract bumped deps + their from/to versions, classifies each as major / minor / patch, and for any feature/core-tier major bump checks that the corresponding trace's last-revalidated field is fresh (set today). If not fresh, exit non-zero with a pointer to the evaluate-library skill.
• CodeQL workflow. Standard GitHub-issued template: language: javascript-typescript, triggers push: branches: [main], pull_request, and weekly schedule. Default queries.
• Pre-commit gitleaks. .husky/pre-commit gains step: gitleaks protect --staged --redact. .gitleaks.toml ships with the repo's allowlist patterns (e.g. test fixtures in __seeds__/ that look like tokens but aren't).
• Reviewer-prompt update. .sandcastle/reviewer.prompt.md gains a "CI security checks" section after the existing library-trace check (from the library-evaluation epic's story 06). The reviewer reads gh run view output for the PR's check suite, looks for Socket findings of severity critical and CodeQL findings of severity error, and rejects the slice if either is present with notes referencing the specific finding.
• Human guide — docs/guides/ci-security.md follows the same shape as docs/guides/coverage.md: overview, per-pillar section, failure-mode hierarchy table, consumer settings list, two worked examples.
• CLAUDE.md update — one bullet in Key Conventions: "CI security + supply-chain enforcement: Renovate for bumps + Action SHA pinning, Socket for supply-chain behavior, weekly trace revalidation, CodeQL + audit signatures + gitleaks. See ADR-023 + docs/guides/ci-security.md."

Trace schema extension (Zod, lifted from ADR-023 §6.3):

filterResults: z.object({
  // ... existing 8 fields ...
  socketRisk: z.union([
    z.literal("clean"),
    z.literal("flagged"),
    z.string(), // human-readable finding summary
  ]),
});
lastRevalidated: z.string().nullable(); // ISO date or null on a fresh adoption

The date field stays mandatory (adoption-provenance); last-revalidated is set on major-bump re-eval (Q3) and on a successful trace revalidation run (J).

Failure-mode hierarchy (lifted from ADR-023 §5): the table is the source of truth referenced by both the reviewer prompt and docs/guides/ci-security.md. Changes to the hierarchy require an ADR amendment.

Sequencing — depends on the library-evaluation epic. This PRD's implementation depends on the in-flight library-evaluation epic:

• Story 01 of library-evaluation (trace schema foundation) must land first — this PRD extends that schema.
• Story 02 of library-evaluation (pre-commit check script) must land first — this PRD extends that script with the major-bump-detection mode.
• Story 04 of library-evaluation (evaluate-library skill) must land first — this PRD adds the Socket filter to that skill.
• Story 06 of library-evaluation (reviewer-prompt update) must land first — this PRD extends the reviewer prompt added there.

Sandcastle dispatch should order this PRD's epic after the library-evaluation epic completes.

Conformance system composition — no new use cases, controllers, manifest entries, audits, events, jobs, or realtime channels. This PRD is workflow/policy implementation, not feature-domain change. The conformance gates apply only to the new TypeScript/JS modules (Zod schema extensions, the revalidate.mjs script, the check.mjs major-bump mode) — they get standard vitest coverage.

Testing decisions

• scripts/library-decisions/schema.mjs extensions — unit tests covering: socketRisk field round-trips for all three variants (clean / flagged / <string>); lastRevalidated accepts ISO dates and null; missing socketRisk on a trace fails validation; lastRevalidated: null is the default for fresh traces.
• scripts/library-decisions/check.mjs major-bump mode — integration tests covering: minor bump on a feature-tier dep → pass without trace refresh; major bump on a feature-tier dep with fresh last-revalidated → pass; major bump on a feature-tier dep with stale last-revalidated → fail with a clear pointer; major bump on an app-tier dep → pass (app tier exempt); patch bump in a Renovate branch → pass; non-Renovate branch with a major bump → pass (the rule is Renovate-PR-scoped).
• scripts/library-decisions/revalidate.mjs — integration tests using a fixture trace directory: trace with no drift → no issue opened; trace with soft drift → dashboard issue created/updated; trace with hard drift → per-dep issue opened with correct labels + title format; trace already covered by an open per-dep issue → no duplicate issue; rejection trace → skipped entirely. Use gh CLI mocks or a fake GitHub API surface for the issue-write side.
• Renovate config — no automated test; verified by Renovate Dependency Dashboard preview run + manual review of the first PR (the SHA-pin sweep).
• Socket CI step — smoke test by adding a known-flagged package fixture to a test branch and asserting CI fails. Captures the socket-cli output format we depend on for parsing.
• CodeQL workflow — no test; the workflow file IS the test (GitHub validates the YAML; CodeQL action either runs or no-ops per consumer plan).
• pnpm audit signatures step — verified by the existence of the step in ci.yml + a smoke test where a deliberately-corrupt signature fails CI.
• gitleaks pre-commit hook — bash smoke test that pipes a staged commit containing a known token pattern through the hook and asserts exit code non-zero. Use a Stripe-style test key as the fixture.
• Reviewer-prompt extension — no automated test in the conformance sense (it's a prose runbook for an agent). Success criterion is manual: dispatch an agent against a PR with a simulated Socket critical finding, verify the agent rejects with the expected notes.
• Prior art — mirror the test patterns from the library-evaluation epic's stories 01–02 (trace schema + check script). The fixture and assertion shape carries over directly.
• Coverage bands — new scripts under scripts/library-decisions/ aren't feature packages, so no per-layer thresholds. Default expectation: 100% statement coverage on the new branches (the scripts are small).

Open questions

• Q1: Should the Renovate config use a branchPrefix other than the default renovate/ to make the check.mjs Renovate-PR-detection more robust against future Renovate refactors? — Recommended: no — Renovate's renovate/ prefix has been stable for years; use the default and detect via that prefix. Future-proofing here is premature.
• Q2: Should socket-cli scan run on every CI PR or only on PRs that touch package.json / pnpm-lock.yaml? — Recommended: only on PRs that touch those files. Use a paths: filter on the step. Cheaper CI; same coverage (Socket can't catch behavior changes in a PR that doesn't change deps).
• Q3: Should the major-bump re-evaluation rule apply when Renovate groups multiple deps in one PR? — Recommended: the rule applies per-dep, not per-PR. If a grouped PR contains 3 minor bumps + 1 major bump, the trace for the major-bump dep needs last-revalidated refreshed; the 3 minor bumps don't trigger. The check.mjs script walks the lockfile diff and validates each bumped dep independently.
• Q4: Should library-policy/re-evaluation issues auto-close when the trace's last-revalidated is refreshed in a subsequent commit? — Recommended: yes, the trace revalidation workflow checks for open issues whose dep names appear in newly-refreshed traces and closes them with a comment citing the refresh commit.
• Q5: Where do the gitleaks allowlist patterns live? — Recommended: ship a minimal .gitleaks.toml at repo root with one explicit allowlist for __seeds__/** test fixtures. Document that consumers extend it for their own custom patterns.
• Q6: Should the dependencyDashboard issue Renovate opens be labeled identically to the trace revalidation library-policy/dashboard issue? — Recommended: no, keep them separate. Renovate's dependency dashboard is about pending bumps; the trace revalidation dashboard is about post-adoption drift. Different queues, different labels (renovate/dashboard vs library-policy/dashboard).

Out of scope (deferred)

• Branch protection rules. Configuring GitHub branch protection to require Socket + CodeQL + audit-signatures + library-trace-check checks before merge is a per-repo settings change, not a tracked file. Document the recommended ruleset in docs/guides/ci-security.md and leave application to consumers.
• Auto-dispatch on library-policy/re-evaluation issues. Decided out of scope in the grill (Q7c); revisit if human triage becomes a bottleneck in practice.
• Renovate Dependency Dashboard → docs/work/ task integration. Surfacing pending bumps as pnpm work tasks would let agents pick them up via dispatch. Interesting but separate.
• OSSF Scorecard integration. Complementary to Socket but duplicates several signals; defer until the four-pillar stack has matured.
• StepSecurity Harden Runner. Adds runtime egress detection on top of Action SHA pinning. Defer; the SHA pins close the primary attack surface.
• Socket Team plan upgrade. Free tier is documented as adequate for this template; consumers upgrade per their own threat model.
• License-checker lockfile-layer enforcement. Defer until the policy has run for some time.

Further notes

• Anchored by ADR-023 — CI security + supply-chain enforcement stack. Read that first.
• Builds on ADR-022 — Library evaluation policy. ADR-022 stays unedited; ADR-023 §6 amends it with major-bump trigger, last-revalidated field, and Socket as the 9th hard filter.
• Builds on PRD library-evaluation-policy — many of this PRD's modules extend artifacts being built by that PRD's epic. The sequencing constraint in §Implementation Decisions is load-bearing.
• Glossary entries for Trace revalidation and Major-bump re-evaluation landed during the 2026-05-14 grill session that produced ADR-023.
• Conversation provenance — the 2026-05-14 grill-with-docs session that produced this PRD is captured in the session transcript; ADR-023 cites the audit of zero security tooling + the tj-actions/changed-files incident as concrete catalysts.