From 9d83a6a5a25eade53772c5c5f657a2ab6f3bd007 Mon Sep 17 00:00:00 2001 From: Danijel Martinek Date: Thu, 14 May 2026 18:12:29 +0000 Subject: [PATCH] docs(ci-security): add CI security guide + CLAUDE.md convention bullet MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Covers the four-pillar stack (Renovate, Socket, trace revalidation, GitHub-native gates), the failure-mode hierarchy table from ADR-023 §5, consumer-toggleable settings, Socket App + gitleaks install instructions, CodeQL private-repo note, and two worked examples (minor-bump auto-merge; major-bump block + hard-divergence revalidation issue). CLAUDE.md Key Conventions gains the CI security bullet pointing to ADR-023 + docs/guides/ci-security.md for agent discoverability. Co-Authored-By: Claude Sonnet 4.6 --- CLAUDE.md | 1 + docs/guides/ci-security.md | 354 +++++++++++++++++++++++++++++++++++++ 2 files changed, 355 insertions(+) create mode 100644 docs/guides/ci-security.md diff --git a/CLAUDE.md b/CLAUDE.md index a0efc28..96bbc07 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -125,6 +125,7 @@ See `docs/guides/coverage.md` for the cookbook and ADR-020 for the full rational - **Self-asserting `bindProductionX(ctx)`** — every feature's bind-production calls `assertFeatureConformance(container, manifest, symbols, ctx)` at its tail. `pnpm dev` refuses to boot on drift - **`pnpm conformance`** — cross-feature event-closure check; fails CI on orphan consumers - **New runtime dependencies require a library trace** — adding a runtime dependency to a feature- or core-tier package requires a trace at `docs/library-decisions/-.md` produced by the `/evaluate-library` skill; see ADR-022 and `docs/guides/adding-a-library.md` +- **CI security + supply-chain enforcement** — Renovate for bumps + Action SHA pinning, Socket for supply-chain behavior, weekly trace revalidation, CodeQL + audit signatures + gitleaks. See ADR-023 + `docs/guides/ci-security.md` ## MCP Servers diff --git a/docs/guides/ci-security.md b/docs/guides/ci-security.md new file mode 100644 index 0000000..758c1c7 --- /dev/null +++ b/docs/guides/ci-security.md @@ -0,0 +1,354 @@ +# CI security + supply-chain enforcement + +Human reading-room for the four-pillar CI security stack. For decision-record density, see [ADR-023](../decisions/adr-023-ci-security-and-supply-chain.md). For the library evaluation policy that this stack extends, see [ADR-022](../decisions/adr-022-library-evaluation-policy.md) and [`docs/guides/adding-a-library.md`](./adding-a-library.md). + +--- + +## Overview: the four pillars + +ADR-022 closes the **decision** gate — every new runtime dependency is evaluated before it enters the lockfile. It does not close the **drift** gate. Once a library is in the lockfile, ADR-022 has nothing to say about CVEs that surface later, supply-chain compromises against trusted upstream maintainers, license relicensing, or EU-residency changes. + +ADR-023 adds four pillars that close the drift gate: + +| Pillar | Mechanism | Primary signal | +| --------------------------- | ---------------------------------------------------------- | ---------------------------------------------------------------------- | +| **1 — Renovate** | `.github/renovate.json` | Keeps lockfile current; SHA-pins GitHub Actions | +| **2 — Socket.dev** | GitHub App + `socket-cli` in CI | Supply-chain _behavior_ — malicious scripts, suspicious network access | +| **3 — Trace revalidation** | `.github/workflows/trace-revalidation-weekly.yml` | CVE drift, license change, EU-residency flip, Socket-flag escalation | +| **4 — GitHub-native gates** | CodeQL, push protection, `pnpm audit signatures`, gitleaks | Static analysis, secret patterns, sigstore provenance | + +Each pillar enforces at a different latency, mirroring the multi-latency pattern of the conformance system (ADR-012): + +``` +pre-commit → CI (per-PR) → server-side (GitHub edge) → weekly cron +``` + +No single pillar sees everything. The composition is the enforcement. + +--- + +## Pillar 1 — Renovate (bumps + Action SHA pinning) + +### What it does + +Renovate bot manages all dependency bumps via `.github/renovate.json`. It: + +- Opens weekly PRs grouping minor + patch bumps by ecosystem cluster (`@sentry/*`, `@opentelemetry/*`, etc.) and auto-merges them when CI is green. +- Opens separate PRs for semver-major bumps. Major bumps do **not** auto-merge — they block until an agent re-runs `evaluate-library` and refreshes the trace. +- Rewrites every `uses: /@` in `.github/workflows/*.yml` to `uses: /@<40-char-sha> # `. This closes the `tj-actions/changed-files` class of supply-chain attack permanently. + +### What it catches + +- **CVE patches** arriving as minor/patch releases — auto-merged once CI is green. +- **License or behavior regressions** hidden behind semver-major bumps — blocked until the trace is re-walked. +- **Action supply-chain attacks** where a compromised maintainer pushes a malicious tag — SHA pins mean the workflow ignores the new tag until Renovate opens a reviewed bump PR. + +### Toggling in a consumer repo + +Renovate requires a GitHub App install (see [GitHub Marketplace — Renovate](https://github.com/marketplace/renovate)) or a self-hosted Mend Renovate runner. The `.github/renovate.json` file ships with the template and works unchanged. To customize bump grouping, edit `packageRules` in that file. + +--- + +## Pillar 2 — Socket.dev (supply-chain behavior detection) + +### What it does + +Socket detects supply-chain behavior, not just CVEs. It inspects what a package _does_ — post-install scripts, environment variable access, network calls at install time — not just what a CVE database _knows_. This catches the `event-stream` (2018) and `ua-parser-js` (2021) class of attacks that had no CVE when they shipped. + +Two layers: + +1. **Socket GitHub App** — posts a risk-score comment on every PR that touches `package.json` or `pnpm-lock.yaml`. Advisory; does not block merge. Free for open-source repos. +2. **`socket-cli` CI step** in `ci.yml`'s `validate` job — runs `socket-cli scan` against the lockfile. Configured by `.socket.json`: + + ```json + { + "issueRules": { + "critical": "error", + "high": "warn", + "medium": "ignore", + "low": "ignore" + } + } + ``` + + `critical` findings hard-block the PR. `high` and below are advisory comments only. + +Socket is also the **9th hard filter** in the `evaluate-library` skill (ADR-022 adds 8; ADR-023 §6.3 adds this one). Trace frontmatter records the result: + +```yaml +filter-results: + socket-risk: clean | flagged | "" +``` + +### What it catches + +- Post-install scripts that write to `~/.ssh` or make outbound network calls. +- Packages with maintainer-account-compromise indicators (new maintainer + immediate publish). +- Dependency confusion and typosquat patterns. + +### Toggling in a consumer repo + +The CI step (`socket-cli scan`) runs unconditionally in CI regardless of App install — no configuration needed beyond the `.socket.json` that ships with the template. Adjust severity thresholds in `.socket.json` to match your threat model. + +### Installing the Socket GitHub App + +The App posts PR comments with a risk summary. It's free for public repos and most open-source use. + +1. Navigate to [socket.dev](https://socket.dev) → **Get started** → **GitHub App**. +2. Select the organization or personal account that owns your repo. +3. Grant access to the specific repositories you want covered (or all repositories). +4. The App will start posting on your next PR that touches `package.json` or `pnpm-lock.yaml`. + +No secrets, no environment variables, and no config file changes are needed — the App reads from your public repo or an authenticated GitHub token. + +--- + +## Pillar 3 — Trace revalidation (continuous ADR-022 validation) + +### What it does + +`.github/workflows/trace-revalidation-weekly.yml` runs every Monday at 06:30 UTC (and on demand via `workflow_dispatch`). For each approved or pre-shipped trace in `docs/library-decisions/`: + +1. Re-runs the trace's `verification-commands` block. +2. Compares output against the stored `filter-results` snapshot. +3. Classifies divergence: + - **Soft** — CVE count changed without crossing severity threshold; maintenance signal downgraded one level; transitive dep count changed. + - **Hard** — license changed; named consumer no longer present; critical CVE disclosed; EU-residency flipped; Socket flag escalated to `critical`. + +**Issue management:** + +- **Soft divergence** — appends to a single rolling "library-trace dashboard" issue (labeled `library-policy/dashboard`), kept open continuously. One issue total; humans skim it periodically; most entries need no action. +- **Hard divergence** — opens a fresh per-dep issue labeled `library-policy/re-evaluation`. Title: `[trace-revalidation] — `. Body cites the trace path, verification output, and diff. + +**What revalidation never does:** + +- Edits a trace file. The re-walk needs the `evaluate-library` skill (8 filters + 3 prompts, with agent judgement). CI catches divergence; the dispatch loop fixes it. +- Fails CI on `main`. Main keeps deploying while traces get re-walked. Blocking main on CVE data would stall release-please PRs every time a CVE drops upstream. +- Auto-dispatches `library-policy/re-evaluation` issues. Issues are a human-triaged queue drained via `pnpm work dispatch`. + +### What it catches + +- CVEs published against a previously-clean dep. +- License relicensing (MIT → BSL, MIT → SSPL) that ADR-022's adoption-time check didn't see. +- EU-residency changes after a vendor infrastructure announcement. +- Socket flag escalation (package that was `clean` is now flagged after a maintainer compromise). + +### Toggling in a consumer repo + +The workflow ships in `.github/workflows/trace-revalidation-weekly.yml` and works without modification. It creates GitHub issues using the `GITHUB_TOKEN` automatically provided in workflows — no additional secrets needed. + +--- + +## Pillar 4 — GitHub-native gates (CodeQL, secret scanning, sigstore, gitleaks) + +### What it does + +Four independent mechanisms in this pillar: + +**CodeQL** (`.github/workflows/codeql.yml`) — static analysis for JavaScript/TypeScript. Runs on push to `main`, on PRs, and weekly on Wednesdays. `error`-severity findings hard-block the PR; `warning` and `note` are advisory. + +**GitHub native push protection** — server-side, GitHub edge. Scans for known secret token patterns (API keys, credentials) before accepting a push. Consumer must enable this in repo settings (see [Consumer-toggleable settings](#consumer-toggleable-settings)). + +**`pnpm audit signatures`** — added as one step in `ci.yml`'s `validate` job. Verifies npm sigstore attestations. Fails CI when a package in the lockfile has an invalid or missing attestation at `--audit-level=high`. Roughly 40% of the registry is signed today and the percentage is increasing. + +**gitleaks pre-commit hook** — catches custom secret patterns that GitHub's allowlist doesn't know about (internal API keys, self-hosted service tokens). The hook in `.husky/pre-commit` runs `gitleaks protect --staged --redact` and exits gracefully if the `gitleaks` binary is not installed, so it never blocks developers who haven't set it up yet. + +### What it catches + +- **CodeQL** — SQL injection, XSS, prototype pollution, command injection, unsafe regex. +- **Push protection** — AWS keys, GitHub tokens, Stripe keys, and hundreds of other known provider patterns. +- **`pnpm audit signatures`** — tampered or unsigned packages in the lockfile. +- **gitleaks** — custom token patterns, internal service credentials, `.env`-style secrets accidentally staged. + +### Toggling in a consumer repo + +CodeQL, `pnpm audit signatures`, and the gitleaks hook all ship with the template and activate without per-consumer config. GitHub native push protection requires a one-time repo settings toggle (see [Consumer-toggleable settings](#consumer-toggleable-settings)). + +#### CodeQL note for private repos + +CodeQL is **free for public repos** and for repos on GitHub Pro/Team/Enterprise plans. Private repos on the free GitHub plan do not have access to GitHub Advanced Security, which CodeQL requires. If you're on a free plan with a private repo, the CodeQL workflow will fail with a clear error message from GitHub rather than silently no-op-ing. Either upgrade your plan or remove the `codeql.yml` workflow from your fork. + +#### Installing gitleaks for developers + +The pre-commit hook exits gracefully if `gitleaks` is not in `PATH` — it prints a one-line prompt to install and continues. No developer is blocked by a missing binary. To activate the hook: + +**macOS (Homebrew):** + +```bash +brew install gitleaks +``` + +**Linux (apt / package manager):** + +```bash +# Ubuntu/Debian — check the gitleaks GitHub releases for the current version +wget https://github.com/gitleaks/gitleaks/releases/latest/download/gitleaks_linux_amd64.tar.gz +tar -xzf gitleaks_linux_amd64.tar.gz +sudo mv gitleaks /usr/local/bin/ +``` + +**Linux (via go install):** + +```bash +go install github.com/gitleaks/gitleaks/v8@latest +``` + +**Verify:** + +```bash +gitleaks version +``` + +After install, the pre-commit hook runs automatically on `git commit`. The `__seeds__` allowlist in `.gitleaks.toml` covers intentional test fixtures — add new allowlist entries there if a false positive blocks a commit. + +--- + +## Failure-mode hierarchy + +Two principles govern what blocks vs. what comments: + +- **Boolean checks** (schema valid, signature verifies, secret present, trace file present) hard-block. They have a definite right answer. +- **Judgment checks** (Socket risk score, CodeQL semantic finding) are advisory unless severity reaches `critical` / `error`. They can have false positives. + +Full table — pillar, gate, trigger, action, GitHub label, who resolves: + +| Pillar | Gate | Trigger | Action | Label | Who resolves | +| ------------------ | ---------------------------------------------------------------- | ------------------------------------- | ------------------------------------------------------------------ | ------------------------------ | --------------------------------------------- | +| Cross-cutting | `pnpm typecheck && test && lint && conformance && coverage:diff` | PR / push to main | Hard block PR | — | Developer / agent | +| Cross-cutting | State-sync guard (pre-commit) | `git commit` | Block commit | — | Developer / agent | +| GitHub-native | gitleaks (pre-commit) | `git commit` | Block commit | — | Developer | +| Cross-cutting | Library-trace presence check (pre-commit) | `git commit` | Block commit | — | Developer / agent | +| GitHub-native | GitHub native push protection | `git push` | Block push at GitHub edge | — | Developer | +| Renovate | Minor / patch bump PR | New dep version available | Auto-merge when CI is green | `renovate/dashboard` | Renovate bot | +| Renovate | Major bump PR | Semver-major dep version available | Block until `evaluate-library` re-run + `last-revalidated` refresh | `renovate/dashboard` | Agent (via `pnpm work dispatch`) | +| Socket | `socket-cli` CI — `critical` finding | PR touching `package.json` / lockfile | Hard block PR | — | Developer / agent | +| Socket | `socket-cli` CI — `high` or below | PR touching `package.json` / lockfile | Advisory CI annotation | — | Developer (optional) | +| Socket | Socket GitHub App PR comment | PR touching `package.json` / lockfile | Advisory comment on PR | — | Developer (optional) | +| GitHub-native | CodeQL — `error` severity | Push / PR / weekly schedule | Hard block PR | — | Developer / agent | +| GitHub-native | CodeQL — `warning` / `note` | Push / PR / weekly schedule | Advisory annotation | — | Developer (optional) | +| GitHub-native | `pnpm audit signatures` failure | PR / push CI | Hard block PR | — | Developer / agent | +| GitHub-native | GitHub Dependabot vuln alerts | CVE publication (server-side) | Advisory alert in Security tab | — | Developer (schedule via Renovate bump) | +| Trace revalidation | Soft divergence | Weekly cron | Append to dashboard issue | `library-policy/dashboard` | Developer (skim; usually no action) | +| Trace revalidation | Hard divergence | Weekly cron | Open per-dep issue | `library-policy/re-evaluation` | Human triage → agent via `pnpm work dispatch` | + +--- + +## Consumer-toggleable settings + +These three settings are not configured by template files — they require one-time actions in your GitHub repo settings or a third-party install. The template documents them here so consumers know what's available and how to activate each. + +### GitHub native push protection + +Blocks pushes at the GitHub edge if they contain known secret patterns. Free for all plan tiers on public repos; available on GitHub Advanced Security plans for private repos. + +**To enable:** Repo settings → Security → Code security and analysis → Secret scanning → Enable "Push protection". + +Once enabled, pushes containing detected patterns are rejected with a link to review the finding. Developers can bypass with a justification if the detection is a false positive. + +### Socket GitHub App + +Adds risk-score comments on PRs that touch `package.json` or `pnpm-lock.yaml`. See [Installing the Socket GitHub App](#installing-the-socket-github-app) above. + +The CI step (`socket-cli scan`) runs independently of the App install and hard-blocks on `critical` findings either way. The App adds the human-readable PR comment layer on top. + +### Branch protection rules for `library-policy/*` labels + +Optionally, configure branch protection to require human review before merging any PR labeled `library-policy/re-evaluation`. This prevents an agent loop from auto-closing a re-evaluation issue without a human sign-off. + +**To enable:** Repo settings → Branches → Add rule for `main` → check "Require approvals" (≥1) → optionally add a status check that verifies the label is resolved before merge. + +This is consumer-optional. The template does not enforce it because the right threshold varies by team. + +--- + +## Worked examples + +### Example 1: Passing Renovate minor-bump PR + +**Scenario:** Renovate opens a weekly PR bumping `@sentry/node` from `8.3.0` to `8.5.0` (a minor bump) and `@sentry/nextjs` from `8.3.0` to `8.5.0` in the same PR (grouped by the `@sentry/*` cluster rule in `renovate.json`). + +**What happens:** + +1. **Renovate opens the PR** — title: `chore(deps): bump @sentry/* packages`, labeled `renovate/dashboard`. +2. **CI runs the `validate` job** — all five conformance gates pass (`typecheck`, `test`, `lint`, `conformance`, `coverage:diff`). `pnpm audit signatures` passes (no tampered packages). `socket-cli scan` returns `clean` (no new install scripts or network behaviors added by the patch). +3. **CodeQL workflow runs** — no new findings. +4. **Socket GitHub App** posts a comment: "No new issues found." Advisory; no action required. +5. **All checks green** → Renovate auto-merges the PR per the `automergeMinor: true` rule. +6. **release-please** reads the `chore(deps):` commit prefix — no version bump (deps bump commits don't trigger a feature-package version). The root template version increments if the commit path is cross-cutting. + +**No human action required.** The full cycle — Renovate opens → CI passes → auto-merge — is self-contained. + +--- + +### Example 2: Blocked major-bump PR + hard-divergence revalidation issue + +This example covers two related failure modes that appear in the same ecosystem: a Renovate major-bump PR that blocks, and a Socket-flagged hard-divergence issue opened by the weekly cron. + +#### Part A — Blocked major-bump PR + +**Scenario:** Renovate opens a PR bumping `zod` from `3.22.4` to `4.0.0` (a semver-major bump). The existing trace at `docs/library-decisions/2026-05-14-zod.md` has `last-revalidated: 2026-05-14` (set when Story 05 backfill ran) and `version: 3.22.4`. + +**What the pre-commit check does (`scripts/library-decisions/check.mjs`):** + +The script detects that `zod`'s version in `package.json` crosses a semver-major boundary relative to the trace's recorded `version` field and that `last-revalidated` predates the bump. The PR comment reads: + +``` +[library-trace] zod: semver-major bump detected (3.22.4 → 4.0.0). +Trace last-revalidated: 2026-05-14. Re-run /evaluate-library before merging. +``` + +The check does **not** auto-fail CI — it opens a blocking PR comment. CI itself passes (the code may build fine). But the sandcastle reviewer prompt rejects the slice until the trace is refreshed. + +**What the agent (or developer) does:** + +1. Invoke the skill: `/evaluate-library zod --tier feature --target packages/auth`. +2. The skill re-walks all 9 filters (including Socket scan of zod v4). +3. If `zod` v4 passes: the trace at `docs/library-decisions/2026-05-14-zod.md` is updated in-place — `version`, `filter-results`, `verification-commands`, and `last-revalidated` are refreshed; the original `date` field is preserved. +4. Commit the updated trace: `docs(library-decisions): re-evaluate zod 4.0.0 after major bump`. +5. Push to the Renovate PR branch. The check re-runs, finds `last-revalidated` is current, and unblocks. +6. CI + Socket + CodeQL all pass. The reviewer prompt approves. Renovate's PR merges. + +If `zod` v4 **fails** a filter (e.g., license changed, Socket finds a new install script), the trace is updated with `decision: rejected` and the Renovate PR is closed. + +#### Part B — Hard-divergence revalidation issue + +**Scenario:** Two weeks later, `socket-cli` flags `lefthook` (a dev dep used in a downstream consumer's fork) with a `critical` finding after a maintainer-account compromise. The weekly cron runs on Monday morning. + +**What the cron does:** + +1. Re-runs `socket-cli scan lefthook` as part of `verification-commands`. +2. Compares against the trace's `filter-results.socket-risk: clean` snapshot. +3. Classifies as **hard divergence** (Socket flag escalated to `critical`). +4. Opens a GitHub issue labeled `library-policy/re-evaluation`: + + ``` + Title: [trace-revalidation] lefthook — socket-risk escalated to critical + Body: + Trace: docs/library-decisions/2026-05-14-lefthook.md + Previous socket-risk: clean + Current socket-risk: critical — malicious post-install script detected + Verification output: [full socket-cli output pasted here] + Next step: run /evaluate-library lefthook and update the trace. If rejection is warranted, remove the package. + ``` + +**What the developer / agent does:** + +1. Triage the issue via `pnpm work dispatch`. The dispatch loop surfaces it as the next ready task. +2. Re-run `/evaluate-library lefthook --tier feature --target packages/auth`. +3. If the Socket finding is confirmed: `decision: rejected`, trace updated, `pnpm remove lefthook --filter packages/auth`, commit. +4. If it's a false positive (Socket has re-classified): `socket-risk: clean`, trace updated with `last-revalidated`, issue closed. +5. The agent closes the `library-policy/re-evaluation` issue with a link to the updated trace commit. + +**The weekly cron never auto-closes issues.** Closing requires an explicit agent or human action — the issue is the queue; the dispatch loop drains it. + +--- + +## Cross-links + +- [ADR-023 — CI security + supply-chain enforcement](../decisions/adr-023-ci-security-and-supply-chain.md) — the authoritative decision record with full rationale, alternatives considered, and failure-mode hierarchy (§5) +- [ADR-022 — Library evaluation policy](../decisions/adr-022-library-evaluation-policy.md) — the adoption-time gate this stack builds on +- [`docs/guides/adding-a-library.md`](./adding-a-library.md) — how to add a new runtime dependency (includes the 9-filter evaluation flow) +- [`.claude/skills/evaluate-library/SKILL.md`](../../.claude/skills/evaluate-library/SKILL.md) — agent runbook for `evaluate-library` +- [`docs/guides/releasing.md`](./releasing.md) — release-please workflow; how Renovate bump PRs interact with versioning +- [`docs/glossary.md`](../glossary.md) — entries for **Library trace**, **Pre-shipped trace**, **Trace revalidation**, **Major-bump re-evaluation** +- [ADR-019 — Sandcastle agent orchestration](../decisions/adr-019-sandcastle-agent-orchestration.md) — the reviewer prompt is the agent-loop enforcement surface for Socket + CodeQL gates (ADR-023 §7)