agentic-dev/docs/guides/ci-security.md

# CI security + supply-chain enforcement

Human reading-room for the four-pillar CI security stack. For decision-record density, see [ADR-023](../decisions/adr-023-ci-security-and-supply-chain.md). For the library evaluation policy that this stack extends, see [ADR-022](../decisions/adr-022-library-evaluation-policy.md) and [`docs/guides/adding-a-library.md`](./adding-a-library.md).

---

## Overview: the four pillars

ADR-022 closes the **decision** gate — every new runtime dependency is evaluated before it enters the lockfile. It does not close the **drift** gate. Once a library is in the lockfile, ADR-022 has nothing to say about CVEs that surface later, supply-chain compromises against trusted upstream maintainers, license relicensing, or EU-residency changes.

ADR-023 adds four pillars that close the drift gate:

| Pillar                      | Mechanism                                                  | Primary signal                                                         |
| --------------------------- | ---------------------------------------------------------- | ---------------------------------------------------------------------- |
| **1 — Renovate**            | `.github/renovate.json`                                    | Keeps lockfile current; SHA-pins GitHub Actions                        |
| **2 — Socket.dev**          | GitHub App + `socket-cli` in CI                            | Supply-chain _behavior_ — malicious scripts, suspicious network access |
| **3 — Trace revalidation**  | `.github/workflows/trace-revalidation-weekly.yml`          | CVE drift, license change, EU-residency flip, Socket-flag escalation   |
| **4 — GitHub-native gates** | CodeQL, push protection, `pnpm audit signatures`, gitleaks | Static analysis, secret patterns, sigstore provenance                  |

Each pillar enforces at a different latency, mirroring the multi-latency pattern of the conformance system (ADR-012):

```
pre-commit → CI (per-PR) → server-side (GitHub edge) → weekly cron
```

No single pillar sees everything. The composition is the enforcement.

---

## Pillar 1 — Renovate (bumps + Action SHA pinning)

### What it does

Renovate bot manages all dependency bumps via `.github/renovate.json`. It:

- Opens weekly PRs grouping minor + patch bumps by ecosystem cluster (`@sentry/*`, `@opentelemetry/*`, etc.) and auto-merges them when CI is green.
- Opens separate PRs for semver-major bumps. Major bumps do **not** auto-merge — they block until an agent re-runs `evaluate-library` and refreshes the trace.
- Rewrites every `uses: <owner>/<repo>@<tag>` in `.github/workflows/*.yml` to `uses: <owner>/<repo>@<40-char-sha>  # <tag>`. This closes the `tj-actions/changed-files` class of supply-chain attack permanently.

### What it catches

- **CVE patches** arriving as minor/patch releases — auto-merged once CI is green.
- **License or behavior regressions** hidden behind semver-major bumps — blocked until the trace is re-walked.
- **Action supply-chain attacks** where a compromised maintainer pushes a malicious tag — SHA pins mean the workflow ignores the new tag until Renovate opens a reviewed bump PR.

### Toggling in a consumer repo

Renovate requires a GitHub App install (see [GitHub Marketplace — Renovate](https://github.com/marketplace/renovate)) or a self-hosted Mend Renovate runner. The `.github/renovate.json` file ships with the template and works unchanged. To customize bump grouping, edit `packageRules` in that file.

---

## Pillar 2 — Socket.dev (supply-chain behavior detection)

### What it does

Socket detects supply-chain behavior, not just CVEs. It inspects what a package _does_ — post-install scripts, environment variable access, network calls at install time — not just what a CVE database _knows_. This catches the `event-stream` (2018) and `ua-parser-js` (2021) class of attacks that had no CVE when they shipped.

Two layers:

1. **Socket GitHub App** — posts a risk-score comment on every PR that touches `package.json` or `pnpm-lock.yaml`. Advisory; does not block merge. Free for open-source repos.
2. **`socket-cli` CI step** in `ci.yml`'s `validate` job — runs `socket-cli scan` against the lockfile. Configured by `.socket.json`:

   ```json
   {
     "issueRules": {
       "critical": "error",
       "high": "warn",
       "medium": "ignore",
       "low": "ignore"
     }
   }
   ```

   `critical` findings hard-block the PR. `high` and below are advisory comments only.

Socket is also the **9th hard filter** in the `evaluate-library` skill (ADR-022 adds 8; ADR-023 §6.3 adds this one). Trace frontmatter records the result:

```yaml
filter-results:
  socket-risk: clean | flagged | "<finding-summary>"
```

### What it catches

- Post-install scripts that write to `~/.ssh` or make outbound network calls.
- Packages with maintainer-account-compromise indicators (new maintainer + immediate publish).
- Dependency confusion and typosquat patterns.

### Toggling in a consumer repo

The CI step (`socket-cli scan`) runs unconditionally in CI regardless of App install — no configuration needed beyond the `.socket.json` that ships with the template. Adjust severity thresholds in `.socket.json` to match your threat model.

### Installing the Socket GitHub App

The App posts PR comments with a risk summary. It's free for public repos and most open-source use.

1. Navigate to [socket.dev](https://socket.dev) → **Get started** → **GitHub App**.
2. Select the organization or personal account that owns your repo.
3. Grant access to the specific repositories you want covered (or all repositories).
4. The App will start posting on your next PR that touches `package.json` or `pnpm-lock.yaml`.

No secrets, no environment variables, and no config file changes are needed — the App reads from your public repo or an authenticated GitHub token.

---

## Pillar 3 — Trace revalidation (continuous ADR-022 validation)

### What it does

`.github/workflows/trace-revalidation-weekly.yml` runs every Monday at 06:30 UTC (and on demand via `workflow_dispatch`). For each approved or pre-shipped trace in `docs/library-decisions/`:

1. Re-runs the trace's `verification-commands` block.
2. Compares output against the stored `filter-results` snapshot.
3. Classifies divergence:
   - **Soft** — CVE count changed without crossing severity threshold; maintenance signal downgraded one level; transitive dep count changed.
   - **Hard** — license changed; named consumer no longer present; critical CVE disclosed; EU-residency flipped; Socket flag escalated to `critical`.

**Issue management:**

- **Soft divergence** — appends to a single rolling "library-trace dashboard" issue (labeled `library-policy/dashboard`), kept open continuously. One issue total; humans skim it periodically; most entries need no action.
- **Hard divergence** — opens a fresh per-dep issue labeled `library-policy/re-evaluation`. Title: `[trace-revalidation] <package> — <reason>`. Body cites the trace path, verification output, and diff.

**What revalidation never does:**

- Edits a trace file. The re-walk needs the `evaluate-library` skill (8 filters + 3 prompts, with agent judgement). CI catches divergence; the dispatch loop fixes it.
- Fails CI on `main`. Main keeps deploying while traces get re-walked. Blocking main on CVE data would stall release-please PRs every time a CVE drops upstream.
- Auto-dispatches `library-policy/re-evaluation` issues. Issues are a human-triaged queue drained via `pnpm work dispatch`.

### What it catches

- CVEs published against a previously-clean dep.
- License relicensing (MIT → BSL, MIT → SSPL) that ADR-022's adoption-time check didn't see.
- EU-residency changes after a vendor infrastructure announcement.
- Socket flag escalation (package that was `clean` is now flagged after a maintainer compromise).

### Toggling in a consumer repo

The workflow ships in `.github/workflows/trace-revalidation-weekly.yml` and works without modification. It creates GitHub issues using the `GITHUB_TOKEN` automatically provided in workflows — no additional secrets needed.

---

## Pillar 4 — GitHub-native gates (CodeQL, secret scanning, sigstore, gitleaks)

### What it does

Four independent mechanisms in this pillar:

**CodeQL** (`.github/workflows/codeql.yml`) — static analysis for JavaScript/TypeScript. Runs on push to `main`, on PRs, and weekly on Wednesdays. `error`-severity findings hard-block the PR; `warning` and `note` are advisory.

**GitHub native push protection** — server-side, GitHub edge. Scans for known secret token patterns (API keys, credentials) before accepting a push. Consumer must enable this in repo settings (see [Consumer-toggleable settings](#consumer-toggleable-settings)).

**`pnpm audit signatures`** — added as one step in `ci.yml`'s `validate` job. Verifies npm sigstore attestations. Fails CI when a package in the lockfile has an invalid or missing attestation at `--audit-level=high`. Roughly 40% of the registry is signed today and the percentage is increasing.

**gitleaks pre-commit hook** — catches custom secret patterns that GitHub's allowlist doesn't know about (internal API keys, self-hosted service tokens). The hook in `.husky/pre-commit` runs `gitleaks protect --staged --redact` and exits gracefully if the `gitleaks` binary is not installed, so it never blocks developers who haven't set it up yet.

### What it catches

- **CodeQL** — SQL injection, XSS, prototype pollution, command injection, unsafe regex.
- **Push protection** — AWS keys, GitHub tokens, Stripe keys, and hundreds of other known provider patterns.
- **`pnpm audit signatures`** — tampered or unsigned packages in the lockfile.
- **gitleaks** — custom token patterns, internal service credentials, `.env`-style secrets accidentally staged.

### Toggling in a consumer repo

CodeQL, `pnpm audit signatures`, and the gitleaks hook all ship with the template and activate without per-consumer config. GitHub native push protection requires a one-time repo settings toggle (see [Consumer-toggleable settings](#consumer-toggleable-settings)).

#### CodeQL note for private repos

CodeQL is **free for public repos** and for repos on GitHub Pro/Team/Enterprise plans. Private repos on the free GitHub plan do not have access to GitHub Advanced Security, which CodeQL requires. If you're on a free plan with a private repo, the CodeQL workflow will fail with a clear error message from GitHub rather than silently no-op-ing. Either upgrade your plan or remove the `codeql.yml` workflow from your fork.

#### Installing gitleaks for developers

The pre-commit hook exits gracefully if `gitleaks` is not in `PATH` — it prints a one-line prompt to install and continues. No developer is blocked by a missing binary. To activate the hook:

**macOS (Homebrew):**

```bash
brew install gitleaks
```

**Linux (apt / package manager):**

```bash
# Ubuntu/Debian — check the gitleaks GitHub releases for the current version
wget https://github.com/gitleaks/gitleaks/releases/latest/download/gitleaks_linux_amd64.tar.gz
tar -xzf gitleaks_linux_amd64.tar.gz
sudo mv gitleaks /usr/local/bin/
```

**Linux (via go install):**

```bash
go install github.com/gitleaks/gitleaks/v8@latest
```

**Verify:**

```bash
gitleaks version
```

After install, the pre-commit hook runs automatically on `git commit`. The `__seeds__` allowlist in `.gitleaks.toml` covers intentional test fixtures — add new allowlist entries there if a false positive blocks a commit.

---

## Failure-mode hierarchy

Two principles govern what blocks vs. what comments:

- **Boolean checks** (schema valid, signature verifies, secret present, trace file present) hard-block. They have a definite right answer.
- **Judgment checks** (Socket risk score, CodeQL semantic finding) are advisory unless severity reaches `critical` / `error`. They can have false positives.

Full table — pillar, gate, trigger, action, GitHub label, who resolves:

| Pillar             | Gate                                                             | Trigger                               | Action                                                             | Label                          | Who resolves                                  |
| ------------------ | ---------------------------------------------------------------- | ------------------------------------- | ------------------------------------------------------------------ | ------------------------------ | --------------------------------------------- |
| Cross-cutting      | `pnpm typecheck && test && lint && conformance && coverage:diff` | PR / push to main                     | Hard block PR                                                      | —                              | Developer / agent                             |
| Cross-cutting      | State-sync guard (pre-commit)                                    | `git commit`                          | Block commit                                                       | —                              | Developer / agent                             |
| GitHub-native      | gitleaks (pre-commit)                                            | `git commit`                          | Block commit                                                       | —                              | Developer                                     |
| Cross-cutting      | Library-trace presence check (pre-commit)                        | `git commit`                          | Block commit                                                       | —                              | Developer / agent                             |
| GitHub-native      | GitHub native push protection                                    | `git push`                            | Block push at GitHub edge                                          | —                              | Developer                                     |
| Renovate           | Minor / patch bump PR                                            | New dep version available             | Auto-merge when CI is green                                        | `renovate/dashboard`           | Renovate bot                                  |
| Renovate           | Major bump PR                                                    | Semver-major dep version available    | Block until `evaluate-library` re-run + `last-revalidated` refresh | `renovate/dashboard`           | Agent (via `pnpm work dispatch`)              |
| Socket             | `socket-cli` CI — `critical` finding                             | PR touching `package.json` / lockfile | Hard block PR                                                      | —                              | Developer / agent                             |
| Socket             | `socket-cli` CI — `high` or below                                | PR touching `package.json` / lockfile | Advisory CI annotation                                             | —                              | Developer (optional)                          |
| Socket             | Socket GitHub App PR comment                                     | PR touching `package.json` / lockfile | Advisory comment on PR                                             | —                              | Developer (optional)                          |
| GitHub-native      | CodeQL — `error` severity                                        | Push / PR / weekly schedule           | Hard block PR                                                      | —                              | Developer / agent                             |
| GitHub-native      | CodeQL — `warning` / `note`                                      | Push / PR / weekly schedule           | Advisory annotation                                                | —                              | Developer (optional)                          |
| GitHub-native      | `pnpm audit signatures` failure                                  | PR / push CI                          | Hard block PR                                                      | —                              | Developer / agent                             |
| GitHub-native      | GitHub Dependabot vuln alerts                                    | CVE publication (server-side)         | Advisory alert in Security tab                                     | —                              | Developer (schedule via Renovate bump)        |
| Trace revalidation | Soft divergence                                                  | Weekly cron                           | Append to dashboard issue                                          | `library-policy/dashboard`     | Developer (skim; usually no action)           |
| Trace revalidation | Hard divergence                                                  | Weekly cron                           | Open per-dep issue                                                 | `library-policy/re-evaluation` | Human triage → agent via `pnpm work dispatch` |

---

## Consumer-toggleable settings

These three settings are not configured by template files — they require one-time actions in your GitHub repo settings or a third-party install. The template documents them here so consumers know what's available and how to activate each.

### GitHub native push protection

Blocks pushes at the GitHub edge if they contain known secret patterns. Free for all plan tiers on public repos; available on GitHub Advanced Security plans for private repos.

**To enable:** Repo settings → Security → Code security and analysis → Secret scanning → Enable "Push protection".

Once enabled, pushes containing detected patterns are rejected with a link to review the finding. Developers can bypass with a justification if the detection is a false positive.

### Socket GitHub App

Adds risk-score comments on PRs that touch `package.json` or `pnpm-lock.yaml`. See [Installing the Socket GitHub App](#installing-the-socket-github-app) above.

The CI step (`socket-cli scan`) runs independently of the App install and hard-blocks on `critical` findings either way. The App adds the human-readable PR comment layer on top.

### Branch protection rules for `library-policy/*` labels

Optionally, configure branch protection to require human review before merging any PR labeled `library-policy/re-evaluation`. This prevents an agent loop from auto-closing a re-evaluation issue without a human sign-off.

**To enable:** Repo settings → Branches → Add rule for `main` → check "Require approvals" (≥1) → optionally add a status check that verifies the label is resolved before merge.

This is consumer-optional. The template does not enforce it because the right threshold varies by team.

---

## Worked examples

### Example 1: Passing Renovate minor-bump PR

**Scenario:** Renovate opens a weekly PR bumping `@sentry/node` from `8.3.0` to `8.5.0` (a minor bump) and `@sentry/nextjs` from `8.3.0` to `8.5.0` in the same PR (grouped by the `@sentry/*` cluster rule in `renovate.json`).

**What happens:**

1. **Renovate opens the PR** — title: `chore(deps): bump @sentry/* packages`, labeled `renovate/dashboard`.
2. **CI runs the `validate` job** — all five conformance gates pass (`typecheck`, `test`, `lint`, `conformance`, `coverage:diff`). `pnpm audit signatures` passes (no tampered packages). `socket-cli scan` returns `clean` (no new install scripts or network behaviors added by the patch).
3. **CodeQL workflow runs** — no new findings.
4. **Socket GitHub App** posts a comment: "No new issues found." Advisory; no action required.
5. **All checks green** → Renovate auto-merges the PR per the `automergeMinor: true` rule.
6. **release-please** reads the `chore(deps):` commit prefix — no version bump (deps bump commits don't trigger a feature-package version). The root template version increments if the commit path is cross-cutting.

**No human action required.** The full cycle — Renovate opens → CI passes → auto-merge — is self-contained.

---

### Example 2: Blocked major-bump PR + hard-divergence revalidation issue

This example covers two related failure modes that appear in the same ecosystem: a Renovate major-bump PR that blocks, and a Socket-flagged hard-divergence issue opened by the weekly cron.

#### Part A — Blocked major-bump PR

**Scenario:** Renovate opens a PR bumping `zod` from `3.22.4` to `4.0.0` (a semver-major bump). The existing trace at `docs/library-decisions/2026-05-14-zod.md` has `last-revalidated: 2026-05-14` (set when Story 05 backfill ran) and `version: 3.22.4`.

**What the pre-commit check does (`scripts/library-decisions/check.mjs`):**

The script detects that `zod`'s version in `package.json` crosses a semver-major boundary relative to the trace's recorded `version` field and that `last-revalidated` predates the bump. The PR comment reads:

```
[library-trace] zod: semver-major bump detected (3.22.4 → 4.0.0).
Trace last-revalidated: 2026-05-14. Re-run /evaluate-library before merging.
```

The check does **not** auto-fail CI — it opens a blocking PR comment. CI itself passes (the code may build fine). But the sandcastle reviewer prompt rejects the slice until the trace is refreshed.

**What the agent (or developer) does:**

1. Invoke the skill: `/evaluate-library zod --tier feature --target packages/auth`.
2. The skill re-walks all 9 filters (including Socket scan of zod v4).
3. If `zod` v4 passes: the trace at `docs/library-decisions/2026-05-14-zod.md` is updated in-place — `version`, `filter-results`, `verification-commands`, and `last-revalidated` are refreshed; the original `date` field is preserved.
4. Commit the updated trace: `docs(library-decisions): re-evaluate zod 4.0.0 after major bump`.
5. Push to the Renovate PR branch. The check re-runs, finds `last-revalidated` is current, and unblocks.
6. CI + Socket + CodeQL all pass. The reviewer prompt approves. Renovate's PR merges.

If `zod` v4 **fails** a filter (e.g., license changed, Socket finds a new install script), the trace is updated with `decision: rejected` and the Renovate PR is closed.

#### Part B — Hard-divergence revalidation issue

**Scenario:** Two weeks later, `socket-cli` flags `lefthook` (a dev dep used in a downstream consumer's fork) with a `critical` finding after a maintainer-account compromise. The weekly cron runs on Monday morning.

**What the cron does:**

1. Re-runs `socket-cli scan lefthook` as part of `verification-commands`.
2. Compares against the trace's `filter-results.socket-risk: clean` snapshot.
3. Classifies as **hard divergence** (Socket flag escalated to `critical`).
4. Opens a GitHub issue labeled `library-policy/re-evaluation`:

   ```
   Title: [trace-revalidation] lefthook — socket-risk escalated to critical
   Body:
   Trace: docs/library-decisions/2026-05-14-lefthook.md
   Previous socket-risk: clean
   Current socket-risk: critical — malicious post-install script detected
   Verification output: [full socket-cli output pasted here]
   Next step: run /evaluate-library lefthook and update the trace. If rejection is warranted, remove the package.
   ```

**What the developer / agent does:**

1. Triage the issue via `pnpm work dispatch`. The dispatch loop surfaces it as the next ready task.
2. Re-run `/evaluate-library lefthook --tier feature --target packages/auth`.
3. If the Socket finding is confirmed: `decision: rejected`, trace updated, `pnpm remove lefthook --filter packages/auth`, commit.
4. If it's a false positive (Socket has re-classified): `socket-risk: clean`, trace updated with `last-revalidated`, issue closed.
5. The agent closes the `library-policy/re-evaluation` issue with a link to the updated trace commit.

**The weekly cron never auto-closes issues.** Closing requires an explicit agent or human action — the issue is the queue; the dispatch loop drains it.

---

## Cross-links

- [ADR-023 — CI security + supply-chain enforcement](../decisions/adr-023-ci-security-and-supply-chain.md) — the authoritative decision record with full rationale, alternatives considered, and failure-mode hierarchy (§5)
- [ADR-022 — Library evaluation policy](../decisions/adr-022-library-evaluation-policy.md) — the adoption-time gate this stack builds on
- [`docs/guides/adding-a-library.md`](./adding-a-library.md) — how to add a new runtime dependency (includes the 9-filter evaluation flow)
- [`.claude/skills/evaluate-library/SKILL.md`](../../.claude/skills/evaluate-library/SKILL.md) — agent runbook for `evaluate-library`
- [`docs/guides/releasing.md`](./releasing.md) — release-please workflow; how Renovate bump PRs interact with versioning
- [`docs/glossary.md`](../glossary.md) — entries for **Library trace**, **Pre-shipped trace**, **Trace revalidation**, **Major-bump re-evaluation**
- [ADR-019 — Sandcastle agent orchestration](../decisions/adr-019-sandcastle-agent-orchestration.md) — the reviewer prompt is the agent-loop enforcement surface for Socket + CodeQL gates (ADR-023 §7)