Files
agentic-dev/docs/work/2026-05-14-ci-security-and-supply-chain/08-reviewer-prompt-update/_story.md

2.8 KiB

id, epic, title, type, status, feature, depends-on, blocks
id epic title type status feature depends-on blocks
08-reviewer-prompt-update 2026-05-14-ci-security-and-supply-chain Sandcastle reviewer prompt update technical-story done tooling
02-socket-integration
06-codeql-and-audit-signatures
09-ci-security-guide-and-docs

Goal

Extend .sandcastle/reviewer.prompt.md with a "CI security checks" section that instructs the reviewer agent to read Socket CI output and CodeQL findings via gh run view, and reject the slice with specific notes if any critical Socket finding or error-severity CodeQL finding is present.

Why

The sandcastle reviewer is the single composable gate for agent PRs (ADR-019 constraint). Without an explicit section, an agent reviewer has no machine-readable instruction to check Socket + CodeQL outputs and may approve a slice that introduced a flagged dependency or a static-analysis error. Landing this after Stories 02 + 06 ensures the reviewer references gates that actually exist in CI.

External dependency: library-evaluation epic story 06 (sandcastle reviewer prompt) must be complete — the reviewer prompt must exist and have the library-trace check section that this story composes with. That epic is marked done.

Done when

  • .sandcastle/reviewer.prompt.md has a "CI security checks" section added after the existing library-trace check section.
  • The section instructs the reviewer to: (a) run gh run view <run-id> --log (or equivalent) for the PR's check suite; (b) scan the output for Socket findings of severity critical — if found, reject with notes naming the finding and referencing the failure-mode hierarchy in docs/guides/ci-security.md; (c) scan the output for CodeQL findings of severity error — same rejection pattern.
  • The reviewer composes these checks with the existing library-trace presence check (both must pass for approval).
  • pnpm typecheck && pnpm lint && pnpm test && pnpm conformance && pnpm fallow:audit && pnpm coverage:diff all pass (prose-only change; no executable code).

In scope

  • .sandcastle/reviewer.prompt.md — new "CI security checks" section only; existing sections unchanged.

Out of scope

  • Automated tests for the reviewer prompt — it's a prose runbook for an agent; success is verified manually (PRD testing decisions).
  • Extending the reviewer for pnpm audit signatures step failures — those surface as standard CI job failures, already handled by the reviewer's existing "all CI checks must pass" instruction.

Tasks

  • Extend .sandcastle/reviewer.prompt.md with a "CI security checks" section after the library-trace check: instruct the reviewer to read gh run view output for Socket critical findings and CodeQL error findings, reject on either with notes naming the finding and citing docs/guides/ci-security.md failure-mode hierarchy; one commit, all gates pass.