fraqtal/agentic-dev
Template
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
Files
c17d3f147d6ffda26adbd93f919f9f3f13181e66
agentic-dev/docs/work/2026-05-14-ci-security-and-supply-chain/05-trace-revalidation-workflow/_story.md
Danijel Martinek c17d3f147d chore(work): decompose 2026-05-14-ci-security-and-supply-chain epic 
Decomposer produced 9 stories under docs/work/2026-05-14-ci-
security-and-supply-chain/, ordered to land the schema foundation
first and the cross-referencing content (reviewer prompt, guide)
last:

01 - trace schema extensions (socketRisk + lastRevalidated)
02 - Socket integration (skill + CI)
03 - Renovate adoption
04 - major-bump re-evaluation flow
05 - trace revalidation workflow
06 - CodeQL + audit signatures
07 - gitleaks pre-commit
08 - reviewer prompt update
09 - CI security guide + docs

Also fixes a one-char status typo in the PRD frontmatter
(\`appoved\` -> \`approved\`) that landed with the decompose run.

Anchored by ADR-023 + the approved PRD at
docs/work/prds/2026-05-14-ci-security-and-supply-chain.prd.md.
Sequencing: depends on stories 01/02/04/06 of the in-flight
library-evaluation epic landing first.
2026-05-14 18:59:12 +02:00

4.2 KiB
Raw Blame History

id, epic, title, type, status, feature, depends-on, blocks
id epic title type status feature depends-on blocks
05-trace-revalidation-workflow 2026-05-14-ci-security-and-supply-chain Trace revalidation workflow technical-story todo scripts
01-trace-schema-extensions
04-major-bump-reevaluation
09-ci-security-guide-and-docs

Goal

Write scripts/library-decisions/revalidate.mjs — a script that walks every approved + pre-shipped trace, re-runs each trace's verification-commands, classifies divergence as soft or hard, and opens/updates/closes GitHub issues accordingly — then wire it into .github/workflows/trace-revalidation-weekly.yml (weekly cron + workflow_dispatch).

Why

ADR-022 traces go stale silently when new CVEs drop or Socket picks up behavioral changes in a package post-adoption. A weekly automated revalidation creates a feedback loop: soft divergence (minor drift) surfaces as a rolling dashboard issue; hard divergence (a re-evaluation is warranted) surfaces as a per-dep library-policy/re-evaluation issue with the trace path + finding + re-walk handoff. No auto-edit of traces and no CI gating on main — the workflow runs in parallel, not on the critical path.

External dependency: library-evaluation epic story 02 (pre-commit check script) must be complete — check.mjs and the docs/library-decisions/ fixture patterns are the prior art this script mirrors.

Done when

  • scripts/library-decisions/revalidate.mjs walks all traces in docs/library-decisions/ whose decision field is accepted or pre-shipped; for each trace, re-runs its verification-commands; classifies divergence (soft: minor discrepancy from expected output; hard: finding that would change the evaluation decision); opens a rolling library-policy/dashboard-labeled issue for soft divergence (creates or updates a single issue); opens a library-policy/re-evaluation-labeled per-dep issue for hard divergence with title re-evaluate: <package>@<version> — <finding>, trace path, and evaluate-library re-walk pointer; closes open library-policy/re-evaluation issues whose dep has since had lastRevalidated refreshed; skips rejected traces entirely.
  • Integration tests use a fixture trace directory (no real gh CLI / no network): no-drift trace → no issue; soft-drift trace → dashboard issue created; hard-drift trace → per-dep issue with correct labels + title format; open per-dep issue already exists → no duplicate opened; lastRevalidated refreshed → open issue closed with comment.
  • .github/workflows/trace-revalidation-weekly.yml triggers on schedule: - cron: "30 6 * * 1" and workflow_dispatch; job steps: checkout, pnpm install --frozen-lockfile, node scripts/library-decisions/revalidate.mjs; permissions: issues: write, contents: read (NO contents: write).
  • pnpm typecheck && pnpm lint && pnpm test && pnpm conformance && pnpm fallow:audit && pnpm coverage:diff all pass.

In scope

  • scripts/library-decisions/revalidate.mjs — the revalidation script.
  • scripts/library-decisions/revalidate.test.mjs — integration tests with fixture directory and mocked gh CLI surface.
  • .github/workflows/trace-revalidation-weekly.yml — the workflow file.

Out of scope

  • Auto-editing trace files — explicitly forbidden (no contents: write).
  • Auto-dispatching on library-policy/re-evaluation issues — human triage required (PRD out of scope).
  • CI gating on main from this workflow — main keeps deploying; revalidation runs in parallel.

Tasks

  • Write scripts/library-decisions/revalidate.mjs (walk approved+pre-shipped traces, re-run verification-commands, classify soft/hard divergence, open/update/close issues via gh CLI; mock-friendly gh surface for tests); write revalidate.test.mjs integration tests with fixture traces covering: no-drift, soft-drift (dashboard issue), hard-drift (per-dep issue with correct labels+title), duplicate-issue guard, stale-issue close on refreshed lastRevalidated, rejected-trace skip; one commit, all gates pass.
  • Create .github/workflows/trace-revalidation-weekly.yml (trigger: schedule: cron: "30 6 * * 1" + workflow_dispatch; steps: checkout, pnpm install --frozen-lockfile, node scripts/library-decisions/revalidate.mjs; permissions: issues: write, contents: read); one commit, all gates pass.