feat(coverage): pnpm mutate (Stryker) + L3 implementation
Lands L3 of the agent-first coverage architecture (ADR-020) — the
mutation-testing layer. Stryker on entities + use-cases (the pure
business-logic surface) catches the third dimension of test quality:
tests that exist + execute the code but assert nothing.
Deps (root devDependencies):
- @stryker-mutator/core ^8.7.0
- @stryker-mutator/vitest-runner ^8.7.0
Shared base: packages/core-testing/stryker.base.json
- testRunner: vitest (uses each feature's vitest.config.ts)
- mutate: src/entities/** + src/application/use-cases/** (excludes
tests, factories, contracts)
- thresholds: high 90 / low 80 / break 80
- reporters: progress + html + json (reports/mutation/{index.html,
mutation.json})
- incremental mode enabled, concurrency 4, timeout 10s
- exposed via @repo/core-testing/stryker.base.json subpath export
Per-feature config: packages/auth/stryker.config.json
- 4-line file that extends the shared base
- Proof-of-concept; other features get a config when L0 unification
closes their existing test gaps
Driver: scripts/coverage/mutate.mjs (zero-dep Node ESM)
- discoverStrykerConfigs: walks packages/* and apps/* for
stryker.config.json
- Supports --filter <name>, --since <ref> (incremental), --json
- Runs Stryker per-feature via node_modules/.bin/stryker run
- Surfaces per-package pass/fail summary; exits 1 on any failure
- Tests: scripts/coverage/mutate.test.mjs (3 tests, all green)
CI: .github/workflows/mutation-nightly.yml
- Cron at 02:30 UTC + workflow_dispatch with filter input
- Uploads reports/mutation/** as artifact (30-day retention)
- On failure, opens a tracking issue labelled mutation-testing
- permissions: contents: read, issues: write
- 60-min timeout (Stryker is slow by design)
Generator: turbo gen feature now scaffolds stryker.config.json from
turbo/generators/templates/feature/stryker.config.json.hbs — new
features ship mutation-ready out of the box.
Guide: docs/guides/coverage.md L3 section fleshed out with run
syntax, config shape, base config inventory, CI behavior, and a
"what you're looking for" primer on mutation scores.
Lockfile churn: pnpm regenerated the lockfile for the new deps;
~5K-line net reduction is collateral (pnpm version drift) but
mechanical.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -197,15 +197,56 @@ git show <sha> -- coverage/summary.json | grep -E '"statements"|"branches"'
|
||||
|
||||
## Mutation testing (L3)
|
||||
|
||||
> Not yet wired. The `coverage.mutationTargets` manifest field is declarative today; the `pnpm mutate` runner lands in a follow-up story.
|
||||
Stryker mutation testing on `entities/` + `application/use-cases/` — the pure-business-logic surface. Not part of `pnpm test` (slow); runs on-demand and nightly via GH Action.
|
||||
|
||||
When it lands, scope is per-feature:
|
||||
### Running
|
||||
|
||||
```bash
|
||||
pnpm mutate --filter @repo/blog
|
||||
pnpm mutate # every feature with a stryker.config.json
|
||||
pnpm mutate -- --filter @repo/auth # one feature
|
||||
pnpm mutate -- --since main # incremental against base ref
|
||||
pnpm mutate -- --json # machine-readable summary
|
||||
```
|
||||
|
||||
Mutations run on `entities/` + `application/use-cases/` (the pure-business-logic surface). Default mutation-score threshold is 80% (override per-manifest via `coverage.mutationScore`). Not part of `pnpm test`; runs on-demand and nightly via GH Action.
|
||||
### Configuration
|
||||
|
||||
Each feature has a slim `stryker.config.json` that extends the shared base:
|
||||
|
||||
```json
|
||||
{
|
||||
"$schema": "../../node_modules/@stryker-mutator/core/schema/stryker-schema.json",
|
||||
"extends": "@repo/core-testing/stryker.base.json"
|
||||
}
|
||||
```
|
||||
|
||||
The base lives at `packages/core-testing/stryker.base.json` and defines:
|
||||
|
||||
- **Test runner**: vitest (uses each feature's `vitest.config.ts`)
|
||||
- **Scope**: `src/entities/**/*.ts` and `src/application/use-cases/**/*.ts` (excludes tests/factories/contracts)
|
||||
- **Thresholds**: high 90 / low 80 / break 80 (`break` is the fail threshold)
|
||||
- **Reporters**: progress, html (`reports/mutation/index.html`), json (`reports/mutation/mutation.json`)
|
||||
- **Incremental mode**: enabled (subsequent runs skip mutants whose source + tests haven't changed)
|
||||
- **Concurrency**: 4 workers
|
||||
|
||||
To override per feature (rare), add fields to the feature's `stryker.config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"extends": "@repo/core-testing/stryker.base.json",
|
||||
"thresholds": { "high": 95, "low": 85, "break": 85 },
|
||||
"mutate": ["src/entities/**/*.ts"]
|
||||
}
|
||||
```
|
||||
|
||||
### CI: nightly run + on-demand
|
||||
|
||||
`.github/workflows/mutation-nightly.yml` runs Stryker across every feature at 02:30 UTC + on `workflow_dispatch`. The dispatch UI accepts a `filter` input (e.g. `@repo/auth`) for targeted reruns. Reports uploaded as the `mutation-reports` artifact (30-day retention). On meaningful score drops it opens a tracking issue labelled `mutation-testing`.
|
||||
|
||||
### What you're looking for
|
||||
|
||||
Stryker's `mutation.json` reports the **mutation score** (killed mutants / total) per file. A surviving mutant means: the mutator changed source code (e.g., `<` → `<=`, `&&` → `||`, removed a line, etc.), reran the tests, and they STILL passed. That's a test that exists + executes the code but doesn't actually assert behavior.
|
||||
|
||||
Fix: read the surviving mutant's diff in `reports/mutation/index.html`, identify the assertion that should have caught it, add the assertion.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
|
||||
Reference in New Issue
Block a user