Implementation seed for ADR-025 Epic A: declarative PII inventory + retention + sub-processor manifests with three generators, pre-commit + CI drift detection, background purge job, and ADR-022 trace frontmatter extension for sub-processor fields (discriminated union). Eleven user stories ordered for the decomposer; five open questions with recommendations. Status: approved — ready for pnpm work decompose.
386 lines
26 KiB
Markdown
386 lines
26 KiB
Markdown
---
|
|
id: compliance-manifests-pii-retention-subprocessors
|
|
title: Declarative compliance manifests (PII + retention + sub-processors) — Epic A of ADR-025
|
|
type: prd
|
|
status: approved
|
|
author: danijel
|
|
created: 2026-05-18T17:52:09Z
|
|
updated: 2026-05-18T17:55:38.523Z
|
|
---
|
|
|
|
## Problem
|
|
|
|
A consumer adopting this template today gets the audit channel (ADR-018), observability PII boundary (ADR-017 §7), EU library residency filter (ADR-022), and supply-chain hardening (ADR-023) — about half the playbook's surface. But three load-bearing compliance artifacts that a DPA auditor (or a partner doing GDPR due diligence) expects are missing:
|
|
|
|
- **Data map** — no structured inventory of "what personal data does this system hold, where, with what retention, exportable by whom"
|
|
- **Retention policy** — no declarative source of truth for how long data lives per collection, no scheduled purge mechanism
|
|
- **Sub-processor inventory** — no record of "which third-party services receive personal data, under what DPA, in what region"
|
|
|
|
Every downstream EU-bound consumer currently has to invent these three artifacts from scratch, and the cost of drift between code (what's actually PII) and documentation (what we _say_ is PII) is high enough that most teams ship with both stale.
|
|
|
|
ADR-025 settled the strategy: tag at the Payload collection/field level + generators emit `compliance/*.yml` + extend ADR-022 traces for sub-processors. This PRD is the implementation seed for Epic A.
|
|
|
|
## Goal
|
|
|
|
Ship the declarative compliance manifests + generators so downstream consumers get a complete, automatically-validated PII inventory, retention policy, and sub-processor record by editing source-of-truth Payload configs and ADR-022 library traces. Drift detection runs in pre-commit + CI; the consumer's `compliance/` directory becomes audit evidence.
|
|
|
|
## In scope
|
|
|
|
- Type primitives in `core-shared/payload/`: `PiiCategory`, `DataProcessingPurpose`, `RetentionAction`, `RetentionTrigger`, `FieldPii`, `CollectionRetention`, `AuthPiiDefaults`
|
|
- TypeScript ambient module declaration extending Payload's `custom: Record<string, unknown>` to type `pii` (per field) and `retention` (per collection)
|
|
- Three generators under `scripts/compliance/`:
|
|
- `emit-data-map.mjs` → `compliance/data-map.yml`
|
|
- `emit-retention-policy.mjs` → `compliance/retention-policy.yml`
|
|
- `emit-sub-processors.mjs` → `compliance/sub-processors.yml`
|
|
- Orchestrator `emit-all.mjs` + `pnpm compliance:*` package scripts
|
|
- `--check` mode on every generator for drift detection
|
|
- Pre-commit hook integration: conditional (runs only when staged files match Payload configs or library traces) auto-regenerate + auto-stage
|
|
- CI integration: `pnpm compliance:emit-all --check` step in `ci.yml`'s validate job, hard-fail on drift
|
|
- New conformance ESLint rule `pii-declaration-must-be-complete` (warn): flags `custom.pii: {...}` blocks missing required sub-fields
|
|
- ADR-022 amendment: trace frontmatter gains discriminated-union sub-processor fields (`is-sub-processor`, `processes-pii`, conditional `data-sent`/`region`/`dpa-signed`/`sccs-required`/`contact`)
|
|
- `/evaluate-library` skill update: prompts for the new fields during trace authoring
|
|
- Background purge job in `core-shared/payload/retention-purge/` using existing `IJobQueue` infrastructure; emits an audit entry per row purged
|
|
- Backfill of existing template collections per Q6 of the Epic A grill:
|
|
- `auth.users`: full PII tagging on `displayName` + `custom.retention` + `custom.authPii` overrides (if non-default)
|
|
- All 6 existing collections: `custom.retention` declared
|
|
- Other collections: PII tags only where unambiguous (e.g., `media.media.uploadedBy` if tracked)
|
|
- `docs/compliance/` reference files: `data-map.example.yml`, `retention-policy.example.yml`, `sub-processors.example.yml`, `README.md` explaining the `docs/compliance/` (templates) vs root `compliance/` (live artifacts) split
|
|
|
|
## Out of scope
|
|
|
|
- DSR scaffold (`@repo/core-dsr`, `IDataExport`/`IDataDelete`/`IDataRectify`/`IProcessingRestriction`) — Epic B
|
|
- Consent abstraction (`@repo/core-consent`, `IConsent`, `ConsentChecked` brand, `requiresConsent` manifest field) — Epic B
|
|
- Cookie consent UI component — Epic B
|
|
- Security headers middleware — Epic C
|
|
- Rate-limit primitive (`IRateLimit`, `RateLimited` brand) — Epic C
|
|
- SBOM generation in CI — Epic C
|
|
- Compliance fill-in docs (runbooks, policies, pre-launch checklist) — Epic D
|
|
- Per-feature PII migration beyond what Q6 specifies — consumers ship their own
|
|
- Pure-HTTP sub-processors with no library trace — allowed as hand-authored `compliance/sub-processors.yml` entries, generator handles "manual entry, no trace" with a CI-visible flag, but no scaffold for editing them
|
|
- Retention enforcement for non-Payload data stores (Redis, S3, log aggregators) — out of scope; the template doesn't ship those abstractions yet
|
|
- Cross-region transfer assessment (Schrems II / TIA artifacts) — partially addressed via `region` field in sub-processor records, but DPIA-style transfer-impact-assessment docs are Epic D's territory
|
|
|
|
## Constraints
|
|
|
|
- **ADR-025** — Epic A's strategy is settled there. Implementation may surface details ADR-025 didn't anticipate; flag those for amendment before proceeding.
|
|
- **ADR-022** — sub-processor trace fields extend ADR-022's frontmatter. The discriminated-union shape and the `/evaluate-library` skill prompts are amendments captured in this PRD.
|
|
- **ADR-018** — purge job emits `IAuditLog.record({ action: "DELETE", reason: "retention-policy" })` per row purged. Uses the existing `core-audit` audit channel; doesn't introduce new audit semantics.
|
|
- **ADR-023** — pre-commit + CI integration follows the existing pattern (`.husky/pre-commit` already runs `bump-updated-timestamps.mjs`; ci.yml already has a `validate` job).
|
|
- **Manifest-first ordering** — the PII type primitives + Payload ambient declaration are the "manifest" for this work; they land first.
|
|
- **Generator-first** — Payload collections do NOT get hand-rolled scaffolding; existing collection files are modified in place per Q6.
|
|
- **`core-shared` must-have boundary** — purge job lives in `core-shared/payload/` because every template consumer uses Payload. Doesn't create a new optional core (per Q5).
|
|
- **No `--no-verify`** — pre-commit hook auto-regenerates compliance YAMLs; developers cannot bypass with `--no-verify` per repo policy. CI re-checks anyway.
|
|
- **Conventional Commits** — every slice lands as one green commit per the established session convention.
|
|
|
|
## Success criteria
|
|
|
|
- `pnpm compliance:emit-all` produces three deterministic YAMLs at `compliance/data-map.yml`, `compliance/retention-policy.yml`, `compliance/sub-processors.yml`.
|
|
- `pnpm compliance:emit-all --check` exits 0 when committed YAMLs match source declarations; exits non-zero with a readable diff otherwise.
|
|
- Pre-commit hook auto-regenerates conditionally — staging only a non-Payload-config / non-trace file does NOT trigger the generators.
|
|
- CI workflow (`ci.yml`'s validate job) blocks merges with mismatched compliance YAMLs.
|
|
- `pii-declaration-must-be-complete` ESLint rule fires on a `custom.pii: { category: "contact-email" }` (missing required sub-fields) in a synthetic Payload collection fixture.
|
|
- `auth.users` has a complete `custom.pii` tag set: `displayName` tagged, `email` covered by `PAYLOAD_AUTH_PII_DEFAULTS`, `password`/`salt`/`hash` excluded by the same default.
|
|
- All 6 existing template collections declare `custom.retention`.
|
|
- A library trace authored via `/evaluate-library` with `is-sub-processor: true` triggers the conditional fields prompt; the trace fails validation if any are missing.
|
|
- The retention purge job, when run via `pnpm work dispatch` or via local `pnpm dev` boot, schedules per-collection deletes; deleted rows produce audit entries with `action: "DELETE"` and `reason: "retention-policy"`.
|
|
- `pnpm typecheck && pnpm lint && pnpm test && pnpm conformance && pnpm fallow:audit && pnpm compliance:emit-all --check` all pass green at every commit boundary.
|
|
|
|
## User stories
|
|
|
|
1. As a **template author**, I want declarative PII + retention + sub-processor inventories so downstream consumers get audit evidence by editing source-of-truth configs.
|
|
2. As a **downstream consumer**, I want to add `custom.pii: { category: "contact-email", ... }` to a Payload field and have it appear in `compliance/data-map.yml` after `pnpm compliance:emit-all`.
|
|
3. As a **downstream consumer**, I want collection-level retention with cron-schedulable purge so old data gets deleted automatically without my writing a custom cron job.
|
|
4. As a **downstream consumer running a serverless deployment**, I want `purgeSchedule` to be runnable by either a process-local scheduler or an external cron — the interface accepts both.
|
|
5. As a **downstream consumer**, I want a CI gate that fails my PR if I add a new Payload field with `pii: true` but forget to regenerate `compliance/data-map.yml` so my audit evidence stays in sync.
|
|
6. As a **downstream consumer**, I want `is-sub-processor: true` traces to drive `compliance/sub-processors.yml` automatically so I don't maintain a separate inventory.
|
|
7. As an **AI agent** scaffolding a new Payload collection, I want the TypeScript types for `custom.pii`/`custom.retention` to be enforced by the compiler so I can't ship an invalid declaration.
|
|
8. As an **AI agent** evaluating a new library via `/evaluate-library`, I want the skill to ask "is this a sub-processor?" upfront so I author a complete trace in one pass.
|
|
9. As a **compliance reviewer**, I want `compliance/sub-processors.yml` to match DPA Section D so a regulator review surfaces zero discrepancies.
|
|
10. As a **template author**, I want the existing `auth.users` collection backfilled with PII tags so the template has a working reference example (not just empty schema).
|
|
11. As a **template author**, I want `password`/`salt`/`hash`/`resetPasswordToken` excluded from the data map by default so downstream consumers can't accidentally ship them as "exportable user data."
|
|
|
|
## Implementation decisions
|
|
|
|
### Module surface
|
|
|
|
- **`@repo/core-shared` modifications** (must-have package):
|
|
- New module `core-shared/payload/pii-types.ts` exporting `PiiCategory`, `DataProcessingPurpose`, `RetentionAction`, `RetentionTrigger`, `FieldPii`, `AuthPiiDefaults`, `PAYLOAD_AUTH_PII_DEFAULTS`
|
|
- New module `core-shared/payload/retention-types.ts` exporting `CollectionRetention`, `ISO8601Duration` (helper)
|
|
- New module `core-shared/payload/retention-purge/` containing `retention-purge.job.ts` + unit test
|
|
- Ambient TypeScript declaration extending Payload's `Field` and `CollectionConfig` `custom?: {}` to type `pii?: FieldPii` and `retention?: CollectionRetention` respectively
|
|
- **No new optional core packages** — Epic A doesn't introduce `core-retention` or similar (per Q5)
|
|
- **`@repo/core-eslint`**: new rule `pii-declaration-must-be-complete` (warn). Extends `_manifest-ast.js` with a Payload-collection field-parser
|
|
- **`scripts/compliance/`**: 4 new mjs scripts (3 emitters + 1 orchestrator) following the established `scripts/<topic>/` pattern
|
|
- **Existing feature packages** (auth, blog, media, marketing-pages, navigation): Payload collection files modified in place to add `custom.retention` (all) and `custom.pii` (auth fully, others sparingly per Q6)
|
|
- **`.claude/skills/evaluate-library/SKILL.md`**: updated with the two new prompt questions and the discriminated-union trace template
|
|
- **`.husky/pre-commit`**: new conditional step for `pnpm compliance:emit-all`
|
|
- **`.github/workflows/ci.yml`**: new step `pnpm compliance:emit-all --check` in `validate` job
|
|
- **`package.json`** (root): new scripts `compliance:data-map`, `compliance:retention-policy`, `compliance:sub-processors`, `compliance:emit-all`
|
|
|
|
### Type primitive contracts (decision-encoding inlined)
|
|
|
|
```ts
|
|
// core-shared/payload/pii-types.ts
|
|
export type PiiCategory =
|
|
| "contact-email"
|
|
| "contact-phone"
|
|
| "contact-address"
|
|
| "identification-name"
|
|
| "identification-username"
|
|
| "identification-government-id"
|
|
| "auth-credential"
|
|
| "auth-token"
|
|
| "network-ip"
|
|
| "network-user-agent"
|
|
| "financial-info"
|
|
| "behavioral-engagement"
|
|
| "document-content"
|
|
| "derived-metric"
|
|
| (string & Record<never, never>); // declaration-merge escape hatch for consumer extension
|
|
|
|
export type DataProcessingPurpose =
|
|
| "account-authentication"
|
|
| "transactional-notifications"
|
|
| "marketing-communications"
|
|
| "analytics-aggregation"
|
|
| "legal-compliance"
|
|
| "service-delivery"
|
|
| (string & Record<never, never>);
|
|
|
|
export type RetentionTrigger =
|
|
| "from-creation"
|
|
| "from-last-access"
|
|
| "after-deletion";
|
|
export type RetentionAction = "hard-delete" | "pseudonymize";
|
|
|
|
export type FieldRetention = {
|
|
duration: string; // ISO 8601 duration, e.g. "P30D"
|
|
trigger: RetentionTrigger;
|
|
action: RetentionAction;
|
|
};
|
|
|
|
export type FieldPii = {
|
|
category: PiiCategory;
|
|
purpose: DataProcessingPurpose[];
|
|
retention?: FieldRetention; // optional; falls back to collection-level when omitted
|
|
exportable: boolean;
|
|
restrictable: boolean;
|
|
};
|
|
|
|
// PAYLOAD_AUTH_PII_DEFAULTS — applied automatically when `auth: true`
|
|
// `null` = excluded from data-map (security material, never PII-export)
|
|
// Consumer overrides via `custom.authPii: { email: { ...override }, totpSecret: null }`
|
|
export const PAYLOAD_AUTH_PII_DEFAULTS: Record<string, FieldPii | null> = {
|
|
email: {
|
|
category: "contact-email",
|
|
purpose: ["account-authentication", "transactional-notifications"],
|
|
exportable: true,
|
|
restrictable: true,
|
|
},
|
|
password: null,
|
|
salt: null,
|
|
hash: null,
|
|
resetPasswordToken: null,
|
|
resetPasswordExpiration: null,
|
|
loginAttempts: null,
|
|
lockUntil: null,
|
|
apiKey: null,
|
|
apiKeyIndex: null,
|
|
};
|
|
```
|
|
|
|
```ts
|
|
// core-shared/payload/retention-types.ts
|
|
export type PurgeSchedule = "daily" | "weekly" | "monthly" | string; // cron expression
|
|
|
|
export type CollectionRetention = {
|
|
activeRetention?: {
|
|
duration: string;
|
|
trigger: "from-creation" | "from-last-access";
|
|
};
|
|
postDeletion?: {
|
|
duration: string;
|
|
trigger: "after-deletion";
|
|
action: RetentionAction;
|
|
};
|
|
purgeSchedule: PurgeSchedule;
|
|
coldArchive?: { duration: string; trigger: "from-creation" };
|
|
};
|
|
```
|
|
|
|
Ambient module declaration:
|
|
|
|
```ts
|
|
// core-shared/payload/payload-custom-ambient.d.ts
|
|
declare module "payload" {
|
|
interface Field {
|
|
custom?: {
|
|
pii?: FieldPii;
|
|
[key: string]: unknown;
|
|
};
|
|
}
|
|
interface CollectionConfig {
|
|
custom?: {
|
|
retention?: CollectionRetention;
|
|
authPii?: Record<string, FieldPii | null>;
|
|
[key: string]: unknown;
|
|
};
|
|
}
|
|
}
|
|
```
|
|
|
|
### Generator contracts
|
|
|
|
Each generator runs in one of three modes:
|
|
|
|
- Default: emit YAML to `compliance/<artifact>.yml`, overwriting
|
|
- `--check`: regenerate in-memory, diff against existing file, exit 0 on match, non-zero with diff on mismatch
|
|
- `--print`: emit to stdout (for debugging)
|
|
|
|
YAML output is deterministic (sorted keys, normalized formatting, trailing newline) so byte-identical runs produce byte-identical output.
|
|
|
|
`emit-data-map.mjs`:
|
|
|
|
- Walks every Payload collection across packages (uses `@repo/cms` to load all configs)
|
|
- For each field in `fields[]`, reads `custom.pii` if present, emits an entry
|
|
- For collections with `auth: true`, applies `PAYLOAD_AUTH_PII_DEFAULTS` then overlays `custom.authPii` overrides; emits entries for non-null defaults
|
|
- Output structure: per-collection block listing fields + their PII metadata
|
|
- Excluded fields (e.g., `password: null`) are documented in a separate `excluded:` section per collection for audit transparency
|
|
|
|
`emit-retention-policy.mjs`:
|
|
|
|
- Walks every Payload collection
|
|
- Emits `custom.retention` block per collection
|
|
- Validates: every collection MUST have `purgeSchedule` declared (failure: print collection name + hint)
|
|
- Output structure: per-collection retention block with purge cadence + activeRetention + postDeletion + coldArchive
|
|
|
|
`emit-sub-processors.mjs`:
|
|
|
|
- Walks `docs/library-decisions/*.md`, parses frontmatter
|
|
- Filters to traces with `is-sub-processor: true`
|
|
- Emits each as a sub-processor entry with conditional fields (`data-sent`, `region`, `dpa-signed`, `sccs-required`, `contact`)
|
|
- Also reads `compliance/sub-processors.manual.yml` (if exists) for pure-HTTP entries with no backing trace; merges into output with `source: manual` flag
|
|
- Output structure: array of sub-processor records, sorted by name
|
|
|
|
`emit-all.mjs`:
|
|
|
|
- Orchestrates the three; supports `--check` mode for all at once
|
|
- Single failure exit code per failed generator
|
|
|
|
### ADR-022 amendment — trace frontmatter discriminated union
|
|
|
|
Every library trace at `docs/library-decisions/<date>-<pkg>.md` MUST declare two boolean fields after the existing ADR-022 fields:
|
|
|
|
- `is-sub-processor: boolean` (does this library send data to an external server it owns/operates?)
|
|
- `processes-pii: boolean` (does this library process personal data inside the calling process?)
|
|
|
|
When `is-sub-processor: true`, the following fields become REQUIRED:
|
|
|
|
- `data-sent: string[]` (references `PiiCategory` values)
|
|
- `region: "EU" | "EEA" | "US" | "UK" | "CH" | "OTHER"`
|
|
- `dpa-signed: ISO-date | null` (null = pending)
|
|
- `sccs-required: boolean`
|
|
- `contact: string` (email or URL)
|
|
|
|
When `is-sub-processor: false`, these fields MUST be absent. Validator enforces.
|
|
|
|
The `/evaluate-library` skill prompts the user for both binary fields during evaluation; when `is-sub-processor: true`, also prompts for the 5 conditional fields. Trace template in `.claude/skills/evaluate-library/SKILL.md` updated accordingly.
|
|
|
|
The weekly trace revalidation cron (ADR-023) checks `dpa-signed` for staleness: DPA dates older than 2 years trigger a re-confirmation issue.
|
|
|
|
### Pre-commit hook integration
|
|
|
|
`.husky/pre-commit` gains a step that runs `pnpm compliance:emit-all` if and only if any staged file matches:
|
|
|
|
- `packages/*/src/integrations/cms/**/*.ts` (Payload configs)
|
|
- `docs/library-decisions/*.md` (library traces)
|
|
- `compliance/*.yml` (the artifacts themselves — protects against manual edits)
|
|
|
|
Output is auto-staged via `git add compliance/`. The conditional check keeps unrelated commits fast (~10ms detection cost).
|
|
|
|
### CI integration
|
|
|
|
`.github/workflows/ci.yml`'s `validate` job gains a step:
|
|
|
|
```yaml
|
|
- name: Compliance manifest drift check
|
|
run: pnpm compliance:emit-all --check
|
|
```
|
|
|
|
Position: after `pnpm conformance`, before `pnpm coverage:diff`. Same severity (hard error). Failure message includes the fix command.
|
|
|
|
### Background retention purge job
|
|
|
|
Lives at `core-shared/payload/retention-purge/retention-purge.job.ts`. Receives `ctx.queue` (`IJobQueue` from `core-shared/jobs`) + `ctx.config` (Payload SanitizedConfig).
|
|
|
|
At app boot, the binder walks every collection, reads `custom.retention.purgeSchedule`, registers a scheduled job per collection with the corresponding cadence. The job body:
|
|
|
|
1. Queries the collection for rows whose `activeRetention.duration` has elapsed (from `createdAt` for `from-creation`, from `updatedAt` for `from-last-access`)
|
|
2. For each row:
|
|
- If `postDeletion.action === "pseudonymize"`: NULL the PII fields, set `processing_restricted: true` (per Epic B's IProcessingRestriction)
|
|
- If `postDeletion.action === "hard-delete"`: cascade delete via Payload's delete operation
|
|
3. Emits one audit entry per processed row: `IAuditLog.record({ action: "DELETE", subject: row.id, actor: "system", reason: "retention-policy" })`
|
|
|
|
When `auditLog` isn't wired (consumer hasn't scaffolded `core-audit`), the audit emission is skipped without throwing.
|
|
|
|
### Backfill scope (per Q6)
|
|
|
|
Stories cover backfill of existing template collections:
|
|
|
|
- `auth.users`: `displayName` tagged as `identification-username` (exportable, restrictable); `role` tagged as null (not PII per template default). Collection retention: `activeRetention: indefinite, postDeletion: 30d hard-delete, purgeSchedule: daily`. `PAYLOAD_AUTH_PII_DEFAULTS` covers `email`/`password`/`salt`/`hash` automatically; no `custom.authPii` override needed unless future custom auth fields are added.
|
|
- `blog.articles`: collection retention only (no PII fields by default — author refs not modeled as PII per template).
|
|
- `marketing-pages.site-settings`, `pages`: collection retention only (no PII).
|
|
- `media.media`: collection retention; if `uploadedBy` exists, tag it as `identification-username`.
|
|
- `navigation.header`: collection retention only.
|
|
|
|
Each backfill is one slice = one commit.
|
|
|
|
### Conformance impact
|
|
|
|
- ESLint rule count: 10 → 11 (adds `pii-declaration-must-be-complete`)
|
|
- Manifest fields: unchanged (no per-use-case manifest fields added by Epic A — declarations are on Payload configs, not feature manifests)
|
|
- New brand: none in Epic A (brands are Epic B/C territory)
|
|
- Boot assertion: extended to validate `custom.retention.purgeSchedule` is parseable when the binder boots in production
|
|
|
|
## Testing decisions
|
|
|
|
- **Type primitives**: vitest tests on `core-shared/payload/pii-types.ts` and `retention-types.ts` — verify TS shape via `@ts-expect-error` on malformed declarations; verify defaults exports.
|
|
- **Generators**: each script gets unit tests covering: happy path, `--check` matches, `--check` mismatch with readable diff, empty input (no collections), auth-managed defaults applied, `custom.authPii` overrides applied, sub-processor frontmatter discriminated union parsing.
|
|
- **ESLint rule**: RuleTester-based fixture suite mirroring `no-undeclared-audit.test.js`. Cover: complete `custom.pii` passes, missing `category` fires, missing `purpose` fires, missing `exportable` fires, non-PII collection no-op, malformed YAML in trace no-op.
|
|
- **Retention purge job**: unit test with in-memory Payload mock; verify schedule registration, row matching, audit emission, pseudonymize vs hard-delete branches, optional `auditLog` graceful skip.
|
|
- **Integration**: e2e test using the existing dev-seed setup — declare a `custom.retention` on a test collection, run `pnpm compliance:emit-retention-policy --check`, assert output. Same for `--check` mismatch with a forced manual edit.
|
|
- **No repository contract suite** — Epic A doesn't introduce a new `IXRepository`.
|
|
- **Coverage**: Epic A's modules join the L0 vitest thresholds (per `coverage.bands` in the affected packages' manifests); L1 `pnpm coverage:diff` gates the slices in dispatch.
|
|
- **Prior art to mirror**:
|
|
- Generator + `--check` pattern: `scripts/coverage/diff.mjs` (similar diff-against-committed-output pattern)
|
|
- ESLint rule shape: `packages/core-eslint/rules/no-undeclared-audit.{js,test.js}`
|
|
- Background job in `core-shared`: existing `packages/core-shared/src/jobs/payload-job-queue.{ts,test.ts}`
|
|
- Ambient TypeScript module augmentation: existing patterns in `node_modules/@types/*` (search for `declare module`)
|
|
|
|
## Open questions
|
|
|
|
- **Q1: Should `retention-must-be-declared` ESLint rule join `pii-declaration-must-be-complete`?** — Recommended: **No, defer.** Q6 already establishes that every existing template collection gets `custom.retention`; making it ESLint-enforced for _all_ Payload collections in _all_ downstream consumers is stricter than the strategy ADR. Add the rule in a follow-up PRD if a consumer feels the pain.
|
|
- **Q2: How does `from-last-access` retention trigger interact with Payload — does Payload track `updatedAt` natively?** — Payload sets `createdAt` and `updatedAt` by default. `from-last-access` reads `updatedAt`. If a consumer needs true "last read" tracking (rare), they add a custom hook updating `lastAccessedAt`; the purge job uses that field via `custom.retention.lastAccessFieldOverride` (deferred — not in Epic A).
|
|
- **Q3: Should the generator emit `compliance/manifest.lock.yml` containing a hash of source declarations, for fast `--check` mode?** — Recommended: **No, defer.** Full regenerate-and-diff is fast enough (~50ms for the template's 6 collections). Reconsider if generators grow slow on a real consumer with hundreds of collections.
|
|
- **Q4: How does the purge job handle race conditions between concurrent purge runs (e.g., misconfigured cron firing daily and weekly simultaneously)?** — Recommended: per-job advisory lock via Payload's job system (existing `IJobQueue` should support this); test for it in the job's unit suite. If not supported, document as a known limitation in the consumer's runbook.
|
|
- **Q5: Does the pre-commit hook trigger on `compliance/*.yml` edits themselves (the auto-stage behavior could feedback-loop)?** — Recommended: **Yes, conditionally.** The hook runs `emit-all` when `compliance/*.yml` is staged because that's the manual-edit case; the generator regenerates the YAML, auto-stages the regenerated version, replacing the developer's manual edit. The developer sees this in their `git status` post-commit and can re-edit if they had intent. Avoids drift via manual edit silently surviving.
|
|
|
|
## Out of scope (deferred)
|
|
|
|
- DSR scaffold and consent abstraction (Epic B)
|
|
- Security headers + rate-limit + SBOM (Epic C)
|
|
- Compliance fill-in docs (Epic D)
|
|
- `retention-must-be-declared` ESLint rule (see Q1 above)
|
|
- `lastAccessedAt` field hook + true "from-last-access" retention (see Q2)
|
|
- `compliance/manifest.lock.yml` for faster `--check` (see Q3)
|
|
- Backfill of any Payload `auth: true` collection beyond `users` (none exist yet)
|
|
- Cross-region transfer documentation (DPIA / TIA) — Epic D's territory
|
|
- Migration tooling for downstream consumers upgrading from a pre-ADR-025 version of the template (the template hasn't been versioned with consumers yet; no migration needed)
|
|
|
|
## Further notes
|
|
|
|
- **Builds on:** ADR-018 (audit channel — purge emits audit entries), ADR-022 (library evaluation policy — extended for sub-processor fields), ADR-023 (CI security + supply chain — pre-commit + CI integration follows the established pattern), ADR-025 (strategy umbrella).
|
|
- **Pairs with:** Epic B PRD `dsr-consent-and-cookie-banner.prd.md` (consumes A's PII tags for DSR cascade); Epic D PRD `compliance-docs-scaffolds.prd.md` (references A's generator output formats in `.example.yml` files).
|
|
- **Sequencing:** Epic A is the dependency-graph root for Epic B. Epic B's PRD will be authored once Epic A's stories are at least partially dispatched (decomposer needs A's type primitives to settle B's interfaces).
|
|
- **Stakeholders:** template authors (most affected — three new generators, ESLint rule, ADR-022 amendment), downstream consumers (positively affected — gain three compliance artifacts), AI agents operating in feature code (positively affected — typed Payload custom config catches invalid declarations at compile time), compliance reviewers (positively affected — `compliance/` directory becomes audit evidence).
|