agentic-dev/docs/guides/audit-and-compliance.md

# Audit logging & DPA compliance

> **Prerequisite:** This guide assumes `@repo/core-audit` is scaffolded. If your
> project started from the slim template, run `pnpm turbo gen core-package audit`
> first, then follow the manual wiring steps in §4 below.

## What DPA requires

A Data Processing Agreement (DPA) typically mandates that any system handling
personal data must keep a tamper-evident record of every access to that data.
The six action types covered by this template are: **VIEW**, **CREATE**,
**UPDATE**, **DELETE**, **EXPORT**, and **PERMISSION_CHANGE**. Each entry must
capture four required fields:

| DPA field | Mapped to |
|---|---|
| **Who** performed the action | `actorId`, `actorType`, `actorRoles` |
| **What** was acted on | `action`, `resource.type`, `resource.id` |
| **When** it happened | `at` (server timestamp, ISO 8601) |
| **From where** the request came | `from.ipTruncated`, `from.userAgent` |

**What NOT to log** — the DPA "exclusion list" is enforced by the `AuditEntry`
type itself: there are no `payload`, `body`, `oldValue`, or `newValue` fields.
UPDATE actions capture only `changedFields` (the names of modified fields, not
their values). This is intentional — storing "what changed" rather than "what
it changed to" prevents the audit log from becoming a secondary store of
regulated data.

**Retention requirements** vary by jurisdiction, but a common baseline is 90 days
in hot storage (queryable via Payload admin or API) and 12 months in cold archive
(shipped to a log aggregator like Grafana Loki or Elasticsearch). The stdout JSON
sink + log shipper pattern satisfies this: Payload holds the hot copy; the
aggregator holds the archive.

**Immutability** is enforced by the Payload collection's `update: () => false`
access rule. No user — including admins — can modify a written entry through the
Payload API. The only write path is `IAuditLog.record()`. Erasure on GDPR request
uses a privileged `overrideAccess: true` path that pseudonymizes or deletes the
`actorId` field rather than altering the event itself.

## The two-pattern model

Two complementary ways to capture a read event:

### Pattern 1 — Use-case-level `record()` calls

In your feature's READ use cases, the developer explicitly calls
`ctx.auditLog?.record({ action: "VIEW", ... })`. This gives you full control
over the WHY (the `reason` field) and works in every context — HTTP requests,
background jobs, CLI scripts, and tests.

```ts
// packages/blog/src/application/use-cases/get-article.use-case.ts
export function getArticleUseCase(
  deps: { articlesRepo: IArticlesRepository; auditLog?: AuditLogProtocol },
) {
  return async (input: GetArticleInput): Promise<GetArticleOutput> => {
    const article = await deps.articlesRepo.findById(input.id);
    await deps.auditLog?.record({
      actorId: input.userId,
      actorType: "user",
      actorRoles: input.userRoles,
      action: "VIEW",
      resource: { type: "articles", id: input.id },
      at: new Date(),
      scope: { feature: "blog", environment: process.env.NODE_ENV ?? "development", tenant: input.tenant ?? "default" },
      from: { ipTruncated: input.ipTruncated, userAgent: input.userAgent },
      containsPii: false,
      outcome: "success",
      reason: "article-page-render",
    });
    return getArticleOutputSchema.parse(article);
  };
}
```

### Pattern 2 — Payload `afterRead` hook (automatic, opt-in)

Install `createAuditAfterReadHook(...)` on a collection's `afterRead` hook list.
This captures **every** read of the collection — including admin UI reads, direct
programmatic reads, and REST API reads — automatically, without per-use-case
instrumentation.

```ts
// packages/blog/src/integrations/cms/articles.collection.ts
import { createAuditAfterReadHook } from "@repo/core-audit/hooks";

export const articlesCollection: CollectionConfig = {
  slug: "articles",
  hooks: {
    afterRead: [
      createAuditAfterReadHook({
        auditLog: ctx.auditLog,
        feature: "blog",
        tenant: "default",
      }),
    ],
  },
  // ...
};
```

The hook fires asynchronously (fire-and-forget) so it never blocks the read
response. It uses the sentinel IP/UA `"system"` / `"payload-admin"` for admin UI
reads where no request context is available.

## When to use which pattern

| Read source | Recommended pattern |
|---|---|
| tRPC procedure (app-facing read) | Use-case-level `record()` call — you have full request context |
| Payload admin UI | Hook automatically captures (no request context needed) |
| Background job / cron | Use-case-level `record()` call with `actorId: "system"`, sentinel IP/UA |
| Direct programmatic / CMS REST | Hook automatically captures |
| CLI / seed script | Use-case-level `record()` call with `actorId: "service-{name}"` |

Use **both** for collections under DPA scope. The hook covers reads you might
forget at the use-case layer; the use-case calls add contextual `reason` and
accurate IP/UA.

## Wiring core-audit into your app (7 steps)

After running `pnpm turbo gen core-package audit`, the package exists in
`packages/core-audit/` but is not yet wired into your app. Complete these seven
steps:

### Step 1 — Set `AUDIT_PSEUDONYM_SALT`

Generate a cryptographically random salt and store it in your deployment secrets
manager. The salt is used for sha256 pseudonymization on GDPR erasure requests.

```bash
export AUDIT_PSEUDONYM_SALT="$(openssl rand -hex 32)"
```

Add to your `.env` (development) and to your production secrets vault. The
`bindAudit()` function throws at startup if `NODE_ENV=production` and this
variable is not set — intentional fail-fast behavior.

### Step 2 — Mount the Payload collection

In `packages/core-cms/src/payload.config.ts`, import and register the
append-only `auditLogsCollection`:

```ts
import { auditLogsCollection } from "@repo/core-audit/collection";

export default buildConfig({
  collections: [
    // ... existing collections ...
    auditLogsCollection,
  ],
});
```

The collection enforces `update: () => false` and `delete: () => false` at the
access-control layer. The only write path is via the `IAuditLog.record()` API.

### Step 3 — Mount the admin tRPC router

In `packages/core-api/src/root.ts`, import `createAuditRouter` and wire it to
the app router. The router requires an `IAuditLog` instance — pass the one
returned by `bindAudit()`:

```ts
import { createAuditRouter } from "@repo/core-audit/api";

// In your router factory (called after bindAudit):
export function createAppRouter(auditLog: IAuditLog) {
  return t.router({
    // ... existing routers ...
    audit: createAuditRouter(auditLog),
  });
}
```

This exposes `audit.eraseSubject` as an admin-only tRPC mutation. Protect it
with your admin auth middleware (the `auditProcedure` base already requires a
caller-supplied auth guard — see `src/integrations/api/procedures.ts`).

### Step 4 — Bind audit in `bind-production.ts`

In `apps/web-next/src/server/bind-production.ts`, call `bindAudit()` before
any feature binder that uses `ctx.auditLog`:

```ts
import { bindAudit } from "@repo/core-audit/di";

// Inside your resolveProductionContext():
const { auditLog } = bindAudit(sharedContainer, {
  payloadConfig: resolvedConfig,
  sinks: ["payload", "stdout"],
});

const ctx: BindProductionContext = {
  tracer,
  logger,
  config: resolvedConfig,
  bus,
  queue,
  realtime,
  realtimeRegistry,
  auditLog,   // <- new
};
```

The returned `auditLog` is already wrapped in `TraceIdEnrichingAuditLog`, so
every entry receives `correlationId` from the active OTel span automatically.

### Step 5 — Install user-collection hooks (DPA recommended)

To automatically pseudonymize or delete a user's audit history when their account
is deleted, install the erasure hook on your users Payload collection. In
`packages/auth/src/di/bind-production.ts`:

```ts
if (ctx.auditLog) {
  const { createAuditErasureHook, createAuditAfterReadHook } =
    await import("@repo/core-audit/hooks");

  // Automatically erase audit entries when a user is deleted:
  usersCollection.hooks ??= {};
  usersCollection.hooks.afterDelete ??= [];
  usersCollection.hooks.afterDelete.push(
    createAuditErasureHook({ auditLog: ctx.auditLog, mode: "pseudonymize" }),
  );

  // Optionally capture VIEW events for user profile reads:
  usersCollection.hooks.afterRead ??= [];
  usersCollection.hooks.afterRead.push(
    createAuditAfterReadHook({ auditLog: ctx.auditLog, feature: "auth", tenant: "default" }),
  );
}
```

This step is gated on `ctx.auditLog` being present, so it's safely skipped in
slim-template projects where `@repo/core-audit` has not been scaffolded.

### Step 6 — Set up a log shipper

The `StdoutJsonAuditLog` sink writes one JSON line per audit entry to process
stdout. A log shipper (Vector or Fluent Bit) reads this stdout stream and
forwards entries to your centralized aggregator (Grafana Loki, Elasticsearch,
Splunk, etc.). See §5 for sample configs.

### Step 7 — Verify

```bash
pnpm install
pnpm lint && pnpm typecheck && pnpm test
pnpm turbo boundaries
```

Run your app in development mode and trigger a VIEW event. You should see an
`_type: "audit"` JSON line in stdout within 100 ms of the action.

## Sample log-shipper configs

### Vector (recommended)

Vector reads the container's stdout stream, filters for audit entries
(distinguished by `_type: "audit"`), and ships them to Grafana Loki in the EU
region.

```toml
# vector.toml
[sources.app_stdout]
type = "stdin"

[transforms.parse_audit]
type = "remap"
inputs = ["app_stdout"]
source = '''
  . = parse_json!(.message)
  if ._type != "audit" { abort }
'''

[transforms.enrich_labels]
type = "remap"
inputs = ["parse_audit"]
source = '''
  .labels = { "env": .scope.environment, "feature": .scope.feature, "tenant": .scope.tenant }
'''

[sinks.loki_eu]
type = "loki"
inputs = ["enrich_labels"]
endpoint = "https://logs-prod-eu-west-0.grafana.net"
auth.strategy = "basic"
auth.user = "${LOKI_USER}"
auth.password = "${LOKI_API_KEY}"
labels.job = "audit"
labels.env = "{{ .labels.env }}"
encoding.codec = "json"
```

Set `LOKI_USER` and `LOKI_API_KEY` in your deployment environment. The filter
`if ._type != "audit" { abort }` ensures only audit entries are forwarded;
other stdout lines (application logs, framework output) pass through unshipped.

### Fluent Bit

```ini
# fluent-bit.conf
[INPUT]
    Name              tail
    Path              /var/log/app/stdout.log
    Parser            json
    Tag               app.stdout

[FILTER]
    Name              grep
    Match             app.stdout
    Regex             _type audit

[OUTPUT]
    Name              loki
    Match             app.stdout
    Host              logs-prod-eu-west-0.grafana.net
    Port              443
    TLS               On
    Labels            job=audit,env=${AUDIT_ENV}
    HTTP_User         ${LOKI_USER}
    HTTP_Passwd       ${LOKI_API_KEY}
    line_format       json
```

For containerized deployments (Docker / Kubernetes), configure the log driver to
write stdout to a file or use Fluent Bit's `docker` input plugin instead of `tail`.

## GDPR erasure

GDPR Article 17 ("right to erasure") requires that a data subject can request
deletion of their personal data. `@repo/core-audit` satisfies this via
`IAuditLog.eraseSubject(actorId, mode)`.

### Trigger via admin tRPC

```bash
# Replace <TOKEN> with a valid admin session token and <ACTOR_ID> with the user ID
curl -X POST https://your-app.com/api/trpc/audit.eraseSubject \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <TOKEN>" \
  -d '{"json":{"actorId":"user_123","mode":"pseudonymize"}}'
```

The `pseudonymize` mode replaces `actorId` in every matching Payload audit entry
with `erased-{sha256(salt+actorId)[0:16]}`. The entry itself is preserved (the
event happened; only the identity is pseudonymized). The `delete` mode hard-deletes
every entry for that actor — use only when the DPA or a court order requires it.

### Trigger via the `afterDelete` hook

If you installed the erasure hook in Step 5, deleting a user via Payload admin
automatically triggers pseudonymization. No manual API call is needed for the
standard deletion flow.

### What `eraseSubject` does NOT cover

`StdoutJsonAuditLog.eraseSubject()` writes a tombstone entry to stdout but cannot
retroactively alter past stdout lines that have already been shipped to your
aggregator. Handle this by issuing a deletion request to Loki/Elasticsearch for
that `actorId` label after the Payload pseudonymization completes:

```bash
# Grafana Loki: delete by label selector (requires delete permission)
curl -X POST "https://logs-prod-eu-west-0.grafana.net/loki/api/v1/delete?query={job=\"audit\"}&start=0&end=$(date +%s)000000000" \
  -H "X-Scope-OrgID: ${LOKI_TENANT}" \
  --data-urlencode 'query={actorId="user_123"}'
```

Consult your aggregator's deletion API for the exact syntax.

## Sample-week audit verification

"Can you tell who accessed any given article last Tuesday?"

1. Open Payload admin → Collections → Audit Logs
2. Filter: `resource.type = "articles"` and `at` between Monday 00:00 and Sunday 23:59
3. Each entry shows `actorId`, `actorType`, `actorRoles`, `action`, `at`, and `from.ipTruncated`
4. Cross-reference `actorId` with the Users collection to get the display name (keep this lookup out of the audit log itself — names are PII)

For bulk queries, use your aggregator's log search. In Grafana Loki:

```logql
{job="audit"} | json | resource_type="articles" | action="VIEW"
  | line_format "{{.at}} {{.actorId}} viewed {{.resource_id}}"
```

The `correlationId` field (populated by `TraceIdEnrichingAuditLog`) links each
audit entry to its OTel trace, enabling pivot from the compliance timeline to the
full distributed trace in Grafana Tempo or Jaeger.

## Hostile-actor immutability test

The append-only guarantee is only as good as its enforcement. Verify it holds:

```bash
# 1. Record a test entry
curl -X POST http://localhost:3001/api/audit-logs \
  -H "Content-Type: application/json" \
  -d '{"actorId":"attacker","action":"VIEW","resource":{"type":"test"}}'
# Expected: 403 Forbidden — the collection's create access is API-only via IAuditLog

# 2. Try to update an existing entry via Payload REST
ENTRY_ID=$(curl -s "http://localhost:3001/api/audit-logs?limit=1" | jq -r '.docs[0].id')
curl -X PATCH "http://localhost:3001/api/audit-logs/${ENTRY_ID}" \
  -H "Content-Type: application/json" \
  -d '{"actorId":"tampered"}'
# Expected: 403 Forbidden — update: () => false

# 3. Try to delete via Payload REST
curl -X DELETE "http://localhost:3001/api/audit-logs/${ENTRY_ID}"
# Expected: 403 Forbidden — delete: () => false
```

Also verify that the stdout shipper is configured with an independent retention
policy that does NOT depend on Payload. If a hostile actor gains DB access and
truncates the `audit_logs` table, the shipped log lines in Loki/Elasticsearch
remain as the authoritative record.

## Common mistakes

**Forgetting `scope.tenant`** — the field is required, not optional. Single-tenant
projects must explicitly pass `tenant: "default"`. TypeScript will catch this at
compile time if you omit it.

**Setting `containsPii: false` on a collection that has PII** — for example, a
"users" profile view logs `resource.type: "users"` but `containsPii: false`.
Even though the audit entry itself doesn't store PII values, the *resource* being
accessed is PII-bearing. Set `containsPii: true` and list relevant categories in
`piiCategories: ["profile", "email"]` so downstream retention systems apply the
correct access policy to the audit entries themselves.

**Trying to add `oldValue`/`newValue` fields** — these fields do not exist on
`AuditEntry` by design. The type system prevents this. If you need to capture the
before/after state of a field for a specific compliance requirement, build a
separate audit-detail mechanism outside this package — do not extend `AuditEntry`.

**Forgetting `AUDIT_PSEUDONYM_SALT` in production** — `bindAudit()` throws at
startup with a clear message. Do not try to catch this error; the intent is to
refuse to start rather than silently use a weak or predictable salt.

**Using `eraseSubject("delete")` as the default** — prefer `"pseudonymize"` for
the standard GDPR erasure path. Hard delete removes all evidence that the events
occurred, which can itself create compliance problems. Pseudonymization preserves
the event record while making the actor unidentifiable.

**Not verifying the log shipper in staging** — deploy to staging with the same
Vector/Fluent Bit configuration you'll use in production. Verify that audit
entries appear in your aggregator before go-live. The `_type: "audit"` filter is
your first line of defense against shipping non-audit data to the compliance log.