diff --git a/docs/guides/runbook.md b/docs/guides/runbook.md index c4fa253..946ddde 100644 --- a/docs/guides/runbook.md +++ b/docs/guides/runbook.md @@ -220,6 +220,183 @@ For the full design see `docs/architecture/agent-first-workflow-and-conformance. --- +## Using Sandcastle for agent dispatch + +[Sandcastle](https://github.com/mattpocock/sandcastle) is the substrate that takes a markdown task description, hands it to a Claude / Codex agent running inside an isolated Docker sandbox, captures the agent's commits, and returns them so the orchestrator can route the diff to a reviewer agent. The repo's `pnpm work dispatch` wraps sandcastle for the manifest-first workflow. + +### When to use Sandcastle + +- **Routine, well-specified tasks** — adding a behaviour slice to an existing use case, migrating a feature to a new convention, scaffolding new packages. The task description is the contract; sandcastle automates the rest. +- **Parallel work** — dispatch multiple independent tasks at once; each runs in its own sandbox branch. +- **Reviewer-loop verification** — the reviewer agent reads the diff against the task spec and either approves or sends feedback for another implementer pass. + +### When NOT to use Sandcastle + +- **Exploratory / design work** — when the right answer isn't known, write it yourself. Sandcastle thrives when the task is "implement this", not "figure out what to do". +- **Cross-cutting refactors** — dispatch is per-task; many tasks that touch unrelated files at once is better done in one human-driven session. +- **First-time integrations** (e.g., adopting a new SDK) — better to walk through it manually, then capture the pattern as a generator for future sandcastle dispatches. + +### Prerequisites + +1. **Docker running** — sandcastle uses Docker for the sandbox by default. `docker info` should succeed. +2. **Agent API key** — set ONE of: + - `ANTHROPIC_API_KEY` (recommended; sandcastle's default agent is `claudeCode`) + - `OPENAI_API_KEY` (alternative) +3. **GitHub token** (optional) — `GITHUB_TOKEN` if you want the orchestrator to create PRs. +4. **`.sandcastle/` config present** — already in tree: + - `Dockerfile` — node:22 + pnpm sandbox image + - `prd-eliciter.prompt.md` — interviews humans to draft PRDs + - `adr-eliciter.prompt.md` — same shape, for infrastructure decisions + - `decomposer.prompt.md` — PRD → epic + stories with generator-first task lists + - `implementer.prompt.md` — executes one task; runs all 5 gates before committing + - `reviewer.prompt.md` — reviews implementer's diff against AC + scope + +### The dispatch flow + +``` +pnpm work next → identifies the next ready story (DAG-aware) +pnpm work dispatch → prints what WOULD be dispatched (no Sandcastle call) +pnpm work dispatch --execute + → invokes sandcastle.run(implementer prompt + task spec) + → sandcastle returns { branch, commits, stdout, ... } + → orchestrator computes `git diff main..` + → invokes sandcastle.run(reviewer prompt + diff) + → reviewer returns approve / reject + notes + → orchestrator prints suggested state mutation + (in v1: human ticks the bullet + commits manually) +``` + +### Worked example — dispatch a real task + +Suppose `pnpm work next` reports: + +``` +auth-v1 / 02-sign-up — Sign up with email and password + status: in-progress, tasks: 3/7 +``` + +The story file `docs/work/auth-v1/02-sign-up/_story.md` has a Tasks list with the next unchecked bullet: + +``` +- [ ] Hash password using injected IPasswordHasher before persisting +``` + +**Step 1 — Plan** + +```bash +pnpm work dispatch +``` + +Output: + +``` +=== Dispatch plan === + Epic: auth-v1 + Story: 02-sign-up — Sign up with email and password + Bullet: - [ ] Hash password using injected IPasswordHasher before persisting + Prompt: .sandcastle/implementer.prompt.md + +To execute this dispatch, run: + ANTHROPIC_API_KEY=... pnpm work dispatch --execute +``` + +This is safe to run anywhere — it never invokes Sandcastle. + +**Step 2 — Execute** + +```bash +ANTHROPIC_API_KEY=sk-ant-... pnpm work dispatch --execute +``` + +The orchestrator: + +1. Builds the task spec (story metadata + the current bullet + full story context) +2. Calls `sandcastle.run({ promptFile: ".sandcastle/implementer.prompt.md", promptArgs: { TASK_FILE_CONTENT: spec }, ... })` +3. Sandcastle pulls the Docker image, mounts the repo into `/workspace`, runs `claudeCode` with the implementer prompt template populated +4. The implementer agent (inside the sandbox): + - Reads the task spec + - Runs `pnpm install --frozen-lockfile` + - Locates the use case: `packages/auth/src/application/use-cases/sign-up.use-case.ts` + - Writes a red test asserting `hasher.hash` is called before `repo.create` + - Runs `pnpm test --filter @repo/auth` — sees the red test fail + - Adds `IPasswordHasher` to the factory deps; calls `hasher.hash(input.password)` before `repo.create` + - Runs `pnpm test --filter @repo/auth` — green + - Runs `pnpm typecheck`, `pnpm lint`, `pnpm conformance`, `pnpm fallow:audit` — all five gates green + - Commits on a sandbox branch (`task/02-sign-up-hash-password` or similar) +5. Sandcastle returns: `{ branch: "task/02-sign-up-hash-password", commits: [{sha: "..."}], stdout: "...", ... }` + +**Step 3 — Review** + +The orchestrator immediately runs the reviewer: + +1. Computes `git diff main..task/02-sign-up-hash-password` +2. Calls `sandcastle.run({ promptFile: ".sandcastle/reviewer.prompt.md", promptArgs: { TASK_FILE_CONTENT: spec, DIFF: diff }, ... })` +3. The reviewer agent reads the diff + task + story; verifies: + - The AC bullet is satisfied (test was added; impl calls `hasher.hash`) + - Nothing in the "Out of scope" section was touched (no drive-by edits) + - All gates were run + - The implementer ran `pnpm fallow:audit` + - Generator-first was respected (no hand-rolled scaffolding) +4. Returns `{ decision: "approve", ac_verified: [4], scope_violations: [], notes: "..." }` + +**Step 4 — State mutation (v1: manual)** + +The orchestrator prints: + +``` +=== Suggested state mutation === + Edit docs/work/auth-v1/02-sign-up/_story.md — tick the bullet: + - [x] Hash password using injected IPasswordHasher before persisting + Then: pnpm work rebuild-state && git add -A && git commit -m "..." + +(Automatic state mutation by the orchestrator is v2.) +``` + +You (the human) then: + +1. Merge the sandbox branch: `git merge --no-ff task/02-sign-up-hash-password` +2. Tick the bullet in the story markdown +3. The pre-commit hook auto-runs `pnpm work rebuild-state` + re-stages `_state.json` +4. Push. CI runs the full gate stack (typecheck + test + lint + conformance + fallow + boundaries + visual regression). + +### Troubleshooting Sandcastle + +**`✗ --execute requires ANTHROPIC_API_KEY or OPENAI_API_KEY in env.`** +— Set one. The default agent is Claude. + +**`Error: Cannot find module '@ai-hero/sandcastle'`** +— Run `pnpm install`. Sandcastle is a dev dependency at the workspace root. + +**`Error: docker: command not found`** or sandcastle hangs at "starting sandbox" +— Docker isn't running. `docker info` to confirm. On macOS, start Docker Desktop. + +**The implementer agent times out** +— Default `idleTimeoutSeconds` is 600 (10 minutes). For complex tasks, increase via `dispatch.mjs` (look for the `run({...})` call and add `idleTimeoutSeconds: 1800`). + +**The reviewer rejects with `generator_skipped: true`** +— The implementer hand-rolled what should have been generator output. Either re-dispatch (it gets the reviewer notes), or delete the implementer's diff and run `pnpm turbo gen ` manually first, then dispatch the customisation as a separate task. + +**The reviewer rejects with `scope_violations: [...]`** +— The implementer touched files outside the AC. Re-dispatch with stricter scope; the rejection notes are passed back as context. + +**Cost control** — each dispatch typically uses 50K–200K agent tokens depending on task complexity. The orchestrator does NOT cap retries; if you want to limit, set `max-attempts: 1` in the task's frontmatter (the orchestrator respects this in v2 — for now, just don't re-run dispatch after a reject). + +### Cost-aware variant: planning-only loop + +If you want sandcastle's structure without the agent spend, use planning mode + manual execution: + +```bash +pnpm work dispatch # prints the plan +# (you implement the bullet manually in your editor) +# tick the bullet in docs/work/.../...story.md +# commit; pre-commit auto-rebuilds _state.json +pnpm work dispatch # prints the NEXT plan +``` + +This gives you the same DAG-aware "what's next?" without invoking any agent. Useful for exploratory work or low-budget contexts. + +--- + ## Troubleshooting **`pnpm dev` refuses to boot with `ConformanceError`**