gh-forgejo-shim testing workflow Codex.app diagnostics

Use Codex traces to adapt the shim to observed behavior

When Codex.app reports unavailable GitHub CLI, missing pull request status, or stale branch data, capture what the app actually ran before changing compatibility code. The trace is the contract; the fix should follow the trace.

The Loop

Capture the exact gh and, when needed, raw git probes Codex runs.
Summarize the trace to find failures, slow calls, unsupported commands, and known Codex probe shapes.
Convert the failing command shape into a focused unit or replay test.
Patch only the mismatch proven by that trace.
Re-run the smoke command, the focused Rust test, and the full Cargo suite.

Rule: do not guess at Codex behavior when a trace can show the exact argv, cwd, route decision, exit code, duration, output sizes, and redacted stderr/stdout excerpt.

Capture Shimmed gh Probes

From the Forgejo checkout you want to diagnose, enable trace capture and run the local probe smoke first.

cd /path/to/forgejo/repo

export FJ_SHIM_TRACE="$PWD/codex-probe-trace.jsonl"
export FJ_SHIM_TRACE_BODY=1

gfj trace smoke
gfj trace summarize "$FJ_SHIM_TRACE"

FJ_SHIM_TRACE is opt-in. With it unset, the shim keeps its normal behavior. FJ_SHIM_TRACE_BODY=1 adds safe, redacted excerpts; leave it unset when output bodies are not needed.

Capture Real Codex.app Behavior

Launch Codex from an environment that includes the trace variables, reproduce the broken UI state, then summarize the same trace file again.

export FJ_SHIM_TRACE="$PWD/codex-probe-trace.jsonl"
export FJ_SHIM_TRACE_BODY=1

# launch Codex from this environment, then reproduce the issue

gfj trace summarize "$FJ_SHIM_TRACE"

Useful reproduction targets include GitHub CLI unavailable, Pull request status unavailable, missing PR board data, and stale branch selector state after creating or fetching branches.

Add Raw Git Recording For Branch UI

Codex branch controls may call git directly instead of going through gh. Use the temporary recorder for one launch session when diagnosing branch selector freshness.

gfj trace git-recorder create "$PWD/codex-probe-trace.jsonl"

The command prints an export PATH=... line, an export FJ_SHIM_TRACE=... line, and a removal command. Launch Codex with the printed environment, reproduce the branch issue, then remove the wrapper.

gfj trace git-recorder remove /path/to/printed/wrapper-dir
gfj trace summarize "$FJ_SHIM_TRACE"

The recorder forwards real git stdout and stderr, appends JSONL records, and removes only directories it created.

Read The Summary

Unsupported command

Add a test for that exact argv, then implement the narrowest routing or parser support needed.

Non-zero exit

Inspect route, host/repo, stderr excerpt, and requested JSON fields. Decide whether the failure is auth, repo detection, schema, or unsupported flags.

Slow call

Check whether Codex has a short timeout. Optimize the exact path, such as version or auth probes, only after the trace proves timing matters.

Raw git stale read

If the trace shows only cached refs or local branch state, the fix may be repo setup or documentation rather than shim code.

Turn A Trace Into A Test

Before changing behavior, copy the observed command shape into the smallest focused Rust test that would fail today. Prefer existing test homes:

crates/gh-forgejo-shim/src/routing.rs for route decisions and host/repo detection.
crates/gh-forgejo-shim/src/read_only.rs and crates/gh-forgejo-shim/src/create.rs for Forgejo command behavior and JSON shape compatibility.
crates/gh-forgejo-shim/src/trace_summary.rs for new summary classifications.
crates/gh-forgejo-shim/src/codex_smoke.rs for changes to the replayed probe family.
crates/gh-forgejo-shim/src/git_recorder.rs for raw git recorder behavior.

# Good adaptation path:
# 1. Add a failing test with the exact traced argv.
# 2. Implement the smallest compatibility fix.
# 3. Re-run the focused test.
# 4. Run the full suite.

cargo test -p gh-forgejo-shim routing::tests::test_name_here --locked
cargo test --workspace --locked

Patch Only Proven Mismatches

The shim should stay narrow. Use the trace to decide which type of change is justified.

Schema mismatch: adjust normalization or JSON filtering for the requested fields.
Missing Forgejo route: extend decide_route() and the matching handler only for the observed command family.
Unsupported flag: parse and ignore harmless GitHub-only filters, or fail clearly when the flag changes behavior the shim cannot emulate.
Auth or identity mismatch: verify GH_HOST, GH_REPO, stored shim auth, and gh api user behavior.
Raw Git cache behavior: do not patch gh if Codex never called it. Fix repo remote/upstream shape or document the refresh workflow.

Safety Notes

gh auth token stdout is always suppressed in trace excerpts.
Known token and auth-sensitive patterns are redacted from argv and body excerpts.
Body excerpts are disabled unless FJ_SHIM_TRACE_BODY=1 is set.
Trace files may still include local paths, branch names, repo names, and redacted command context. Treat them as diagnostic artifacts, not public logs.

Recommended Validation

gfj trace smoke
gfj trace summarize "$FJ_SHIM_TRACE"
cargo fmt --all -- --check
cargo clippy --workspace --all-targets --locked -- -D warnings
cargo test --workspace --locked
git diff --check

For GUI-facing changes, install or refresh the native shim before re-testing Codex.app, then capture a new trace from the app itself.