Release diagnostics
Start with the records directory printed by the failed command. Diagnose the failing boundary before deciding whether to resume. Use release recovery for the commands that act on the result.
What does the observation establish?
| Observation | Interpretation and next check |
|---|---|
| Waiting for approval | No automatic permission is granted. Check the displayed operation and private approval log; do not approve just to clear a stalled terminal. |
| A broadcast journal exists | It may exist after rejection. Its existence alone does not establish whether a transaction was sent. Check the approval audit, sender nonce and actual chain outcome before retrying. |
| An RPC submission returned a hash | Look for a successful mined receipt. Acceptance by an RPC is not contract execution. |
| A submission response was lost | The outcome is uncertain. Reconcile the retained transaction before approving another creation. |
request not forwarded | Local admission or deadline failure occurred before forwarding. Check active/queued counts and the request's deadline. |
upstream returned HTTP … | The provider returned an HTTP failure. Distinguish that from local capacity rejection. |
| Cleanup is unconfirmed | Inputs and diagnostic evidence are intentionally retained. Restore engine access and identify the owned project; do not report removal as successful. |
Private logs and their limits
The ordinary release launcher collects approval-diagnostics.log in the signing phase's output directory before teardown: state and bounded tails for the approval tool and its signing/artifact RPC proxies. The file is capped at 64 KiB with at most 16 KiB per service. Collection is time-limited and cannot replace an existing file; a collection failure is reported separately from cleanup.
The approval audit, if created, is approval-audit/approval.jsonl beneath that same phase directory. The phase records reference identifies the business, projection and CAM outputs. A missing audit does not establish that no transaction was sent.
Automated approval rehearsals additionally retain:
| File | Evidence |
|---|---|
approval-pty.log | Live private terminal transcript, at most 8 MiB. Exceeding the limit fails the rehearsal. |
approval-rpc.log | Live RPC-only extract, at most 1 MiB. |
approval-pty-tail.log | Final 512 KiB, including shutdown after a capture-limit failure. |
approval-diagnostics.json | Prompt milestones, simulation mode, last RPC, counters and diagnostic result. |
signer-tail.log, approval-tail.log | Best-effort state and last 80 output lines before teardown. |
source-commit, simulation-mode | The exact private fixture candidate and chosen experiment. |
The live PTY/RPC files are created exclusively with mode 0600. Full logs can contain transaction details and approval challenges; do not post them publicly without review. Progress summaries omit those values. Capture exhaustion does not stop draining the PTY during cleanup. Container tails may be unavailable after the relevant container has already been removed.
RPC capacity is part of the deadline
Each release RPC proxy executes at most two connections concurrently and queues up to 32. Queue waiting consumes the original 40-second connection deadline; starting a worker does not reset it. The approval tool allows 45 seconds for a proxy response. Its separate operator-decision timeout is 180 seconds. Each dispatched request has one upstream attempt: the proxy does not silently retry a transaction submission.
Listener-assigned connection/request numbers correlate reads, writes and elapsed time without logging caller IDs, parameters or response bodies. write=flushed says the socket callback completed, not that Forge consumed the reply. Five-second summaries distinguish empty connections from bytes that never formed a valid request.
Malformed frames report framing, size, UTF-8, JSON or admission stage. JSON failures also identify a fixed reason and location. Parameters/results accept finite JSON numbers, but request IDs must be strings, null or safe integers. Transaction quantities remain hexadecimal strings; diagnostic changes do not relax signed transaction validation.
Detail traces are bounded to 128 lines per second and 448 KiB per listener, with 64 KiB reserved for summaries. Backpressure drops trace lines rather than accumulating them. More than 256 connection attempts in one second closes admission and the approval owner with a connection-storm failure; it is not evidence that the client's underlying defect was repaired.
Simulation experiments are fixture-only
An unsupported optional RPC method alone does not establish why a client stalled. Follow the standard reads and their responses in the correlated trace. For example, the pinned Foundry account loader can probe eth_getAccountInfo and fall back after an error; inspect what happened next rather than widening permissions.
Only after examining a failed normal approval-start rehearsal, compare a separate disposable run:
make escrow/release/approval/start/check \
DOCKER_CONTEXT=your-local-context \
RELEASE_APPROVAL_IMAGE=sha256:<reviewed-image-id> \
RELEASE_APPROVAL_SIMULATION=skipnormal remains the default. The explicit skip experiment is admitted only by the startup check and changes one signer command in its new private fixture source. It does not edit the operator checkout or production launcher. Approval isolation, automatic fixture rejection and cleanup remain enforced.
A passing skip run identifies a dependency on normal simulation/preparation, not a root cause or a safe production workaround. Never add an automatic skip fallback or bypass approval to obtain a green result. Compare the two runs' approval-diagnostics.json and approval-rpc.log.
Source: bounded rehearsal diagnostics, RPC proxy implementation and the account-loader fallback.