Skip to content

Release diagnostics ​

Start with the records directory printed by the failed command. Diagnose the failing boundary before deciding whether to resume. Use release recovery for the commands that act on the result.

What does the observation establish?

ObservationInterpretation and next check
Waiting for approvalNo automatic permission is granted. Check the displayed operation and private approval log; do not approve just to clear a stalled terminal.
A broadcast journal existsIt may exist after rejection. Its existence alone does not establish whether a transaction was sent. Check the approval audit, sender nonce and actual chain outcome before retrying.
An RPC submission returned a hashLook for a successful mined receipt. Acceptance by an RPC is not contract execution.
A submission response was lostThe outcome is uncertain. Reconcile the retained transaction before approving another creation.
request not forwardedLocal admission or deadline failure occurred before forwarding. Check active/queued counts and the request's deadline.
upstream returned HTTP …The provider returned an HTTP failure. Distinguish that from local capacity rejection.
Cleanup is unconfirmedInputs and diagnostic evidence are intentionally retained. Restore engine access and identify the owned project; do not report removal as successful.

Private logs and their limits

The ordinary release launcher collects approval-diagnostics.log in the signing phase's output directory before teardown: state and bounded tails for the approval tool and its signing/artifact RPC proxies. The file is capped at 64 KiB with at most 16 KiB per service. Collection is time-limited and cannot replace an existing file; a collection failure is reported separately from cleanup.

The approval audit, if created, is approval-audit/approval.jsonl beneath that same phase directory. The phase records reference identifies the business, projection and CAM outputs. A missing audit does not establish that no transaction was sent.

Automated approval rehearsals additionally retain:

FileEvidence
approval-pty.logLive private terminal transcript, at most 8 MiB. Exceeding the limit fails the rehearsal.
approval-rpc.logLive RPC-only extract, at most 1 MiB.
approval-pty-tail.logFinal 512 KiB, including shutdown after a capture-limit failure.
approval-diagnostics.jsonPrompt milestones, simulation mode, last RPC, counters and diagnostic result.
signer-tail.log, approval-tail.logBest-effort state and last 80 output lines before teardown.
source-commit, simulation-modeThe exact private fixture candidate and chosen experiment.

The live PTY/RPC files are created exclusively with mode 0600. Full logs can contain transaction details and approval challenges; do not post them publicly without review. Progress summaries omit those values. Capture exhaustion does not stop draining the PTY during cleanup. Container tails may be unavailable after the relevant container has already been removed.

RPC capacity is part of the deadline

Each release RPC proxy executes at most two connections concurrently and queues up to 32. Queue waiting consumes the original 40-second connection deadline; starting a worker does not reset it. The approval tool allows 45 seconds for a proxy response. Its separate operator-decision timeout is 180 seconds. Each dispatched request has one upstream attempt: the proxy does not silently retry a transaction submission.

Listener-assigned connection/request numbers correlate reads, writes and elapsed time without logging caller IDs, parameters or response bodies. write=flushed says the socket callback completed, not that Forge consumed the reply. Five-second summaries distinguish empty connections from bytes that never formed a valid request.

Malformed frames report framing, size, UTF-8, JSON or admission stage. JSON failures also identify a fixed reason and location. Parameters/results accept finite JSON numbers, but request IDs must be strings, null or safe integers. Transaction quantities remain hexadecimal strings; diagnostic changes do not relax signed transaction validation.

Detail traces are bounded to 128 lines per second and 448 KiB per listener, with 64 KiB reserved for summaries. Backpressure drops trace lines rather than accumulating them. More than 256 connection attempts in one second closes admission and the approval owner with a connection-storm failure; it is not evidence that the client's underlying defect was repaired.

Simulation experiments are fixture-only

An unsupported optional RPC method alone does not establish why a client stalled. Follow the standard reads and their responses in the correlated trace. For example, the pinned Foundry account loader can probe eth_getAccountInfo and fall back after an error; inspect what happened next rather than widening permissions.

Only after examining a failed normal approval-start rehearsal, compare a separate disposable run:

make escrow/release/approval/start/check \
  DOCKER_CONTEXT=your-local-context \
  RELEASE_APPROVAL_IMAGE=sha256:<reviewed-image-id> \
  RELEASE_APPROVAL_SIMULATION=skip

normal remains the default. The explicit skip experiment is admitted only by the startup check and changes one signer command in its new private fixture source. It does not edit the operator checkout or production launcher. Approval isolation, automatic fixture rejection and cleanup remain enforced.

A passing skip run identifies a dependency on normal simulation/preparation, not a root cause or a safe production workaround. Never add an automatic skip fallback or bypass approval to obtain a green result. Compare the two runs' approval-diagnostics.json and approval-rpc.log.

Source: bounded rehearsal diagnostics, RPC proxy implementation and the account-loader fallback.