021 — Portable deployment and persistent scientific runners

Started 23 October 2026; updated as implementation progresses. Compose runners and shared delivery are deployed locally and in production. One local API prediction qualifies the new provider-to-relay-to-API path; broader failure and multi-provider exercises are deferred in note 022. The local delivery checkpoint records exact identities and numerical comparisons; the implementation record below distinguishes source checks, installed witnesses and outstanding work.

Initial planning source checkpoint: 87c3bad. The user requested a portable, copyable architecture following sibling precedent, then a thorough note with transition distinguished from end-state. Writing this initial plan did not start a deployment, change credentials or create API Jobs.

This plan supersedes 004's systemd/host-supervisor choice. It retains that record's historical evidence. The operator guide now describes the maintained Compose commands.

Shared result delivery — design, 24 October 2026

Deployed 24 October 2026; current evidence follows. Run one delivery service per Docker host, shared by its providers and runners. It owns durable acceptance and API publication of final immutable completion or failure requests. Providers own scientific decisions, execution and cleanup. This replaces provider result-file dumps and completed-payload history; it does not change molecular inputs, prediction, result formatting or the API contract. Earlier implementation and retention descriptions below record the preceding design.

Observed shared-delivery cutover, 24 October 2026

Runtime/configuration checkpoint cb31917 is deployed at https://storage.example with [local NMRNet provider], and at https://storage.example with [production NMRNet provider]. Each Docker host has its own nmr-api-delivery project, private queue volume and provider-scoped socket; only NMRNet is connected so far. All seven NMRNet roles and both long-running shared roles started successfully. The finite volume initializer exited zero. Production logs also observe transient CONNECT 502 refusals followed by read recovery; healthy containers alone do not prove uninterrupted API availability.

The new local path executed exactly one fixed chloromethane Job. The delivery worker reported a verified, non-replayed COMPLETE receipt for body sha256:6491f7b2e300e832b81a6031deb3ad72e46359a8e7fcfb333c786a0bff890c52. The first public client stopped on a GET with HTTP 401 authentication_window_closed. No mutation was retried. The maintained JOB_RECEIPT readback-only mode at 54fc49f then used four GETs to confirm exactly one succeeded Attempt and one result, with live interpretation and the unchanged scientific oracle. Received result SHA-256 is d9794208571eb728be539136a3527c6d7890c6d97d5c57c68582022f590d55d3. Private receiving evidence remains in .work/qualification/shared-delivery-api-cb31917 and .work/qualification/shared-delivery-readback. The independent oracle is artifacts/pasted-scientific-oracle.json; expected values and tolerances were unchanged. No new production Job or scientific matrix was run for this cutover.

Read-only queries against the live databases observed zero outstanding delivery rows and zero producer checkpoints after completion. A maintained local delivery/deployment/down then up preserved the queue volume, reopened it with zero rows, and left NMRNet running with restart count zero. The actual queue directory is mode 0700 and its database mode 0600. This proves ordinary restart and accepted-payload removal for this case; it does not prove every pending-send interruption timing.

The clean cut follows the user's later instruction and supersedes the earlier migration plan: no importer, archive or compatibility path was introduced. At cutover the displaced file stores contained completed history only, with no pending local files. Their result history and operational deployment dumps were removed; intentional scientific qualification evidence and private input credentials remain. Unused prototype Engine-handoff files were removed. The deployed path uses direct maintained Compose calls, actual frozen configuration and required persistent volumes, with bounded Docker logs.

Source review of 15f06db..54fc49f found no rollout-blocking issue in the direct grants, bounded source reads or GET-only receipt branch. Earlier focused core tests below are checkpoint-specific evidence, not a claim that obsolete full-file-store suites pass. Remaining test ports, wider failure/resource scenarios and detailed SDK-reference updates are explicitly deferred in note 022. The following paragraphs retain the dated implementation history.

Completion-tail source 3126062 adds an offline operator listing/discard lane. The queue's original exclusive lock requires an explicit daemon stop. Listing projects only configured-binding held metadata; deletion matches that binding, Attempt, SHA-256 and held state. Pending rows and other bindings remain unchanged. No producer deletion endpoint, second registry, API request or automatic stop/restart is added. Five installed storage witnesses pass, including the new scoped deletion/refusal case and deleted-payload exclusion. The actual maintained discard target was also refused with EAGAIN while the live daemon held the lock. The broader unit aggregate's four stale-fixture errors are recorded in note 022.

Final image refresh at 3126062, 22:15 UTC: both deployments again started all roles successfully, with initializer exit zero and restart count zero. Offline authoritative listing showed no held rows on either host; a stopped-daemon discard of the already completed local Attempt correctly refused because no matching held row existed. An authenticated production inventory GET reported zero unfinished Attempts and no continuation cursor. After the final refresh, both live queues again contained zero outstanding rows. No further scientific run was performed.

Observed local images: controller sha256:e9d3ae933be317d0de96be09ec3aaabd0b7802b723405009a82e77cabbe22628, RDKit sha256:a555ff8f0a3750a4b6cbc2d3b58388cfaa245b047e55906c91be0c1290d5037a, prediction sha256:3cb21ad27b269f403ead6514b105e369453dddfb2e03ec72aab611bfea43ecee, delivery sha256:e28174442d2388e79ccc65fff9b8e7840b026662ad4350348c2109296cd47a34. Observed production images: controller sha256:930badd2530eb8e886ab533bdbbb0c670f37d7a3158327d57c9a16d5562b91b7, RDKit sha256:c234d42eca9268f103051ae9309d3a88341dc4abb7c3d2aacc3a32a5e3563759, prediction sha256:c3be40636fe29ea56c2b14f8c039ef9547613bcfe07856d2d6a27a3c6540b1bb, delivery sha256:876c15a076ff96739e758fcb5384cf826c7d64b2abe174d38aa809cdca0f322b. These are independently built image identities, not a claim of byte-identical cross-host builds. The frozen execution declarations retain caller provenance; the receiving scientific result above belongs to its original smoke checkpoint.

Foundation checkpoints: d4238ce implements pending-only transactional SQLite storage (nine focused checks); 48beaf0 adds fixed-destination Unix HTTP through the existing exchange and retirement owner (13 connected transport checks and 31 unit/network checks). The local inbox/client adds durable-before-acknowledgement submission, exact outstanding deduplication, provider-scoped metadata-only inventory and safe binding-refusal diagnostics. Seven focused installed-package IPC checks cover lost acknowledgement, full capacity, read-versus-submit effect reporting, stalled connection closure and original SQLite write settlement across waiter/shutdown cancellation. These checks use synthetic outcomes and local sockets, without model calls or scientific execution. The per-binding delivery worker now uses ProviderAPI directly: only a matching parsed receipt permits transactional removal; exact-byte retry uses the API owner's settled-I/O permission and capped jittered backoff. Ten connected TLS worker checks additionally cover fresh signing after a lost reply, queue reopen, one held item followed by another publication, concurrent progress on another binding, interrupted-send uncertainty, topology failure without moving pending items into hold, and receipt truth across independent storage failures, including cancellation during final retirement. All 17 delivery checks pass through make test/integration/provider-core/delivery WHEELHOUSE=<path>. The shared service lifecycle now connects those components under one database owner. Six additional checks exercise real producer socket to signed API publication, original commit settlement after interrupted shutdown, stop during listener startup without launching delivery, scoped paused-publisher status, provider shares fitting the host budget, and changed-binding startup refusal that preserves original pending bytes. The connected delivery lane now passes 23 checks; the affected storage lane passes nine. These prove local delivery components; daemon packaging, producer handoff, cutover and deployed recovery remain outstanding.

The descriptor-based startup reader and existing API/TLS settings now live in provider_core.startup_file and provider_core.api_config. NMRNet delegates to them while preserving its own configuration error context. This allows the delivery image to admit the same private credentials and exact CA bytes without importing scientific provider modules. The existing 23 configuration checks and eight diagnostic checks pass; the move adds no network effects, new trust defaults or credential permission exceptions.

provider_core.delivery_config.load_delivery_service admits the closed frozen service document and constructs unstarted owners. Each binding requires its explicit proxy socket and the existing verified API credential principal/profile. Missing/extra fields, private credential modes and invalid time budgets refuse before SQLite/listener acquisition. Three focused configuration checks bring the delivery unit/storage lane to 12 passing checks; the 23 connected delivery checks remain green. Deployment generation of this document, the daemon entrypoint and its image are still pending.

provider_core.diagnostics now owns the existing bounded, redacted error graph and stderr writers. NMRNet supplies only its reviewed scientific details and relationships through two explicit projection seams; core imports no scientific provider modules. Three new core recipient checks cover API correlation/effect plus independent cleanup, unknown/private text exclusion and a full stderr pipe. All 15 delivery unit checks and 66 affected process receiving checks pass. This extraction preserves the existing NMRNet writer contract; it does not impose whole-record pipe atomicity on the new daemon's logging design.

tools.core_environment now owns the existing offline locked wheel build/install. tools.provider_environment adds NMRNet binding modules only for controller and scientific images; the delivery image can use core installation directly. All three existing image COPY, ignore and committed-input closures include the shared helper. Twelve focused image tests and all 50 native-image convention checks pass, including refusal of changed/missing wheels and existing scientific dependency conflicts. Review caught and corrected the inference selector list's required sorting. No scientific dependency versions or inference calculations changed.

The installed provider_core.delivery_main command now consumes the frozen configuration, establishes private creation modes, starts shared delivery and handles SIGTERM through its original retirement owners. A real local Unix CONNECT/TLS fixture qualifies one producer handoff, one signed API publication, receipt-based deletion and clean signal retirement. A separate malformed-configuration process check proves safe diagnostics with no runtime state. All 27 connected delivery checks and 18 unit checks pass.

Logging follows the ordinary StreamHandler precedent in Magnet's process logger. The container owner supplies the existing bounded, non-blocking Docker log policy; Python neither promises retained logs nor detects Docker's full-buffer drops. See Docker's delivery-mode contract. There is no new logging queue, pipe-size gate, framing protocol or file fallback. The queue alternative adds a sentinel and unbounded thread join (CPython 3.12.12 QueueListener.stop); process shutdown also acquires handlers and flushes streams (logging.shutdown). Review therefore retained the ordinary Docker-backed writer. Observed local output failures appear in scoped status without reversing API acceptance or deletion; a connected broken-stderr witness proves that separation. Diagnostic display fields do not gate ownership/capacity reads. The daemon image, producer integration, cutover and deployed recovery remain outstanding.

make delivery/image/build WHEELHOUSE=<path> now uses the same committed-input capture, locked wheel contexts, bounded native build, immutable-ID and role-label admission as the controller. The shared owner is tools/core_image.sh; role wrappers keep their own selectors, records and guidance. Controller selector exports remain available to deployment capture. Fifteen focused image checks and 50 native-image convention checks pass. Delivery source capture excludes scientific namespaces and remains unchanged by unrelated NMRNet binding edits.

The delivery Dockerfile installs core directly, with only the existing three standard-library routing modules for the shared project's egress role. Its build checks both entrypoints and refuses importable NMRNet, RDKit or Torch modules in the installed daemon environment. The first image build and NMRNet's producer handoff are the next runtime checkpoints; no deployed provider uses this service yet.

The first cold build exposed a hidden NMRNet logging import in the proxy entrypoint. The proxy now configures only its own ordinary logger while retaining the existing dependency WARNING floor and verbosity setting; NMRNet's executable logging policy stays at its owner. Sixty-five focused proxy checks pass. The proxy also uses core's readiness owner, so both roles run under the installed core interpreter. Build qualification constructs and closes a private proxy listener as well as checking command help; no DNS, API request or server loop runs during that check.

The maintained offline build now succeeds from source 4dcfba9, producing delivery image sha256:3abea5961f7be6fda5e6d00814a557ac6dee59b8ef9b26530d0e6afc73223fe3. Both entrypoint checks, the scientific-namespace refusal check and proxy constructor check passed. Build progress and failures use the existing native stdio mode because the build receives only public committed source and locked wheels; immutable image admission still reads its private ID file. This is image qualification, not deployment or NMRNet producer integration.

Acquisition review then added configured binding/socket context to listener startup failures, including observed parent ownership/mode, while preserving already-open listener cleanup. A failed private-file inspection now closes its original descriptor before handoff; the real-FD witness observes EBADF. The affected connected lane passes 28 checks and the unit lane passes 19. These follow-up source changes require the normal matching image rebuild when the producer integration is ready; no live service changed.

provider_core.producer_checkpoint now defines compact control states and their codec. They preserve exact small START bytes and scoped admission metadata, then prepared-identity digest, selected terminal-command identity and actual relay acceptance as distinct states. The selected operation/hash/size must be retained before the first enqueue: after a lost enqueue reply and producer crash, an absent row plus an in-progress API Attempt cannot authorize a replacement failure or another scientific run. This is local control metadata, not a staged/ready relay protocol.

Suppression records only its disposition and existing control identity. This deliberately supersedes note017's P3 full-candidate restart retention; neither a full prepared identity nor a suppressed scientific body may enter the new producer store. Seven focused codec checks pass through make test/unit/provider-core/checkpoints WHEELHOUSE=<path>. They prove metadata roundtrip and payload exclusion, not checkpoint I/O or runtime handoff.

provider_core.checkpoint_store now owns private, single-writer SQLite control metadata with conditional START, admission, execution-entry, terminal-selection, delivery-acceptance and suppression transitions. It ports the original-operation wait/settlement gate from attempt_store at 5229b60 and private-file/durability admission from delivery_store at 71dd6a8. A cancelled or expired wait keeps the same transaction and owner; it cannot submit a replacement or close ahead of settlement. Schema and version initialization commit together. Reopening checks retained item and total metadata limits. Clearing an accepted checkpoint deletes its metadata without an archive; the caller must first confirm cleanup. API-terminal recovery and terminal START replay need their own honest lifecycle resolution, never a fabricated relay acceptance. Eighteen focused checks pass, including restart before acceptance, conflicting selection, interrupted writes, initialization rollback, capacity and protected error rendering. START/execution consumer integration and deployment remain outstanding; this store alone changes no running provider.

The START owner now uses those explicit checkpoint transitions. It retains the pending request before POST, then binds and persists the verified API admission before returning the actual received acknowledgement. Restored metadata is never converted into a live acknowledgement. A late ACK write settles its original operation without another POST; a storage refusal retains its actual reason rather than being relabelled as a missing record. The focused unit lane passes eight checks. The isolated core-only TLS lane passes its receiving-side witness: both original and replay POSTs have the same retained bytes before dispatch and valid independently checked request signatures. Commands are make test/unit/provider-core/starting and make test/integration/provider-core/starting, each with an explicit WHEELHOUSE. This is a producer migration in progress: executor, controller, recovery and service consumers still need their direct checkpoint/relay cutover. Do not publish or deploy this intermediate branch as a complete replacement of the file-backed producer.

The executor now persists the prepared identity digest and selected operation/hash/size, then hands the approved body to the scoped relay client. Native retirement starts independently before selection storage or IPC can block. Full capacity paces the same body with capped delivery backoff. Failed cleanup blocks reuse and claim release while the original approved handoff can still proceed; it is not a publication veto. Explicit caller cancellation separately stops further submissions. A verified relay acknowledgement immediately releases producer outcome/body references, before acceptance metadata persistence. Protected diagnostics distinguish that acknowledgement from API publication and native retirement. Suppression retains only its disposition; late writes settle the same operation and release RAM evidence after confirmed suppression. The focused delivery unit lane passes 31 checks, including entry-write refusal, independently blocked selection/cleanup, capacity returning, cleanup failure before/after handoff, explicit versus native cleanup cancellation, failed acceptance metadata and interrupted suppression. These recording scientific sessions do not qualify deployment or repeat inference. Controller/recovery/service cutover remains active.

Urgent implementation direction, 24 October 2026: finish the working end state and use minimal essential checks now. Record non-blocking refinements and additional testing in running note 022. Preserve scientific admission and actual ownership; defer broad legacy test ports, extra failure timings and polish rather than delaying the connected service. The clean-cut decision below removes legacy data conversion.

Runtime cutover source, 25 October 2026: controller, recovery, admission, factory and process now use compact checkpoints and independent relay clients. Successful handoff plus native cleanup clears producer control without history. Restart reconciles positive relay ownership or terminal API lifecycle state without recreating science or claiming an API publication receipt. Admission excludes the relay's exact current Jobs and Attempts; it keeps no completed-Job list. File-backed storage, producer terminal publication and stopped-file recovery commands are removed.

The host deployment owner derives API-only inputs from installed provider bundles. Its stable project and queue volume are independent of checkouts and provider generations. Delivery owns separate API proxies. Each producer mounts only its own external socket volume, with an explicit supplementary group; queue and credentials remain private. Producer control also uses a named volume scoped to its API, provider, lane and generation, mounted only by that producer. A finite initializer receives queue/control/socket volume roots and CHOWN capability. It never changes original credentials or existing volume contents. The installed-image build and minimal operational/API smoke are the next qualification; broader legacy test ports and refinements are explicitly deferred in note 022.

Small deployment and explicit ownership

Direct deployment correction, 25 October 2026: the maintained provider and shared-delivery targets now call Docker/Compose directly with explicit project, image, volume, resource and network configuration. They do not create Engine-observation snapshots, checker handoff files, command receipt trees or success-marker gates. The finite initializer receives its UID/GID and volume roots as command arguments. Required execution source hashes are written directly into actual runtime configuration; there is no intermediate source inventory/hash handoff. Normal diagnostics stream to the caller and bounded Docker logs. This supersedes the observation/checker deployment workflow described in earlier implementation checkpoints.

Use the existing provider-core package for a small delivery daemon and client. The daemon has no model, RDKit, GPU or Docker-engine access. It is the sole writer of one SQLite database in a named Docker volume. Copy Magnet's private transactional outbox boundary, including durable commits, but omit its retained confirmed payloads. Reuse core's API signing, request/receipt parsing, effect classification and permitted retry behavior; replace the publisher's current dependency on a producer's full Attempt record.

Use SQLite journal_mode=DELETE, synchronous=EXTRA and secure_delete=ON. EXTRA supplies the directory sync for rollback-journal commit durability; secure deletion clears removed content in ordinary database tables. These are built-in settings, not a per-Job vacuum or erasure subsystem. Sources: SQLite commit synchronization and deleted-content clearing.

Give each configured producer its own Unix socket directory, mounted only into that producer and the service. That mount is the handoff capability. Trusted deployment configuration binds the socket to an API origin, expected topology, provider identity, credential and admitted outbound route. Derive those bindings from the existing provider configuration instead of asking operators to maintain a second copy. Producers cannot choose another destination or signing identity, and do not mount the delivery database. Reconfiguration must not retarget queued bodies to another API/provider; a different identity gets a separate binding. Credential rotation may keep the same API/provider identity. Native prediction children continue returning results to their provider.

Handoff and ordinary delivery

1. The provider finishes its existing result-acceptance decision and encodes the final API request once. It submits those exact bytes to its socket. Delivery does not wait for native cleanup; cleanup still gates safe reuse.
2. The service admits the complete/fail request using existing core parsers, enforces its byte/item limits and commits the immutable body before acknowledging acceptance. The key is API origin, provider and Attempt; complete and fail are competing choices for that same key.
3. A repeated outstanding submission with the same operation/body is acknowledged without creating another item. A conflicting outstanding submission is refused without replacing the original.
4. After the durable acknowledgement, the provider drops its body and the service alone owns publication. It signs each API exchange afresh while preserving the submitted body. Existing API rules decide when exact replay or reconciliation is permitted; a timeout alone grants neither.
5. After validating the matching API receipt, delete the queue row and body in one transaction. Keep no completed-result archive, permanent tombstone or second producer acknowledgement protocol. Subsequent duplicate submissions can be queued again: the API remains the final authority for already-completed replay and conflict.

The client operations are submit, status for a selected Attempt, and a scoped pending inventory with its capacity summary. Submission carries the operation, Job and Attempt references and exact request bytes; routing comes from the socket binding. Status/inventory return identity, disposition, body digest and safe delay/failure facts, never molecular payloads. The Job reference supplies the existing API read/reconciliation binding. There is no arbitrary HTTP forwarding or model-specific plugin interface.

Retention, capacity and failures

The database holds only outstanding request bodies, their binding/Attempt and Job identity, and retry or held-disposition facts. It does not copy raw Job text, Uploads, preparation arrays, unrounded scientific evidence or transcripts. Bodies may themselves contain the coordinates and shifts selected for the public result. Normal confirmation removes them; an unresolved permanent refusal retains its body for an explicit operator disposition.

Allocate the host's byte/item budget among providers and bound send concurrency. Check capacity before additional computation; an unaccepted result stops further admission and remains bounded by producer concurrency. Enqueue is the authoritative capacity check, not a promise made by an earlier status read. Queue-full or unavailable storage returns no acceptance acknowledgement. There is no direct-send fallback or silent result eviction. Pace permitted retries with capped jittered backoff and request deadlines. A held item must not block other items; one provider's outage must not occupy all delivery slots or consume another provider's capacity.

Keep any necessary execution checkpoint with the producer in managed volume storage, without result payloads or completed history. One concrete need is the original small START request/key when a process dies after sending START but before receiving its reply. Retain only current unresolved control identity and entry/handoff facts needed by actual restart code; obtain remote facts from the API where it already owns them. The delivery service does not become an execution journal or infer that missing state permits another execution.

Use Docker restart policy and bounded Docker logs. Status/logs identify the provider, Attempt, queue age, retry delay, safe cause and required action; distinguish service availability, durable enqueue and API acceptance. Permanent authentication/configuration failures pause the affected binding with a useful diagnosis. A local volume covers container/process replacement, not loss of the host disk; no replication or host failover is proposed.

Implementation handoff

The maintained Make flow below is implemented in source; installed proof is pending. Build from the existing shared wheelhouse, install the providers' frozen bundles, then derive the host delivery bindings and start the service. Each successful command should print its next useful action. Provider setup mounts its own endpoint and preserves the selected host interface where needed.

make delivery/image/build WHEELHOUSE=/data/cache/wheelhouse
make delivery/deployment/install DEPLOYMENTS="provider-name ..."
make delivery/deployment/up
make delivery/deployment/status
make delivery/deployment/logs
make delivery/deployment/down

For a permanently held item, status must explain the refusal and the exact next action. A proposed make delivery/discard PROVIDER=<binding> ATTEMPT=<reference> removes only the explicitly selected held item when the operator chooses to abandon delivery; it does not claim API acceptance.

Clean cutover — user decision, 24 October 2026: replace the old delivery path directly. This supersedes the earlier plan to import pending terminal bodies or convert producer control records. Stop the old publisher; remove its displaced result/control storage and completed-payload history along with the old writing paths. Build no legacy importer, conversion, compatibility or dual-write machinery. An ordinary API inventory at cutover must report any actual unfinished Attempts honestly; it cannot invent outcomes or authorize another scientific execution. Preserve unrelated services and intentional qualification evidence, which are separate from operational delivery storage.

Use a few connected checks at the changed boundary: lose the enqueue reply; lose the API reply and restart delivery; keep a second provider progressing while the first backs off; exercise queue-full refusal; and confirm accepted payload removal and absence from logs. Verify unchanged bytes and one scientific execution, using controlled API fixtures. No scientific benchmark, new real Job or exhaustive fault matrix is needed to design this service.

Sources: core at publication ownership and retry, exact request parsing and matching receipts; Magnet at private SQLite ownership and commits, retained requests and confirmation and Docker volume and logging. Durable handoff, payload deletion, provider isolation and independent cleanup are adaptations agreed here; Magnet's retained confirmed rows are not copied.

Implementation start: interpreter fallback failures now have an explicit production operator recipient as well as their existing protected evidence snapshot. The NMR interpreter reports each failed endpoint once, including after successful fallback; repeated retirement does not repeat notices. Factory and runtime composition carry the existing protected process reporter with Job/endpoint identity. No core logging hook or extra diagnostic history was added. Focused contained checks passed: interpreter 39, runtime 16, native factory 14, service 53 and process 66. The process witness also required updating its actual interrupted-model composition to supply the mandatory recipient. The delivery service itself is not implemented by this preliminary slice.

Delivery queue implemented: the shared package now owns a private single-writer SQLite queue with exact immutable completion/failure bodies, duplicate/conflict handling, provider/host item and byte limits, and held-item accounting. Matching publication removal deletes the row and clears deleted table content; no confirmed rows or payload history remain. It uses the agreed DELETE/EXTRA/secure-delete settings and the existing request parsers. Receipt verification belongs to the next API delivery owner, not a second parser inside the database.

The new contained test/unit/provider-core/delivery lane passes nine focused checks: exact bytes across reopen, duplicate/conflicting submissions, byte/item capacity, held-provider isolation, payload removal, corrupt storage, single-writer ownership and useful missing-volume failure. Independent review narrowed reads to serialized access without write transactions and corrected storage messages so they do not negate prior enqueue acknowledgements or API receipts. No service, producer cutover or deployment is claimed by this storage slice.

Local delivery transport implemented: an explicit UnixHttpTransport now reuses the existing bounded exchange and retained cleanup owner. Its fixed local HTTP origin cannot select TCP, another socket or a CONNECT proxy. External HTTPS construction and TLS admission remain unchanged. Common transport diagnostics say HTTP rather than incorrectly claiming TLS for local IPC. Thirteen connected transport and 31 unit transport/network checks passed, including exact raw request bytes, response limits and cancellation with late original closure. Independent review found no authority or lifecycle regression. This is the local carrier; the delivery protocol and daemon follow next.

Decision, purpose and completion contract

The product remains a provider of atom-indexed H/C chemical shifts from admitted molecular geometries. Operating it should require a named deployment, its private inputs and a Docker engine, rather than maintaining an application-specific host daemon. Copy the established Compose/container/session-supervisor boundaries; adapt them to NMRNet's existing scientific and durable execution owners.

Target platform: Linux with Docker Engine and Compose v2. Explicit host-interface binding, Unix sockets and the donor supervisor's Linux process facilities are requirements. This is portability between supported Linux installations, not a claim of Windows, macOS, rootless-engine, ARM or arbitrary orchestrator qualification. Preserve the currently qualified image architecture; declare and verify any further platform separately.

1. A finite deployment command starts the admitted stack, verifies startup and returns. Containers continue after the invoking shell exits and recover after engine/host restart according to their declared policy.
2. No production component depends on a permanent host Bash monitor, a custom systemd unit, a Docker socket grant or per-Job Docker commands.
3. One serial provider session owns each runner session. Native child exit and request completion have different evidence. Neither socket EOF nor a healthcheck proves child retirement or successful science.
4. The existing controller/store remain the sole owners of admission, durable execution entry, retained outcomes and permitted API replay. A container restart creates no new authority to repeat interpretation or computation.
5. Scientific inputs, atom order, units, conformers, model/scaler selection, full-precision evidence and output formatting retain their existing meaning. Model identity remains tied to the bytes actually verified and loaded.
6. Ordinary successful Jobs do not exhaust a lifetime allocation counter or accumulate unbounded handles, processes, source files or diagnostic history.
7. One documented deployment workflow remains after cutover. Obsolete production launchers, selectors, tests and guidance are removed, with historical rationale preserved.

Initial implementation baseline — 87c3bad

This section describes the initial planning checkpoint, before the removals and qualification recorded below. At that revision, tools/provider.sh::provider_host_monitor repeatedly services provider_workers.sh's allocation requests. The host creates separate native containers, publishes receipts and proves exact container retirement. NativeAttemptFactory connects admitted Attempts to these ports. deployment/nmrnet-provider@.service keeps that host process alive; provider/deployment/install freezes inputs and installs its service profile. This is why replacing only systemd with a Docker restart option would be incomplete. Source N1.

At that revision the scientific connection is single-use. PredictionSession consumes one prepared handle; SelectedReleaseModels explicitly releases model/scaler references between heads. No current warm-model qualification is implied. The existing private protocol token correlates preparation and execution on a native connection; it is not the removed model-facing submit/token round trip. Source N2.

Two refinements to the preceding discussion are therefore necessary:

• Fresh processes for every Job and retaining loaded models across Jobs are alternative inference lifetimes. The chosen end-state below reuses a healthy supervisor and container, but uses a fresh scientific child for every native operation. Neither inference-process nor model-cache reuse is required.
• “Restart does not repeat a Job” does not prohibit the already admitted single replacement of pure prediction within the same Attempt. Preserve that narrow policy and its original causes; do not broaden it into whole-Job retry. Source N3.

End-state ownership and layout

Provider and scientific runners remain network-isolated except for their private Unix channels; only the existing proxy roles receive the routing authority they need. Keep the established external/mapped-route distinctions and exact signed/TLS origin. Compose must not silently substitute service DNS names, ports or bridge routing for the operator's interface selection. Host-network access, if required by the existing interface-bound proxy, is confined to that role and admitted explicitly.

Use the existing three fixed native roles: prediction, reader and smiles. Each has one serial operation slot; the provider retains its serial Attempt execution policy. Reader and smiles share an image but have separate fixed commands, sockets and declared container budgets. This avoids a new scientific dispatcher inside provider-core. A finite busy/readiness wait consumes the caller's deadline; there is no hidden unbounded queue. Parallel pools, remote workers, Kubernetes and automatic device scheduling are outside this change.

compose/provider.yml                         # persistent service and grant declarations
tools/provider_deployment.sh                 # finite host engine calls
tools/provider_deployment.py                 # contained config/bundle/plan admission
config/deployments/<name>/provider.toml       # public provider definition
config/deployments/<name>/deployment.toml     # ignored host paths/resources/routes
secrets/deployments/<name>/                  # ignored private inputs and frozen bundles
packages/provider-core/src/provider_core/    # shared session/process lifecycle only
nmrnet_provider/                             # NMRNet runner composition and clients
nmrnet_inference/                            # scientific computation and release ownership

These are responsibility boundaries, not a mandatory new file for each box. Adapt the existing deployment helper and protocol clients; preserve a single owner for each rule. Put the small copied process/session supervisor in provider-core, retaining donor provenance and its tests. Keep chemical admission, model loading, process argv, device settings and public result policy in the binding. The shared package must not acquire RDKit/Torch dependencies or become a Compose SDK.

Installed image path: build the ordinary provider-core wheel and install its complete declared dependency closure offline into both scientific Python 3.12 runtimes, alongside their pinned scientific dependencies. Follow provider_environment.py's hashed-lock install, wheel install and pip check; using --no-deps for the wheel is valid after its complete runtime lock has been installed. Do not omit declared dependencies, add a minimal extra, copy a second supervisor or create another package. Install the required binding modules and support files into the runtime using the same existing helper pattern. Capture the complete build source closure/image identity and test installed origins. No runtime /workspace source mount remains. Combining locks must not upgrade scientific versions silently. Source N6.

Copy the donor supervisor's separation of lifecycle observation from scientific wire handling (source P1). The permanent supervisor runs a serial accept → fixed child spawn → wait/reap → scratch cleanup → retirement reply → accept-next loop. A normal operation does not restart the container: Docker restart/backoff is for abnormal container exit, not the ordinary throughput loop. Accepting a new operation does not retry an earlier one. Keep the lifetime lock held throughout the supervisor's lifetime and inherited by its one active child.

Separate lifecycle and scientific channels: one accepted control connection owns one operation. The supervisor creates a scientific socketpair, passes the child end and operation identity to the fixed native child, and sends the provider end as exactly one SCM_RIGHTS descriptor with its bounded startup greeting. The greeting identifies generation, admitted lock and operation. The child alone reads scientific requests and emits result/refusal frames. After the bounded retirement sequence, the supervisor sends the correlated lifecycle reply with the separate observed facts on the control channel. It never reads scientific payloads. A private prepared handle still authorizes only that operation's execution.

This separation is necessary: the current science framing is length-prefixed. If a child dies during a frame, appending retirement to that damaged stream cannot reliably establish its boundary. The control connection remains independently readable. This adapts P1's protocol-blind process ownership and S1's socketpair separation; descriptor transfer is new, explicitly qualified work, not an existing donor guarantee. Source N7.

Receive the descriptor with bounded recvmsg before any asyncio stream buffering can consume its ancillary-data byte. Admit exactly one AF_UNIX stream descriptor with close-on-exec reception; refuse ancillary truncation, unexpected control data and wrong descriptor types, closing every received descriptor on rejection. Transfer rights once even if greeting bytes need partial-write continuation. The supervisor closes scientific endpoint duplicates after spawn/transfer. The child inherits only its science descriptor and lifetime lease, never control/listener or provider-side endpoints. Interrupted socketpair/spawn/greeting/transfer setup remains owned and reaps any child before reporting completed cleanup. Primary API reference D2.

Observed scientific startup: the fresh child sends a binding-owned startup message on the science channel before preparation. Move the existing worker_main.py::_prediction_handler Python/distribution observations from its metadata file into that message. The client admits them into NativeAttemptFactory._prediction_runtime and the existing runtime_expectations before the current dependent Attempt setup/START boundary. The generic supervisor neither reads nor invents these facts. A replacement runtime mismatch still blocks prediction. Model/scaler byte verification and actual load observations remain at the release owner; startup does not load models to manufacture readiness. Startup failure retains its scientific cause and still requires independent lifecycle cleanup. Source N1.

Bounded control output: there is one bounded greeting and one bounded retirement reply, not an output queue. Use deadline-bounded socket writes within the remaining startup/cleanup budget, including partial-write continuation; no write may wait indefinitely for a connected reader. Failed startup delivery owns and reaps any child it created. Failed retirement delivery after known child/scratch cleanup closes control and reports the owned delivery failure through Docker logs; the supervisor may then accept another operation. The original client's obligation remains uncertain until its bounded recovery settles it. Message facts, successful receipt and client-side grant release are three different observations. Lost receipt does not undo supervisor-known cleanup or require holding the entire healthy runner. Qualify a non-reading owner without adding an output scheduler.

Lifecycle frame encoding/admission has one shared owner in provider-core; scientific frame semantics remain in the binding. The minimum shared API consists of the fixed-child session supervisor, greeting/retirement facts and their bounded client admission. No generic RPC dispatcher, arbitrary argv request or second molecular parser is needed. Two fresh operations and a failed/retired operation are the first installed package integration contracts.

Worker lifetimes and scientific identity

A scientific result is not retirement. The client receives its result/refusal and the supervisor's independent retirement reply. The reply carries separate facts for child reaping and operation-scratch release. A successful retirement requires both; child exit status alone does not establish scientific success. The core AnalysisSession still retires its owned native work, local I/O and source grants before the Attempt can complete. The successful path does not leave an inference child alive. Retain original primary and cleanup failures separately if the scientific reply arrives but retirement does not.

Connection owner: retain NativeAttemptFactory's existing ownership pattern: it opens one operation session, then transfers the client and retirement owner to the AnalysisSession/source operation. It retains incomplete setup only. There is no service-lifetime scientific connection or new warm-worker manager. Completed handles are released, not appended to a lifetime history. The existing process.py::_settle_borrowers ordering stops/settles controllers, then the factory's incomplete obligations, before storage/transports close. Failure stays attached to its actual session/factory and reaches the existing process error renderer. Source N5.

Cancellation: immediately half-close the control socket's write side and retain its read side for retirement. Owner hangup causes the protocol-blind supervisor to stop/reap the child; it sends retirement only afterward. Settle the original scientific and control I/O owners, including partial frames and a result racing cancellation, before releasing grants. WorkerClient.close closes science only; the operation owner separately signals/waits on control. Full control loss can prevent acknowledgement and must retain uncertainty. Do not cancel and forget original readers or wait for hung science before requesting termination.

Operation scratch: the supervisor owns one current operation directory under an explicitly bounded, container-private tmpfs and supplies it as the fixed child's working directory/TMPDIR. The lifetime-lock file is outside this tmpfs on its persistent private local mount. Source bytes enter through the scientific channel; no host source tree, credential, durable store or output archive is part of scratch. After child reaping, remove the owned directory without following links outside it, then acknowledge successful retirement and accept the next operation. All scratch cleanup consumes the same finite cleanup budget.

Scratch admission has distinct owners: the finite host deployment command admits the exact container's Docker tmpfs declaration, image and grants. A bind-backed directory fails that mount-kind admission even if private, empty or located on a host tmpfs. The shared supervisor owns safe use of the granted root: hold a no-follow directory descriptor, check owner/mode and require an empty operation root at startup. Its correlated greeting reports those local readiness facts alongside generation and lock identity, not an independent mount-type attestation. The trusted Engine preserves its admitted configuration across restart. A runtime mountinfo parser would duplicate the host's decision without proving old-mount destruction, so it is deliberately excluded. Source D3.

The binding consumes this supervisor observation and the qualified Engine/lifetime contract. It does not independently observe destruction of an earlier mount. After old lifetime holders have stopped and the successor's declared private tmpfs has been admitted, old scratch is no longer granted or reachable to supported native work. This is not a secure-erasure claim. The installed witness must leave scratch remnants, crash the supervisor, observe actual Docker restart and fresh-root admission, and separately prove the host refuses the wrong scratch mount. No host daemon, mount registry or path-name-only proof is added.

If scratch removal fails after reaping, report “child stopped; scratch cleanup unconfirmed” with its bounded owned cause. Preserve the successful reaping fact, hold the client's remaining grants and stop ordinary admission. The supervisor exits so Docker tears down the private tmpfs and restarts the container. A successor needs both qualified lifetime-lock acquisition and admission of the fresh scratch root in the exact declared tmpfs/runtime before treating old runner scratch as no longer granted or reachable to supported native work. Lock identity alone does not prove filesystem cleanup. No per-operation scratch manifest or crash-cleanup registry is introduced; the supervisor owns its current directory and container teardown owns crash remnants. Controller-side source and store grants still need their own owners' settlement.

Provider dies while the runner survives: the supervisor observes control-stream closure, stops and reaps the active child, settles its scratch, then releases that operation session and accepts another. The new provider cannot bypass this ordering by reconnecting. Failed retirement keeps that runner unavailable and the obligation retained; accepting a second owner is forbidden.

Runner/supervisor dies: the proposed replacement barrier extends P1's existing exclusive lifetime lock. Keep one private lock file per fixed runner role on a local filesystem. Normal start, stop, restart and replacement preserve the same file/inode; neither supervisor nor cleanup unlinks or replaces it. Admit the file through the donor's no-follow, owner/mode and descriptor checks. Pass the held lock descriptor into the fixed native child; both retain it until process exit and close descriptors rather than explicitly unlocking. A successor supervisor may publish its handshake only after acquiring that same exclusive lock. Primary lock-semantics reference D2.

Keep each role's lock and socket in secrets/deployments/<name>/runtime/<role>/, outside immutable bundles, temporary socket trees and container tmpfs. Install creates these private leaves and lock files once under the deployment command lock; later commands validate and preserve them. Mount only the role leaf into its runner read-write and the provider read-only, never the parent secrets directory. Use the admitted nonroot caller identity. Supervisors may replace their owned socket, not the lock inode. Ordinary install/up/down/restart never delete these leaves. Explicit decommission may remove them only after all exact owners and outstanding Engine effects have confirmed retirement; no routine garbage collector is needed.

The supervisor is the container init's main child, following the donor's init: true posture. Supervisor death must make init exit; container teardown kills the remaining namespace processes, and Docker then restarts the runner. The inherited lock blocks overlap during that teardown. The installed qualification must demonstrate this whole path with a blocked/TERM-ignoring native child, not merely show a successor waiting forever for its lock. No extra child parent-watcher, host monitor or provider engine grant is introduced.

The binding associates its active operation with the original owner session and admitted lock identity. A trusted successor handshake after exclusive acquisition of the same inode establishes that the previous protected process scope no longer holds its lifetime lease. Together with settlement of the binding's own I/O and confirmed runner-scratch release under the recovery contract above, that permits release of the old operation's grants. It does not supply a waitpid exit status, completed result or permission to retry. Missing/mismatched lock identity keeps the old obligation held. The core contract requires confirmed stopped work and released grants; it does not require inventing an OS exit code. Source N5.

Lost-control recovery owns its new session: obtaining that handshake also creates a fresh operation and scientific child. The existing factory must retain the entire connection/descriptor handoff as incomplete recovery setup before admission. Validate role, lock identity, supervisor generation and fresh operation identity. In the same supervisor generation, the serial loop accepts another operation only after reaping the previous child and cleaning its scratch. In a successor generation, exclusive acquisition of the same lifetime lock and fresh declared tmpfs/scratch-root admission supply that barrier. Neither path recovers an old result or authorizes scientific replay.

Send no prepare/execute or other scientific command to this proof session. Immediately half-close its control channel, confirm its fresh child's retirement, confirm scratch release and settle both the original and proof-session I/O before releasing the old grants. Failed proof-session cleanup remains the factory's owned obligation and blocks replacement. Use the remaining cleanup budget, without resetting it or entering an unlimited reconnect loop. This adds no registry or second startup protocol. Test zero scientific commands and retained ownership when the proof child cannot retire.

This inherited-lock extension is proposed work, not a qualified donor feature. The current P1 lock is supervisor-owned. T1 must test supervisor death with a live TERM-ignoring child, refusal of takeover until child exit, the successful successor handshake, and refusal after lock replacement. The initial native contract is fixed single-process workers with library threads. Inspected NMRNet scientific/provider Python has no direct subprocess, multiprocessing or DataLoader launch, but that source observation does not establish the behavior of all installed native libraries. Qualify the supported installed runtime before accepting the contract. This is a supported pinned-runtime qualification assumption, not a runtime detector that can refuse arbitrary library descendants. The existing image/source/lock admission verifies the selected closure; it does not prove that closure's process behavior. A worker-code or native-dependency change requires review and the maintained installed lifecycle lane before its qualification can be carried forward. That lane observes the exercised native paths; it cannot prove absence of spawning on every unexercised library path. Unexpected descendants fail the declared lifecycle qualification and require a narrower supported runtime or an explicitly reviewed lease-preserving design. Do not add a process scanner, second proof registry or general descendant manager. Failure to establish the supported assumption blocks T1/T3 closure rather than weakening retirement.

Existing scientific lifetimes: construct fresh SelectedReleaseModels, PredictionSession and ReleasedPredictionSession state in each native child. Keep the existing verify/load/release sequence for every head. Do not reset the old session's one-execution flag. New images or model selection require explicit deployment replacement, not mutation of a running session.

This is the complete intended end-state, not a temporary cold-worker mode. It removes host per-Job container creation/retirement and the permanent host monitor while keeping the native process lifetime close to the qualified implementation. The first draft required a one-head cache; the next proposed process/import reuse. Independent clarification exposed extra control and release semantics for those optimizations. They are not necessary to achieve the deployment goal.

Deferred optimization: process/import reuse or decoded-model caching requires a separately justified workload and measured benefit. A later proposal must specify quiescence, warning/artifact provenance and resource bounds. Repeated mixed H/C workloads would evict/reload a one-head cache anyway. This migration adds no reuse selector, dormant cache or warm-request protocol.

RDKit gets a new child for each parse/generation operation, including each correction attempt. Use bounded immutable bytes and fixed installed handlers; do not share a host source tree between Jobs. Scratch is private and bounded, and no new operation can use it until the previous child is reaped. A fresh child in a persistent container is not a fresh container or a hostile-code sandbox. Persistent runner isolation is an intentional change; document the common UID, filesystem grants and the supported untrusted-data threat model accurately.

Failure, recovery and diagnostics

Preserve the existing pure-prediction retry: at most one replacement execution in the same Attempt, only for the failure classes admitted by session.py::_predict. Require confirmed retirement, identical retained request bytes/model selection, new worker-local preparation, remaining budget and the existing diagnostic observer. Protocol/admission errors, cancellation and observed stops do not become retryable through this migration. Container availability recovery and this operation retry are separate decisions. Source N3.

Idle versus work deadlines: the supervisor can wait indefinitely for an operation connection, without an idle scientific child or a daemon-work timeout. Once attached, the operation has a finite outer lifetime cap from its trusted role profile. Preserve the provider's finer phase deadlines; on expiry it half-closes control and observes retirement. The supervisor's independent cap also stops a child if its owner stays connected but stops making progress. If prediction allocation remains before interpretation, derive that cap to include the admitted pre-science preparation/interpretation wait; do not apply a short prediction-only receive timeout during that wait. No arbitrary Job value selects the command, resource grant or an unbounded deadline.

The child's initial receive must use that compatible bound too. Binding startup composition, using the existing NativePolicy/configuration and worker_main handler responsibilities, derives the fixed prediction profile before launch. Its child io_seconds covers the admitted allocation-to-START API/store/retry work, preparation/interpretation window and native work; the supervisor owns the corresponding absolute lifetime cap and cleanup margin. Pass the derived value in the fixed child argv. The first nmrnet_inference.worker.serve_connection receive cannot retain the shorter prediction-only timeout while valid interpretation is still running. Client phase deadlines and existing overall phase budgets remain unchanged; repeated messages do not restart the supervisor's absolute cap. This is a binding-owned profile correction, not a new timeout framework. Test a legitimate pre-science delay longer than prediction_seconds, followed by successful prepare and execute, and separately prove the active prediction deadline still stops work. Source N1; source N7.

Start-to-finish budgets include queue/readiness, parsing/generation, inference, any admitted replacement and result handling. Derive finite outer deployment/test waits from their owned inner budgets, with explicit cleanup grace. Do not reuse one arbitrary numeric bound for bytes, calls and seconds. No unlimited daemon-work timer is required; individual operations and shutdown remain bounded.

Both process streams go to Docker logs. For the portable end-state, use the sibling Compose json-file policy with explicit rotation (max-size: 10m, max-file: 3) per service, preserving mode: non-blocking and max-buffer-size: 1m from the current logging owner. Nonblocking delivery prevents log backpressure from stopping lifecycle progress, but drops new messages when that buffer is full. Durable outcomes and effect receipts retain their independent authority. Delivery semantics: source D1. Use this per-container policy rather than requiring host journald. Retain existing redaction, safe owned causes, request/Attempt correlation, primary-versus-cleanup distinctions and actual API acknowledgement facts. Do not restore application log dump files or generic exception stringification. Source M1; source N4.

Log rotation is a finite diagnostic window, not scientific preservation. Restart of the same container retains its log history within that window; removal/recreation can lose it. Stop preserves containers for inspection; explicit replacement/removal documents that consequence. Durable result/recovery records remain in their existing store and selected qualification evidence follows U19. There is no new log archive service. A healthcheck reports usable readiness; Docker restart is driven by process exit, not merely an unhealthy flag. Source D1.

Deployment and operator contract

make provider/deployment/install DEPLOYMENT=<name>
make provider/deployment/up      DEPLOYMENT=<name>
make provider/deployment/status  DEPLOYMENT=<name>
make provider/deployment/logs    DEPLOYMENT=<name>
make provider/deployment/down    DEPLOYMENT=<name>

Proposed final commands, not current instructions. Install freezes and admits a configuration bundle without starting. Up selects that complete bundle and starts the stack; it neither builds nor downloads. Down stops provider admission first, keeps workers/proxies available for bounded drain, then stops them after confirmed provider exit. Ordinary down preserves containers, bundles and durable state. Explicit reconfiguration uses down/install/up. It never starts a second provider while the previous owner may still be active. Source S1.

Concurrent commands: every finite install/up/down command takes a nonblocking exclusive flock on an open descriptor for the stable private secrets/deployments/<name>/ directory before reading the selected bundle or issuing Engine effects. Create it only below validated private parents, reject symlinks, and verify the held directory's identity, owner and mode. Preserve that directory inode across commands. Contention fails before effects; it must not queue an old up request that starts the service after down returns. Hold the descriptor through publication and bounded command settlement. This copies the existing host directory-lock mechanism, not an additional lock-holder process or registry. Read-only status/logs do not hold the mutation lock. The private caller-owned directory and trusted Engine administrator remain the trust boundary; this does not contain a hostile same-UID administrator. Source N1, provider_workers_open; source P2, _locked_parent.

Find the actual stack independently of the selected bundle. Keep the Compose project identity stable for the deployment and canonical checkout, not the bundle version. Adapt S1's inventory: inspect actual full container IDs, exact project/service/oneoff/working-directory labels, fixed roles and duplicates. Add the deployment identity and retained bundle name as fixed labels. Resolve that name only below this deployment's admitted bundle root, never as a caller-supplied host path. Retain the normalized Compose plan used for creation inside the existing immutable bundle. Compare each actual container's image and grants against that original plan before controlling it. Engine inspection owns live IDs; no second live-container manifest is introduced. Source S1, ComposeProject.inventory.

Thus down still finds and stops stack A after install selects bundle B. Install refuses running or transitional ownership and never implicitly shuts down work. After an explicit acknowledged down, it may fence remaining late-stop restrictions by removing only already-stopped original IDs, warning of Docker-log loss and confirming absence. A stopped snapshot without an exact-ID/grace stop acknowledgement is insufficient; request explicit down instead. Thus down/install/up does not need an extra startup to clear bookkeeping. Otherwise install preserves A's stopped containers and bundle while selecting B. A unique admitted subset left by interrupted creation can be torn down; foreign/duplicate roles, missing original plans or mismatched grants are refused with actual IDs, without guessed stop/delete commands. Preserve and settle original receipts for uncertain Engine effects before any competing mutation. Releasing the finite command lock does not establish that an interrupted Engine request finished.

Interrupted commands must remain discoverable. The current deployment helper creates a random operation directory and reports its location to its caller; that does not supply cross-command recovery. Place the finite mutator's existing private receipt root at secrets/deployments/<name>/operation/ instead. Under the deployment lock, every install/up/down checks this single current root before effects. Durably publish the admitted command, bundle and pending phase before dispatching an Engine mutation, including creation of a contained helper. Use atomic publication and persistence of the file and containing directory; inability to preserve intent blocks dispatch. Retain original bounded response/client receipts at that owner. This is a required discovery extension, not an existing implementation guarantee. Source N8.

Recover ordinary lost replies without requiring manual repair. Original responses are evidence, not the only possible authority. Under the same deployment lock and finite command budget, reconcile the retained operation against exact Engine identity and grants. Give each create a unique predeclared name for its immutable generation/role; subsequent lifecycle effects target full IDs, never reusable names. Recover individual phases, not an interrupted composite compose up that might recreate or start services again. Primary Engine sources D4; owned-name reconciliation precedent M3.

Do not overwrite the current operation root during recovery. Persist recovered observations and any remaining restrictions there; no second registry is needed. Same-intent readiness may be reported while an older start/stop restriction remains recorded, but an opposing transition must complete the request or remove its old target first. Missing PID, CLI timeout, daemon restart or an empty inventory alone cannot clear uncertainty. Unknown composite effects, unmatched identities and exhausted reconciliation budgets remain held with the safe reason and receipt path. Clear the current root only after all effects are settled or fenced by confirmed old-ID removal, following the existing evidence-retention policy. No append-only journal, background recovery process, Job retry or model replay is introduced.

Recovery removal is an explicit exception to log preservation. Ordinary down preserves stopped containers. When an unresolved old-ID start/stop would otherwise conflict with shutdown or subsequent up, recovery may remove that exact container after stopping it, confirming absence before proceeding to dependent teardown. In particular, fence a pending provider start before stopping its runners or proxies; an exited snapshot alone does not prevent a late provider restart. Report the affected ID and loss of its Docker logs before removal; retain control receipts and durable Attempt state, without a log-file dump. Never remove a container merely because a response is missing if observed-state recovery already suffices. Qualify this exception and its late-request ordering on the supported Engine version.

Down confirms container ownership, not just provider PID exit. Issue Docker stop for the exact admitted provider container with the declared drain grace: TERM permits application settlement, then Docker forcibly kills it if that grace expires. The finite Engine-client budget must cover the grace and response margin. This handles the provider's deliberate wait when cleanup remains held; down must not wait forever for voluntary exit. Inspect the exact container for confirmed exit and non-restarting state, and settle or fence any pending start as above, before the same finite host command stops and observes every admitted runner, then the proxies. Preserve stopped containers and their logs except for the explicit recovery-removal case above. Reuse the current controller-before-native teardown order and exact retirement checks, not its container-removal step. Source N8.

Report forced physical shutdown separately from a clean application drain. Confirm their stopped/non-restarting state and release of their admitted ephemeral grants through the qualified Engine teardown contract. Only then return zero for physical service shutdown. If physical exit, a conflicting Engine effect or grant release remains uncertain after bounded recovery, return nonzero with the exact held IDs and causes; subsequent up refuses overlapping ownership. A clean physical stop can still leave unresolved Attempt or API outcomes. Report those distinctly and preserve the store: its existing recovery owner, not down's success, decides what may be replayed or admitted next. Qualify provider cleanup remaining held through the drain deadline, forced exit, then runner teardown and a subsequent up that cannot bypass unresolved ownership.

Startup result: up returns zero only after local component readiness and an accepted API Hello within its finite startup budget. If local components are ready but the API remains unavailable, it returns nonzero with “running; API announcement pending” evidence and leaves the stack running. The existing service recovery owner continues bounded/backed-off Hello attempts; no host retry or teardown resets that work. Status subsequently reports recovery from the provider's current observation, never by inferring acceptance from logs or a live container.

Read-only status surface: adapt Magnet's atomic bounded status projection and separate healthcheck/status reads, not its multi-lane status board. The provider process projects its existing startup, Hello, held and stopping facts into one private current record under container-local /run tmpfs. Each supervisor similarly projects listener/idle/busy/held facts. Healthchecks read those records without acquiring the exclusive operation socket or starting science. A runner's status is listener/runtime readiness, never proof of loaded models. Source M2.

The projection contains only admitted state, process-start identity and monotonic observation time; it has no history, transcript, credential or scientific arrays. Initialize unready at process start before ordinary startup work. The healthcheck verifies the live process identity and record age in the exact admitted container; a previous process's record cannot be reused. Refresh at most every 10 seconds and expire after 30 seconds, allowing three refresh periods. These are explicit health-observation constants, not new operator selectors or scientific deadlines. Update also when an owned status fact changes. One bounded lifecycle-owned refresh task/loop expires on shutdown; there is no separate monitoring daemon.

Atomic replacement prevents partial records; it cannot guarantee removal of an old ready record after a write failure. On projection failure, preserve diagnostics, stop refreshing and settle the process; readers refuse a mismatched/dead process or expired observation. Until expiry they may see a last-known state up to 30 seconds old, which status must label with its observation age. Up and healthcheck use this same bounded observation contract. It is never authority to dispatch/replay work: actual service admission and recovery still consult their in-memory/durable owners. Test failed replacement while an earlier ready record exists, including held cleanup.

Repeated up: for the same admitted installed plan and already running stack, inspect and wait within the startup budget only. Do not create, restart or replace containers, reset recovery, or submit a probe Job. Return zero when all required readiness facts hold, otherwise the explicit pending/failure result. First apply the bounded phase recovery above when the original command was interrupted; this is not permission to replay the whole up operation. A different plan or partial stack with running, transitional or unresolved conflicting ownership is refused for explicit down/install/up; foreign or ambiguous ownership always requires resolution. After down confirms retirement and install selects B, up(B) may replace uniquely admitted, confirmed-stopped A containers, including a stopped subset from interrupted creation. Validate them against A's retained plan and settle outstanding Engine effects before replacement. This replacement removes their Docker logs as documented above. Down uses the actual admitted container ownership, not guessed names or a newly edited configuration.

Keep the public provider definition, ignored machine settings, ignored secrets and frozen bundle structure already implemented. Consolidate the old service.env host settings into the one deployment settings owner; no permanent systemd-profile projection or second UNIT identifier remains. API identity is a configuration fact, not inferred from the directory name. Preserve literal command-line-only Make captures, hierarchical names, honest help and .PHONY declarations.

The host wrapper uses Bash/engine commands only; configuration parsing and plan rendering run in the existing contained helper. The rendered Compose document is derived from admitted inputs, with literal values protected from interpolation, never a second manually edited configuration authority. Fix image IDs, installed source manifests, model declarations, resources, service membership and grants; inspect the launched containers against that plan. Do not trust caller JSON as observed runtime attestation. Source S1; source N1.

For a new stack, create the fixed containers first and inspect their actual image/configuration/grants against the admitted plan before starting the provider. Start proxies/runners and admit their runtime readiness, then start the provider and observe Hello. This keeps API effects behind deployment admission. Existing same-plan healthy stacks follow the inspect-only repeated-up contract; explicit replacement first confirms all old owners stopped. No extra host lifecycle service is required for these finite stages.

Container users must match admitted secret/store ownership. Retain the credential 0400/0600 checks, private directories, read-only secret grants, read-only roots, dropped capabilities, no-new-privileges, finite CPU/memory/PID/tmpfs budgets and zero network for scientific work. Establish private umask before creation and verify privacy inside the mounted subtree. Never fix an ownership mismatch by making a private key group/world-readable.

Restart policy is unless-stopped for persistent services. Operator stop remains respected. Docker daemon startup is a host prerequisite; no NMRNet unit configures it. Startup checks separately report container posture, usable worker readiness, proxy readiness and API announcement. Model-load readiness must not silently change the existing “admit input before model decoding” order: report channel/runtime readiness separately from the first actual model use. Status does not submit a probe Job or charge a model call.

Keep a single provider/store claim, existing retention owners and bounded active operation state. Remove the obsolete host allocation lifetime counter. A healthy completed request frees its slot without manual history deletion; unresolved state is retained and blocks only what its ownership requires. Continued operation still needs finite disk/retention policy, not a larger default counter. Source U27.

Resource observations must retain their actual scope. A persistent container's cgroup peak and cumulative counters describe its lifetime, not one Job. Do not relabel them as per-operation measurements. Use a clean disposable instance for the focused capacity qualification; label any before/after counter deltas as such. Correlate routine Docker diagnostics with the operation identity inside the stable container rather than relying on the former one-container-per-operation convention.

Transition only: implementation and cutover

The following steps are temporary work sequencing, not additional supported production architectures. Use a focused implementation branch with meaningful commits. The existing deployment remains the sole active owner until cutover. Temporary adapters/test entrypoints must have named removal points; the completed change must not ship two launch modes or an allocator-mode selector.

T2 changes the launch/transport boundary and therefore includes request-state isolation and repeated-use proof. Process reuse and decoded-model caching are not completion gates. A useful partial commit is a development checkpoint, not end-state completion. No optional package extraction or new generic engine framework is a prerequisite for connecting the actual stack.

Cutover safety: do not run old and new providers against the same credential/store concurrently. If old retirement is unconfirmed, preserve it and stop cutover. First startup failure leaves the new containers and original store intact for diagnosis. Returning to the previous revision is an explicit deployment action only after the new owner is confirmed stopped and stored records are shown compatible; never automatic rollback that could repeat API/scientific effects. No store-format change is planned.

Deletion list: remove the production provider_host_monitor/provider_workers_service loop and its allocation request/receipt protocol; the custom systemd template and service installer/control targets; UNIT selection; ordinary runtime journalctl dependency; host-allocation counters; tests asserting those obsolete implementation details; stale operational guidance. Preserve historical notes and receipts. One-shot build/acquisition/test helpers may retain engine ownership where necessary; do not delete useful finite qualification or image-admission code merely because it shared a former launcher.

Focused tests and acceptance evidence

Write behavioral tests before each corresponding implementation slice. Follow the existing scope/purpose structure: package unit normative/adversarial and contract tests; binding unit tests; installed integration tests; narrowly scoped contained deployment qualification. Keep native science and paid/live API lanes explicit and outside default offline aggregates. Add maintained Make lanes only where an actual repeatable workflow needs one, with finite deadlines and cleanup.

1. Session and ownership: a second owner is refused; disconnect reaps the active child; cancellation with nested operation/cleanup failure reaches the real protected renderer. A held child blocks reuse and retains its input. A stale socket, prior prepared token and old operation reply cannot authorize work. Supervisor death exercises the inherited-lock replacement barrier, preserving uncertain scientific outcome separately from confirmed lifetime release. Kill the native child midway through a scientific header and body: independent control retirement still arrives and the scientific failure remains visible. Exercise partial descriptor transfer, excess/truncated ancillary descriptors, cancellation and failed spawn; no leaked FD or unowned child may remain. Lost-control proof sessions must issue zero scientific commands and retain both old and newly acquired obligations until complete settlement. Fail scratch deletion after child reaping and distinguish stopped work from unreleased resources. A non-reading owner must not indefinitely block greeting/retirement output or leave a child unowned; observed cleanup and delivery failure remain distinct.
2. Two successful operations: every role uses distinct reaped children, while its container/supervisor stays the same. There are no old result/input references. Model-load observations retain their existing per-head meaning. Include several short RDKit operations to prove ordinary throughput does not depend on container restart/backoff. A short bounded serial run checks active-handle/scratch counts return to baseline; no arbitrary RSS-equals-zero assertion against Python/Torch allocators.
3. Scientific repeated use: successive same-head and mixed H/C cases preserve ordered predictions against fixed independent existing outputs, exact artifact identity and truthful warning provenance. Never regenerate expected values or tune tolerances. Measure the existing 128-atom case under unchanged admitted limits because image dependencies and container resource composition change; avoid rerunning every unchanged format/model case. The installed child's startup observations must reach runtime_expectations unchanged; an admitted replacement with mismatched observed runtime is refused before prediction.
4. Failure and replacement: injected execution failure, bounded OOM stand-in and actual process exit prove retirement before replacement. At most two pure executions per eligible Attempt; zero repeated interpretation/generation. Distinguish an actual container OOM observation from an injected exception. A standalone failed Attempt does not require manual restart before the next Job.
5. Publication/restart: lose a completion acknowledgement after retaining the outcome, restart the provider, then use existing exact recovery. Counters show zero new scientific/model effects. Restart the runner during active work; retained uncertainty is diagnosed and new work waits for known ownership.
6. Actual deployment grants: no engine socket; no runner network or credentials; correct image/UID/mounts/resources; synthetic secret read succeeds only for the admitted user and stays read-only; mounted scratch stays private under permissive host umask. Test stale/malformed configuration before engine effects.
7. Deployment serialization and retained identity: overlap install/up/down and show contention causes zero publication/Engine effects, no mixed generation and no delayed up after down. Select bundle B with stopped A still present; down must inspect A against A's retained plan and up(B) may replace A only after confirmed retirement. Exercise partial creation and refusal of ambiguous labels or a bundle name escaping its root. Verify native lock inodes survive configuration changes and runner replacement; unresolved ownership prevents deleting their directories. Kill the host command before dispatch and after handoff but before response publication; the next mutator must find the retained operation without a caller-provided path. A late create must not race new dispatch or be ruled out by an earlier empty inventory. Failed intent publication causes zero Engine effects. A released command lock must not bypass an uncertain Engine effect. Also lose replies after successful create/start/stop and prove bounded state recovery without manual repair. Refuse a foreign occupant; never restart a completed finite helper. Delay an old stop past a new up, and an old start past down: require original settlement or old-ID removal before the opposing transition and before dependent grant teardown. Verify the replacement has a different ID and cannot receive delayed old effects. Test ordinary log preservation and the reported exceptional removal separately.
8. Operation and diagnostics: deployment start returns, status is read-only, down drains in order and forcibly stops a provider held past its grace before retiring runners. Unconfirmed stop preserves grants and blocks progress; confirmed physical stop does not clear durable Attempt uncertainty. Logs contain safe primary/cleanup/API evidence, and configured canaries remain absent. Actual rotation settings are inspected; no file dump path returns. A failed start is diagnosable without inventing a healthy API offering. Verify restart behavior in a disposable engine environment; do not restart a shared daemon merely to qualify this repository.
9. Final service witness: one connected transaction through the final Compose stack, followed by another Job and bounded recovery where required. Record image/source/config/model identities, request/result association and what was actually observed. Scripted interpretation proves plumbing; actual API and actual selected-model interpretation are separate claims, not inferred from TLS.

Batch coherent reviews and validation. After each meaningful slice, review its changed ownership, failure recipient and forbidden effects against the relevant U/A items; use independent audits when implementation is delegated. Do not repeat a completed scientific matrix for shell wording, logging or unchanged admission code. Existing evidence is reusable only for genuinely unchanged scientific/runtime facts.

U/A review and evolution checks

The governing meanings remain in 013. This plan groups them by what the implementation must prove; it does not replace them with a smaller checklist or dilute a now-red check.

Before closure, walk these ordinary changes through the finished code:

• Deploy on a second Linux host: change private paths/interface/UID/resources, admit the supported images and credentials, and use the same targets. No Python source edit, unit installation or relaxed credential permissions.
• Change the selected checkpoint or image: freeze a new admitted bundle, stop the old deployment, replace it and observe the new identity. No stale release declaration, mixed-generation sockets or in-place edits to running artifacts.
• Add an accepted structure format: change the existing reader and its tests. No deployment service or generic lifecycle schema change for another parser in the same already admitted scientific dependency boundary.
• Adopt the small supervisor in another provider: supply its fixed child command, session contract and budgets, without NMRNet imports or chemical policy. Consumer requirements are documented; no sibling repository edit is part of this plan.
• Fail after bundle creation, container start, scientific completion or terminal publication: the actual owner explains retained state and the next permitted action. No second registry must be repaired to reconstruct authority.

Precise sources and adoption rationale

Inspected sibling precedents, requested by the user. Revisions identify the inspected source, not claims that every behavior has been newly run here.

M1 — Magnet deployment, 0619d9ffd92a4326758b442c532aa09496161098

compose/provider.yml and G01 runner serve.compose.yml: persistent provider/offline runner, private Unix channel, engine restart, read-only grants, finite resources and rotated Docker logs. Copy these boundaries, not G01's chemistry or its exact CPU/memory/UID choices.

P1 — NMRPeak, 1697dc84dd1d76ae25b04efc8183fd80614000e5

compose/provider.yml; owner_session_supervisor.py, module contract, supervise_owned_worker, endpoint ownership and terminate/reap path: donor for lifecycle-only supervision and exclusive provider ownership. Its scientific worker alone owns protocol bytes; do not port HF/CHF policy into provider-core.

P2 — NMRPeak deployment serialization, same P1 revision

provider_deployment.py::_locked_parent, _locked_deployment_state, _ensure_deployment_state_root: private deployment-state admission and held descriptor serialization. Use N1's existing Bash directory-lock mechanism here; do not introduce host Python or copy the donor's separate lifecycle-lock file.

M2 — Magnet current status projection, same M1 revision

provider_status.py::_replace_status/read_provider_status; provider_process.py healthcheck/status cases: one atomic current projection and distinct silent healthcheck/operator read. Copy that mechanism; the explicit process identity/freshness contract is local proposed work, not a claim about the donor or permission to import its status board.

M3 — Magnet owned-name reconciliation, same M1 revision

provider-deployment.sh::reconcile_owned_provider_validation_container: resolve a reserved name through exact labels, then target its full container ID. Reuse that identity check. Its absence check alone does not settle an earlier unknown create or establish ordering against a delayed opposing request.

S1 — SECS, 7356c354368f2355d3ab5cef00cf48c67cea9c2c

worker.py, WorkerClient, _serve_session, _child_loop and child-stop path: warm serial worker precedent and the rule that EOF is not retirement. deployment/compose.py, ComposeProject.inventory/start/stop: exact project/service/oneoff/working-directory label inventory, finite Compose calls, literal rendering, exact image checks and ordered stop preserving durable data. Retained bundle labels/plans are proposed local additions. Do not copy its whole relay/deployment framework.

N1 — NMRNet current host and deployment owners, 87c3badfeb4c1bb2e5336a44e7bc53c148bf9484

provider.sh, provider_host_monitor and startup/grants; provider_workers.sh; NativeAttemptFactory; worker_main.py::_prediction_handler; freeze_deployment; deployment host wrapper: map existing authority to replacement and preserve bundle/secret admission rather than create another configuration or launch owner.

N2 — NMRNet scientific lifetimes, same checkpoint

predict.py::PredictionSession, release.py::SelectedReleaseModels/ReleasedPredictionSession, worker.py::serve_connection: one-use handles, per-head loading and per-request observations. Preserve these scientific ownership rules while qualifying the changed process lifetime.

N3 — Existing pure-prediction replacement and durable recovery

session.py::_predict and _replace_prediction; 013 U27 and its 23 October refinement; 017 P3 recovery evidence. Preserve the narrow admitted retry and exact durable replay; neither engine restart nor this note creates a new retry policy.

N4 — Diagnostics and permission requirements

018 error/permission audit; 019 Docker-backed logging; current journald configuration. Preserve useful safe evidence and strict grants; replace the host journal dependency explicitly, without claiming rotation preserves logs after container removal.

N5 — Existing service settlement and shared retirement contract, same checkpoint

process.py::_settle_borrowers and run_provider; analysis.py::AnalysisSession: factory/service cleanup ordering and the binding's obligation to establish stopped work, released grants and qualified quiescence before reuse. P1's _acquire_supervisor_lock uses flock, validates the opened descriptor, never unlinks its lock file and releases by close; inheritance into the child is the explicit proposed extension, not a claim about P1's current behavior.

N6 — Installed runtime/image ownership, same checkpoint

provider_environment.py::install_provider, provider-core pyproject.toml, inference Dockerfile and geometry Dockerfile: current complete core dependency declaration, offline installation and matching Python 3.12 base. Final combined installed locks and image-owned binding modules still require qualification; Python version compatibility alone is insufficient.

N7 — Current scientific framing and channel cleanup, same checkpoint

worker_protocol.py::frame/receive; worker_client.py::WorkerClient; retirement.py::retire_worker_channel: length-prefixed scientific frames, retained original I/O and current full-channel close. These justify independent control acknowledgement and an explicit cleanup adaptation; the old close-then-host-receipt sequence cannot simply be reused.

N8 — Finite Engine commands and held provider shutdown, same checkpoint

provider_deployment.sh operation-root creation; worker_retirement.sh::worker_retirement_command/worker_container_retire; inference_native.sh local-versus-Engine retirement contract; provider.sh::provider_host_drain/provider_host_finish; process.py held-cleanup wait: bounded private receipts, explicit limits of client retirement and controller-first emergency teardown. Reuse those semantic owners; stable cross-command receipt discovery/durable intent and stop-without-removal are proposed adaptations.

U/A — Repository practices and scientific evidence

013 Use/Avoid; 003 and 017 evidence limits: reuse existing fixtures, fixed references and recipient proofs only within their actual scope. Do not reinterpret historical qualification as a new warm-runner run.

D1 — Docker primary documentation

Start containers automatically (inspected 23 October 2026): engine restart policy, manual-stop behavior and the conflict caused by combining container restart with a host process manager. Concrete log rotation settings are copied from the pinned Compose source M1.

Configure logging: delivery mode (inspected 24 October 2026): blocking default, bounded nonblocking buffer and dropped messages under saturation. Preserve N4's existing nonblocking delivery while replacing host journald retention with explicit container rotation.

D2 — Descriptor transfer and lifetime-lock semantics

Python 3.12 socket.recvmsg and socket.sendmsg: ancillary descriptor receipt/transfer and message flags; Linux flock(2), DESCRIPTION: duplicated/inherited descriptors share the open-file-description lock, which is released by explicit unlock or closure of all duplicates. Inspected 23 October 2026. These support the proposed mechanism; they do not prove descriptor discipline, child-tree behavior or cancellation in this implementation. The versioned Python documentation currently renders 3.12.15; this is not a runtime upgrade from the pinned 3.12.12 images.

D3 — Scratch mount lifecycle and observation

Docker tmpfs mounts: container-stop lifetime and distinctions from persistent bind mounts/volumes; Linux proc_pid_mountinfo(5), DESCRIPTION: inspected while considering and rejecting duplicate runtime mount-type validation. Inspected 23 October 2026; the manual page is from Linux man-pages 6.19. Host admission owns mount kind; supervisor checks only safe use of its held root. Neither is an independent proof of old-mount destruction or secure erasure. Docker also notes that tmpfs usage consumes the container memory limit; scratch capacity must remain inside the admitted resource budget.

D4 — Engine operation semantics, Moby v28.3.0 / API v1.51

ContainerCreate name input, create outcomes and start/stop outcomes; start.go::validateState/containerStart; stop.go::ContainerStop/containerStop; delete.go::containerRm and cleanupContainer (inspected 24 October 2026): already-running/stopped replies, start's locked state check, stop continuing after cancellation, and removal of the exact old object. These sources justify investigating bounded same-intent recovery and old-ID removal before conflicting transitions. The recovery policy above is our design inference, not an Engine guarantee that arbitrary concurrent commands converge. Installed-version and delayed-request witnesses remain required.

Plan status and implementation evidence

Plan authored against the stated checkpoint. Implementation has started with the shared lifecycle slice below; production runner/Compose migration remains incomplete. Record subsequent slice commits, focused test results, cutover observations and remaining limits here. End-state closure requires the completion contract above, not merely completion of the first transition slice.

T1, 24 October 2026: added the installed shared runner supervisor, bounded lifecycle/descriptor protocol and synchronous operation-session owner in provider-core. The binding still owns the upcoming asynchronous task handoff. The supervisor admits a pre-existing lifetime lock, inherits it into each fresh child, separates control/science, bounds operations and control delivery, reaps before descriptor-relative scratch deletion, and stops admission on failed cleanup. Protected failure projection retains stopped-child/unreleased-scratch facts.

make test/unit/provider-core/runner WHEELHOUSE=<wheel-pool> passed 13 installed-package tests: successive fresh children; damaged scientific frames with independent retirement; TERM-ignoring child and connected-owner cap; extra descriptor rejection without leaks; absolute partial-frame timeout; actual child descriptor/lease observation; supervisor death followed by child-held lock and a successor greeting; wrong role/inode refusal; and received scratch-cleanup failure. Two independent source reviewers found no further actionable T1 issue after repairs. The successor fixture explicitly supplies fresh scratch; these tests do not prove Docker init/namespace teardown, scientific runtime descendants, asynchronous task settlement or lost-control proof-session recovery. Those remain T2–T4 obligations.

The installed package contract lane passed 13 tests, including exact wheel payload, full dependencies, isolated imports and operational import guards. The scoped repository lane ran 50 tests and exposed four existing deployment integration failures: two launcher-fixture cases omit the deployment helper, repository input grants omit the config/deployment public roots, and legacy service help uses brace shorthand that hides its actual target names. Keep these failures visible; T3/T6 must fix their owning wiring/help while replacing the legacy launch path. No check was relaxed and this checkpoint does not close the migration.

T2 transport owner, 24 October 2026: the binding's NativeOperation retains original asynchronous opening, control retirement, proof and science-close tasks across cancelled waiters. It revokes control before joining blocked setup, uses one absolute cleanup deadline, and retains a failed proof after closing its locally owned descriptors. Lost control can be settled through one zero-command proof operation under the existing lease/serial barrier. Core connection readiness waits are bounded and limited to missing/refused private endpoints.

make test/integration/provider-core/runner WHEELHOUSE=<wheel-pool> passed five focused installed-core/binding tests: cancelled opening and retirement, cancelled proof recovery with an actual zero-scientific-input observation, failed proof setup revocation, and exhausted cleanup refusing new proof acquisition. Two independent reviewers found no further actionable transport-owner issue after deadline and failed-proof cleanup repairs. Factory/production worker/image wiring and Compose qualification remain incomplete; this is a development checkpoint.

Iterative clarification with six independently briefed 6.1-Sol reviewers at medium reasoning addressed native image installation, exact channel ownership, handler selection, idle/operation deadlines, supervisor death, status freshness, API outages and repeated up. The fresh review led to persistent supervisors with fresh children for all three fixed roles, removing both process-reuse and cache scope. A later coordinator counterexample (death during a scientific frame) led to the independently reviewed control/science split. Inherited-lock and descriptor transfer behavior remain named proof obligations, not already demonstrated facts. A further fresh review made scientific startup observations, scratch ownership, bounded control delivery and the pinned-runtime process assumption explicit. The next clarified host-versus-supervisor scratch admission, physical shutdown versus unresolved outcomes, and the prediction child's initial receive timeout. A final U15/A05 check removed the proposed runtime mountinfo parser because it duplicated host mount-kind authority without adding a required consumer decision. The fifth reviewer clarified finite-command serialization, actual-stack discovery across bundle changes and the stable location/retention of native role locks. Existing directory locking and Engine inventory supply these owners without an extra lock file, live-ID manifest or lifetime registry. The sixth found that existing random operation roots did not establish discovery after host-command death, and that held provider cleanup could prevent voluntary shutdown. The plan now requires one discoverable current receipt root before Engine dispatch and bounded forced provider stop before runner teardown. It explicitly retains indeterminate Engine effects and unresolved durable Attempt outcomes. A follow-up U27 challenge rejected the blanket original-response-only settlement rule. Primary Engine code and sibling reconciliation now ground bounded recovery from observed state, while late opposing requests require completion or removal of their old target. The sixth reviewer challenged and reread this refinement too. These are implementation contracts and evidence limits, not new runtime proof.

T2 binding checkpoint, 24 October 2026: the production factory now connects to the fixed prediction/reader/SMILES runner endpoints instead of requesting per-operation host containers. Successful handoff releases its factory references; the transferred native owner retains original setup, control, proof and transport-close tasks. Cancellation during transport adoption closes the returned writer. A departed queued caller no longer stops an otherwise healthy supervisor after its child and scratch have been retired.

Settled independent failures reach the process's existing protected operator notice recipient, including an opening failure arriving after cancellation. String facts are redacted before output and before retaining a failed output; successful resource settlement remains distinct from diagnostic delivery. Focused contained checks passed: core runner 13, native runner integration 9, native factory 11, and process/receiving reports 64. The existing 48 service tests passed after removal of the displaced native-history callback. Independent lifetime, scientific-fidelity, simplicity and diagnostic-recipient reviews found no further material issue in this slice after corrections. These are local lifecycle/binding tests, not qualification of the future Compose stack.

Native image composition now installs the complete locked core closure and binding into each role's own interpreter; isolated children need no source mount or PYTHONPATH. The baseline locates the same installed upstream package. Inference's committed selector declaration is its sole authored-input selection owner, with package roots expanded to committed leaves and missing selectors refused. Host file privacy does not have to equal Git's materialized 0644/0755 modes: only executable-bit agreement is required before the captured bytes are given their committed modes. Image construction and connected runtime proof remain outstanding at this checkpoint.

Image-input qualification passed 50 focused contained source/capture/metadata checks, 15 controller-image checks (including refusal of an installed scientific dependency conflicting with the core lock before any installation), seven runner startup checks and ten baseline model checks. No dependency pin, scientific fixture output or numerical tolerance changed. The local build pool combines existing cached wheels and the separately verified, commit-keyed Uni-Core archive; no dependency download was required.

T2 image construction, source e1984c46f00190b1dc51a3774ea62a53f7608a4b: offline maintained builds produced prediction sha256:9ac60cd489f1f2f872b2b93ccd4354ede56a153799a7c66f7f116ef8d4be7200, RDKit sha256:99e8b492d055a8c4e00937d6bd53ce11a7174f1fe32c1102060416a2f87bfb2b and controller sha256:ca94138fa7af98514bf2bc4eab0bc1698dd0ad11510ce534e129cd0acb5b55d2. make packages/import passed for the prediction image. Its build and the RDKit build also executed the installed isolated runner entrypoint's help command. These establish installed dependency/code composition, not a connected scientific transaction or deployment.

T3 plan/freezer checkpoint: the contained owner renders seven fixed roles from the actual admitted frozen configuration. It retains exact credentials, execution/model declarations, inference inputs, Compose plan, DNS configuration and source receipt in the immutable generation. One private deployment.toml owns machine settings: store location, startup byte ceiling, three explicit routes and non-loopback IPv4 resolvers when needed. The freezer no longer accepts or emits service.env or UNIT. Host selection still requires successful helper exit, confirmed helper retirement and its published bundle-name; a completion marker alone is not authority.

The plan preserves each original HTTPS authority and explicit interface; local routes require the caller-observed full network ID and complete address grants, mapped routes retain the public authority, and native/controller roles have no network. Host observation/admission of these declared grants remains required before launch. The stable project identity survives bundle refresh. Provider diagnostics and scientific scratch are private per-start tmpfs.

Ten contained plan/bundle checks passed, including immutable refresh, exact role grants, invalid credential/route/DNS refusal and explicit local networking. Independent code, containment and lifetime reviews found no remaining material issue after the proxy corrections. This checkpoint does not implement the host command's Engine inventory, request settlement or connected Compose transaction; those remain T3 work.

Stop allowances follow actual sequential service/process cleanup owners. Runner SIGTERM during greeting can spend startup twice, around TERM/KILL/scratch cleanup; each stage receives one fifth of the existing allocation allowance. Provider stop allowance includes bounded startup admission, drain, settlement observations, incomplete setup and final dependency retirement. Proxy stop allows five seconds for immediate SIGTERM/init scheduling. These are physical process stop limits, never evidence of durable Attempt completion. The finite host command must add a separate response margin and enforce provider, then runners, then proxies; the dependency graph alone only establishes provider-first.

Persistent proxy restart now holds a nonblocking flock on its private route directory before admitting/removing stale private socket/readiness leaves. A live owner prevents any removal. Bind uses umask 0177, so a crash before activation cannot strand a public socket that the next owner must refuse. FD-relative serving/publication retains the admitted directory. The 63 focused proxy tests pass, including actual child-process crash/restart, a live contender, malformed leaves, private socket birth and failed activation cleanup. Ten native runner tests pass, including a delayed replacement supervisor: the existing core ENOENT/ECONNREFUSED wait continues under the original cleanup deadline, with one retained proof task and no scientific command. No additional retry owner was added.

T3 host admission and readiness checkpoint, 24 October 2026: the finite deployment command now freezes its helper from the committed source capture, normalizes the recipe with the selected Compose CLI before sealing, and admits actual container identities, original bundle grants, image defaults, namespaces, resources, complete mounts and networks before lifecycle control. Private machine settings remain in config/deployments/<name>/deployment.toml; credentials and sealed generations remain under the private deployment root. Mutation and query helpers have separate discoverable receipt roots; queries do not take the deployment mutation lock. Interrupted-effect reconciliation is still incomplete, so this checkpoint is not ready for production cutover.

Creation proceeds one fixed role at a time, with full admission between creates; the provider is last, after its six dependencies already exist. Immutable generation-specific names supply discovery after a lost create reply. Shutdown is explicitly provider, then native runners, then proxies, retaining Docker containers/logs. New Make targets expose install, up, status, logs and down. The observed local tools are Docker Engine 29.9.2 and Compose plugin 5.5.1; Compose v2 execution has not been demonstrated. Actual connected-stack and delayed-request qualification remain outstanding.

Each role projects one private atomic current observation in its per-start /run tmpfs. The record contains role, PID, Linux process start ticks, monotonic update time and phase. Existing loops refresh every five seconds; provider owns one joined refresh task. Readers reject dead/reused processes and observations older than thirty seconds. Up/status read the current record and age rather than trusting cached Docker health. Provider readiness follows accepted initial Hello and subsequent Hello failure/recovery. Failed atomic replacement can leave the earlier ready record visible until expiry; a focused witness preserves that limitation. No readiness check opens an operation socket or submits science. Repeated up on the same running stack only observes readiness and cannot create a replacement Attempt store.

Focused contained checks at this checkpoint: runner/status 17, proxy 64, Compose plan/admission/bundle 16, deployment/client 6, installed package contract 13, provider/process reports 65. The latter includes actual periodic Hello failure/recovery changing the current readiness record. These tests cover private status replacement, process identity/expiry, acquired-listener cleanup, exact local network identity/member addresses, bounded host handoff, literal selectors, per-role startup budgets and refusal of cached health after observation expiry. Independent lifetime and containment review corrected listener cleanup and repeated-up store creation. This is local source/fixture evidence, not a connected Compose transaction, scientific accuracy rerun or deployment proof.

T3 completed-effect recovery, 24 October 2026: a failed read/readiness command no longer requires manual receipt removal when all persistent mutations have their original successful replies and every finite helper has its original confirmed-retirement marker. Successful helper create or start alone cannot release its grants. Nine focused launcher/client tests pass, including create-only and started-but-unretired helpers, a missing reply before an opposing down, and automatic reuse after known completed effects. Independent lifetime review found and corrected the helper acknowledgement versus retirement distinction. Missing/failed mutation replies still remain held; observed-state recovery and old-ID fencing are not yet implemented.

T4 first connected Compose transaction, 24 October 2026: make test/integration/provider/deployment passed at bc6e840760ed4b53f60025fa671a27f2878272aa. A disposable signed loopback API and scripted interpreter exercised the actual installed seven-role Compose path, supplied nm table, native reader, released H/C prediction, API result publication, ordered shutdown, restart and observe-only repeated up. The peer counted exactly one model exchange, one Attempt start and one result publication across fourteen observed Hello requests. Startup readiness separately confirmed the accepted initial Hello on both process starts. Retained bounded Docker logs showed one prediction child, one reader child and no SMILES child; restart added no child. All final role stops were observed without OOM. Prediction's cumulative cgroup peak was 2,786,230,272 bytes under its 4 GiB limit.

Inputs reuse NM_TABLE, nm_conversion_oracle and tests/fixtures/preparation/expected.json; the witness compares coordinates against that independent binary64 conversion bound, atom order, H/C output population and finite shifts. It does not establish numerical shift accuracy. Exact image IDs, receipt hashes, tool versions and remaining limits are recorded in the Compose qualification packet. Private data/receipt retention is separate from application logs, which remain in bounded Docker logs. Four-reference and real 128-atom path proofs, interrupted-effect recovery, named cutover and legacy deletion remain required; neither live-model nor external API/GUI completion is claimed.

T3 acknowledged-helper interruption recovery, 24 October 2026: recovery now reuses the original private CID plus complete matching create reply as cleanup authority, following tools/worker_retirement.sh's existing created-container owner. It never reruns or starts that helper. Under one sixty-second recovery deadline it stops the exact old ID, confirms stopped/non-restarting state, delivers bounded protected diagnostics, removes that ID, confirms absence, then publishes the original retirement marker. Removal fences a delayed start for that old ID. A failed log sink preserves the stopped helper; failed stop/removal/absence preserves the original receipts. No duplicate CLI parser, grant-policy copy or recovery-helper recursion was added.

Ten focused launcher/client tests pass, including original create-only or started-helper receipts, successful retirement without another create/start, independent stop/removal/absence/log failures, and retained missing replies. Independent lifetime review found no further material issue in this bounded correction. These synthetic Engine controls qualify host ownership logic; delayed-request qualification against the installed Engine remains required. Lost create replies still require original-name discovery/admission; persistent runtime requests with missing replies still require their own settlement/fence. This checkpoint does not close the complete interruption contract.

Stopped-stack reuse, 24 October 2026: up now skips Compose create for roles already admitted from the selected immutable bundle. It still starts exact IDs, observes ordered current readiness, and performs final inventory admission. This removes seven no-op creates and their repeated helper inventories on a complete stopped-stack restart. The new regression failed on the old create dispatch and passed after the guard; all eleven focused launcher/client tests pass. Independent lifetime and scientific reviews found no additional material issue. The earlier connected packet remains evidence for its recorded revision; this optimization adds no scientific or deployment claim.

Transient startup recovery, 24 October 2026: readiness now waits for the same admitted ID through a known Engine restart under the original role and whole-start deadlines. It sends no additional start and does not reset either allowance. Changed identity, malformed process state and failed inspection still refuse. A controlled restart-to-ready witness failed before the correction; all twelve focused launcher/client tests now pass. Independent lifetime review found no further material issue.

Portable scientific inputs, 24 October 2026: promoted the original four-reference and synthetic H128/C128 XYZ/manifest exports into tests/fixtures/scientific_inputs, preserving all four files verbatim. Their SHA-256 pins and historical producer identities are explicit; current execution provenance remains separate. The pure loader reuses existing input, population and capacity-recipe owners, never the legacy lifecycle loader that equates historical producer and current worker images. Eighteen focused Compose/input tests pass, including byte-change refusal and the unchanged independent 001/002 packet checks. Independent scientific review confirmed exact promotion and no regenerated expectations. This qualifies input integrity only; actual connected numerical/capacity execution remains the next step.

T4 connected reference and capacity proof, 24 October 2026: both opt-in Compose scientific targets passed at 8f952b2c122981b3934c20f4e2c46a5840ea0527. Four original records reproduced all 137 predictions (82 H, 55 C) and the historical 64-position mask exactly against the unchanged 001/002 packets. The reference interpreter explicitly selects seventeen significant digits to preserve the original binary64 values; product output defaults and comparison tolerances are unchanged. A one-ULP negative control qualifies the actual receiving comparator.

The separate synthetic H128/C128 case consumed both original 128-atom geometries and returned every original index/nucleus with finite shifts. It proves input/output capacity, not chemical accuracy or the old tensor-shape observation. Both cases preserve exact source hash, record/line correspondence, atoms, coordinates and units. Each acquired the Upload once, used two scripted exchanges (inspect, prepare), started one Attempt and published once across stop/restart and repeated up. Actual store peers require the exact download bearer and refuse API signing/model credentials; twenty focused fixture/Compose checks and fifteen runtime-port checks passed before native execution.

Independent scientific and privacy review found no material issue in the retained scientific qualification packet, which links exact result/receipt bytes, historical input manifests and safe lifecycle/resource observations. Reference shutdown reported a cancelled periodic Hello with unconfirmed acceptance and proxy EPIPE; those observations remain explicit despite successful physical exit. No live model, external API/GUI deployment, lost-reply completion or named production cutover is claimed. Successive distinct Jobs in one production factory and the remaining T3/T5/T6 obligations still require closure.

T6 dependency removal groundwork, 24 October 2026: moved unchanged population/capacity comparisons and the supplied nm table/conversion oracle into the existing pure reference_fixtures.py owner. Exact signed-request verification now lives in the existing API TLS fixture owner. Current consumers import those canonical definitions; they no longer need the displaced native launch harness merely to use scientific/API checks. A cold import regression enforces that dependency boundary. Twenty-one focused Compose/input checks and sixty-five process/receiving checks pass; independent scientific and diagnostic review confirmed unchanged function bodies and no weaker checks. No unchanged numerical execution was repeated for these moves.

The documentation check correctly refused the newly introduced fixture README. Its provenance/rationale now has one home in the existing fixture guide; no repository rule or historical-source exception was weakened. This is deletion groundwork, not completion of the named legacy cutover or removal.

T3 lost helper-create reply recovery, 24 October 2026: helper creation now retains one original declaration of image, command, environment, isolation, resources and complete bind grants before dispatch. Normal receiving and original-name recovery share that comparison. An admitted received inspection is retained before stopping/removing its exact ID; a missing name alone cannot fence a delayed create. Acknowledged creator-owned IDs remain cleanup authority even when execution grants are rejected. Joined read-only retries retain one latest observation; uncertain stop/remove receipts are never compacted. Protected diagnostic paths follow the retained observation. Query serialization now holds the stable operation-parent inode rather than a replaceable receipt leaf.

Eighteen focused contained deployment/client tests pass, including altered environment/command/bind/name refusals, original-name recovery, a complete identity followed by a partial observation, bounded read-only retry retention and a controlled displaced-inode lock race. The common admission also passed an actual Engine status inspection of the stopped first connected deployment. Two independent ownership and simplicity reviews found no remaining actionable issue in this slice. These checks do not yet qualify lost persistent-runtime replies or complete T3/T5/T6.

T4 successive Jobs and restart, 24 October 2026: make test/integration/provider/deployment/successive passed at f1b4f2b1b65bea04abe881c53c35748ed1adfc40. Two distinct Jobs and Attempts used the existing fixed chloromethane nm input and independent conversion/population oracle through one uninterrupted provider process (observed Engine restart count zero). Both the reader and prediction runner started and retired two distinct native children before intentional shutdown. Down/up/repeated-up/down kept the same native container IDs and identical projected child events. Final peer counts were two model exchanges, two starts, two completions, no Upload downloads and eighteen observed Hello requests. This qualifies successive local production-factory use and restart without a third execution; it does not qualify a live model or deployed API/GUI.

The pinned qualification source owns the exact receiving checks. The retained successive evidence packet identifies both unmodified public results, exact hashes, fixed lifecycle projections and scope. Children retired with return code -15 after scientific replies and owner-session closure; this is neither a failed prediction nor a normal child exit. All seven containers are stopped with bounded Docker logs retained. No application log dump is preserved.

T3 installed recovery and runner crash, 24 October 2026: the runtime recovery packet records lost create/start/stop replies, same-intent startup of all seven actual containers, and absent old-ID fencing before an opposing command. Public up/down preserve the original pending restriction until settlement. Install refuses live prior work; after acknowledged down it may fence stopped pending IDs without starting or stopping another owner.

The installed crash packet records the unchanged reader image, private tmpfs admission, a child still alive after STOP/TERM, and independent exit observation through its original pidfd after supervisor death. Engine restarted the same container; scratch was empty before successor admission, and the lease inode was unchanged. The successor used a new generation and operation. No scientific input was sent. The old child's exit code is unknown; exec status 137 and socket EOF are not retirement proof. A mutated inspection with bind scratch was refused, which is admission evidence rather than an actual bad-container launch.

T4 prediction replacement connection: the focused installed-core runner lane now connects the real factory, NativeOperation, supervisor, PredictionClient, framed scientific server and NMRNetSession. One authored execution failure triggers one replacement only after original physical retirement and local I/O settlement. Both children receive the original frozen request bytes, and the first diagnostic survives. Authored outcomes qualify this connection and ordering, not OOM behavior or released-model accuracy. Existing exact terminal replay and service recovery tests retain their separate API/store claims. Named deployment cutover and removal of displaced launch paths remain outstanding.

T6 systemd surface removed: the user-service template, installer, environment profiles, UNIT selectors and service targets are gone. The shell-tail lifetime test moved to the still-retained foreground caller's focused suite (28 tests passed). Current operator instructions use named Compose commands and bounded json-file logs. Historical systemd/journald notes and observations remain scoped to their original architecture. The foreground allocator remains temporarily while finite API and recovery qualification callers are migrated; its deletion is still required.

T6 actual-API qualification caller migrated: the opt-in API witnesses now stage known logical input paths, original credential bytes and explicit routes, then install/up a unique named Compose deployment before their existing one-Job client runs. Down settles before the scripted model peer can retire. A failed up prevents Job creation; a failed down retains containers, dependencies and receipts and returns a nonzero named recovery handoff. Only an unretired local process requires keeping its host owner alive; daemon-owned obligations do not require an idle Bash process. Seventeen focused contained staging/composition tests passed, including all seven existing profiles, exact clock/grant checks, and failed install/up/down controls. The staging container receives two narrow writable output leaves, never the host receipt root. This is fixture/source evidence; no additional actual API Job or model call was issued for this migration.

T6 finite-helper logging: retained finite callers now use the same json-file 10 MiB × 3 and nonblocking 1 MiB policy as production. Shared exact-container retirement confirms stop, delivers the last 80 Docker log messages directly to protected stderr, and only then removes the container. Failed log delivery retains the stopped ID and original receipts. The authored wheel-capture preview includes that receiving step. Twenty-eight host-owner and 57 still-retained allocation tests passed, including failed delivery with no remove. Fifteen image/wheel-helper tests also passed, retaining original build failures across cleanup. The journal-only log command, post-removal qualification and Make selectors were removed. Note 019 remains historical; current proofs make no claim that logs survive removal or demonstrate new fatal-science delivery.

T6 synthetic OOM fixture port: the maintained opt-in prediction-recovery lane now uses the installed serial supervisor and actual factory/client/session connection, replacing the host request/receipt pump. It reuses the finite component's fixed 512 MiB/zero-additional-swap profile; a bounded 1 GiB touched allocation must produce one kernel OOM kill and an independently reaped child return code -9. Original scratch and channel settlement gate one successor in the same runner generation. Its requests and synthetic 1.25 ppm outcome are unchanged. The provider image lacks the scientific wire server, so the fixture retains its original narrow frozen worker.py source mount. No released model or API is granted.

Physical OOM qualification passed: make test/integration/provider/prediction-recovery at 5ddd393ffd720c7646f4a116d83843e0a82084bd observed kernel counter 0 → 1, child return codes -9 and -15, two distinct operations in one runner generation, and Engine restart count zero while that runner remained running. The successor's -15 followed its successful synthetic reply and owner closure. The pinned receiving source owns the exact request, reap, scratch and channel checks. artifacts/runner-oom-qualification.json identifies unchanged result/diagnostic/request evidence and hashes; independent reviews matched all five public files to their private originals. Both finite containers retired after direct bounded log delivery; no application log dump remains.

T4 recovered-store connection passed: at 7ec1494456a74a6c10c845971055e9e313f2f687, the retained test/integration/provider-service/recovery lane performed credential-free stopped-store inspection, exactly one retained COMPLETE replay, then current named Compose startup against that same store. Empty discovery supplied no Jobs; the three native event projections contained no child activity. After shutdown and joined TLS handlers, active inventory was empty and the exact original confirmed-file bytes survived in its canonical private history entry. The publication body and protected evidence stayed unchanged. artifacts/compose-store-recovery-qualification.json records hashes and limits. Inspection/replay/peer helpers retired; the seven stopped runtime containers retain bounded Docker logs.

Two failed fixture checkpoints are preserved as source history. The first, 30ddf9135a4f7c4671daf66f1982780d4b8473cb, lacked the finite proxy's newly required status-directory argument and stopped before replay. The second, c70a5fb496cf495d78784a8aedebf46b661b5235, reached successful replay and restart but expected the confirmation in active inventory; existing service archival had correctly moved it into history. The receiving check now compares the original stored bytes across that move rather than regenerating an expected record. Neither correction changed scientific outputs, tolerances or replay content.

T6 shell entrypoint lifetime: the old host's live-file-tail witness was ported to the actual finite deployment caller. It failed before the fix: an appended checkout tail executed after the original command returned. The entrypoint now explicitly exits with that command's status, preserving its original EXIT cleanup owner without reading another tail. All 31 focused deployment/client tests passed. This qualifies that invocation boundary, not arbitrary changes to a running checkout.

T6 foreground startup authority removed: deleted provider/run, its host main/pump, native cleanup/capacity branches, service-mode routing/monitoring and the 17-field handoff emitter. The obsolete snapshot capture API/CLI is gone. Source-only finite component helpers remain; stopped inspection/replay uses tools/provider_recovery.sh with exactly seven fields and only the controller/API route. Current execution-byte capture, private/path admission, prediction lifetime and CONNECT authority checks remain at their actual consumers. No compatibility launch alias was introduced.

Focused checks passed: finite owner 24, current configuration 23, Compose freeze/admission 22, literal operator inputs 33 and documentation 3. The operator check exposed stale preparation-trace assertions from the earlier shared-dependency step; their expected failure prefixes now include that step and still forbid later image/model dispatch. The store-recovery lane is now test/integration/provider/deployment/store-recovery; its earlier packet preserves the exact command executed at the recorded revision. Native allocation protocol modules and old scientific launch harnesses remain the next deletion slice. Final images must be rebuilt after that closure; prior source/image qualification checkpoints remain unchanged.

Legacy protocol deletion: removed the file-based allocation/retirement client, host request pump, old live-service/worker harnesses and their Make targets. Supplied-format correction and ordinary signed Upload tests now use their existing deterministic fixtures directly. Reference/population and capacity tests import the existing pure scientific owners; only tests of the removed receipt protocol were deleted. Historical packets and their source hashes remain unchanged. Focused checks passed: molecular lifecycle 16, Upload 39, finite host 24, launcher 20, operator inputs 32 and image admission 12. These are source checks, not a new numerical qualification.

Final source and installed SMILES connection, 24 October 2026: 4b164b4 removes the displaced allocator/pump/harnesses; independent absence review found no executable legacy path. 0479193 makes ordinary up preserve matching running/restarting components, excludes nested /run store mounts and publishes state-appropriate next actions only after cleanup/completion. A failed hint explicitly preserves completed action truth. dc7f57c joins retry diagnostics to the original native operation, generation and observed return code after retirement; an unknown original exit stays null even when a successor proof has an exit code. Focused checks passed: deployment 33, process/reporting 64, runner connection 11, Compose admission 22, pasted 7, runtime 15, service 48, prepared identity/recovery 46, operator 32 and documentation 3.

The images built from these committed sources passed the maintained inference import check. At af74064, the final Compose SMILES lane generated two CCl conformers with RDKit and predicted eight finite H/C shifts using released fold-0 CPU inference. Authored source atom/H-parent facts, exact request bytes, binary32 projection, count and population passed. A down/up/repeated-up/down sequence added no interpretation, Attempt start or completion. Fixed bounded-log projections show one generator and one prediction child, and no reader child. Scientific replies preceded child retirement and scratch release. This is no independent geometry/shift-accuracy or live-model/API/GUI claim. Exact image IDs, source/fixture/result hashes, measurements and limits are retained in artifacts/compose-smiles-qualification.json. Earlier numerical, capacity and recovery packets retain their original scopes.

The bounded kernel-OOM lane also passed with the final installed image: one independently observed kernel OOM kill, original child -9, one successor after reap/scratch/channel settlement and no supervisor restart. Its retry receipt now joins the actual original operation/generation/-9 to the synthetic Job/Attempt and original cause. Both request observations and the synthetic result remain byte-identical to the earlier packet. See artifacts/runner-oom-linked-qualification.json; this does not qualify released-model OOM or external delivery.

Named-deployment delivery caller: provider/qualify/api/existing-pasted owns only a finite public client. The existing deployment owner admits its original provider/prediction bundle and observed image; a captured offline exporter projects nonsecret client expectations. Availability/delivery belong to public discovery and the received result, not a second readiness gate. Client output is fresh and separate from original store/control roots; it may use an ignored checkout leaf. The lane creates no second provider, store or model peer and never resubmits an uncertain Job. Current image admission is separate from unchanged historical scientific comparisons. Configured correction/fallback retains actual counts and routes; it is not rejected merely for taking another bounded turn.

The original pasted receipt was promoted byte-for-byte from .work/qualification/provider-service-nmrnet-native-service-wMwkAIMq/scientific-result.json to artifacts/pasted-scientific-oracle.json: 23,060 bytes, SHA-256 9ae5017c42af5c5691554eb3ff6dced7895c6a00ab177975d8aa15ae0c58f59a. Its historical source is f01e52d0e91b7afd23af7465bc27da38aaa8b9e1, image sha256:cabb4fe3e36541b0fefe54ca7c16fb21f9a60112a1e3d47992d4aa18343ecd08, Python 3.12.12 / Torch 2.13.0+cpu. Original warnings and obsolete execution identities remain unchanged. Note 003, lines 431–439 already records this exact hash and producer location. This preserves an existing prediction/reference authority and removes a hidden untracked prerequisite; it is neither new expected output, experimental accuracy evidence nor current runtime provenance. Independent privacy/scientific review found no credential, endpoint, host path or personal identity in the receipt.

Finite recovery-fixture command stop/join now lives with that fixture; removing the displaced foreground helper must not delete its live finite inspect/replay cleanup obligation. Its focused cancellation witness confirms the original command is joined before peer release.

Unattended terminal availability, 24 October 2026: removed the aggregate terminal-publication retry window and its obsolete configuration field/expiry exception. Once science has retired, the original publisher keeps paced, cancellation-responsive retries only while the existing replay owner permits the same durable COMPLETE/FAIL bytes. Individual HTTPS/storage/settlement bounds, permanent refusals and cleanup/storage failure gates remain intact. This changes availability policy, not scientific or API authority. The tests-first regression advanced the event-loop clock by 31 seconds after science retirement: the old controller failed; the revised controller confirms the unchanged terminal body without another scientific call. Focused checks passed: core recovery 86, connected service 48, configuration 23 and process/reporting 64. Independent ownership/error review found no regression.

START and active-science observation remain separate. Active freshness expiry cannot authorize continued or renewed science. Continuous START retry also needs initial prediction acquisition moved after verified START admission: at that checkpoint, its scientific idle timeout can expire during an extended START outage. That resource-ordering issue still blocked unattended closure; merely making its window infinite would have been incomplete. The later source-ordering repair and continuous START retry are recorded below. The next qualification priority was one real local API Job (now passed below), then named production delivery.

Local legacy teardown, 24 October 2026: startup preflight missed an active legacy user-service instance; the candidate Compose stack briefly overlapped it before both were stopped. No qualification Job had been submitted, both stores contained no files, and legacy native inventories showed no scientific operations. The corrected maintained definition preserves [local NMRNet provider], lane:nmrnet-local and generation:local-20261023, plus the original one-endpoint OpenAI configuration. Its legacy configuration SHA-256 was baa22f26ddbd26ca8cd41a8201a4def55bbd46a1a8fa04a4a674ad6124c6e9ad; private input/store paths were rebased into the maintained config/secrets layout, with obsolete terminal-window policy removed. Current authored authority is config/deployments/nmr-localhost/provider.toml and its ignored machine settings/private inputs; generated frozen bundles remain execution snapshots.

The legacy service was stopped/disabled, all 20 recorded CIDs were confirmed absent, and a complete container mount inventory found no remaining legacy input/launcher grants. Independent review found no scientific outputs or unresolved native requests in the old launcher trees. The obsolete user unit, environment profile, input/config copies and four run trees were removed. Credentials were preserved in maintained private inputs. The original empty store directory was moved intact to its selected data location; device/inode, caller ownership and mode 0700 continuity were verified. No duplicate store or symlink remains. Candidate obsolete frozen generations still require pruning after their stopped container references and original control obligations settle.

Real local API delivery, 24 October 2026: the maintained provider/qualify/api/existing-pasted lane submitted exactly one demo Job to the existing seven-role nmr-localhost deployment and real API at https://storage.example. Source revision was 40ef43221f19134daae5da9fd4053542e9844cca. The live OpenAI gpt-5.6-luna interpreter selected prepare_pasted in one exchange, with zero corrections; interpretation was not scripted. The finite client recorded 32 API requests, one Job creation, successful delivery and retirement. It did not start another provider or retry Job creation. The named Compose deployment remained running.

The received scientific result was 21,337 bytes, SHA-256 722c2e0ba255017053c0c574fb2e9a17b9ce40c3406ac26cd18bc12d710812ee. Selected models, runtime, prepared request, request hash, float32 coordinates, prediction units and ordered predictions matched the unchanged historical oracle above exactly. For supplied chloromethane, ordered atom indices 0, 2, 3 and 4 received respectively 3.062568426132202, 32.309967041015625, 3.0625784397125244 and 3.0625784397125244 ppm; the chlorine context atom received no shift. The fixture explicitly requested 17 significant digits for exact comparison; the service default remains four. This is delivery and numerical repeatability evidence for released fold-0 CPU inference, not experimental accuracy or production/GUI qualification. Existing scaler-version warnings remain applicable.

Observed deployment image identities were controller sha256:2ca028e960fd83d763198195fd8e9194090245f0c39d0af84a504ec4896709a4, geometry sha256:3bbe17339db1da5d646d39f1105139cd06a3e901ac3b167972294dbb66d4917a and prediction sha256:94e9bf764c09dd4cd8e0c65e1de5d3dbea2f8db5d9d929ab77f155809f9ab870. The result labels its execution declaration as caller-supplied; the client independently admitted the installed prediction image. Private receiving evidence was retained under .work/qualification/local-api-existing-40ef432/client/, including passed.json and scientific-result.json. The essential findings and immutable identities are recorded here so they survive removal of that scratch directory; headers, signatures and interpreter transcripts are not copied into this note.

Installed legacy-removal check, 24 October 2026, 09:51 UTC: independent read-only inspection inside those same three existing role containers found nmrnet_provider.allocation absent and HostRetirement absent from the retirement module in every image. This checks the installed removal as well as source deletion; it launched no additional container, scientific operation or API request. Named production delivery, START resource ordering and safe superseded-bundle cleanup remain separate outstanding obligations.

Settled command scratch, 24 October 2026: successful deployment mutations now remove their own copied inputs and inspection receipts after durable completion and successful next-action reporting. Failed commands, failed reporting and original late-request restrictions retain their receiving evidence. The tests-first regression failed on the old retained scratch and passed after the change; all 35 focused contained deployment tests passed. Independent review found no lost ownership or evidence obligation. Attempt state, immutable bundles, runtime leaves, containers and bounded Docker logs are outside this removal root. Completed qualification cleanup remains separate work.

Superseded input generations, 24 October 2026: successful mutations now prune unreferenced generation directories through the existing deployment owner. Its mutation lock and the existing query lock exclude concurrent control/readers; unresolved query or mutation effects suppress deletion. A fresh admitted Engine inventory retains bundles referenced by stopped as well as running containers, alongside the validated current link. Only canonical caller-owned generation directories with the expected private modes are removed; abandoned unsealed generations are included. Unexpected names, modes and symlinks are left alone. No new lifetime registry is added.

The tests-first pruning witness failed before implementation; all 36 focused contained deployment checks now pass, including busy-query, pending-effect, inventory-failure, stopped-reference and unsafe-candidate controls. Independent ownership review found no issue. Cleanup errors report the deployment action as completed while retaining original operation receipts. These are source checks. The maintained idempotent provider/deployment/up was then exercised against the healthy local stack: it requested no role starts, kept generation-HZlSw8Un selected and running, removed the unreferenced failed-cutover input copy generation-GYcTeKGk, and left no settled mutation scratch. No new Job or scientific operation was requested for this cleanup.

Completed transport-close errors, 24 October 2026: a real asyncio transport reset reproduced the reported false hold: its completed wait_closed error stopped independent control cleanup and was then counted again through the client. The native operation now owns the sole observation of its exact StreamWriter. A completed exceptional task remains protected diagnostic evidence; a pending task, cancellation or deadline expiry still prevents settlement. The original and proof-session controls are closed independently after confirmed child retirement.

The regression forces a real local transport reset and lost original control, confirms proof-session retirement/scratch release, receives the reset cause, and verifies one close observation across repeated retire/settle calls. A separate held-close deadline control confirms that unfinished closure is not released. Focused contained checks passed: native runner 13 and native factory 11. Independent resource/error reviews found no regression. No scientific accuracy matrix was repeated for this lifecycle-only change.

A follow-up ownership review found that the control session still retained the raw scientific socket after asyncio adopted it. Closing control during a pending transport wait could therefore close asyncio's descriptor behind its owner. Successful adoption now immediately clears the control session's socket reference. The strengthened gated-close witness first reproduced the premature descriptor closure, then passed with the descriptor still open under its transport until that owner closed it. All 13 native-runner tests passed again; independent review found no further issue in this handoff.

The displaced nmrnet_provider/retirement.py helper subsequently had no production callers and was removed. Recording ports keep their separate channel/retirement observations in a test-only helper; there is no runtime compatibility wrapper. Affected contained checks passed: output 27, Uploads 39 and native factory 11. Independent review found no dangling production dependency or weakened failure coverage. The installed module observations above describe their earlier image checkpoint, not this later source deletion.

Prediction acquisition after input admission, 24 October 2026: controller assembly no longer allocates a prediction child. One Job-bound allocator supplies both initial and replacement prediction ports. Initial acquisition follows the acknowledged START and successful input interpretation, reading or generation, immediately before result planning. Its actual runtime observation becomes the result-plan expectation; replacements must match it exactly. Scientific preparation and shift inference still follow core's durable prepared-identity write. The old preallocated constructor and separate recovery policy carrier are removed rather than retained as compatibility modes.

A pending initial handoff prevents session retirement. A pair returned after retirement begins is adopted before dispatch is refused, so its actual owner can still retire it. Rejected input, cancelled interpretation, controller construction failure and terminal START replay acquire no predictor. Cleanup witnesses that require a real prediction obligation now explicitly acquire it first. The SIGTERM receiving fixture injects independent cleanup failure into the input owner that exists during interpretation, preserving interruption, EIO evidence and the held-store assertion without inventing a predictor.

The tests-first factory witness reproduced allocation before START. Focused contained checks passed: factory 12, output/preparation/prediction recovery 40, configured runtime 16, native runner 13, Uploads 39, process/receiving reports 64 and documentation 3. Initial late-handoff and terminal START-replay witnesses cover the new ordering; existing exact frozen request, runtime-drift, one-interpretation and retirement-before-replacement controls remain. Independent ownership and scientific-order reviews found no remaining signal. The installed local deployment still uses its earlier admitted images; this source repair requires the final image refresh and does not claim new installed scientific qualification.

Continuous reviewed START availability, 24 October 2026: removed the aggregate START retry window, expiry exception and obsolete configuration field. The same starter settles original I/O and checks the existing API replay owner before paced retries of the exact durable command with fresh signing. Resolution or cancellation ends availability waiting; received-but-not-durable acknowledgement, permanent refusal, storage failure and unconfirmed cleanup still prohibit replay. Individual exchanges and cleanup remain bounded. Active-science freshness is unchanged.

The tests-first signed START witness advanced the event-loop clock by 31 seconds in each retry notice, before science. The old aggregate cutoff failed; the new policy delivers one durable admission and one scientific execution, preserving exact request bodies and distinct authentication on each send. Late certificate and cleanup controls still prevent replay. Removed settings are refused explicitly, not accepted and ignored. Deferred prediction acquisition also permits its supervisor lifetime calculation to exclude START, input reading and chooser time; its remaining post-acquisition bounds stay finite.

Focused contained checks passed: core recovery 85, configuration 23, connected service 48, process/receiving reports 64, Compose admission 23 and documentation 3. Independent ownership/lifetime and scientific/error reviews found no issue. Service fixtures now distinguish constructed input/session ports from acquired predictors: input refusals and pre-START cancellation acquire no prediction child; independent failed-factory cleanup remains covered at its existing input owner. No released-model accuracy matrix was repeated.

Ordinary restart with a retained terminal command remains a distinct open recovery obligation. AttemptExecutor persists TerminalPending before native retirement, so that record alone cannot authorize publication after a crash. The existing provider_core.recovery.AttemptRecovery/TerminalPublisher can replay exact terminal bytes once original provider authority is gone, runner retirement/lease barriers are proved and the complete scoped inventory is admitted. Store-lock acquisition, health and EOF do not prove that barrier. Entered science with an unknown outcome, corruption or conflicting identities must not become automatic scientific re-execution. This remains separate from START availability and from active-observation expiry; unattended recovery is not claimed complete here.

Production prerequisites, 24 October 2026: the clean deployment checkout was advanced to f8b25c0 through a 142,130-byte Git bundle. The existing explicitly selected shared wheelhouse supplied offline builds; no dependency download was needed. Controller, RDKit and inference builds completed. The existing API-client image passed all 39 contained client checks. These establish source/build/client prerequisites, not a running production deployment or delivered production prediction. Actual retained-state admission and cutover remain required.

A post-commit consumer scan found the removed START window still present in config/provider.toml.example; it is now removed there too. Held-cleanup guidance names the supported make provider/deployment/down DEPLOYMENT=<name> action and its provider/native-runner containers, rather than the removed host worker owner. Internal protected effect/retirement facts retain their existing field names. These wording/example corrections do not claim another scientific run.

Production runtime cutover, 24 October 2026: source 6369e131511af82aa4c6eda8246f33cde1851cb2 was built offline and installed as generation-2ADWCLjN for nmr-dapp32-com. Inference imports passed. The seven-role stack became ready and received its initial Hello acknowledgement from https://storage.example, preserving [production NMRNet provider], lane:nmrnet-pa-a, generation:nmrnet-pa-a-run and the original selected store. The old inactive unit/template, environment profile and empty launcher directory were removed. Both old frozen credential sets matched maintained private inputs byte-for-byte; a one-time cutover mount inventory found no old input/launcher grants. Ordinary pruning remains scoped to the supported named-deployment/query/original-effect owners; it is not coupled to unrelated containers through a host-wide negative proof.

Admitted images were controller sha256:8ddec8762ffbb49ae1e50f4e1c3e22f73961b4849b790645eee2f64e45bbad2d, RDKit sha256:8e6296ea65ddcd34ba66fa4bc5e1af040c7b5cd5281dff9e367ab316abb07324 and prediction sha256:400e51d38537c87d5cdca7b946aa5f136118e4e26cdb121047eb32e7476960c8. First installation correctly refused the group-writable public definition (0664); only that caller-owned file was tightened to 0644 before installation succeeded. Credential checks were not relaxed. A first client-witness preflight stopped at a missing private evidence parent before creating its client or submitting a Job. The parent was then prepared with mode 0700 and the same single-Job witness started. That startup/Hello checkpoint is distinct from the subsequently received production prediction below.

Real production API delivery, 24 October 2026: the maintained client-only witness completed one Job against the production API. It recorded 24 client API requests, including readback polling, and did not resubmit the Job. The first configured qwen3.8-flash-next endpoint returned HTTP 400; gpt-5.6-luna then selected prepare_pasted. There were two model exchanges and zero input corrections. OpenAI fallback succeeded; the primary endpoint's rejection requires separate diagnosis; its cause is not inferred from HTTP 400.

The delivered scientific result was 21,448 bytes, SHA-256 338f1b692c5de39b83a88055abc3148be0140a962c8f86e4315f28de99648b9c. The unchanged historical oracle matched selected model artifacts, runtime, frozen requests, float32 coordinates, units and ordered predictions exactly, including the four chloromethane shifts recorded in the local checkpoint. Runtime was Python 3.12.12, Torch 2.13.0+cpu, NumPy 2.4.1, scikit-learn 1.8.0 and joblib 1.4.2. The fixture requested 17 significant digits; the normal default remains four. This is repeatability and actual API delivery evidence for the supplied-coordinate path, not experimental accuracy, five-checkpoint ensemble reproduction, production Upload/generated-geometry coverage or GUI qualification.

The finite client retired, and a subsequent admitted status query found all seven production roles running, non-restarting and ready from the same generation-2ADWCLjN. It was the only remaining input generation; settled mutation scratch was absent. Private receiving evidence is under .work/qualification/production-api-existing-6369e13/client/. The Job, source revision, image identities, result hash, runtime, selected interpreter route and numerical comparison are retained in this note so its essential findings do not depend on keeping that scratch. No signatures, credential bytes, raw response dumps or model transcripts are copied here.

Independent read-only production inspection at 11:32 UTC observed Engine 29.9.1 / Compose 5.6.0, all seven actual roles healthy, and every role's actual log configuration as json-file, 10 MiB × 3, nonblocking with a 1 MiB buffer. The legacy unit's load state was not-found. Isolated imports inside the actual controller, prediction and RDKit images found both nmrnet_provider.allocation and nmrnet_provider.retirement absent. Image identities matched the build outputs and the received prediction image above. This later installed removal check made no additional scientific/API request or file write.

Operator follow-through after the production witness: the client-only lane now creates missing evidence parents with mode 0700 only after original deployment metadata and disjoint-path admission. Its final output leaf still uses exclusive creation, preserving uncertain-Job protection. The tests-first nested-parent witness reproduced the observed refusal. Recovered startup probes deliver a protected 8 KiB cause window before their temporary receipt is removed, retaining the failed phase/status without pointing to deleted files. If delivery fails, the completed action remains explicit, its native reporting failure is shown and original cause receipts stay intact. Genuine command failures and late-effect restrictions retain their protected receipt pointers. A phase/status-only intermediate fix was rejected because it erased useful stderr evidence.

The unchanged recording shell-composition module now runs in the existing source-only deployment lane, alongside its related lifecycle checks. Installed configuration staging remains in its separate provider-image lane. All 44 focused deployment/client-composition checks and all three documentation checks passed. Independent operator and ownership reviews found no issue. Build follow-up text names the maintained configuration/secrets layout and provider/deployment/install/up; it forwards an already selected WLAN_INTERFACE or leaves the existing acquisition prompt available. No new selector, automatic interface discovery, API Job or model request was introduced for these corrections.

Protected model-rejection detail, 24 October 2026: the successful fallback exposed a reporting gap: the carrier retained the HTTP body in memory, but its durable/operator projection retained only status and correlation. The shared carrier now extracts only string error.message/type/code/param from complete JSON within 8 KiB. Unreadable detail preserves the received status with an explicit explanation; HTML, partial JSON and arbitrary body fields are not projected.

This narrowly ports SECS rejection/diagnostic handling at 7356c354, lines 211–288. Original overlapping matches are merged before removing complete prompt/tool-argument echoes and the endpoint key, including JSON-escaped spellings. The consumer redacts all configured keys and recognizable credential fields before its display bound. Endpoint text remains explicitly unverified and protected; this is not a detector for paraphrased input. Public interpretation/exception text and one-send carrier behavior remain unchanged.

The tests-first missing-detail witness failed, then focused checks passed: chat/conversation 33 and process/receiving reports 65. Receiving controls cover useful server reason/code/parameter/request ID, credential-field canaries, another configured key crossing the display bound, and excluded body/prompt echoes. Independent disclosure/ownership review found no issue. This cannot reconstruct the earlier production server reason. The subsequent operator instruction disables Qwen rather than authorizing further availability probes; no new API Job or model request was made for this repair.

Qwen disabled by operator instruction: the production definition now retains OpenAI as its sole active endpoint. The original Qwen stanza is preserved in config/deployments/nmr-dapp32-com/qwen.toml.disabled as an explicitly inactive reference. This deployment loader reads only provider.toml; renaming that reference does not activate it. Re-enabling requires restoring its stanza and reinstalling. The observed HTTP 400 does not establish an offline endpoint or its rejection reason. No endpoint-discovery mechanism, availability probe or additional Job is needed for this configuration change. Existing private keys are retained.

Applied through the maintained production down/install/up/status workflow at source 4dac150: bundle generation-Vgg1crHN, controller image sha256:f64ca4bca38f2628b1b3774bc89b7c091691cea070d3462d34e9635ad015752f. All seven roles reported running, not restarting, with fresh ready observations. The frozen configuration contains only OpenAI and its frozen secrets directory excludes Qwen's key; the original private key remains retained. The existing Attempt store was preserved. This is configuration and startup evidence, not another scientific/API prediction witness.

Subsequent production stability check: the fresh ready observations above did not establish sustained operation. At 12:17–12:20 UTC the provider's restart counter advanced from 10 to 14. Its bounded Docker logs showed idle Attempt/Job-list failures caused by httpcore.ProxyError, with no retained origin response head. The API proxy's inspected tail reported forwarding BrokenPipeError/EPIPE; that does not establish the original connection or DNS cause. Store metadata showed the earlier history file and no active record. No reset, record deletion, further Job or blind restart was used for this diagnosis. Production is therefore unstable at this checkpoint, despite the earlier actual prediction and startup proof.

Next ownership sequence: ordinary terminal restart recovery. Before archive, fresh admission or Hello, the already-open store must admit its complete inventory as only TerminalPending or TerminalConfirmed, then reserve it in the factory's existing claim registry. Unknown entered work, conflicting identities and corruption still block startup. A pending terminal record was written before native retirement, so the record alone cannot prove physical cleanup.

The existing prediction, reader and smiles runners must each establish their serial lease/session barrier and retire the proof child without a scientific command, scientific-channel adoption or prediction startup read. Each original connection/retirement task remains with its existing native owner until settled; a proof child's status is not a historical science child's exit status. Only then may the existing AttemptRecovery/TerminalPublisher replay the exact durable terminal request. Continuous terminal pacing moves from the controller into the publication owner so both callers share one policy.

Startup recovery must use the existing cancellable work-task lane, retaining its current recovery and inventory claims through shutdown. It cannot use the non-cancelled maintenance task: an API outage there could prevent stop. Shutdown settles the original recovery before releasing claims or retiring shared API/storage. Confirmed recovery permits the existing archive and admission path; no interpreter, prepared-session reconstruction or inference retry is involved. Focused proofs must cover exact replay after outage, zero science, malformed/mixed inventory before effects, failed native proof, and cancellation during proof/send/backoff. Active observation expiry is a separate disposition question: retirement alone does not grant permission to discard an unknown acknowledged outcome or admit another Job.

First restart slice implemented: NativeOperation.connect_for_retirement retains its original connection task before waiting and never adopts the scientific socket. The factory proves prediction, reader and smiles in order, retires each proof child and releases only settled setup references. Failed cleanup uses the existing factory settlement path. Proof children still perform their normal startup; the claim is no scientific command, not no child process. The tests-first missing-method witnesses failed before implementation; focused runner 15 and factory 14 checks then passed. Real supervisor tests cover no scientific input and cancellation with a late original connection; factory controls cover all three roles, no clients and held cleanup followed by settlement. Independent lifecycle and scientific reviews found no issue. This slice is not yet wired into service startup or terminal replay.

Shared terminal pacing implemented: the existing controller loop and its TerminalRecovery policy now belong to TerminalPublisher. Both the controller and an explicitly opted-in AttemptRecovery use that owner; stopped recovery remains one-send by default. Settlement precedes API retry permission, and received but unretained outcomes, permanent refusal and cleanup uncertainty still prevent replay. The two new tests failed before implementation; all 87 focused publication/recovery/controller/projection tests then passed. Controls cover exact retained bytes after outage without a scientific session and stop during backoff with the pending record and original failure preserved. Independent ownership and maintenance reviews found no issue. Startup must still establish native retirement before selecting this policy.

Ordinary startup recovery connected: service startup now reads and claims the complete admitted terminal inventory, establishes all native barriers through the factory, and replays pending bytes through the same retained recovery/publication owners before archive, Hello or fresh admission. Startup uses the cancellable work-task lane; shutdown joins it, settles its exact recovery and releases only unborrowed claims. No new lifecycle registry, scientific reconstruction or inference path was added.

Focused contained service 52 and foreground/receiving process 65 checks passed. New controls cover ordinary exact terminal replay with no second scientific allocation, native proof refusal before API effects, and stop during proof, terminal send and retry backoff. Existing mixed/corrupt and unknown-work holds remain. Earlier manual-only startup expectations now exercise the actual shared claim exclusion or unconfirmed native barrier; their source/output identity and no-repeat assertions remain.

Independent review caught a stale-cause risk introduced by retaining startup in its own task: cancelling a later storage read could inherit a previous API 503. The read owner now links only a newly installed API failure and preserves an existing cause. Its regression checks clean stop during storage after API retry. Hello retains its separate uncertainty report and clean shutdown behavior; actual child API cancellation remains visible alongside caller interruption. These are composition/diagnostic proofs; the new recovery code has not yet been installed in production or used to repeat a scientific witness.

CONNECT gateway repair: pinned httpcore 1.0.9 parses a failed proxy tunnel's response, then raises ProxyError without preserving structured status. The transport now retains only the first parsed CONNECT status via its existing trace extension, separately from the origin response head. It adds no parser, raw trace history, reason text, headers or body capture. Inspected source: CONNECT extension forwarding and refusal and parsed header trace; the verified wheel SHA-256 is 2d400746a40668fc9dec9810239072b40b4484b640a8c38fd654a024c7a1bf55.

The existing API operation owner permits gateway recovery only for the exact proxy exception with received CONNECT 502/503/504 after confirmed I/O retirement. Missing status, CONNECT 403, cancellation or failed cleanup grant no retry permission. CONNECT 200 cannot authorize a subsequent TLS failure. Origin response handling and operation-specific mutation gates remain with their existing owners. A proxy gateway response does not prove its underlying DNS or connection failure is temporary: independent production monitoring also observed DNS timeouts among forwarding failures.

Focused contained checks passed: real Unix proxy transport 9, API 66, service 53, foreground/receiving process 66. Controls distinguish proxy status from API status, exclude private reason text, retain cleanup and cancellation gates, and preserve certificate refusal. A connected proxy/TLS/API/service witness receives CONNECT 502, restores the route and reaches Hello in the same service with no scientific allocation. Independent ownership and disclosure reviews found no issue. Production installation and sustained stability remain the next separate proof.

Production idle-read recovery delivered: the maintained down/install/up/status workflow installed source 8e14575 as generation-Mx7vsMhh. Its controller image is sha256:2ef651375082a816c27e28c0b8c31f0790b57cb5d68d0985c6f3cf414ec59d4a, RDKit image sha256:c9abeca51e12ea7c7bf56e96e1fc7308fc40044887351757be5f711f9effcb81, prediction image sha256:1cad4c1ee5462702b8c8efacf36d42364e93dc86ae7933c63c4bc2fc3fbd2b13. The provider's start time remained 2026-10-24T13:27:56.909182163Z, with restart count zero at independent 13:28:54, 13:30:49 and 13:32:57 UTC checks. All seven roles were healthy at 13:30:49. This five-minute window spans repeated actual CONNECT 502 failures and paired api_read_retry/api_read_recovered notices in the same process, with origin status absent. It proves scoped idle-read recovery, not every active-Attempt or model-outage path.

The original Attempt store stayed mounted and its saved Job record SHA-256 remained dbce9c97930a14cf076cccf42e729cc734cfb99041a5378b1741fab4dadeacc7 before and after installation. Docker logging remains bounded json-file, 10 MiB × 3, nonblocking with a 1 MiB buffer. No additional Job, model request or scientific witness was submitted. Current operator history guidance now describes archive followed by trim and persistent-runner session ownership; removed host allocation/release mechanisms remain only in their dated historical records.

Earlier documentation-only validation at source c9f654856f8a7992525693afe51d492b9315c316, 24 October 2026: make test/repository/documentation, using the existing offline checker container, passed all three checks (HTML links/anchors, one maintained prose home, preserved historical source bytes). This validates the document, not the proposed runner architecture. At that checkpoint, source review recorded the one-use model lifecycle, bounded pure-prediction retry and then-current journald dependency; that documentation check performed no runtime or scientific qualification. Later installed and real-API evidence is recorded above.