Reproducing the numerical evidence

Use this reference to acquire the pinned release, reproduce the bounded CPU calculation, inspect diagnostics and run the maintained checks. The scientific guide explains what those comparisons establish; the provider reference covers API operation.

Prerequisites

Run make help or simply make to list every maintained command. Command discovery uses Make and its shell without starting Docker or Python. Commands that acquire packages or scientific artifacts list their required NETWORK_INTERFACE. The proxy and Debian Python base have separate bootstrap download commands using Docker's default routing; the two generic base-image pull names still print guidance. ARCHIVE and STAGE also require explicit Make command-line assignments. Environment-only selectors are not accepted. Make passes admitted values literally; it does not expand selector text as Make functions.

Use Linux x86-64, Bash, Make, Git, and local Docker with BuildKit named-context support. Run as a non-root user able to access the local Docker engine. Python, dependency operations, archive processing, tests, and inference run in containers.

Owned Python workers require NMRNET_IN_CONTAINER=1, supplied explicitly by their maintained container launchers. Use the Make targets rather than invoking Python directly. The shared guard prevents accidental host execution; Docker's configured grants enforce isolation. The environment marker is not proof that a process is contained.

Initialize the pinned source:

git submodule update --init --recursive

Python workers use this pinned Debian base:

docker.io/library/python:3.12.12-slim-bookworm@sha256:593bd06efe90efa80dc4eee3948be7c0fde4134606dd40d8dd8dbcade98e669c

make packages/base-images/python/download

This explicit bootstrap target uses Docker daemon routing; NETWORK_INTERFACE does not control daemon pulls. Package and artifact acquisition then uses the interface-bound proxy installed in the selected shared runtime image. Build or load that image through nmr-provider-tools and pass its immutable ID as SHARED_RUNTIME_IMAGE on acquisition commands. Ordinary builds use local inputs with networking disabled; they do not pull missing images.

Repository checks use a separate image containing pinned Git, Make and Ruff. Prepare it explicitly after provisioning that base image:

make checks/dependencies/download WHEELHOUSE=/absolute/cache/root NETWORK_INTERFACE=YOUR_SETUP_INTERFACE SHARED_RUNTIME_IMAGE=sha256:YOUR_SHARED_IMAGE
make checks/dependencies/check WHEELHOUSE=/absolute/cache/root
make checks/image/build WHEELHOUSE=/absolute/cache/root

The dependency declaration and advisory record identify the exact tool closure; preparation notes explain its sources and accepted offline scope. Ordinary checks install nothing. They admit the recorded immutable image and read only public source, sanitized Git metadata and the two local object stores. Original Git configuration and hooks are not granted. Test containers use an init process to reap orphaned probe children under the existing resource limits.

Acquire and prepare

Every networked acquisition requires a selected interface. Pass NETWORK_INTERFACE=YOUR_INTERFACE on the Make command line for automation, or omit it to choose from the host's available interfaces on the controlling terminal. The numbered menu has no default, even when only one interface is listed. It lists available devices without guessing connectivity or a route; hardware/VM adapters appear first, followed by other interfaces. Each entry shows its type and kernel link state; virtual links and loopback remain selectable. Loopback serves local destinations. An environment-only setting does not select the interface. Without a terminal, an omitted value fails before image checks or network activity and lists explicit selections to use. HTTP(S) connections and IPv4 DNS queries use the selected device through the contained proxy. DNS requires a configured non-loopback IPv4 resolver; no unbound system resolver fallback is used. Offline commands need no interface setting.

Archive downloads report the request/resume phase, then percentage and byte counts about every five seconds as data arrives, followed by checksum verification. Successful download and extraction messages give the host output paths.

For an interactive wheel acquisition, run make packages/wheelhouse. It asks first for WHEELHOUSE, then NETWORK_INTERFACE, before inspecting images or acquiring files. The selected interface is passed through the preflight and execution owners, so they do not ask again. The menu follows the sibling interface inventory and its controlling-terminal choice owner. The explicit Python-base bootstrap uses Docker daemon routing, as described above.

Use the checked-in dependency lock:

make packages/wheelhouse WHEELHOUSE=/absolute/cache/root NETWORK_INTERFACE=YOUR_SETUP_INTERFACE SHARED_RUNTIME_IMAGE=sha256:YOUR_SHARED_IMAGE
make packages/image WHEELHOUSE=/absolute/cache/root
make packages/import

The wheelhouse workflow applies across repositories: each project supplies its locks, target Python/platform and package sources. WHEELHOUSE selects the cache; NETWORK_INTERFACE selects acquisition routing. Each project uses its own locks to acquire missing packages and consume compatible cached files. The pool has no global completeness marker.

make packages/wheelhouse prepares the locked packages for the inference, geometry and provider images, the pinned Uni-Core source, and the build streaming tool in that pool. make packages/preparation/wheelhouse prepares the geometry image's locked packages and SDK dependencies when only that image is needed. Each target resolves its locks separately. The SDK wheel comes from its pinned GitHub release; its source archive is acquired separately for consumer tests as described in the shared runtime dependency guide.

WHEELHOUSE selects a shared dependency cache. There is no default: when omitted, Make asks for a path on the controlling terminal. Both the prompt and WHEELHOUSE= accept absolute or relative paths, including ./, ../ and trailing slashes. Relative paths are resolved once from Make's working directory (after any -C option) into the absolute path used internally. For example, make packages/image WHEELHOUSE=../shared-wheels uses a cache beside the checkout. This follows GNU Make's working-directory semantics; GNU realpath normalizes the path, resolving operator-selected symlinks and allowing cache directories not yet created. Docker receives that resolved absolute target. Without a terminal, pass the path on the Make command line; piped input and environment variables do not select it. Wheel files live directly in this one root. Different projects, versions and platforms can share it: each consumer uses its own hash-locked requirements and pip's compatibility selection. Image builds select the needed files and stream them with the captured build inputs to Docker. Builds install offline and do not mount the live pool into BuildKit.

The pinned Uni-Core source is also stored here, as Uni-Core-<commit>.tar.gz. It is verified separately against its source declaration, including its byte size and SHA-256; it is not treated as a wheel.

The terminal prompt follows the established operator-input pattern: command-line values are passed literally, and environment variables cannot choose a cache. The shared publisher verifies and publishes acquired artifacts in the selected pool. It coordinates publication with managed checks and build streams through locks on the pool. The pool is not replaced or cleared, and existing complete artifacts remain in the same location for later acquisitions.

A complete cached lock needs no index query. With a partly populated pool, the shared publisher selects compatible cached wheels and acquires missing locked dependencies. The target Python environment verifies the selected wheels against the locks. If an acquisition is interrupted, completed files remain in the pool for a retry; no second HTTP cache or wheel staging directory is created. Acquisition may still contact package indexes; checks and builds select locally with --no-index and install nothing from the network. See pip installation reports.

For the provider, acquire the two model archives and extract their declared members:

for archive in weight.zip checkpoints.zip; do
    make weights/download ARCHIVE="$archive" NETWORK_INTERFACE=YOUR_INTERFACE || break
    make weights/extract ARCHIVE="$archive" || break
done

Downloads verify their archive bytes; extraction verifies the archive and selected members again as it reads them. The standalone weights/check and packages/wheelhouse/check commands remain available for optional offline inspection. Successful operator commands show the relevant next action after cleanup; those hints do not run the suggested command.

For baseline reproduction, also acquire the reference dataset. This is a separate path from running the provider:

make weights/download ARCHIVE=data.zip NETWORK_INTERFACE=YOUR_INTERFACE
make weights/extract ARCHIVE=data.zip
make baseline/prepare

All three baseline archives total about 6.96 GB. Complete-archive checksum verification is required before extraction, although the reference consumes only a dictionary, labelled validation data, two fold-0 checkpoints, their scalers, and release logs. Allow additional space for extracted artifacts, packages, images, and results.

Downloads resume retained .part files. Prepared directories are published without replacing existing results; extraction and baseline/prepare refuse an existing destination. Generated files remain under ignored .work/. Preparation selects two intact labelled examples for each element, with at most 128 atoms and 130 tokens. It preserves original record bytes and atom order. Its provenance.json also records the preparation image ID, repository revision and size/SHA-256 of the mounted Python tools. These tool hashes identify the executed checkout bytes, including uncommitted edits that Git HEAD alone omits.

Keep the operational .work/ roots (downloads, prepared, preparation, inference, baseline, packages-stage and checker) at their maintained paths. Completed captures belong under .work/qualification/<topic-or-run>/; the original .work/qualification/20261014/ baseline evidence stays in place. Captured scripts are evidence of historical commands and paths; use maintained Make commands to rerun workflows. Retain reusable inputs while offline execution is needed, qualification evidence while its claims are published, and debug captures until the issue is resolved or reviewed replacement evidence is accepted. Review package stages before adoption or removal. The selected cache's preparation/ set holds the reviewed RDKit dependency wheels for reading molecular structures and generating conformers offline, separately from its inference/ set. Its evidence and limits are in the preparation dependency record.

Run make test/integration/rdkit-runtime to check the retained RDKit dependencies offline in a temporary Python environment. It checks the complete installed package list and native extension origins; it does not read molecular files or generate conformers. This lane requires the Debian Python base listed above.

Build and check the RDKit image before testing native molecular readers:

make geometry/image/build
make test/integration/rdkit-worker

The build copies committed image inputs and only the compatible, hash-locked wheels it selects, verifies the complete lock, and installs only those wheels into an isolated Python environment. Docker builds without network access or implicit image pulls. The recorded immutable image ID is checked against its local image and a label identifying the committed build inputs. A changed lock or image recipe requires a rebuild; unrelated commits do not. The check runs offline with read-only inputs and temporary scratch, using that image's installed packages. It checks the lock, interpreter isolation and native extension origins. Molecular reader tests and conformer generation require their own behavioral evidence.

make test/integration/mol-reader runs the direct V2000 MOL reader checks in that recorded image. The reader preserves supplied atoms and coordinates, requires declared source units and a 3D header, and rejects V3000 before native parsing. MOL/SDF reading is also available over the private worker protocol. The shared Upload/interpreter binding accepts declared formats. This lane checks direct MOL reading; the service's XYZ comparison is listed under Additional comparisons.

make test/integration/sdf-reader checks ordered SDF record selection in the same isolated image. It preserves supplied atoms, coordinates, original record numbers and the whole-file digest. Selected records must contain supported 3D V2000 MOL structures; a failed selected record rejects the whole selection. Associated data is kept outside geometry input. The reader does not check whether records describe the same molecule. This lane checks direct SDF reading independently of the XYZ service comparison.

make test/integration/native-geometry-worker runs real MOL/SDF parsing across the private socket in the recorded RDKit image. It checks original atom/record identity, source hashes, unit conversion, selected record order and whole-selection rejection. The controller process does not import RDKit. Native retirement has separate Compose checks below; the native socket tests run child processes inside one contained test.

make test/integration/provider-core/runner checks native connections, cancellation and lost-control retirement. make test/integration/provider/deployment/recovery checks lost Docker replies, old-container-ID fencing and an installed reader-supervisor crash without scientific input. These checks establish control and retirement behavior; they do not demonstrate an observed RDKit hang or scientific accuracy.

Retrieve failure diagnostics

When a command reports a retained diagnostic path, inspect it first. Acquisition receipts under /tmp/nmrnet-proxy.* include create stdout/stderr, preload-N.stderr, container IDs and readiness inspect.stderr, logs.stdout and logs.stderr, when produced. Checker and baseline image-admission receipts are stdout/stderr under /tmp/nmrnet-checker-admission.* and /tmp/nmrnet-baseline-admission.*. Each metadata output file is capped at 4096 bytes. Failed captured operations can retain combined stdout/stderr in /tmp/nmrnet-inference-native.*/command.log, capped at 8 MiB by the native client owner. Failed Docker client preparation can retain /tmp/nmrnet-docker-client.*/preparation.log, capped at 4096 bytes, alongside an empty or partial config directory. Directory-allocation failure has no private receipt path. Storage failure can also prevent the final reporter from starting; the command's exit status remains available. These directories have mode 0700; read them as the user who ran the command. Capped files may be incomplete; storage failures can prevent retention.

Cancellation can finish silently. Ordinary acquisition preflight exits 130/143 also stay silent while retaining inspection receipts. Find surviving temporary diagnostic directories owned by your user:

find /tmp -maxdepth 1 -type d -uid "$(id -u)" \
    \( -name 'nmrnet-proxy.*' -o -name 'nmrnet-inference-native.*' \
    -o -name 'nmrnet-docker-client.*' \
    -o -name 'nmrnet-checker-admission.*' -o -name 'nmrnet-baseline-admission.*' \) -print

The contained test launcher can retain failed checker-image inspection receipts under .work/checker/.inspect.*/: observed-id and diagnostic, each capped at 4096 bytes. Setup or storage failure can leave either file absent or incomplete; directory-allocation failure establishes no inspection receipt. A wrapper setup failure can also leave a captured cause in the native client owner's command.log, described above. These private receipts can survive silent cancellation too. From the repository root, when .work/checker exists:

find .work/checker -mindepth 1 -maxdepth 1 -type d -uid "$(id -u)" \
    -name '.inspect.*' -print

Successful inspection receipts are normally removed during launcher cleanup. Failed inspections retain their receipts for review; no automatic lifetime is promised. A failed removal can leave partial evidence or no files, so check which paths survive before relying on them.

Several attempts may coexist; a matching name alone neither identifies the attempt nor authorizes process/container cleanup. If retirement is unconfirmed, preserve resources and receipts until ownership and state are established.

Acquisition workers stream output live; the launcher does not retain a transcript. Save stdout and stderr when running the command for later use. Manual or system cleanup or reboot can remove temporary diagnostics. Copy needed evidence to a private location and review it before deleting it.

Run the reference

Arriving directly at this example? First check the container and source prerequisites and complete artifact acquisition and image construction. This command uses the already prepared inputs; it does not download or build them.

After image construction and artifact preparation, the maintained offline command preflights both H/C models, runs each twice and compares source-aligned predictions:

make baseline/run RUN_ID=reference-cpu-1 LABEL_ATOL=0.0001 ATOL=0 RTOL=0

Use a fresh RUN_ID for each attempt; environment-only settings are not accepted. RUN_ID must start with a letter or digit and contain 1-80 letters, digits, dots, underscores or hyphens. Each tolerance must be finite and nonnegative. LABEL_ATOL allows absolute rounding error when checking labels after the upstream float32 scaler round trip against the original record labels. ATOL and RTOL govern repeated predictions, with each atom requiring abs(repeated - reference) <= ATOL + RTOL * abs(reference).

These tolerances were declared before execution. The label allowance exceeds the calculated float32 scaler-rounding bound for the selected labels; repeat tolerances require exact equality in the recorded CPU runtime. The qualified run used Python 3.12.12, Torch 2.13.0+cpu, batch size 1 and two computation threads. Cross-platform or GPU equality has not been established. Investigate differences rather than relaxing tolerances to make a comparison pass. Note 001 records the arithmetic rationale, actual errors, scaler-version warning and scientific source review.

Both baseline commands use the immutable image ID recorded by packages/image. Before importing scientific code, they compare the image's embedded inference.lock and unicore-source.json against the current declarations in requirements/. A mismatch requires rebuilding the image: the package lock alone does not identify the separately built Uni-Core source. baseline/run also requires the image's upstream-revision label to match the recorded submodule commit and records the source declaration's hash. These checks are implemented in tools/baseline.sh and tools/baseline_run.py. Both commands run offline with read-only inputs.

Evidence is retained under .work/baseline/runs/<RUN_ID>/, including model preflights, separate H/C run-1 and run-2 outputs and logs, verification receipts and comparisons. attempt.json and a successful complete.json record the execution image ID, repository revision and mounted tool hashes. The run's prepared_receipt hash links back to the unchanged preparation provenance; tools/baseline_prepare.py computes these file identities inside the container. Existing destinations are refused. Failed attempts preserve their available evidence; use a new RUN_ID for another attempt. After creating a new attempt directory, the runner retains caught failures and their exception chains in failure.log, synchronizes it, then re-raises the original error. Admission failures create no attempt; abrupt termination or output-storage failure can prevent diagnostic retention. complete.json is published only after execution, output verification and repeatability checks succeed. Its pending_review entries remain explicit: this execution receipt alone does not establish scientific qualification. For the checked-in reference, the separate source/scaler review and independent runtime audit recorded in note 001 close those reviews without rewriting the original receipt. This is a fold-0 execution reference, not a five-fold ensemble or a benchmark accuracy claim.

Additional comparisons

Use the maintained lanes to check distinct boundaries:

These Compose lanes use the production factory and installed serial runners with local signed API peers and scripted interpretation. A passing lane establishes its recorded comparison, not experimental accuracy or cross-platform equality. The scientific guide explains how the numerical references differ from published benchmark errors. The operator reference covers deployed API checks and the experiment record retains individual run outcomes and source/image identities.

Prediction and input preparation run in separate offline containers. Only the prediction worker receives checkpoints; the controller owns API credentials and durable state. The native supervisors own child termination, reaping and scratch cleanup; the host controls the retained runtime containers. Successful runs retain source-hashed summaries under .work/qualification/; failed runs retain private receipts and diagnostic evidence at the reported paths. Runtime output stays in bounded Docker logs; it is not exported beside those receipts. Preserve retained owners before attempting recovery.

Checks and dependency updates

make test                 # offline unit, repository, core contract and integration suites
make test/unit
make test/repository
make test/probe

These commands run offline with read-only inputs/root, a non-root identity, dropped capabilities, no-new-privileges, and bounded CPU, memory, PIDs, and scratch. They do not execute real checkpoints or establish scientific accuracy. The repository suite includes strict conventions imported from sibling repositories. Their provenance and review decisions are recorded in note 011, along with source crosswalks, execution requirements and exact run results. The prepared checker supplies Git, Make and Ruff 0.15.15; the unit lane keeps the smaller Python base. Repository conventions check early shared guards across owned Python sources, hierarchical Make targets and the test aggregate. New files under tools/, repository_checks/ and nmrnet_inference/ enter that inventory automatically. The suite records the pinned NMRPeak and Magnet precedents for its organization and source checks.

For an intentional dependency update, stage a new lock and advisory receipt:

make packages/lock/stage NETWORK_INTERFACE=YOUR_INTERFACE

Review them, then apply offline with make packages/lock/apply STAGE=lock.ID, using the printed stage ID. Lock resolution also requires the pinned UV image declared in tools/packages.sh to be preloaded. The advisory check covers exact-version PyPI metadata; it is not a complete security audit.

Image preflight and stage acquisition each wait for their controller's cleanup path to return on cancellation; that path can report cleanup failure. An unresponsive controller can keep that wait running indefinitely. Any captured stage inputs remain available after failure or cancellation. The successful-stage Review instruction follows only a zero controller result.

Package staging accepts edits to the requirements/ intent files inference.in, build.in and unicore-source.json. It captures the committed resolver recipe and tools, plus the current intent, in the stage's private .inputs/ directory. Retain that snapshot with the review stage after success or failure. Apply checks a fresh snapshot of current intent against the reviewed report; edits since staging require another stage. Each snapshot reflects admission for that invocation; edits across the three files are not one transaction.