Runner protocol 1
The runner is the node-side half of a module. The agent execs it once per job attempt. It speaks no network protocol:
its whole interface is argv, a fixed environment, files in a workdir (including the control document) and an exit code.
So a runner can be written in any language, for any platform (platforms.md). Models:
oarbank_sdk.runner_protocol. Schemas: runner-*-1.schema.json.
Invocation
Section titled “Invocation”<runner.exec...> run --spec <W>/spec.json --workdir <W> --out <W>/result.json [--events <W>/events.ndjson]<runner.exec...> doctor --jsonrunner.exec(or the variant for the node’s platform) is resolved as manifest.md says: thepythontoken becomes the module environment’s interpreter,{bundle}the bundle’s absolute path.- Arguments: the agent passes an argv array, never through a shell. On Windows it builds the command line with the
CommandLineToArgvWquoting rules. - Process container: the runner and every process it starts live in one process container (platforms.md) that the agent can stop, freeze and kill as a whole. A runner never detaches.
--eventsis passed only to a runner that declaresprogress_eventsorcheckpoint.- cwd: the work directory for
run; the module’s data directory fordoctor.
Environment
Section titled “Environment”The agent passes exactly these variables, plus the conventional ones derived per OS (platforms.md),
then the manifest’s [runner].env with the platform variant’s env merged over it (never a reserved name:
platforms.md). Nothing is inherited. The runner’s home and every per-user
location (HOME and the XDG directories; USERPROFILE, APPDATA, LOCALAPPDATA) point into its work directory, so
they are discarded with it: platforms.md, “What the home is”.
| Variable | Stability | Meaning |
|---|---|---|
OARBANK_WORKDIR |
stable | The job’s work directory <W>. |
OARBANK_TMP |
stable | <W>/tmp, private to the job (also TMPDIR, or TEMP/TMP on Windows, which the AppContainer start moves into the work directory’s container folder: platforms.md). |
OARBANK_MODULE_DATA |
stable | The module’s data directory on this node, kept across jobs (caches, locks). Besides the work directory, the only place a runner may write. Executing files from it needs the exec_writable grant (sandbox.md). |
OARBANK_PLATFORM |
stable | The node’s platform token, e.g. linux-amd64. |
OARBANK_MODULE |
stable | The module’s short name (the last part of module.id). |
OARBANK_ATTEMPT_ID |
stable | Unique per attempt. Use it to label anything created outside the workdir. |
OARBANK_PROTOCOL |
stable | The runner protocol major the agent chose. |
OARBANK_POOL_<NAME>_TOKENS |
stable | For each pool the node offers: its token count (upper-case name). |
OARBANK_DISABLED_SERVICES |
beta | Comma-separated names of this module’s services the owner disabled on the node. |
OARBANK_SETTINGS_FILE |
stable | A UTF-8 JSON object with the module’s node-scoped settings for this node: each key the settings schema declares with "x-oarbank": {"scope": "node"}, the owner’s value for this node (resolved over fleet, node group and node) or else its default; a key with neither is absent (manifest.md). A job of a campaign that overrides a key declared "x-oarbank": {"campaign": true} gets the campaign’s value for it. It is a file because environment size limits differ per OS. |
OARBANK_SECRETS_FILE |
beta | Set only when the job’s stage lists secrets (stages[].secrets, core 2.5): a UTF-8 JSON file {"<name>": "<value>"} with those secrets’ values for this node, owner-only, inside the work directory and deleted with it. oarbank_sdk.secrets.get(name) reads it. Never log a value or write it into the result or artifacts. |
OARBANK_BROKER |
beta | The job’s container broker endpoint, unix:/path or npipe://./pipe/<name> (sandbox.md). Set only for a module approved for containers. |
OARBANK_SERVICE_<NAME> |
beta | The job’s connector to its module’s endpoint service <name> (upper-cased), fd:<n> or handle:<n>, for each endpoint service providing a pool its stage reserves (service-protocol.md). Use oarbank_sdk.service_endpoint. |
OARBANK_TOOLS_FILE |
stable | A UTF-8 JSON file {"<tool id>": [{"path": "<canonical path>", "version": "17.0.12", "arch": "aarch64"}]} for the module’s approved [sandbox].tools resolved on this node: the one detected installation per tool that satisfies the request (sandbox.md), exactly the paths the sandbox grants (canonical; conventional symlinks such as /opt/homebrew/opt/... are not readable inside the sandbox). oarbank_sdk.tools.path(id) and tools.version(id) read it. |
OARBANK_FOLDERS_FILE |
beta | A UTF-8 JSON file {"<folder id>": {"path": "<canonical path>", "access": "read" | "write"}} for the runner’s granted [sandbox].folders on this node (an empty object when there are none): exactly the paths the sandbox grants. oarbank_sdk.folders.path(id, access) reads it. |
HTTPS_PROXY, HTTP_PROXY, ALL_PROXY |
stable | Set for egress-allowlist: the agent’s local proxy, the only network route (sandbox.md). |
OARBANK_CONTROL_EVENT |
stable | Windows: the handle, in decimal, of the auto-reset event the runner inherits and the agent sets after every change to control.json (Control). |
PYTHONUTF8=1 |
stable | For Python runtimes, on every OS. |
A module gets host tools through [sandbox].tools and finds them in OARBANK_TOOLS_FILE; there are no tool-specific
variables such as JAVA_HOME.
A job of a bootstrap stage gets less (sandbox.md): no OARBANK_MODULE_DATA, no
OARBANK_BROKER, an OARBANK_TOOLS_FILE that lists no tools, an OARBANK_FOLDERS_FILE that lists no folders and an
OARBANK_SETTINGS_FILE holding {}, and no OARBANK_SECRETS_FILE.
All of this runs under the module sandbox (sandbox.md).
Workdir
Section titled “Workdir”The agent creates <W> fresh for each attempt, and deletes it after the result is recorded. Every path the runner
names is a PortablePath, /-separated (platforms.md).
| Path | Written by | Contents |
|---|---|---|
spec.json |
agent | The spec envelope (envelopes.md). |
<mount>/... |
agent | Each dataset in spec.datasets, under spec.mounts[id], as read-only regular files (never symlinks). How they are placed (clone, hardlink or copy) is the agent’s business. Modifying one is a fault, and the agent may verify. |
inputs/<name>/... |
agent | Artifacts of the upstream stage (spec.inputs). |
checkpoint/... |
agent | [beta] When spec.resume is set: the job’s latest checkpoint, each file under its name, as read-only regular files (Checkpoints). |
control.json |
agent | The control document, always present (Control). |
.grants/ |
agent | The files the environment names (tools, settings, and for a stage that lists secrets secrets.json, mode 0600). Not output. |
result.json |
runner | The result envelope, written atomically (a temporary file, then rename) before exiting 0. |
failure.json |
runner | {reason, detail, fault?, retryable?}, written atomically before a non-zero exit (fault and retryable: Exit codes). reason is one of the agent’s end reasons (bad_input, mode_mismatch, oom, doctor, no_metrics), which the coordinator maps to its reason codes, or the module’s own <module-short>/<code>, which ends the attempt as exit_nonzero with the code in its detail. |
events.ndjson |
runner | One UTF-8 event per line (LF; CR tolerated): log, progress, metric, checkpoint or phase. The agent reads complete lines only. |
phase |
runner | Optional: one line naming the current phase, replaced atomically. |
Writers replace atomically. A replace that fails because the other side has the file open (Windows) is retried for up
to 2 s. oarbank_sdk helpers do this. Output artifacts are listed in result.json as
artifacts[].files[] = {path, local}, with local relative to the workdir. The agent uploads each file by content
digest and replaces local with digest and size. Artifacts carry no file modes. A file may name a thumbnail: {local}: a small preview image the runner made of it (PNG, JPEG, WebP or AVIF, at most 1 MiB), uploaded the same way,
which media components show (spec/ui-contract.md, “Media”).
Bootstrap results
Section titled “Bootstrap results”A bootstrap stage’s runner writes a result whose payload is {} and whose artifacts are the datasets it fetched, one
artifact per dataset, each holding exactly the files of one [[datasets.pinned]] entry (the same paths; the agent’s
digests and sizes must be the pinned ones). The host checks that instead of calling result.evaluate, registers each
dataset, and stores nothing else of the result (effective and provenance are dropped). A result that is not exactly
pinned datasets fails with pin_mismatch, a job fault that never counts against the node.
Cleanup runs
Section titled “Cleanup runs”The agent runs a module’s cleanup stage (manifest.md) on a node before it deletes the module’s data there, with the cleanup grants (sandbox.md). The invocation, workdir, environment and control document are a job’s, with these differences:
- The spec envelope names the cleanup stage in
stage, has nodatasets,mountsorinputs, an emptyjob_key,schema<module-short>/spec@1, andpayload{"reason": "disabled" | "removed" | "retired" | "purge"}: the module was turned off on the node, is no longer in its release, the node was retired, or the owner purged it. OARBANK_ATTEMPT_IDis0, andOARBANK_DISABLED_SERVICESis not set.- The hard limit is the stage’s
timeout_sfor the node’s platform, at most 600 s. When it runs out, the agent kills the process container. - Exit 0 means done; a result is not needed, and the agent ignores one. Any other exit, with or without
failure.json, is a failed cleanup: the agent logs the reason and detail and reports them to the owner, and deletes the module’s data anyway. - A cleanup run is never retried. It runs once per removal, so it must be safe to run when there is nothing to release.
Exit codes
Section titled “Exit codes”| Code | Meaning | Attribution |
|---|---|---|
| 0 | result.json was written. |
The module’s result.evaluate decides. |
| 2 | The spec can never succeed. | Job fault: not retried elsewhere. |
| 3 | A node dependency is missing. | Host fault: the node is re-doctored and loses the capability. |
| 75 | Transient failure (EX_TEMPFAIL). | Retried; not a job fault. |
| other, or crashed | Failure. | The core’s breaker rules. |
Codes 2, 3 and 75 count only when failure.json was written. On Windows a C/C++ abort() exits 3, and argparse
exits 2 everywhere. Without the file, any non-zero exit is “other”. failure.fault (job, host, transient)
overrides the code, and so does retryable. The agent records separately whether it terminated the job itself or the
job crashed (a POSIX signal, or a Windows exception code).
fault says who is at fault; retryable says whether another attempt can help. Together (core 2.10.4):
fault |
retryable |
The job | The node |
|---|---|---|---|
transient, or none |
true |
Released and retried; nothing is spent. | Not charged. |
transient |
unset | Released and retried; nothing is spent. | Not charged. |
job (or exit 2) |
unset or false |
Ends failed at once: never retried elsewhere, never quarantined. |
Not charged. |
job |
true |
Spends one of its stage’s retry.max_attempts; quarantined once they are spent. |
Not charged. |
host (or exit 3) |
unset or true |
Sent to another node; its retries are untouched. | Re-doctored. |
host |
false |
Ends failed at once. |
Re-doctored. |
| none | false |
Ends failed at once. |
Not charged. |
| none | unset | Spends a retry; quarantined once they are spent. | The breaker rules. |
So write fault: "job" (or exit 2) only for a failure that is the same on every node, such as a configuration error,
an input the module rejects, or a parameter combination the tool refuses. A failure that might pass on a retry is
fault: "job" with retryable: true, or no failure.json at all. The coordinator keeps the failure’s reason
and detail on the job. The job’s views show them, and so do a module’s campaign.tick and host.jobs.query rows,
as failure (module-protocol.md). A golden job that ends failed this way fails the
module’s certification on that node at once. A module cannot skip its goldens with it.
Control
Section titled “Control”<W>/control.json is a Control document: {seq, stop, pause, threads?, gpu_duty?, reason?, checkpoint?}.
- The agent writes it for every job, atomically (a temporary file, then rename):
{"seq": 0}before the runner starts, then a newerseqon every change. - After every change the agent nudges the runner:
- POSIX: SIGUSR1 to the runner process, never to its children. The runner starts with SIGUSR1 ignored, so a nudge that comes before it is ready is lost harmlessly: a runner installs its SIGUSR1 handler first, then reads the document. (A shell cannot trap a signal ignored on entry: a shell runner execs a program that handles it.)
- Windows: it sets the auto-reset event whose handle the runner inherits, named by
OARBANK_CONTROL_EVENT. The sandbox passes the runner that handle and its standard handles, nothing else. A nudge stays set until the runner waits on the event.
- The runner re-reads the document when nudged and acts on it at its next safe point. It has no reason to re-read it otherwise: nothing changes without a nudge.
- Workloads use the same document (workload-protocol.md), at
OARBANK_CONTROL_FILE, with the same nudges, and also re-read it at least every 2 s: a workload outlives agent restarts, and after one its Windows control event is gone.
oarbank_sdk.control.Control is a stdlib-only reference implementation (vendorable). It reads the document when
created and again only after a nudge, and holds a pause by blocking on the next nudge; on POSIX create it on the main
thread (it installs the signal handler):
ctl = Control() # OARBANK_WORKDIRctl.phase("work") # <W>/phase: the job is underwaytry: for chunk in work: ctl.safe_point() # raises Stopped on a stop request; holds while paused do(chunk, threads=ctl.threads or default)except Stopped: sys.exit(ctl.acknowledge_stop()) # failure.json {reason: "<module-short>/stopped", fault: "transient"}; 75- Safe point: any place where holding or stopping the job changes nothing already written: between work chunks, between tool invocations, before a checkpoint.
- Stop (
stop: true): finish withinrunner.stop_grace_s, exiting non-zero, or write the result if it is already done.- On POSIX the agent also sends SIGTERM to the process container.
- After the grace period it kills the container: SIGKILL, or
cgroup.killon Linux, orTerminateJobObjecton Windows. - A runner that declares
cancellablehonours stop promptly. At its next safe point it acknowledges the stop: it writesfailure.jsonwithfault = "transient"(Control.acknowledge_stop; reason<module-short>/stopped), or the result if it was already finishing, and then exits (75 afterfailure.json) withinstop_grace_s. The acknowledgement is the runner’s own observable reaction; the process exit that follows includes interpreter and tool teardown, which load on the machine stretches, so onlystop_grace_sbounds it. Keep safe points at most about a second of work apart. - The conformance kit checks both: it sends the stop once the runner has written its first
phase(so start-up is never timed), expects the acknowledgement within 2 s of the nudge (a bound that holds on a loaded machine, and far below any reasonablestop_grace_s), and expects the exit withinstop_grace_s.
- Pause (
pause: true, forcooperative_pauserunners): hold at the next safe point until a newer document clears it.- Runners that declare
freeze_okmay instead be frozen at any instruction, with the platform’s mechanism: SIGSTOP, the cgroup freezer, or suspending every process of the Job Object on Windows. - A pause lasts at most 10 minutes (an owner may set less on a node). After that the agent releases the attempt (not a
failure), and the job runs again elsewhere or later: a runner that declares
checkpointis asked to checkpoint first and resumes from it on any node (Checkpoints); one that declaresresumablecontinues from its own data-directory checkpoint on the same node only.
- Runners that declare
- Throttle (
cooperative_throttle):threadsis the maximum number of active compute threads from the next safe point on.gpu_dutyis the maximum GPU duty fraction, from 0 to 1: over any second, the share of time the runner keeps the GPU busy (its submitted work executing) stays at or under it. Hold it by keeping each unit of GPU work short (a smaller batch, fewer ops per command buffer) and idling between units for the rest of the second; absent means unlimited. The agent writes it for GPU work beside a protected GPU program: at the level a co-tenancy measurement tests, and at a co-tenancy certificate’s level under the node’sgpu_jobs = "certified"(core 2.10). Only runners that declarecooperative_throttleget it and may run beside such a program; others are paused there. The agent measures each process’s GPU time, and a runner that does not hold the duty is treated as one that cannot (Holding a GPU duty).- Time spent held is left out of any speed the result reports.
Holding a GPU duty
Section titled “Holding a GPU duty”oarbank_sdk.duty.Duty (also oarbank_sdk.gpu.Duty; standard library only, vendorable with control.py) holds
gpu_duty in the runner’s own loop:
ctl = Control()duty = Duty(ctl, events=events_path) # reads ctl.gpu_duty; reports what it heldfor batch in batches: ctl.safe_point() with duty.unit(): # one unit of GPU work, finished on the GPU when the block ends out = model(batch) mx.eval(out); mx.synchronize() # MLX; llama.cpp: one eval call; Metal: waitUntilCompletedduty.close()-
A unit is timed from its start to the end of the GPU’s work (end the block only once the GPU is done). After a unit of
bseconds at dutydthe governor idlesb·(1−d)/d, in slices of at most 0.25 s, applying a newer control document between slices: a stop raisesStopped, a new duty changes the idle. Duty 0 holds before the next unit until a document with a duty above 0 (or a stop) arrives (Control.wait); no duty runs flat out. -
Keep units short. A unit longer than a second is one burst that long whatever the duty:
duty.scale()says by how much the next units should shrink (a batch, a prefill step, a ubatch) for one to fitdof a second. -
Reports. With
events, it writes{"t", "kind": "metric", "name": "gpu_duty", "value": <busy share>, "data": {"asked", "busy_s", "elapsed_s", "units", "device", "threads"}}every 2 s (and when the duty changes).deviceisgpu, orcpufor a stand-in whose units run on the CPU. The conformance kit checks these against what it measures (conformance.md,duty_specs). -
What the agent checks (core 2.10): every two seconds it reads each job’s GPU time (all processes of its container). A job over its
gpu_dutyplus 0.1 for three consecutive samples, judged from the second sample after it was given a duty, is paused for 60 s (GPU_DUTY_EXCEEDED); the third such pause of the same module’s work within an hour stops it, and that module’s jobs get no duty for an hour, so they wait beside protected GPU work like a runner that cannot throttle. A job whose GPU time the node cannot read is paused the same way (GPU_DUTY_UNVERIFIED): a duty nobody can check is not granted. A runner that cannot be paused is stopped instead. -
The duty itself is the agent’s: under
gpu_jobs = "certified"it is the node’s GPU budget (an AIMD loop on the protected program’s progress, never above the certified level) divided among the cooperative GPU jobs; during a co-tenancy measurement it is the level under test. -
Checkpoint, then stop (
stop: truewithcheckpoint: true, runners that declarecheckpoint): write a checkpoint at the next safe point, then acknowledge the stop as above, withinrunner.checkpoint_grace_sinstead ofstop_grace_s. The agent sends no SIGTERM with this request (Checkpoints).
Host protection prefers cooperative runners: it throttles before it pauses, and pauses before it evicts; a checkpointing runner is asked to checkpoint before it is evicted.
Checkpoints
Section titled “Checkpoints”A runner that declares the capability checkpoint, on a stage with checkpoint = {max_mb, min_interval_s}
(manifest.md), keeps portable checkpoints: the job’s next attempt, on any
node, resumes from the latest one.
- Announce. The runner writes a checkpoint’s files into the workdir, then appends a
checkpointevent:{"t", "kind": "checkpoint", "files": [{"path": "<workdir path>", "name": "<checkpoint path>"}], "data": {...}}.pathnames a regular file in the workdir (never a symlink),namewhere it appears in the checkpoint (default:path); both are PortablePaths and names are unique.datais at most 4 KiB of JSON, given back on resume. - Hand over. Once the event is written, its files belong to the agent: it moves them out of the workdir, uploads them by content digest and records them. Write each checkpoint to new paths and never touch a named file again. Only the latest checkpoint is kept.
- Limits. The agent skips (and logs) a checkpoint over the stage’s
max_mb, or one that comes sooner thanmin_interval_safter the last one it uploaded, unless it answers a checkpoint-then-stop request. - When the job must leave the node (a pause past its limit, an eviction, an owner’s hard cap, a drain, its deadline), the agent sends checkpoint-then-stop (Control), takes the last checkpoint the runner wrote before it exited, and releases the attempt only after the upload.
- Resume. The next attempt finds the checkpoint under
<W>/checkpoint/<name>(read-only regular files), and its spec envelope carriesresume: {from_attempt, digest, data};digestis the sha256 of the canonical JSON list of the files’{name, digest, size}sorted by name (oarbank_sdk.runner_protocol.checkpoint_digest). Withoutresume, start from the beginning. - Not for goldens or bootstrap jobs. A golden runs whole (certification never resumes one) and a bootstrap job keeps nothing: the agent ignores their checkpoint events and stops them like any other job.
- Determinism. A resumed attempt gives the same result as an uninterrupted one:
results.determinismand the stage’s apply to it like any other result, and the conformance kit replays an interrupted run to check.
oarbank_sdk.control.Checkpoints (stdlib only, vendorable) writes each checkpoint into a fresh directory and appends
its event, and reads resume; Control.checkpoint_requested says a stop asks for a checkpoint first:
ckpt = Checkpoints(workdir, events_path, spec)step = int(ckpt.resume_data().get("step", 0)) if ckpt.resume() else 0try: while step < n: ctl.safe_point() work(step); step += 1 if step % 50 == 0: with ckpt.write({"step": step}) as d: save_state(d / "state.bin")except Stopped: if ctl.checkpoint_requested: with ckpt.write({"step": step}) as d: save_state(d / "state.bin") sys.exit(ctl.acknowledge_stop())GPU use
Section titled “GPU use”runner.gpu.use (none, shared or exclusive) declares whether jobs use a GPU. The core leaves GPU jobs pending
(GPU_BLOCKED) while the node may admit none: the owner set gpu_jobs = "never", or a protected process uses that GPU.
apis_any [beta] names the GPU APIs the runner can use; its jobs run only on nodes providing one of them. The set is
open (^[a-z][a-z0-9]*$); a core detects six:
| API | Detected on | Counts when |
|---|---|---|
metal |
macOS | MTLCopyAllDevices returns a device |
cuda |
Linux, Windows | cuInit and cuDeviceGetCount > 0 (libcuda.so.1, nvcuda.dll) |
rocm |
Linux, Windows | the HIP runtime’s hipGetDeviceCount > 0 (libamdhip64, amdhip64_7.dll or amdhip64_6.dll) |
vulkan |
every OS | the loader lists a physical device that is not a CPU device and has a compute queue (MoltenVK on macOS) |
opencl |
every OS | a platform lists a device of type GPU or accelerator (not a CPU device, not Windows’ Basic Render Driver) |
directml |
Windows | DirectML.dll loads and a hardware DXGI adapter supports Direct3D 12 at feature level 11_0 |
A node’s agent reports the APIs it provides on the host and inside its containers in its doctor report (the agent
command oarbank-agent gpu-apis prints them; oarbank-sdk gpu-apis runs the same probes on a developer’s machine, and
oarbank_sdk.gpu.detect() returns them). What a job of a stage needs on a node of a platform
(Manifest.gpu_needs(stage, platform)) is a list of “any of” groups:
- the runner’s
gpufor that platform (a[runner.variants.<key>].gpureplaces it whole), whenuseis notnoneandapis_anyis not empty: within_container = falseit is checked against the host’s APIs, for every stage; within_container = trueagainst the containers’ APIs, only for stages reserving the agent’sgpupool (sandbox.md); - each service on that platform with
gpu.usenotnoneandapis_anyset that provides a pool the stage reserves (requires.pools), against the host’s APIs.
A node runs the job only when every group shares an API with its list (explain: GPU_API_MISSING). A module whose
runner needs an API the node lacks on the host is not certified there, and its doctor should say undetected;
NodeClass.gpu_apis lets golden.list give nodes of different APIs different goldens. min_vram_gb is carried and
selects nothing yet.
doctor
Section titled “doctor”doctor --json prints a DoctorOutput to stdout and exits 0, even when unhealthy. It reports:
runner_protocol.supported: the protocol majors the runner speaks;health, the verdict on the module as a whole:healthy,unhealthy(should work here but something is broken; this alerts) orundetected(this node cannot do the module’s work; no alert). Only ahealthymodule is certified on the node (its goldens run), so only then do the stages that need certification run there;capabilities,attrs(includeplatform) andchecks, each{name, ok, detail}.
A doctor that prints a DoctorOutput shows that the module’s runner starts on the node, so the agent offers the module
whatever its health, and the stages that need no certification (bootstrap stages, and stages that compare nothing and
need no capability or pool: manifest.md) run there. Checks decide
which: a check named after a capability (a probe’s name, or a capability one of the module’s services provides)
proves that capability for the module, and when it fails the node lacks it for the module’s stages, whatever the probe
or service reports; a stage requiring it waits, any other stage keeps running. Name a check after the capability it
proves (java17 for the JDK a java17 probe also finds), and give checks that no stage depends on other names.
runner_protocol.disproved_capabilities(checks) is the reference. A doctor that crashes, hangs or prints no
DoctorOutput gets the module offered nowhere on that node.
detail says why, in full: the agent and the coordinator keep it as it is and show it whole (oarbank node show, the
console’s node page). If a runner shortens a long one, it keeps the end, where the cause of a failed import or command
usually is.
The agent runs doctor at install, after upgrades, when capabilities change, and on a slow timer. It must finish within 60 s.
Requirements the conformance kit checks
Section titled “Requirements the conformance kit checks”- Idempotent retries: the same spec in a fresh workdir, or after a kill mid-run, gives an equal result.
- Deterministic results: with
determinism = "exact", runs in fresh workdirs with different locales, CPU counts, path lengths and spaces in paths give the same digest. They are compared only within one platform whenresults.determinism_scope = "platform". - Writes: the runner writes nothing outside the work and data directories.
- Environment: the runner does not depend on inherited environment, the current directory or the network, unless the manifest declares it.
doctoris honest: it reportsundetectedorunhealthywheneverrunwould fail with exit code 3.
Source: spec/runner-protocol.md in the oarbank-sdk repository