Module protocol 1
The coordinator side of a module is a separate process. The core (oarbankd) spawns it and talks to it over newline-delimited JSON-RPC 2.0 on stdin/stdout: one JSON object per line, UTF-8, no embedded newlines. Anything the module writes to stderr goes to the module’s log. No module code ever runs inside oarbankd. Models: oarbank_sdk.module_protocol. Schemas: module-*-1.schema.json.
Lifecycle
Section titled “Lifecycle”- Spawn. The core starts the process with
coordinator.execfrom the installed bundle, in the module’s own venv (forruntime.kind = "uv"), with a clean environment and the module’s install directory as cwd. It applies the coordinator variant for its own platform (coordinator.for_platform), and the environment carriesOARBANK_PLATFORMand the module’scoordinator.env. - Handshake. Host → module:
initialize {protocol_versions, host, module_digest, settings}(settings: every declared key at fleet scope, the owner’s value else its default), wherehostis{name, version, capabilities, platform}(see “Host capabilities”). The module answers{protocol_version, module, capabilities}: it picks one major fromprotocol_versions, or fails with-32001anddata.supported. The host then sends theinitializednotification. - Verbs. Requests flow, at most
coordinator.concurrencyin flight at once, each bounded bycoordinator.timeouts_s. - Shutdown. The host sends the
shutdownnotification and closes stdin. After 5 s it terminates the process container (POSIX: SIGTERM, then SIGKILL; Windows: Job termination). Never rely on a notice.
The host restarts crashed modules with backoff. Repeated crashes put the module in the module_fault state. Its jobs stop being scheduled, jobs already running finish, and their results wait for evaluation. A module fault never counts against a node or a job.
The module is stateless
Section titled “The module is stateless”The host owns leases, attempts, retries, certification, replication, disputes and every row in the database. Each call carries everything it needs. Verbs must be pure functions of their input: same input, same output, with no filesystem or network access outside the declared host callbacks. The conformance kit checks this. That lets the host evaluate completions outside its write lock, cache verb results, and restart a module at any time. A module that needs durable state (a search’s trials, say) keeps it in its store: it writes documents through store.write effects and reads them back with host.store.*, so a restarted module picks up exactly where it was.
Verbs (host → module)
Section titled “Verbs (host → module)”| Method | Required | Purpose |
|---|---|---|
params.check |
yes | Validate and normalise a parameter set before any job exists. It returns ok, normalized_params and errors[] {path, message, code}. Search strategies call it for every sampled config. |
job.plan |
yes | Turn (params, datasets, study) into the evaluations to run. Each is a PlanItem: key_inputs (canonical; the host computes job_key = H(module_id, compat, key_inputs)), datasets, stages (the manifest stages to run, for example ["call","score"] or ["eval"]), per-stage resource overrides within the manifest’s bounds, and [beta] a group (at most 64 characters) and platforms (tokens or OSes). |
spec.build |
yes | Turn plan items into spec payloads, one per stage, at target_spec_version or lower. |
result.evaluate |
yes | Judge one completed attempt from (spec envelope, result envelope, stage). It returns verdict, reason, value, digest, digest_version, summary and fields. Never called for a bootstrap job: the host’s pin check is its verdict (manifest.md). |
golden.list |
yes | Golden jobs for a node class: platform, os_version, cpu, gpus, [beta] gpu_apis ({host, containers}: the GPU APIs the node provides, from core 2.5), capabilities (list) and pools (name → 1 if the node has it, 0 if not), never node identity. Each golden has key_inputs, datasets, stages and expected (at least digest and digest_version, or keys your golden.compare checks), and [beta] platforms (only nodes of these tokens or OSes get it) and expected_by_platform (the expected value per token or OS; the host resolves the token, then the OS, then expected). A verb may read its own bundle’s files (they are immutable), for example golden fixtures in goldens/*.json stored as Golden JSON; [goldens].fixtures only tells tools where they are; oarbank_sdk.goldens.load(root, glob, node_class) reads them, filtered and resolved for the node class. It must never read host state except through callbacks. |
result.merge |
multi-stage | Build the canonical result of an evaluation from its accepted stage results. |
result.upgrade |
optional | Upgrade a stored result payload from version N to version M. |
golden.compare |
optional | Custom golden comparison, used when goldens.compare = "verb". |
study.metrics |
optional | Values for the manifest’s ui.study_columns, per trial. |
params.distance |
optional | Distance between two parameter sets, used by search strategies and deduplication. |
ui.view.compute |
optional | Compute a declared module view from the inputs the host resolved (UI contract 1). |
op.plan / op.apply |
optional | Preview and apply one of the module’s own [[operations]]; op.apply returns effects (below). |
integrity.check |
optional | Verify the module’s own state through host callbacks; see “Integrity checks” below. |
move.preflight / move.postflight / move.cancelled |
optional | The module’s part in a coordinator move; see “Coordinator moves” below. |
campaign.tick |
optional | Advance one of the module’s running campaigns. The host calls it every few seconds per running campaign, with the campaign (including [beta] its placement {mix, unit, class, state}) and all its jobs and canonical results (each with the platform that produced it); it returns effects limited to coordinator.campaign_effects. With the campaign.tick.results capability each done job also carries its canonical result (see “Results in campaign.tick”). A campaign the module leaves running with no open jobs stays running until the module marks it done. |
workload.report |
optional | What one of the module’s workloads observed: the report events the agent shipped for it, in order. It returns effects limited to coordinator.workload_effects; see “Workload reports” below. |
An optional verb is available only if the module lists it in initialize.capabilities. A call to an unadvertised verb fails with -32003.
Verdicts
Section titled “Verdicts”| Verdict | Meaning | What the host does |
|---|---|---|
accept |
The result is valid. | Records it. It then goes through replica and quorum rules like any other result. |
reject |
The result is wrong for this spec: wrong mode, malformed, or a missing artifact. | Counts it against the node, as a bad result, and re-runs the job elsewhere. |
retry |
The failure was transient and says nothing about the node. | Re-runs the job without counting it as a fault. |
fail_permanent |
The job can never succeed. | Quarantines the job. The node is not blamed. |
reason is a core reason code or a module code of the form <module-short>/<code>.
Results in campaign.tick
Section titled “Results in campaign.tick”A campaign’s jobs carry their canonical result’s value, digest and declared fields (scalars). A module whose tick
needs the whole result, such as an ingestion job’s list of new items and the files it uploaded, declares the
coordinator capability campaign.tick.results (it needs campaign.tick and requires.core >= 2.3; a Python module
advertises it with Module(..., features=(mp.CAP_TICK_RESULTS,))). Then:
- At acceptance the host checks the payload of every evaluation’s result (a job of the campaign, its chain’s
merged result included) against
results.schema(JSON Schema) andresults.max_inline_kb(compact UTF-8 JSON). A result that fails is refused with the reasonresult_invalid: the attempt ends failed and spends the job’s stage retry (never the node’s breaker), so nothing invalid is ever canonical. - In the tick each done job carries
result: {payload, artifacts: [{name, files: [{path, digest, size}]}]}. The files are blobs the coordinator holds, sodatasets.createandfiles.putcan name them. A result accepted by a module version without the capability (a result-cache hit from an older version) is not delivered (result: null). One tick carries at most 8 MiB of results (mp.TICK_RESULTS_BUDGET), newest done jobs first; a done job left out hasresult_omitted: true. Keep what you have consumed in your store, and let results you no longer need age out.
Why a job failed
Section titled “Why a job failed”[beta] In campaign.tick and host.jobs.query, a failed or quarantined job carries failure (core 2.10.4). An older
core sends only the state. failure is {reason, code, detail, fault, retryable, final, attempt_id, node_id, at}:
reasonis the attempt’s end reason (exit_nonzero,bad_input,oom, …).codeis the runner’s ownfailure.jsonreason, such as<module-short>/<code>.detailis its detail, at most 512 characters here. The job’s views show it whole.faultandretryableare what the runner declared.finalis true when the runner declared that the job can never succeed (fault: "job", orretryable: false). Such a job endsfailedat once and is never retried (runner-protocol.md).
A score job whose call failed carries the call’s failure. Use failure to react in the module, for example by not
enqueueing the same configuration again or by marking the trial as rejected. jobs.retry clears it.
Workload reports
Section titled “Workload reports”A workload with reports = true (manifest.md) appends report events to its events file
(workload-protocol.md); the agent numbers them (seq, rising per workload on its node)
and ships them to the coordinator, at most 60 a minute per workload. The coordinator calls workload.report (capability
workload.report, core 2.10) with them:
- Params:
{workload, node, seat_id, fence, events: [{seq, t, data}]}, at most 200 events, oldest first.seat_idandfencename the seat the workload ran in (nullfor a workload without an identity);datais the event’s object, as the workload wrote it (at most 4 KiB). - Result:
{effects, message}. The effects are limited to the kindscoordinator.workload_effectslists, which are only the module’s own records:store.write,store.delete,files.write,files.put,files.delete. The host applies them atomically with the delivery and refuses the whole answer if one is not declared. - Idempotent by seq. A delivery may repeat after a failure: the same
seqfor the same workload on the same node is a retry of something already handled. Key what you write by it, or by the source’s own reference insidedata(a store document per report, keyed<node>.<workload>.<seq>or by that reference), so a repeat writes the same thing again. An error answer is not an acknowledgement: the host delivers the events again later. - Keep your own records. Amounts are integers in the smallest unit, each record keyed by the source’s own reference; a correction is a new record that reverses the old one. A module asks its owner only for public identifiers, never for a key that controls funds.
Host callbacks (module → host)
Section titled “Host callbacks (module → host)”A module can make requests back to the host while it is handling a verb, if the manifest lists the matching coordinator.permissions entry. Any other call fails with -32002.
| Method | Permission | Returns |
|---|---|---|
host.datasets.query {kind?, ids?, attrs, limit, with_files?} |
datasets:read |
Datasets matching the query (by kind and equality on attributes, or by id), each {id, kind, attrs} (with with_files, also files: {path, digest, size} and, when it has any, origins): the module’s own datasets and the operator’s unowned ones. kind is the short kind, as in [datasets].kinds and datasets.create: a dataset’s owning module scopes its kind, so kinds are never namespaced. attrs are the dataset’s meta. |
host.blobs.stat {digest} |
blobs:stat |
Whether the coordinator holds a blob, and its size. |
host.settings.get {key} |
settings:read:self |
A value from this module’s own settings at fleet scope: the owner’s value, else the key’s default; null for a key with neither or one the schema does not declare (manifest.md). |
host.secrets.get {name} |
secrets:read:self |
{set, value?}: the module’s value of a declared secret (manifest.md); node values never reach the coordinator side. Core 2.5. Never log it, return it from a verb or put it in a spec: a spec reaches every stage, and the conformance kit fails a verb whose answer holds one. Host.secret(name). |
host.store.get {collection, key} |
store:read:self |
One document from this module’s store, or null. |
host.store.query {collection, where, limit} |
store:read:self |
This module’s documents in a collection; where is equality on top-level fields. Each document carries its _key. |
host.nodes.query {certified_for_self, platforms} |
nodes:read |
Ready nodes (node_id, hostname, online, module_state, and with nodes.platform also platform, os, arch, os_version), by default only those certified for this module; platforms (tokens or OSes) filters them. Host.fleet_platforms() counts them per platform. |
host.files.list {prefix, limit} |
files:read:self |
This module’s files under a path prefix, sorted: path, digest, size, updated_at. |
host.files.stat {path} |
files:read:self |
One file’s entry, or exists: false. |
host.files.read {path, offset, length} |
files:read:self |
Up to 1 MiB of a file as content_b64, with the file’s size, digest and eof. Host.files_read reads a whole file. |
host.jobs.query {campaign_id, states?, limit} |
jobs:read:self |
This module’s jobs in one of its campaigns: job_id, job_key, state, kind, stage, dataset_id, labels, group, target_node, and for done jobs the canonical result’s value, digest, fields, node_id and platform. [beta] For failed or quarantined jobs (core 2.10.4), failure: why the job ended, as in campaign.tick (see “Why a job failed”). |
Effects
Section titled “Effects”op.apply and campaign.tick never write. They return effects, which the host checks (declared kind, ownership) and applies on its single writer inside the operation’s transaction, together with its audit row. A module can only touch its own campaigns, jobs, store documents and settings.
| Kind | Arguments | Effect |
|---|---|---|
campaigns.create |
campaign_id (a lowercase letter, then 3–40 of [a-z0-9_]; c_… by convention), name, priority, weight, labels, [beta] placement {mix, unit, bind, pin}, [beta] settings {key: value} |
A running campaign owned by the module. An id that already exists is an error and the whole operation fails: derive ids so a repeat is intended, or check your store first. placement may be stricter than the manifest’s, never looser (422 placement_looser_than_manifest); pin binds the campaign to one class now (platforms.md). settings (host capability campaign_settings.v1) overrides settings while the campaign runs: each key is one of the module’s own settings declared "x-oarbank": {"campaign": true} or a core key a campaign may override (replica_rate, jobs, run_on_battery, user_present_slots, which may only tighten: a value looser than the fleet’s is refused); a key the owner locked, an undeclared key or an invalid value is 422 bad_settings and the whole operation fails. |
campaigns.update |
campaign_id, and any of state (running, paused, done), name, priority, weight, labels (merged), message, [beta] settings {key: value | null} |
Changes the campaign. done is how a module finishes one. settings changes its overrides as campaigns.create does; null resets a key to what the campaign inherits. |
campaigns.cancel |
campaign_id |
Cancels it and its open jobs. |
jobs.enqueue |
campaign_id, jobs[] of {job_key (exactly what keys.job_keyreturns: 64 lowercase hex characters, optionally:; anything else is 422 bad_job_keyand nothing is enqueued), spec, stage, spec_version (default 1), labels (your grouping: shown in views and returned byhost.jobs.query; part of the no-op check), dataset_id, datasets, mounts, resources (what the job reserves; default: the stage's manifest resources; it must fit a node), timeout_s, priority, subpriority, target_node, name, group, platforms} (at most 5000; [beta] stage: run exactly this standalone stage, never the chain (host capability jobs.stage; 422 bad_stage for an unknown or chain stage); [beta] group: at most 64 characters, the unit of a group placement; platforms: tokens or OSes the job may run on, at least one of which its stage runs on; [beta] images: at most 16 digest-pinned container images its runner may run, each one of the module’s containers or inside one of its container_sets, for a stage that reserves the containers pool (422 image_not_approved, bad_images; core 2.5); a job runs set images only if it lists them) |
Adds jobs. Without stage, a job runs the default stage, or the chain head → tail when the module’s pipeline is split. With stage, it reserves that stage’s resources and runs only where that stage may run. spec is the stage payload; see “Jobs on the wire” below. The same (job_key, labels) twice in a campaign is a no-op. An untargeted job whose key already has a canonical result of this module is done at once (result cache). A done campaign runs again. |
jobs.cancel |
job_ids[] |
Cancels the module’s open jobs. |
store.write / store.delete |
collection, key, doc (an object, at most 256 KiB) |
Upserts or removes a module document. |
datasets.create |
dataset_id, kind (a short kind from [datasets].kinds; 422 undeclared_kind otherwise), meta, files[] of {path, digest, size, origins?} (digest and size always), [beta] platform (a token; required for kinds in [datasets].platform_bound) |
Registers a dataset. A file without origins names a blob the coordinator holds and the module can see, at exactly size. [beta] A file with origins (1 to 8 https URLs on public host names; host capability datasets.origins) may name a blob the coordinator does not hold: nodes fetch it from an origin and check its digest, and the coordinator fetches it only when every origin failed for a node (see “Datasets by origin”). An origin the URL rule or the operator’s origin host policy refuses is 422 origin_refused. A dataset with a platform is used only by jobs on that platform. An id that already exists: with the same kind, meta, files and platform, the effect is skipped (a repeat is harmless); with anything else it is an error (409 dataset_exists) and the whole operation fails; another owner’s id is 409 dataset_owned. A pinned id ([[datasets.pinned]]) is registered only with its pinned kind, files, platform and (when the pin sets it) meta, else 422 pin_mismatch. |
datasets.update |
dataset_id, meta |
Merges meta into one of the module’s datasets key by key at the top level; a key set to null is removed. kind, files and platform never change (422 dataset_immutable): nodes stage a dataset’s files by id, so new files need a new id. Another owner’s dataset is 403 not_owner, an unknown id 404 unknown_dataset. Core 2.3. |
datasets.delete |
dataset_id |
Removes one of the module’s datasets; an unknown id is a no-op. Refused while a pending or leased job names it (409 dataset_in_use) and for the host’s artifact datasets (art:…, 422 host_dataset). Blobs stay. Another owner’s dataset is 403 not_owner. Core 2.3. |
files.write |
path, content_b64 (at most 1 MiB decoded) |
Stores a small file in the module’s files. |
files.put |
path, digest |
Names a blob the coordinator already holds (a job artifact, a dataset file) as one of the module’s files. |
files.delete |
path, or prefix |
Removes files (the blobs stay; other rows may name them). |
module_settings.update |
{key: value} |
Sets the module’s own settings at fleet scope, key by key (null resets a key to inherit). Every key must be declared in the settings schema and every value valid against it, or the whole effect is refused (manifest.md). |
oarbank_sdk.effects builds them: fx.campaign_create(cid, name, priority=0, weight=1, labels=None, placement=None),
fx.placement(mix, unit=None, pin=None, bind=None), fx.jobs_enqueue(cid, jobs) with
fx.job(job_key, spec, *, stage=None, group=None, platforms=None, target_node=None, ...) items, and
fx.datasets_create(dataset_id, kind, files, meta=None, platform=None) (it checks each file with
oarbank_sdk.origins.file_problem), fx.datasets_update(dataset_id, meta) and fx.datasets_delete(dataset_id). Unset
fields are left out.
Datasets by origin
Section titled “Datasets by origin”Most batch work starts from a public download (model weights, reference data) or the user’s own files. A dataset names every file by sha256 and size, so where its bytes come from never changes what it is:
fx.datasets_create("model:llama-3-8b", "model", [ {"path": "model.safetensors", "digest": "<sha256>", "size": 16_060_000_000, "origins": ["https://huggingface.co/<org>/<repo>/resolve/<rev>/model.safetensors"]}])- Each node downloads from an origin straight into its blob cache (resumable, digest-checked) and keeps it for every job that mounts it. No byte passes through the coordinator unless every origin fails for that node; the coordinator then fetches the blob from the origins itself, once, and serves it.
- Origins are
httpsURLs on public host names (never IP literals or local names;oarbank_sdk.origins.url_problem), and whoever fetches refuses a name that resolves to a non-public address. The operator may restrict origin hosts (settings.origins.update), which applies when a dataset is registered and whenever its files are served. - Files the user has rather than an origin arrive by upload (
oarbank dataset upload <dir> --kind <kind>, or the console), as the operator’s dataset or a module’s (--module). A module’s importer is an operation withtarget = "dataset"that reads the uploaded dataset withhost.datasets.query {ids, with_files: true}and registers its own datasets naming the same blobs. - A dataset’s files and a campaign’s artifacts come back as files with
oarbank dataset downloadandoarbank campaign download(resumable, digest-checked) or the console’s download links: modules need no export step of their own.
Host capabilities
Section titled “Host capabilities”initialize’s host.capabilities names the host features a module may rely on. A module that uses one of these only
at run time checks it with ctx.host_has(name) (constants in oarbank_sdk.module_protocol); a manifest that depends on
them needs requires.core >= 2.2 instead.
| Capability | The host |
|---|---|
placement.v1 |
Honours campaigns.create placement, jobs.enqueue group and platforms, datasets.create platform, and reports placement and job platform in campaign.tick and host.jobs.query. |
nodes.platform |
Returns platform, os, arch and os_version from host.nodes.query and accepts its platforms filter. |
goldens.by_platform |
Passes the node’s platform and os_version in golden.list’s node class, drops goldens whose platforms exclude the node, and resolves expected_by_platform. |
coordinator.variants |
Starts the coordinator side with its variant for the host’s platform, sets OARBANK_PLATFORM and coordinator.env, and sends host.platform. |
jobs.stage |
Honours jobs.enqueue items’ stage (core 2.3). A 2.2 host ignores it and runs the default form. |
datasets.origins |
Accepts datasets.create files with origins for blobs it does not hold (core 2.5). An older host refuses them (422 unknown_blob). |
campaign_settings.v1 |
Honours campaigns.create and campaigns.update settings and sends campaign.settings in campaign.tick (core 2.9): the module’s own settings at fleet scope with the campaign’s overrides on top. A campaign’s overrides of node keys reach its jobs’ runners in OARBANK_SETTINGS_FILE. |
host.platform [beta] is the coordinator host’s platform token (ctx.host_platform). Verbs keep job keys
platform-independent: never put it in key_inputs.
Module files
Section titled “Module files”A module that needs durable files on the coordinator keeps them in its files: a private namespace of
relative paths (plain segments, no . or .., at most 512 characters), each naming a content-addressed blob.
Module code never touches the coordinator’s disk. Verbs read through host.files.* and write by returning
files.* effects, which the host applies atomically with the operation’s audit row. Small files arrive inline
(files.write, at most 1 MiB). Larger ones are produced by a job: its runner writes them as artifacts, and the
module binds them with files.put. Files, store documents, datasets and settings are everything a module
keeps, and all of it moves with the coordinator. oarbank_sdk.effects builds these effects.
Integrity checks
Section titled “Integrity checks”integrity.check {scope, deep, move_id, now} asks the module to verify its own state, reading only through host
callbacks. It returns ok, a list of named checks (error checks decide ok; warn and info are shown),
and an optional fingerprint: a digest of the state the module considers its own (oarbank_sdk.effects.fingerprint).
The host adds its own checks of the module’s files (every blob present, sizes or, with deep, digests), records
every outcome and shows the last one on the module’s page.
| Scope | When |
|---|---|
routine |
Daily. A failure opens an integrity_failed:<module> alert; a later pass resolves it. |
on_demand |
An operator ran oarbank module check or pressed “Check integrity”. |
move_source |
The old coordinator, frozen for a move. A failure aborts the move unless the operator forced it. |
move_target |
The target, on its verified copy before it reports ready. The fingerprint must equal the source’s, or the move aborts. On the target, files that your move rules rebuild or drop are listed but cannot be read. |
The fingerprint is compared across machines: derive it from your state only (sorted paths and digests, store documents), never from time, paths on disk or anything your move rules leave behind.
Coordinator moves
Section titled “Coordinator moves”The coordinator can move to another host (oarbank coordinator prepare, move, finalize). Everything a module
keeps through the SDK moves with it and is verified by digest. A module takes part in three ways.
Rules (manifest [coordinator.move]). Each rule names a files prefix or a store collection and a class. The first
matching rule wins; anything unmatched is carried.
| Class | What happens |
|---|---|
carry (default) |
Moved and verified. |
rebuild |
Not transferred; deleted on the target and listed in move.postflight’s skipped, so the module recreates it (requires move.postflight). |
drop |
Scratch: deleted on the target. |
coordinator.move.effects lists the effects the move verbs may return.
Verbs.
| Verb | Called | Returns |
|---|---|---|
move.preflight {move_id, to_url, not_before, phase, now} |
phase = planned when the move is reviewed or requested (effects ignored); phase = draining every few seconds once the time lock passed and dispatch stopped. |
blockers[] {code, message}: while any remains, the cutover waits (up to 10 minutes, then the move aborts, unless the operator forced it). checks[], and effects (applied while draining, e.g. pause a campaign). |
move.postflight {move_id, from_url, epoch, skipped[], now} |
Once, on the new coordinator after it took over. | effects (resume work, rebuild what was skipped) and checks[]; a failed error check raises an alert. |
move.cancelled {move_id, reason, now} |
On the old coordinator when the move was cancelled or aborted before the commit decision. | effects (resume what preflight paused). |
The order on a move: preflight (planned) at request time → preflight (draining) until no blocker → freeze →
integrity.check {move_source} → the target copies and verifies → integrity.check {move_target} on the copy →
fingerprints compared → commit → move.postflight on the new coordinator. Any failure before the commit
decision thaws the old coordinator and calls move.cancelled.
Jobs on the wire
Section titled “Jobs on the wire”- Keys.
job_key = H(module_id, compat, key_inputs), computed withoarbank_sdk.keys.job_keyby a module that enqueues through effects. The result cache and replica checks are scoped to the module. - What a job carries. A
jobs.enqueueitem’sspecis the module-owned payload. The envelope fields (datasets,mounts,resources,timeout_s) belong on the item. If they appear insidespec, the host lifts them out.compatandexpectedinsidespecare host-side and never reach the runner. - What the runner receives. The agent always writes a
SpecEnvelope(envelopes.md) at grant:schema = <name>/spec@<spec_version>, the module’s id and version, the job key, the stage, the datasets, mounts and stage inputs, the resources the stage reserves, and the payload. - Stage chains. A module whose manifest has a stage
Bwithafter = "A"can run a job as the chain A → B. The host creates the A job with key<key>:A, feeds A’s artifacts to B asinputs, and asksresult.mergefor B’s canonical result. Its single-stage form is the default stage: the stage markeddefault = true, or the only standalone stage (neitherafteranother nor depended on). - Bootstrap jobs. A job that names a bootstrap stage may run on a node before the module is certified there (wherever its runner starts), with the bootstrap grants; the host registers its pinned datasets itself, and the module sees the job done (with
campaign.tick.results, its empty payload and its artifacts). Nothing a bootstrap node sends reaches module code otherwise. - Staged jobs. A
jobs.enqueueitem withstageruns exactly that standalone stage (a utility job such as an ingestionsync), whatever the pipeline mode: it is never expanded into the chain. Its envelope andresult.evaluatecarry its stage, except the default stage, which stays absent as for any single-stage job. The result cache is keyed byjob_keyalone, so key a staged job with its stage:keys.job_key(id, compat, key_inputs, stage). - Replicas and comparisons. The host re-runs a sample of finished jobs on another node and compares the digests (else
the values to 6 decimals); disagreement opens a dispute settled by a third node, and a node that disagrees with itself
is convicted. A job of a stage that does not compare (its effective determinism,
stages[].determinismor elseresults.determinism, isnone) is exempt: it is never replicated, a late second result is recorded and never compared, it neither takes nor serves a result-cache hit, and no golden runs it. It is otherwise an ordinary job: fenced by generation, with its stage’s retry and placement. When it needs no capability and no pool it also runs before the module is certified on a node, wherever its runner starts (manifest.md); otherwise it runs only on nodes certified for the module. - Goldens. The host builds each golden’s spec by calling
spec.buildwith aPlanItemof the golden’skey_inputs,datasetsandstages. It runs the stage the golden names; with no stage named, the one stage the spec has. It then accepts the result whengolden.comparesays so, or, for a module without that capability, when the evaluated digest equalsexpected.digest. A golden for a stage that does not compare is a module error: nothing is queued.
Operations: plan and apply
Section titled “Operations: plan and apply”op.plan is called for every operation of tier T2 or above, and for any operation with preview = true;
its summary, diff and effects are what the operator reviews. op.apply does the change. To refuse invalid
input in either verb, raise invalid params (-32602, RpcError(mp.ERR_INVALID_PARAMS, message)); the host
shows the message. op.apply can also answer response = "errors" with per-field issues.
Digests
Section titled “Digests”Compute a digest over a canonical encoding: oarbank_sdk.keys.canonical_json is RFC 8785 (JCS), the one the host uses for
job keys, with three refusals (NaN and infinities, integers beyond ±2^53, non-string keys) and shared vectors in
vectors/ so every language computes the same bytes. Send numbers that must compare exactly beyond 2^53 as
decimal strings.
Notifications
Section titled “Notifications”| Method | Direction | Params |
|---|---|---|
initialized |
host → module | none |
$/cancel |
host → module | {id}: stop working on that request. The module answers it with -32004. |
log |
module → host | {level, msg, data}: goes into the module’s event journal. |
shutdown |
host → module | none |
Errors
Section titled “Errors”| Code | Meaning |
|---|---|
| -32700, -32600, -32601, -32602, -32603 | Standard JSON-RPC errors: parse error, invalid request, method not found, invalid params, internal error. |
| -32001 | Unsupported protocol version; data: {supported, requested}. |
| -32002 | Permission denied for a host callback. |
| -32003 | The verb’s capability was not advertised. |
| -32004 | Request cancelled. |
An error response to result.evaluate is not a verdict. The host keeps the completion pending and retries it after the module recovers. So a module bug can delay results, but cannot corrupt them.
Example exchange
Section titled “Example exchange”{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocol_versions":[1],"host":{"name":"oarbank","version":"2.0.0"},"settings":{}}}{"jsonrpc":"2.0","id":1,"result":{"protocol_version":1,"module":{"id":"dev.codonic.oarbank.toy","version":"0.1.0"},"capabilities":[]}}{"jsonrpc":"2.0","method":"initialized","params":{}}{"jsonrpc":"2.0","id":2,"method":"params.check","params":{"params":{"n":-1}}}{"jsonrpc":"2.0","id":2,"result":{"ok":false,"errors":[{"path":"/n","message":"n must be >= 0","code":"toy/negative_n"}]}}Python runtime
Section titled “Python runtime”oarbank_sdk.server.Module implements this protocol for Python authors:
- handshake and capability advertisement;
- typed params and results;
- host callbacks through
ctx.host(andctx.host.fleet_platforms()); - the host’s capabilities and platform through
ctx.host_has(name)andctx.host_platform; - cancellation through
ctx.cancelled; ctx.log;- stdout protection:
printgoes to stderr. - capabilities that are not verbs (
campaign.tick.results) throughModule(..., features=...).
oarbank_sdk.client.ModuleClient is the matching unsupervised host, for tests and tools. The reference module is examples/toy.
Modules in other languages implement the wire format above directly. The JSON Schemas in schemas/module-*-1.schema.json describe every message.
Driving a module from tests
Section titled “Driving a module from tests”oarbank_sdk.client.ModuleClient runs your coordinator as the host does and serves host callbacks from a
dict:
from oarbank_sdk.client import ModuleClientc = ModuleClient.spawn([sys.executable, "-I", "primes_module.py"], cwd=module_dir, callbacks={"host.settings.get": lambda p: {"value": None}}, permissions={"settings:read:self"})info = c.initialize() # an InitializeResult (pydantic model)r = c.call("params.check", {"params": {"n": 10}}) # plain dicts in and outc.close()Source: spec/module-protocol.md in the oarbank-sdk repository