Launch is a midpoint.

Stage contract

• Owns: execution controller + Nodus runner
• Before: generation.start_requested for a reserved lease
• After: customer-visible running only after verified runner heartbeat
• Transport: Kafka command + bidirectional gRPC/mTLS (runner ↔ gateway)
• Durable ack: desired-generation record persisted; runner proves identity before process start

A provider reporting “VM created” is not sufficient to advance customer-visible state.

Pilot today Runner control is versioned HTTPS JSON (protocol_version: runner-v1) against /v1/runner/*, not Kafka commands over bidirectional gRPC and mTLS. Shared wire types live in one package so both ends move together. The transport swaps later without touching what this note is about: generation fencing, identity before running, and manifest semantics are transport-independent by design.

Reconciliation model

The execution controller continuously compares desired and observed state:

Desired:  stage generation N is running under lease L
Observed: provider runtime, runner identity, heartbeat,
progress, checkpoint state
Action:   provision | start | stop | fence | recover

That makes controller restarts and duplicate messages safe. An API request never waits for a workload to finish. Desired state is durable, observed state derives from supplier and runtime signals, and every action is safe to repeat. Correctness never depends on in-memory task ownership.

Provisioning flow

lease reserved
  → adapter provisions runtime
  → controller injects short-lived generation credentials
  → Nodus runner authenticates
  → runner heartbeat establishes generation ownership
  → user process starts

Runners are immutable, signed, versioned artifacts. They take short-lived generation-scoped credentials, run in a least-privilege sandbox, emit structured telemetry, and are denied broad customer storage access. The customer process is isolated from Nodus control credentials wherever the runtime allows.

Runner responsibilities

• Pull image and materialize authorized inputs
• Launch the customer process
• Emit heartbeats, progress, logs, resource telemetry
• Request / coordinate safe checkpoints
• Publish signed lifecycle events
• Enforce generation-scoped output permissions
• Report observed usage for settlement

Continuity during the run

The controller and runner honor the envelope’s continuity mode:

• checkpointed: coordinate verified manifests; reclaim → restore gen N+1
• restartable: track shard / job cursor; reclaim → requeue unfinished units
• ephemeral: no durable progress required; interrupt ends or retries from origin per policy

“When capacity disappears, execution keeps moving” holds only where the chosen mode has a defined recovery path. Checkpointing runs concurrently with execution, never as a post-run stage.

Budget and cancellation

The controller tracks observed against predicted spend, and can warn, stop launching stages, request a checkpoint, or cancel per policy. A resumable workload gets a checkpoint request before termination whenever the underlying capacity allows it.

Operational requirements

• Correlation ID through API, route, lease, runner, checkpoint, and invoice records
• Heartbeat timeout policy with jitter; a missed heartbeat does not immediately imply data loss
• Tenant-aware log/metric retention, redaction, and cost controls
• Controller actions rate-limited and backpressured so a supplier incident cannot trigger an uncontrolled recovery storm

Multi-stage DAGs are first-class, not a later phase. A workload compiles at submit time into an immutable plan: one implicit main stage when nothing is declared, an explicit acyclic fan-in DAG when something is. Every ready stage gets its own envelope, route, lease, and generation. Handoffs are manifest-only, so a downstream stage resolves its inputs from the upstream stage’s committed outputs map before it starts. A missing output fails the handoff rather than falling back to something convenient, and an upstream stage never re-runs because a downstream one was reclaimed.