rwo-support

Application-facing RWO ownership

Documentation home

Application-facing RWO is implemented beside RWX. Historical local/kernel and single-node lab cases retain their recorded version boundaries; multi-host Kubernetes acceptance remains unverified. See release policy for build and publication scope.

Contract

An RWO volume has one active node mount session. Several application Pods on that node can publish the same staged FUSE connection. This follows Kubernetes RWO usage; it is not ReadWriteOncePod. The CSI controller and node advertise SINGLE_NODE_MULTI_WRITER, accept that explicit access mode and the legacy SINGLE_NODE_WRITER, and reject SINGLE_NODE_SINGLE_WRITER. The two accepted single-node modes select the same immutable RWO contract. See the CSI access modes and NodePublishVolume rules.

Provisioning persists spec.request.accessMode: rwo. The manager sends the immutable mode in volume context and worker configuration. An absent request field denotes existing RWX volumes; RWX configuration and journal identity bytes remain compatible. Requesting RWX for an RWO volume, or RWO for an RWX volume, fails validation. Mount flags, fsType, fsGroup, raw block and RWOP remain unsupported. Backing local PVC access mode is independent of the application's access mode.

The manager continues to advertise no controller attach capability. Ownership is established by the normal NodeStageVolume operation in the volume's Raft group. Kubernetes owns placement and lifecycle, and stores no per-write ownership or file data.

Ordering and failover

claim_mount, renew_mount and release_mount are quorum-ordered commands. A claim contains the node identity and a random mount epoch. Its lease lasts 90 seconds from committed coordinator time; the mount owner renews every 20 seconds. An active epoch excludes every other epoch, including another frontend session on the same node. Multiple Pods share their node's one staged epoch.

Every normal file/metadata read is checked against the current epoch after the existing verified-quorum read barrier. Every normal mutation, including open, close and handle renewal, is checked in deterministic committed order. The check also precedes returning a cached successful mutation, acquisition or renewal response. Volume fencing and quorum-ordered handle reclamation remain available without a client epoch. Release is idempotent and cannot clear a successor's ownership; it needs no additional dedup entry.

After an unrenewed lease expires, another node can commit a new epoch through the existing replica quorum. The former epoch then fails on data and metadata operations even when its Pod or FUSE connection remains present. No Kubernetes force-detach or Pod deletion establishes authority. An expired epoch cannot renew or silently reacquire ownership. Applications must unpublish/unstage the old mount before staging a new epoch. A stale-epoch error does not establish that an earlier ambiguous write failed; request-outcome and recovery rules still apply.

Lease time uses coordinator wall time clamped by the durable command-time floor, as in handle leases. Synchronized clocks are required. Clock jumps or a mount-owner pause can fence a healthy mount and reduce availability. Safety comes from quorum ordering and epoch checks, not a client's clock or the expected duration of a failure. The 90-second value is not an end-to-end application recovery promise: scheduling, quorum, disk state and mount recovery can add delay. Replica count and quorum are never reduced.

The immutable mode is bound into the original journal identity before service starts and into baseline/checkpoint identity. Checkpoints carry current owner and expiry; replay retains the same ordering. Reopening retained storage with the other access mode fails before service. Old runtimes cannot open new RWO journal identities; rollback of RWO data to an RWX-only runtime is unsupported.

Retry, restart and cleanup

Before acquisition the mount owner atomically publishes and fsyncs a mode-0600 <staging_target_path>.rwo-session record beside the staging directory. It holds volume/generation, node, endpoints, epoch and a completed-stage flag. It contains no TLS credentials or bearer tokens and is never logged. Credential recovery uses the trusted provider. Session files reject symlinks and incompatible volume identity. Normal retries reuse the stored epoch.

The completed flag is durable before CSI reports staging success. If an initial stage never succeeded and its mount is positively absent, a retry can release an expired pending epoch and atomically create a fresh one. A stage that previously succeeded requires normal unstage recovery after expiry; it does not silently replace its identity. Following mount-owner restart, stale FUSE connections follow the existing unpublish/unstage recovery path, using kernel mount ownership and the persisted epoch. In the deployed split-service layout, replacing only the CSI RPC frontend leaves the surviving mount owner and its renewal running.

Unstage checks publication references and scans kernel mount sources even if the staging mount has disappeared. It releases ownership only after the connection and every volume reference are gone. An unmount or reference-inspection failure leaves ownership metadata intact. Remote release is attempted with a five-second budget. If credentials, workers or quorum have disappeared, local cleanup still completes and the old lease expires through normal authority checks. The mount_release_deferred operation records this fallback in JSON logs and bounded Prometheus operation labels. No allocation edits or finalizer removal are part of normal recovery.

Operating boundary

The ownership protocol is implemented. Historical native/kernel and same-host observations do not establish multi-host Kubernetes takeover or mount propagation for the current floating dependency build. There is no fixed application failover SLO. An RWO claim can retain multiple Pods on its owning node; independent nodes must not be treated as simultaneous writers. Maintain synchronized clocks and preserve the local durable session until normal unstage completes.

See supported limits and historical release observations. No test harness or pending test procedure forms part of the operator instructions.