CSI and FUSE sessions

Documentation home · Filesystem semantics · RWO ownership

The node image supports a kubelet-facing CSI frontend and a separate FUSE mount owner. The filesystem is implemented in-process; runtime images contain no mount binary, shell or external FUSE helper.

Split process boundary

The frontend uses --mount-service=unix:///… and forwards node calls over the shared Unix socket. The owner runs /node without that option and holds the kernel FUSE connections, direct authenticated worker clients, credential refresh, handle renewals and RWO sessions. Both processes need the correct node identity, shared socket visibility and kubelet mount propagation.

Restarting only the frontend can preserve a surviving owner's established mounts and descriptors. Losing the owner, entire Pod, kernel or host disconnects those sessions. A fresh process does not inherit another process's FUSE connection; normal unpublish/unstage and remount are required. Existing I/O does not traverse the frontend or the Kubernetes API, although background credential refresh does.

Stage and publish

1. NodeStageVolume validates the volume identity, generation, endpoints, access mode and trusted credential namespace, then creates the staging filesystem.
2. For RWO, the owner persists its local session record and acquires the ordered ownership epoch before reporting a successful stage.
3. NodePublishVolume bind-mounts that staged filesystem into the Pod's target. Multiple publications can share one stage; read-only publication is enforced on the bind mount rather than inventing a separate filesystem copy.
4. NodeUnpublishVolume removes only the matching owned publication. Unstage is refused while matching bind/subPath references remain.
5. NodeUnstageVolume tears down the owned stage, stops renewal and releases the local session. RWO remote release may defer safely to lease expiry if quorum or credentials are unavailable after local references are gone.

A stage retry must match its existing volume, generation and mode. An ordinary directory, a foreign mount or an incompatible session is not accepted as an already healthy filesystem.

Mount identity and path validation

Targets must be clean absolute descendants of the configured kubelet mount root. Parents containing symlinks, unexpected files or foreign mount state are rejected. The driver inspects kernel mount metadata, including source volume, filesystem fuse.tak-os-replicated, device and mount root, before accepting or removing an existing mount. It also scans volume references when the stage path disappeared.

A disconnected FUSE mount can return ENOTCONN or block filesystem calls. Kernel mount metadata lets cleanup inspect ownership without walking through that dead filesystem. Keep consumer references closed; do not replace normal CSI cleanup with forced unmounts or deleting staging directories/session files.

File RPC and retries

The mount owner speaks authenticated HTTPS to eligible worker endpoints. An immutable volume/generation identity and request ID accompany operations. A single logical call has a 15-second budget with three-second HTTP attempts; retries keep the exact serialized request and identity. Endpoint switching and coordinator hints cannot change membership or permit minority service.

Client admission is bounded to 64 concurrent calls. Reads/writes carry at most 1 MiB of data per protocol request; a large application operation can become multiple ordered FUSE requests. There is no multi-request transaction guarantee. Direct I/O and zero entry/attribute/negative-cache lifetimes provide the stated operation-level visibility. See uncertain results before retrying an application mutation after a timeout.

Credentials and permissions

The owner retrieves <volumeID>-auth only from its configured manager namespace, not from a caller-selected Secret namespace. It retains the file-service token, CA, leaf certificate/key and overlap fields needed by its client. Credentials refresh outside ordinary I/O; errors are not hidden by disabling TLS checks.

Basic mode, ownership and umask are supported, and Linux default_permissions enforces application access at the FUSE boundary. fsGroupPolicy: None means existing directory ownership is not recursively rewritten. Mount flags, fsType, CSI mount-group requests, raw block and unsupported access modes are rejected. The worker authenticates volume clients, not individual Unix application users; a trusted privileged node remains inside the volume trust boundary.

Operational signals

Structured logs record operation, outcome, duration, node/volume identity, request correlation and bounded error categories. Metrics expose stage/publish/ unmount failures, active mounts, logical RPC retries and endpoint changes without paths or request IDs as metric labels. No credentials or file contents belong in logs. Use mount recovery for disconnected sessions and observability for signal interpretation.