Administrator guide

Documentation home · User guide · Operator runbook

Use this guide to plan installation, expose storage to applications, monitor it and coordinate maintenance. The detailed references linked here define lifecycle, filesystem and recovery contracts. Application teams can start with the user guide after you provide a working StorageClass.

The documented release is 4.1.1, a documentation update to the 4.1.0 runtime. The shared NODE runtime requires local CSI 2.1.0 and is for staging evaluation. Repository records establish compilation and static review, not cluster runtime recovery or footprint measurements for this release. Confirm actual image publication before promotion; see builds and images. Earlier runtime reports retain their original source/image scope.

Plan the installation

The intended audience is small to medium VM deployments, roughly 4–16 GB RAM and 4–16 vCPUs per node. These are design targets, not measured capacity guarantees. Shared workers accept a common process failure boundary. Read the support scope before choosing workloads or availability expectations.

The required majorities for one through four replicas are respectively one, two, two and three. RF1 has no redundancy and RF2 needs both members. The driver never silently lowers quorum. Permanent original-node loss can leave replacement and reclamation blocked; plan for external recovery.

Install through the deployment repository

This source repository supplies runtime code, a CRD and integration fragments. Installation is managed in the separate Orbit GitOps repository. The orbit-deployment symlink points outside this repository; deployment edits follow that repository's instructions and operational authorization.

Follow getting started in order:

1. Establish the local driver and delayed-binding backing class. For shared mode, deliver local CSI 2.1.0 consumer publication and its explicit consumer root.
2. Install the current CRD before provisioning or transitioning volumes. Preserve its immutable request and runtime rules.
3. Configure manager, external-provisioner, CSI NODE frontend, FUSE owner, registrar, service accounts and RBAC. The provisioner needs --extra-create-metadata. Set CSIDriver attachRequired: false and fsGroupPolicy: None.
4. Supply matching manager, node and replica images. Retain --worker-image for legacy workers. Select --replica-runtime explicitly: it defaults to shared for new generations; pod is the legacy compatibility path. Existing activated volumes retain their persisted mode until an explicit transition.
5. For shared mode, merge the NODE integration fragment into the existing DaemonSet. Preserve frontend, mount owner and registrar. Add /node --role=replica-host, downward-API Node/Pod identity, host label, local-CSI socket, persistent state/mount roots and additive RBAC.
6. Configure real cluster DNS, network access, resources and health monitoring consistently, then deliver through the deployment repository's process.

The source YAML fragments are incomplete installation examples. The shared replica host needs HostToContainer propagation for the same backing directory that local CSI publishes with Bidirectional propagation. Preserve its private state root across restarts. Mount-owner FUSE privileges and kubelet propagation remain separate requirements. Use configuration for flags, defaults, socket paths and DNS settings.

Every scratch image needs an explicit executable command and contains no shell. The NODE image supplies both FUSE/client roles and the shared replica host; the standalone replica image remains for legacy compatibility. A new shared volume has individual backing PVCs and worker endpoints without per-volume worker or controller Pods.

Publish application storage policy

Create a replicated StorageClass through your managed deployment. The StorageClass example uses three copies, Immediate binding, Retain reclaim and disabled expansion.

Choose and communicate:

• parameters.replicas from one through four; omission defaults to one.
• parameters.backingStorageClass naming the existing local class.
• Outer Retain or Delete based on the intended application lifecycle.
• Whether teams may override replicas with the creation-time PVC annotation.
• Shared UID/GID, directory initialization, backup and support procedures.

Keep the outer class distinct from the delayed-binding local backing class. Caller topology constraints, fsType, mount options, clones and unrecognized StorageClass parameters are rejected. Replica count and access mode are immutable creation inputs; edits do not resize membership or migrate data.

Application teams use RWX for multiple nodes or RWO for one owning node. RWO is not a single-Pod policy. ReadWriteOncePod, ReadOnlyMany provisioning and raw block are unsupported. Communicate the workload exclusions and permissions in the user guide before handing over a class.

Inspect a deployment and volume

Use an explicitly selected authorized kubeconfig for every cluster command. Replace these example values with your deployment's identities:

CSI_KUBECONFIG=/absolute/path/to/authorized-kubeconfig
CSI_CONTEXT=your-context-name
CSI_NAMESPACE=kube-system
APP_NAMESPACE=your-application-namespace
CSI_CLAIM=site-content
kubectl --kubeconfig "$CSI_KUBECONFIG" --context "$CSI_CONTEXT" get nodes \
  -o custom-columns=NAME:.metadata.name,UID:.metadata.uid
kubectl --kubeconfig "$CSI_KUBECONFIG" --context "$CSI_CONTEXT" \
  -n "$APP_NAMESPACE" get pvc "$CSI_CLAIM" -o wide
kubectl --kubeconfig "$CSI_KUBECONFIG" --context "$CSI_CONTEXT" \
  -n "$CSI_NAMESPACE" get replicatedvolumes.replicated.csi.tak-os.org
kubectl --kubeconfig "$CSI_KUBECONFIG" --context "$CSI_CONTEXT" \
  -n "$CSI_NAMESPACE" get pods -o wide

Resolve the bound PV and exact CSI handle, then inspect that generation:

CSI_PV=$(kubectl --kubeconfig "$CSI_KUBECONFIG" --context "$CSI_CONTEXT" \
  -n "$APP_NAMESPACE" get pvc "$CSI_CLAIM" -o jsonpath='{.spec.volumeName}')
kubectl --kubeconfig "$CSI_KUBECONFIG" --context "$CSI_CONTEXT" get pv "$CSI_PV" \
  -o custom-columns=NAME:.metadata.name,DRIVER:.spec.csi.driver,HANDLE:.spec.csi.volumeHandle,RECLAIM:.spec.persistentVolumeReclaimPolicy

Continue only after the claim is bound and the PV's driver is csi.replicated.tak-os.org. Set CSI_VOLUME to the displayed handle; the ReplicatedVolume UID is its protocol generation:

CSI_VOLUME=rv-your-exact-volume-handle
kubectl --kubeconfig "$CSI_KUBECONFIG" --context "$CSI_CONTEXT" \
  -n "$CSI_NAMESPACE" get replicatedvolumes.replicated.csi.tak-os.org "$CSI_VOLUME" \
  -o custom-columns=NAME:.metadata.name,GENERATION:.metadata.uid,RUNTIME:.spec.runtime,PHASE:.status.phase,REASON:.status.reason

Keep the source PVC UID, volume/generation, original Node UIDs, backing PVC/PV identities, membership and CA through maintenance. Names alone do not establish ownership. Record actual running image identities/build metadata: the CSI vendor-version string still reports 3.2.0 for this image release.

Monitor and diagnose

Readiness has several layers. A Ready supervisor Pod does not establish worker quorum, and a reachable endpoint does not prove the volume can serve. In shared mode, inspect the per-replica EndpointSlices and correlated host/worker logs as well as ReplicatedVolume phase/reason. ReplicaStateLost indicates known retained replica-state loss with unknown application-file extent; unavailable nodes or one missing redundant copy alone do not prove total data loss.

Use observability for metrics, JSON log fields, dashboards and alerts, and capacity for storage pressure and implementation limits. Monitor inventory freshness, desired/ready members, credential renewal, quorum, backing free space, journal/checkpoint/rebuild progress and shared host resources. Health probes must not restart workers solely because quorum or backing storage is unavailable.

Collect bounded control-plane diagnostics from a checkout with Python and kubectl available:

KUBECONFIG="$CSI_KUBECONFIG" python3 tools/support-bundle.py \
  --context "$CSI_CONTEXT" --namespace "$CSI_NAMESPACE" \
  --volume "$CSI_VOLUME" --output /private/path/incident-volume.json

Run source tool commands from the repository root. The output path must be new and its parent must exist. The bundle excludes Secret bytes, file contents and application logs. Review infrastructure identities before sharing it. For an incident, use the runbook to distinguish dependency, quorum, credentials, disk-full, permission and ownership failures.

Repair and maintain existing volumes

The repair tool renders intent by default. Applying an operational request requires the intended maintenance authorization; an accepted API patch does not prove repair completed. Shared transitions are one-way and require --quiesced when applying. Active mixed-version upgrades and shared-generation downgrades are unsupported.

Preserve uncertain storage and restore original dependencies. Never reset a journal, lower quorum, substitute Node names for original UIDs, force-delete workers or strip finalizers to make an operation finish. The runbook maps bounded cleanup reasons to their required dependencies. Permanent-node disposal is parked.

Back up, retire and uninstall

Establish external backups before an incident. The backup/recovery guide documents Python 3.11+ archive tools, application quiescence, checksums, independently protected storage and restoration to an empty new volume. Restore requires recorded isolation of old authority and an accepted recovery point; restoration does not dispose of the old storage.

For normal retirement, stop consumers and inspect the actual outer PV policy. Delete requests durable volume fencing and normal backing reclamation. Retain keeps original CR, trust, assignments and backing storage for deliberate handling. Direct deletion of the original ReplicatedVolume is destructive intent even for retained storage. Follow lifecycle rather than deleting backing claims individually.

Keep manager, NODE/legacy cleanup components, local CSI, CRD, RBAC and Secrets until deletion completes or retained authority is deliberately preserved. Shared cleanup must stop the worker, unpublish via local CSI and record exact backing release before ordinary reclamation. Follow the uninstall sequence before removing runtime resources through GitOps.