Getting started

Documentation home · User guide · Administrator guide · Configuration reference

Use this guide to install the driver and provision application filesystem volumes. If your cluster already has a replicated StorageClass, start with the user guide. Version 4.0.0 introduced the documented support scope; 4.1.0 adds a shared NODE runtime for staging evaluation with no recorded cluster runtime observations. Installation is managed through your Orbit GitOps repository; this source repository is not a standalone Helm chart or one-command installer.

Requirements

Linux Kubernetes nodes with kernel FUSE, /dev/fuse, working mount propagation into kubelet paths and permission to run the privileged FUSE mount owner.
An existing tak-os-local-csi installation and a backing StorageClass using csi.local.tak-os.org, WaitForFirstConsumer binding and Delete reclaim. Shared runtime in replicated CSI 4.1.0 additionally requires local CSI 2.1.0 consumer publication and the reviewed NODE/RBAC/mount configuration.
One distinct Ready, eligible local-storage node per configured data replica. Application-only mount nodes do not need to host a backing replica.
DNS and network reachability between clients, workers and worker peers. Set the actual cluster DNS suffix consistently. Keep node clocks synchronized.
Enough real backing filesystem space for each full data copy, journals, checkpoints and transfer spools. PVC requested bytes are not enforced quotas.

One replica has no storage redundancy. Two replicas require both to serve. Three or four replicas can continue quorum I/O with one unavailable current member. Distinct Kubernetes nodes do not prove independent physical failure domains; see availability limits.

Install through GitOps

Install the local driver and backing class first. Preserve its installation for the entire lifetime of replicated volumes, including cleanup.
Add the current ReplicatedVolume CRD to the deployment source. Its schema must include immutable accessMode support before provisioning RWO volumes and the runtime/ownership fields before selecting shared mode.
Configure the driver namespace, service accounts/RBAC, manager Deployment, external-provisioner, CSI node DaemonSet and node-driver-registrar. The external-provisioner needs --extra-create-metadata so creates include the original PVC name and namespace.
Configure the CSIDriver as csi.replicated.tak-os.org, with attachRequired: false and fsGroupPolicy: None. Provide the kubelet host paths, /dev/fuse, mount propagation and shared Unix socket needed by the node frontend and mount owner. Kubernetes supplies each image's command explicitly.
Supply all three role images from the same release after publication. Set --worker-image for legacy replica compatibility and explicitly select the manager's --replica-runtime. In 4.1.0 the default is shared for new generations; pod selects legacy workers. Existing activated generations retain their persisted runtime. --controller-image is deprecated; the manager no longer generates node-controller Pods.
For shared mode, first configure local CSI 2.1.0 consumer publication, then merge the shared NODE integration fragment into the existing DaemonSet. Add the replica-host container, Pod/Node identity, additive RBAC, persistent host state and consumer mount/socket paths. Preserve the frontend, FUSE owner and registrar. Existing volumes require the guarded quiescent transition.
Merge resource, DNS, probe and rollout settings into the actual Orbit manifests, then commit through the deployment repository's own process. The source runtime profile omits installer essentials such as RBAC and host mounts; applying it alone does not install a functioning driver.

Source references: CRD and runtime profile. See image names and contents.

The worker trust domain is created internally per volume. It does not depend on public ACME certificates or cert-manager issuing an internet-valid certificate. Any application's public ingress certificate is a separate platform concern.

Define a replicated StorageClass

This example uses three full copies and retains the outer volume when its claim is released. Replace the backing class name with your existing local class.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: replicated-content
provisioner: csi.replicated.tak-os.org
parameters:
  replicas: "3"
  backingStorageClass: tak-os-local
reclaimPolicy: Retain
volumeBindingMode: Immediate
allowVolumeExpansion: false

The replicated class deliberately differs from its delayed-binding local backing class. The manager chooses storage placement internally; application-supplied CSI topology requirements are not supported. Do not add allowedTopologies, filesystem types, mount options or unrecognized parameters.

Use Delete on the outer class only when normal claim deletion should request fenced destruction of the volume and its backing copies. Retain preserves original authority and storage; it is not a backup.

Request RWX content storage

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: site-content
spec:
  storageClassName: replicated-content
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi

Each backing claim requests that capacity. The requested size does not reserve space or stop application writes at 10 GiB. See capacity.

To override the replica default at creation, set the annotation below on the application PVC. The accepted strings are exactly "1", "2", "3" and "4". Changing it after creation does not resize membership.

metadata:
  annotations:
    csi.replicated.tak-os.org/replicas: "3"

Mount from an application

The following is a fragment to merge into your application's Pod template:

spec:
  containers:
    - name: application
      image: registry.example.com/your-site:latest
      volumeMounts:
        - name: content
          mountPath: /srv/content
  volumes:
    - name: content
      persistentVolumeClaim:
        claimName: site-content

Replace the example image and application path. RWX allows multiple nodes to mount the same volume, but applications must coordinate conflicting edits. Prefer complete-file publication with atomic rename. File-backed databases, filesystem locks and shared mmap are not supported; keep databases on an appropriate separate service. Initialize directory ownership before dropping privileges: the driver does not recursively apply fsGroup ownership.

Choose RWO instead

Use a separate claim with accessModes: [ReadWriteOnce]. The backing class and replica configuration remain independent of the application's access mode. Several Pods may use that claim on its one owning node; RWO does not mean one Pod. ReadWriteOncePod and ReadOnlyMany capabilities are unsupported.

The volume mode is immutable. RWO ownership is enforced by quorum-ordered leases, not controller attach/detach. A second node is denied while an owner remains active. Expiry and takeover require actual replica quorum and normal old-mount recovery; read the RWO contract before relying on failover.

Observe provisioning and operate safely

With an explicitly selected authorized kubeconfig:

CSI_KUBECONFIG=/absolute/path/to/authorized-kubeconfig
kubectl --kubeconfig "$CSI_KUBECONFIG" get pvc,pv
kubectl --kubeconfig "$CSI_KUBECONFIG" -n kube-system get replicatedvolumes

A pending claim can mean insufficient eligible storage nodes, missing local CSI, backing capacity/binding problems, unavailable quorum or an incompatible schema. Use the operator runbook instead of editing allocation metadata or removing finalizers. Before uninstalling or changing mount-owner images, follow the lifecycle order.

getting-started