Configuration reference
Documentation home · Getting started
Configure roles through the Orbit-owned manifests. Shared-runtime options below require replicated CSI 4.1.0 and the companion local CSI 2.1.0 extension; 4.0.1 images do not implement them. Generated per-volume resources are managed by the driver; do not hand-edit their identity, membership, access mode or credential files.
StorageClass and PVC inputs
| Input | Default / behavior |
|---|---|
| Provisioner | csi.replicated.tak-os.org. |
parameters.replicas |
1; integer default from one through four. |
PVC annotation csi.replicated.tak-os.org/replicas |
Overrides the default at creation; exact strings 1, 2, 3, 4. |
parameters.backingStorageClass |
Manager's --backing-class, default tak-os-local. |
| PVC access mode | Exactly RWX or RWO; no mixed-mode volume, RWOP, raw block or ReadOnlyMany capability. |
| Requested storage | Passed to each backing claim; not a quota or capacity reservation. |
| Outer reclaim policy | Choose Retain or Delete for application-volume lifecycle. |
| Backing class | Local driver, delayed binding and Delete reclaim; it must not recursively use this replicated driver. |
Replica count, application mode and original claim identity are creation inputs.
Editing the PVC or class later does not migrate an existing generation. Unknown
StorageClass parameters, caller topology constraints, clones, filesystem types,
mount flags and CSI mount-group requests are rejected. The external-provisioner's
--extra-create-metadata supplies the required original PVC name and namespace.
Manager and node-controller options
Both roles use /manager from the manager image.
| Option | Default | Meaning |
|---|---|---|
--role |
manager |
manager; node-controller is retained only for legacy compatibility and is no longer generated. |
--replica-runtime |
shared |
New generation runtime, shared or explicit legacy pod; existing activated generations keep their persisted mode. |
--node-id |
Empty | Required assigned node for a node-controller. |
--endpoint |
unix:///csi/csi.sock |
Manager CSI socket. |
--metrics-listen |
:9906 |
Metrics and health HTTP listener. |
--kubeconfig |
Empty | In-cluster credentials by default; optional external file. |
--namespace |
kube-system |
ReplicatedVolume and generated-resource namespace. |
--local-namespace |
kube-system |
Existing local-CSI namespace. |
--local-selector |
app=tak-os-local-csi |
Local-CSI Pod identity selector. |
--local-service-account |
tak-os-local-csi |
Required local-CSI service-account identity. |
--worker-image |
Empty, required | Replica image for generated workers. |
--controller-image |
Empty | Deprecated compatibility argument; no node-controller Pods are generated. |
--controller-service-account |
tak-os-replicated-csi-manager |
Original account used to verify ownership before retiring legacy controller Pods. |
--backing-class |
tak-os-local |
Default local backing StorageClass. |
Manager and each node-controller role elect their background actor using a Kubernetes Lease. Lease timing is 20-second duration, 12-second renewal deadline and 3-second retry. Losing leadership cancels the actor and exits the process; Lease ownership is not a data-plane serving fence.
CSI frontend and mount-owner options
Both roles use /node from the node image. Omitting --mount-service makes the
process own its FUSE mounts; setting it creates a forwarding frontend.
| Option | Default | Meaning |
|---|---|---|
--endpoint |
unix:///csi/csi.sock |
Absolute Unix CSI/RPC listener. |
--node-id |
NODE_NAME environment variable |
Required Kubernetes node name. |
--namespace |
kube-system |
Trusted namespace for volume credentials. |
--mount-root |
/var/lib/kubelet |
Allowed root for stage/publish paths. |
--metrics-listen |
:9906 |
Metrics and health listener. |
--mount-service |
Empty | Absolute unix:///… endpoint of a separate mount owner. |
For a split deployment, give the mount owner its own Unix socket, configure the frontend to reach that socket, and expose distinct metrics ports when both share a Pod network namespace. Preserve shared sockets, mount namespaces and kubelet mount propagation across the two processes. Scratch images contain no shell or mount helper; use explicit executable commands, not shell wrappers.
Shared NODE replica host
Add a separate /node --role=replica-host container to the existing NODE Pod,
retaining the CSI frontend, FUSE owner and registrar. The
source integration patch documents the
mounts, downward-API Pod identity, resource profile and additive RBAC. It is a
fragment for review in the external deployment repository, not a standalone
install or authorization to apply it.
| Option | Default | Meaning |
|---|---|---|
--role |
node |
Use replica-host for the shared backend process. |
--local-csi-endpoint |
Empty, required for host | Explicit absolute Unix socket to local CSI. |
--replica-state-root |
/var/lib/tak-os/replicated-csi/state |
Persistent, private hostPath for mount identity and initialization history. |
--replica-mount-root |
/var/lib/tak-os/replicated-csi/backing |
Same opt-in consumer root configured in local CSI. |
--max-workers |
128 |
Admission limit per node; valid range 1–1024. |
--metrics-listen |
:9906 |
Set a distinct address such as :9908 in a shared Pod. |
Supply NODE_NAME, POD_NAME and POD_UID using the downward API, namespace and
cluster domain consistently. Label the Pod
replicated.csi.tak-os.org/replica-host: 'true'. Host readiness means the
supervisor is running; it does not depend on every worker having quorum.
Local CSI must run the consumer-publication extension and enable
--consumer-mount-root=/var/lib/tak-os/replicated-csi/backing. Its container must
mount that host directory with Bidirectional propagation; the replica host uses
HostToContainer. The host calls the original CSI NodePublish/NodeUnpublish APIs
for already-bound PVCs; it does not create local volumes directly. Consumer
ownership blocks competing mounts and deletion until release, even after a crash.
The host checks its own mount namespace before opening a worker; failed mount
propagation leaves it unavailable instead of initializing the directory beneath
the mount. Keep this configuration available until all consumer publications
are released.
The manager needs EndpointSlice GET and backing-PVC UPDATE access in addition to existing allocation permissions; the source patch includes additive roles. The NODE account needs the narrowly scoped reads/updates in the source patch. Kubernetes Secret GET remains whole-object access; only the leaf, CA certificate and independent bearer/overlap values are written to host-private credential generations, never the CA signing key.
Each worker gets dynamically assigned HTTP/Raft/metrics ports and a separate Prometheus registry; selectorless per-replica Services/EndpointSlices retain the original ports and DNS names. Permit Pod-to-Pod traffic to those backend ports in NetworkPolicy, rather than assuming only 9900/9901/9906 on the NODE Pod.
Replica configuration
/replica --config=/config/worker.json reads a generated JSON configuration.
It binds volume, generation, member id, backing root, peer identities,
HTTP/Raft listeners and credential-file paths. singleNodeWriter binds the RWO
mode; join and advertised peer identity are managed during replacement.
The insecureLoopback field is not a supported deployment configuration.
maxJournalBytes defaults to 256 MiB and cannot be configured below 16 MiB. It
controls journal pressure/compaction admission, not a lifetime write allowance,
PVC quota or disk-space reserve. Worker metrics default to 0.0.0.0:9906.
DNS, networking and clocks
REPLICATED_CSI_CLUSTER_DOMAIN defaults to cluster.local. Set the real suffix
consistently on manager, node and mount-owner processes; it propagates into
generated roles. Explicit invalid/empty values fail startup. Service dials use
absolute DNS names while preserving the original authenticated TLS origin.
Changing the suffix requires a quiescent role restart and remount.
| Generated worker port | Purpose |
|---|---|
| TCP 9900 | Authenticated Raft peer transport. |
| HTTPS 9901 | File/control, coordinator discovery and checkpoint/rebuild operations. |
| HTTP 9906 | Prometheus, /healthz and /readyz; restrict monitoring access. |
Allow DNS, necessary API access, client-to-worker and peer-to-peer traffic. Synchronize node clocks for retry expiry, handle leases and RWO ownership. Metrics/health ports are diagnostics, not authenticated file-service endpoints.
Resources and credentials
The source staging profile sets generated worker requests to 50m CPU/64 MiB and
limits to 2 CPU/512 MiB, with GOMEMLIMIT=384MiB. Node controllers request
20m/64 MiB and limit 1 CPU/256 MiB, with GOMEMLIMIT=192MiB. These are deployment
settings, not capacity or performance guarantees. Inspect the actual rendered
manifests and tune them for the working set without removing protocol bounds.
Worker projected credentials exclude the volume CA signing key. The trusted
mount owner obtains client credentials from <volumeID>-auth in its configured
namespace. Do not expose Secret contents in logs, support bundles or examples.
Normal renewal retains the original CA; see security.
Version identification
Images for release 4.1.0 use the role tags :4.1.0 after publication. Floating build
inputs can change the result of a later rebuild of that tag; record the running
image identity and /licenses/buildinfo.txt when investigating a deployment.
The current manager/node GetPluginInfo vendor-version string remains 3.2.0;
it does not identify the 4.1.0 OCI release. Use image/build metadata instead.