Testing

Testing

Go checks

Run make fmt, make vet and make test before submission. Unit tests cover handlers, templating, facts, admission and reconciliation using temporary host roots and fake Kubernetes clients/runners. They do not execute package managers or restart services on the development machine.

Image build checks

Run make docker-build (or docker build -t tak-os/keel:latest .) to run the checks in the Dockerfile's build stage. After downloading modules and copying the source, the build checks formatting with test -z "$(gofmt -l .)", runs go vet ./..., then runs go test ./.... Any failure stops the build before the release binary is compiled. Formatting is checked without modifying source files; use make fmt locally to fix it.

The runtime stage copies the binary from the checked build stage, so these checks are required for every image build, including GitLab CI builds. Docker may reuse successful check layers when their inputs have not changed. Use docker build --no-cache -t tak-os/keel:latest . to rerun all build steps. The Go toolchain, source and test files stay in the build stage and are not copied into the runtime image. The kind suite is separate because it requires a Docker daemon capable of running privileged Kubernetes nodes.

Full apply and restart with kind

Run make test-e2e on Linux with Docker, kind and kubectl on PATH. Docker must allow privileged kind nodes. The script creates a uniquely named disposable single-node Kubernetes cluster, uses a temporary isolated kubeconfig, builds the current image and loads it into kind. It applies the production CRD, RBAC and DaemonSet (substituting only the image). It never applies to your current kubeconfig context, mounts the development machine root, or touches services outside the disposable node container.

The test requires a kind node image containing real systemd (default kindest/node:v1.37.0) and proves:

  1. A NodeConfig writes the node file with the desired contents and mode.
  2. Its service list restarts a real systemd oneshot unit, observed through a host file recording each ExecStart invocation.
  3. Repeated no-op reconciles leave the restart count unchanged.
  4. External file drift is repaired and causes another restart.
  5. A failed restart sets error and restartPending status; installing the missing unit lets a retry complete without a new file change or duplicate restart.

The full script output is saved to run.log. Cleanup exports kind logs, agent logs, NodeConfig status and Events, then deletes the cluster and its locally built image even on failure. Artifacts default to test/e2e/artifacts/; set E2E_ARTIFACTS_DIR to override. KIND_NODE_IMAGE selects a different compatible systemd node image. Network access is needed to pull node and build images and Go modules. Each wait is bounded; inspect saved logs if the cluster cannot become ready. A stale shared kind bridge can cause Docker errors attaching a veth interface (Device does not exist). Run E2E_ISOLATED_NETWORK=1 make test-e2e to create and clean up a unique test network. This opt-in uses kind's experimental KIND_EXPERIMENTAL_DOCKER_NETWORK override, which upstream explicitly treats as unsupported; see the pinned provider source. By default the test uses the daemon's default kind network. The test does not modify existing Docker networks.

The complete suite passed on 31 October 2026 with kind v0.33.0 and the pinned Kubernetes v1.37.0 node image using E2E_ISOLATED_NETWORK=1. The hardened DaemonSet ran successfully with the Alpine 3.24 runtime. This host's Docker daemon delayed node exit events during deletion; the exact node and test network were removed after exit, and a final inventory confirmed no disposable test containers or networks remained. The script now retries delayed node removal and fails if cleanup leaves resources behind, naming them in the log. Application assertions are reported separately from successful cleanup.

GitLab CI

GitLab CI uses the shared container build template to build the Dockerfile, which runs the Go checks above. There are no separate go-fmt, go-vet, go-test or kind-e2e jobs. The test stage remains for GitLab's SAST security scan; documentation publishing is unchanged.

Run make test-e2e separately on a host with the prerequisites above. Its logs remain in the local artifacts directory; GitLab no longer collects them. The script defaults to the pinned Kubernetes v1.37.0 kind node image, matching the Kubernetes client minor version. Update kind, kubectl and node versions together after verifying the target combination. This integration test validates API machinery, runtime image tools, RBAC, host mounts and real systemd behavior that fake-client tests cannot establish.