TAK-OS Ingress administrator guide

Use this guide to install and operate the controller. You need cluster-level permissions for the Namespace, RBAC, DaemonSet, Service, and IngressClass resources. Application owners can then follow the user guide.

Deployment model

The shipped deployment runs a DaemonSet in namespace tak-os-ingress, including tolerations for control-plane nodes. Each Pod runs the same routing controllers and proxy. Backend requests go to Service DNS names and Service ports; Kubernetes Service routing handles the backend endpoints. There is no per-application proxy configuration file to reload.

The DaemonSet does not use host networking or host ports. Clients reach container ports 80 and 443 through a Service or another explicitly configured network path. Port 9090 serves /metrics and /healthz. The public Service exposes only HTTP and HTTPS.

Prepare the following before installation:

• A cluster supporting networking.k8s.io/v1 Ingress resources and working Service DNS and networking.
• An accessible controller image and registry credentials when needed.
• An address and Service exposure method suitable for the cluster, plus DNS records pointing at that address.
• cert-manager and an Issuer or ClusterIssuer if automatic Ingress certificate issuance is required. Existing TLS Secrets can be used without issuance.
• Gateway API v1 CRDs if GatewayClass, Gateway, and HTTPRoute support is required.

The manifests do not install cert-manager, its issuers, or Gateway API CRDs. Select dependency versions compatible with your cluster. The Go dependencies and build toolchain requirement are recorded in go.mod.

Install with Orbit Deploy

When Orbit Deploy is already installed, save and apply this resource:

apiVersion: k8s.tlspu.com/v1
kind: OrbitDeployment
metadata:
  name: tak-os-ingress
  namespace: kube-system
spec:
  source: https://gitlab.com/tak-os-platform/tak-os-containers/tak-os-ingress.git

Orbit renders the manifests in default using its deployment values and image mapping in deployment.json. The image mapping uses latest; manage release selection through your deployment process when you need a fixed version. Configure the external addresses used by Orbit and inspect the rendered Service before relying on it for public access.

Orbit owns reconciliation of these resources. Persist configuration changes in the deployment source or values so that reconciliation does not overwrite them.

Install manually

The file default/020-daemonset.yaml is an Orbit template, so it cannot be applied unchanged. Its image, image-hash annotation, and external addresses contain <!...!> placeholders. The Service template also uses ExternalIPs rather than the Kubernetes field externalIPs and includes externalTrafficPolicy on a ClusterIP Service. Prepare a valid manifest before applying it.

The following renders a local manifest with an internal ClusterIP Service. It requires Python 3, leaves the repository template intact, and removes the two Service fields that need an environment-specific exposure decision. Replace YOUR_RELEASE_TAG with the release you intend to deploy.

INGRESS_IMAGE=registry.gitlab.com/tak-os-platform/tak-os-containers/tak-os-ingress:YOUR_RELEASE_TAG \
python3 - <<'PY'
import os
from pathlib import Path

manifest = Path("default/020-daemonset.yaml").read_text()
manifest = manifest.replace("<!tak-os-ingress!>", os.environ["INGRESS_IMAGE"])
manifest = manifest.replace("<!tak-os-ingress-hash!>", '"manual"')
manifest = "\n".join(
    line for line in manifest.splitlines()
    if "ExternalIPs:" not in line and "externalTrafficPolicy:" not in line
) + "\n"
if "<!" in manifest:
    raise SystemExit("Unresolved deployment placeholder")
Path("/tmp/tak-os-ingress.yaml").write_text(manifest)
PY

kubectl apply -f default/000-namespace.yaml
kubectl apply -f default/010-rbac.yaml
kubectl apply --dry-run=server -f /tmp/tak-os-ingress.yaml
kubectl apply -f /tmp/tak-os-ingress.yaml
kubectl -n tak-os-ingress rollout status daemonset/tak-os-ingress

The resulting Service is internal. To expose it, edit the rendered manifest for your cluster before applying it: use a supported LoadBalancer or NodePort Service, or configure valid externalIPs with the required network routing. Declaring an external IP does not allocate or advertise it. The controller does not provision an external address itself.

The template retains internalTrafficPolicy: Local. Internal Service traffic must reach nodes with a local ready ingress Pod. If using externalTrafficPolicy: Local with a supported external Service configuration, ensure the load balancer sends traffic only to nodes with a local ready ingress Pod. This can preserve source addresses, depending on the upstream network path. Verify the observed client IP before depending on per-client rate limits or forwarding headers.

Verify installation

kubectl -n tak-os-ingress get daemonset,pods,service -o wide
kubectl get ingressclass tak-os -o yaml
kubectl -n tak-os-ingress logs -l app=tak-os-ingress --tail=100 --prefix
kubectl -n tak-os-ingress port-forward daemonset/tak-os-ingress 9090:9090

In another terminal:

curl --fail http://127.0.0.1:9090/healthz
curl --fail http://127.0.0.1:9090/metrics

/healthz returns 200 OK; it is a process health endpoint, not a check of backend availability, informer synchronization, or certificate readiness. Verify the data path separately using a known application route from the user guide. You can forward ports 8080:80 and 8443:443 from a controller Pod when public exposure is not ready.

Configure the controller

Set flags in the container args list. Keep --ingress-class=tak-os when using the shipped IngressClass. Updating process flags requires a Pod rollout; application route and Secret updates are applied live.

When changing listen ports, also update container ports, Service target ports, probes, and scrape configuration. Gateway listener validation currently accepts HTTP port 80 and HTTPS port 443; changing process ports does not extend that support.

The response limit is a buffering threshold, not a maximum download size. Larger responses stream with their original Content-Length. Unlimited buffering can consume unbounded memory. Request and response limits are global process flags, not per-Ingress annotations. The memory budget must account for concurrent requests, buffers, connections, and authentication work.

The client request read timeout measures inactivity while reading headers and bodies. Backend connect/read/send defaults are configured separately through route annotations. Keep-alive client connections also have a 60-second idle timeout.

Manage TLS and authentication

For automatic Ingress issuance, install cert-manager, create an issuer, and let application owners select it with cert-manager.io/cluster-issuer or cert-manager.io/issuer. ClusterIssuer takes precedence when both are set. Without either annotation, TAK-OS selects ClusterIssuer letsencrypt-staging. For ACME HTTP-01 issuance, ensure challenge traffic reaches the solver through the configured ingress class and public port 80.

Only Ingress TLS blocks without secretName trigger TAK-OS Certificate creation. For Gateway HTTPS listeners, provision a TLS Secret separately, for example with a cert-manager Certificate. The Gateway controller does not request certificates. See Gateway TLS.

The HTTPS server requires TLS 1.2 or later. An unknown SNI or unusable Ingress TLS Secret uses a self-signed fallback generated at process startup. Gateway HTTPS listeners with invalid certificate references do not install their routes.

Basic and OAuth configuration Secrets must live in the route's namespace. Secret updates reconcile affected routes. OAuth signing keys must decode to exactly 32 bytes; generate one with openssl rand -base64 32. Rotating that key invalidates sessions signed by the old key. Check authentication Secret formats before rotation or migration.

The runtime image contains the system CA bundle for backend and provider TLS validation. Private certificate authorities require an appropriate trust bundle in the container; there is no per-route CA Secret annotation. OAuth callback scheme and forwarding headers derive from the actual connection, so the expected edge deployment terminates TLS at this controller.

Operate with appropriate access controls

The shipped ClusterRole reads Services, Secrets, and Namespaces across the cluster; watches Ingresses, IngressClasses, Certificates, and Gateway resources; creates and updates Certificates; and writes Gateway resource statuses. It does not write Ingress load-balancer status.

Cluster-wide Secret access includes TLS private keys and authentication credentials. Adding narrower namespace Roles does not restrict an existing ClusterRole grant. Restricting this controller to namespace-scoped Secret access requires both RBAC changes and changes to its cluster-wide informer watches.

The shipped container runs as UID/GID 65532, with a read-only root filesystem, privilege escalation disabled, RuntimeDefault seccomp, and all capabilities dropped except NET_BIND_SERVICE. The scratch image has no shell. Use logs, port forwarding, or an authorized diagnostic Pod for troubleshooting.

Port 9090 has no authentication. Restrict it to the monitoring and management networks. Allow ingress Pods to reach the Kubernetes API, cluster DNS, backend Service ports, and any authentication providers. Apply network policies that match your cluster networking and exposure method.

The proxy rebuilds X-Forwarded-For, X-Real-IP, X-Forwarded-Proto, X-Forwarded-Host, and X-Forwarded-Port from the immediate connection and request host, and removes supplied Forwarded headers. It strips supplied X-Auth-Request-* headers before authentication. Deployments behind another proxy will report that proxy's address when it is the TCP peer.

Monitor the controller

The Pod template includes Prometheus scrape annotations for port 9090. Configure your monitoring system to discover those Pods; annotations alone do not install a scraper. Import the JSON files in dashboards into Grafana and select the Prometheus data source.

Request metrics use matched configuration hosts, so they do not count all unmatched requests. Authentication throttling is separate from the route limiter; the route rate-limit metric is not a count of all possible 429 responses. Each Pod exports its own counters and gauges; aggregate them with that scope in mind.

Monitor Pod readiness and restarts, CPU and memory, backend error rates, certificate expiry and issuance failures, and Gateway conditions. The shipped resource requests are 100m CPU and 128 MiB memory, with limits of 2 CPUs and 1 GiB memory. Tune them against representative traffic; the performance tests provide a test environment.

Upgrade and roll back

Record the current image and configuration before changing the release. Check that OAuth signing keys are 32 bytes and that HTTPRoutes use supported matches. Rules containing only unsupported method, query, or regex matches install no route. This behavior is especially relevant when migrating from older releases.

For Orbit-managed installations, update the deployment source or release through Orbit. For manually managed installations, update the desired manifest and apply it, then check rollout and a known route:

kubectl -n tak-os-ingress rollout status daemonset/tak-os-ingress
kubectl -n tak-os-ingress rollout history daemonset/tak-os-ingress

If a manually managed rollout must be reverted, reapply the saved manifest and image, or use kubectl -n tak-os-ingress rollout undo daemonset/tak-os-ingress when the desired revision is available. A mutable latest tag cannot reliably identify the previous image for rollback.

The process handles SIGTERM/SIGINT with a shared 30-second graceful shutdown window. The shipped Pod termination grace period is 60 seconds. Long-lived streams may reconnect during a rollout. Route updates alone do not require Pod restarts.

Diagnose controller problems

Start with Pod events, logs, and the affected resource:

kubectl -n tak-os-ingress describe daemonset tak-os-ingress
kubectl -n tak-os-ingress get events --sort-by=.lastTimestamp
kubectl -n tak-os-ingress logs -l app=tak-os-ingress --tail=200 --prefix
kubectl -n apps describe ingress my-app
kubectl -n apps get service my-app -o yaml
kubectl -n apps get endpointslices -l kubernetes.io/service-name=my-app

For client-visible symptoms, use the application troubleshooting table. /healthz can remain OK while route reconciliation or backend access is failing.

Remove the installation

First migrate or remove application routes and DNS records, and stop Orbit reconciliation if it owns this installation. For a manually managed deployment, delete the resources from your rendered manifest, then remove the RBAC resources from default/010-rbac.yaml once they are no longer used. Deleting namespace tak-os-ingress also deletes all other resources in it, so inspect it first. Application Secrets, cert-manager, and Gateway API CRDs have separate lifecycles; retain shared dependencies and certificates that other applications still use.