Kubernetes in production

Kubernetes is the right orchestration layer for multi-tenant SaaS products that need horizontal pod autoscaling, rolling deploys with zero downtime, and container-level isolation between tenants or environments. Avo knows when to reach for it and when not to.

For Avo's own Argus and Apex services, Kubernetes was evaluated and not selected. The reason is that the Avo stack runs on a single Hetzner AX102 bare-metal server running 103 services. Kubernetes adds a control plane (etcd, API server, scheduler, controller manager) that consumes 2 to 4 GB of RAM and introduces a network overlay (CNI) that adds latency to every service-to-service call. On a single-server, single-tenant system, systemd unit files give the same restart-on-failure and dependency-ordering guarantees with zero overhead. The TOKIO_WORKER_THREADS tuning and bounded channel back-pressure that matter most for Avo's latency profile are invisible to Kubernetes anyway: they live inside the process, not at the orchestration layer.

For client work, Avo builds on Kubernetes when the client is already on AWS EKS, GCP GKE, or Azure AKS and needs a new service wired into that environment. The most common pattern is a stateless API service that needs to scale horizontally under variable load.

Example workflow: deploying a new stateless API service to an existing client EKS cluster. 1. Write a Helm chart with a Deployment, Service, and HorizontalPodAutoscaler. Set minReplicas: 2 so the service survives a node failure without downtime. 2. Configure the HPA to scale on CPU utilization (target 70%) with a custom metric from Prometheus for queue depth if the service processes async jobs. 3. Write the Deployment with a readinessProbe on /api/health returning 200 before the pod receives traffic. Write a livenessProbe with a longer failure threshold so a slow startup does not cause a restart loop. 4. Use cert-manager with a ClusterIssuer to provision a Let's Encrypt TLS certificate for the ingress. No manual certificate rotation. 5. Deploy with Argo CD pointing at the Helm chart in the client's Git repository. Argo CD syncs on every push to main; no manual kubectl apply in CI. 6. Configure resource requests and limits explicitly (e.g., requests: cpu 100m, memory 256Mi; limits: cpu 500m, memory 512Mi). Without limits, a single runaway pod can starve other services on the node.

Tradeoffs to think through. Kubernetes on a managed provider (EKS, GKE) costs $70 to $150 per month for the control plane alone before any nodes. For a simple API with predictable load, a Hetzner VPS with systemd or a Fly.io deployment is 10x cheaper. K8s earns its keep when horizontal autoscaling genuinely matters (traffic spikes above 5x baseline) or when the client's ops team already knows it and will own the cluster after delivery. If Avo builds on K8s for a client, Avo documents the runbook in Notion so the client can operate it without us. Vendor lock-in through managed Kubernetes services is real: EKS-specific features (IAM roles for service accounts, EBS CSI driver) do not transfer to GKE without rework.