k3s is tiny and cute

After KubeCon I've decided to give k3s a try. And I'm impressed!

K3s is a small distribution of Kubernetes (k8s), Linux container orchestrator system. It's really tiny while being functional. One starts with a single 52MiB binary and after few seconds there's a functional installation with half a dozen of system pods. It's a far cry from OKD and its resource hunger. Of course, compared to OKD, there's much less functionality in k3s, but enough for most cases (including mine).

First, I'm writing now, because only recently cgroupsv2 support was added to k3s. Previously it just didn't work on modern systems, like Fedora.

Second, the etcd database, widely perceived as a mandatory part of k8s, is optional in k3s! By default embedded SQLite is used – enough for simple scenarios. I'm particularly happy for Postgresql support. Yes, you can have your Kubernetes working with pgsql.

But K3s is not a single-node solution only. Adding worker nodes is simple; High-Availability solutions for control-plane looks sound (haven't tried yet, but it's on my TODO).

For networking one can easily encrypt inter-node traffic using WireGuard. It's a matter of single switch for provided flannel network backend. Higher level needs? k3s ships with Traefik (which supports ACME for getting TLS certificates) and klipper-lb.

Unfortunately at the lowest level, networking still depends on iptables. This was horrible choice in the beginning of Kubernetes, already 15 years obsolete when it was selected. With known performance problems: nf-hipac tried to solve the problems when, in 2002? Huawei replaced iptables with Linux IP Virtual Server in 2017 to have k8s scale. But the solution is still not default in upstream Kubernetes.

Anyway, for small cluster k3s with iptables should work fine, but it has a potential to demolish your carefully tuned firewall configuration. So beware. For the positive aspect, go and read klipper-lb entire source code. This is world championship in simplicity and getting things done with existing infrastructure.

Keeping cluster up-to-date can be automated with system-upgrade-controller, which downloads new version and restarts the services. Simple.

k3s is provided by our Rancher friends at SUSE. I strongly recommend giving it some attention (at the moment INSTALL_K3S_COMMIT=fadc5a8057c244df11757cd47cc50cc4a4cf5887 works for me). Besides, I vaguely remember one needs k8s cluster to build some Fedora content. ;)