After KubeCon I've decided to give k3s a try. And I'm impressed!
K3s is a small distribution of Kubernetes (k8s), Linux container orchestrator system. It's really tiny while being functional. One starts with a single 52MiB binary and after few seconds there's a functional installation with half a dozen of system pods. It's a far cry from OKD and its resource hunger. Of course, compared to OKD, there's much less functionality in k3s, but enough for most cases (including mine).
First, I'm writing now, because only recently
cgroupsv2 support was added to k3s. Previously it just didn't work on modern systems, like Fedora.
etcd database, widely perceived as a mandatory part of k8s, is optional in k3s! By default
embedded SQLite is used – enough for simple scenarios. I'm particularly happy for Postgresql support. Yes,
you can have your Kubernetes working with pgsql.
But K3s is not a single-node solution only. Adding worker nodes is simple; High-Availability solutions for control-plane looks sound (haven't tried yet, but it's on my TODO).
For networking one can easily encrypt inter-node traffic using WireGuard. It's a matter of single switch for
flannel network backend. Higher level needs? k3s ships with
Traefik (which supports ACME for getting
TLS certificates) and
Unfortunately at the lowest level, networking still depends on
iptables. This was horrible choice in the
beginning of Kubernetes, already 15 years obsolete when it was selected. With known performance problems:
nf-hipac tried to solve the problems when, in 2002? Huawei replaced iptables with Linux IP Virtual Server in 2017
to have k8s scale. But the solution is still not default in upstream Kubernetes.
Anyway, for small cluster k3s with iptables should work fine, but it has a potential to demolish your carefully tuned firewall configuration. So beware. For the positive aspect, go and read klipper-lb entire source code. This is world championship in simplicity and getting things done with existing infrastructure.
Keeping cluster up-to-date can be automated with
system-upgrade-controller, which downloads new version
and restarts the services. Simple.
k3s is provided by our Rancher friends at SUSE. I strongly recommend giving it some attention
(at the moment
INSTALL_K3S_COMMIT=fadc5a8057c244df11757cd47cc50cc4a4cf5887 works for me).
Besides, I vaguely remember one needs k8s cluster to build some Fedora content. ;)