How Jared Rodriguez and I reproduced a Kubernetes control-plane failure with 200 large objects, and what the December 2020 code reveals about LIST requests, cache initialization, and resource exhaustion. [Read more...]
With the last feature release of Helm V2 just published, and Helm V3 on the horizon, we’re still stuck with managing CRDs. CRD management in helm is, to be nice about it, utterly horrible. The Problem with helm When you have a chart that both installs a CRD and installs a resource that requires that CRD, there’s a race condition where the resource installation can fail because the CRD is not yet recognized as defined by the API server. To work around this, helm added a hook. [Read more...]
Selecting a container runtime for use with Kubernetes: a comparison of docker, rkt, containerd, and cri-o [Read more...]
Configuring kubernetes authentication using pki issued certificates with an intermediate CA. [Read more...]
The kube-apiserver establishes some network connections when it first starts up and only when it first starts. This is how to diagnose them. [Read more...]
Sometimes you just can't use multicast. Some cloud providers just do not provide it. In that scenario, you need to configure your vxlan layer using unicast addresses. This is done easily using iproute2. [Read more...]
There are a number of applications that open a log file and just keep writing to it. This is, of course, a problem when you rotate logs as the application just keeps writing to the renamed log instead of opening the new one. [Read more...]
I filed a bug back in July 2013 asserting that a "feature" in the gluster CLI was a potential security bug. I've recently had that bug rejected as "Not a bug" so here it is. [Read more...]
I had a user report that one node couldn't access the voip server over the ipsec vpn, but all the rest would. Perplexing. Not only could any other node on the remote network connect to the server, but I could ping the node that wouldn't talk from that server. [Read more...]
Split brain. Sounding like something from a B movie about zombies, it's probably more terrifying to data storage people than flesh eating undead would be. [Read more...]
