Our Kubernetes platform test suite creates namespaces with their corresponding contents, then deletes everything during cleanup. We noticed a strange problem: namespace deletion would sometimes get stuck indefinitely. The root cause was surprising — we had to clear the contents before deleting the namespace! We also learned that getting stuck isn’t the only issue that can occur if we don’t do this.
We ran into a weird pod scheduling error on Amazon Elastic Kubernetes Service (EKS). Some pods, which scheduled just fine in the past, now stay in Pending with the following event:
Failed to schedule pod, incompatible with nodepool “default”, daemonset...
I have been researching how the Kubernetes Ingress system works. My use case is to setup an autoscaled Nginx cluster that reverse proxies to Pods in multiple Deployments. It wasn’t immediately obvious how to do this. By default, Pods in Kubernetes are not supposed to be reachable from outside the cluster. One makes them reachable either by associating those pods with a Service of the right type (i.e. either NodePort or LoadBalancer), or by defining an Ingress. But what is an Ingress? How do I put Nginx in between an Ingress and a set of Pods? This post describes my journey through the jargon-loaded Kubernetes documentation which does not hold any hands, as well as my journey through the Kubernetes source code, all in a quest to find answers.
This post a bit long, so if you just want a summary then you can skip straight to the conclusion at the bottom.
