Last month we got a Pull Request with a new feature merged into the Kubernetes Nginx Ingress Controller codebase. This feature request came from a client that needs a specific behavior of the Load Balancer not available on any Ingress Controller.
Link to the PR: https://github.com/kubernetes/ingress-nginx/pull/3396/
Our client’s software is a multi-tenant machine learning application that caches and store some information in local memory. Each tenant has multiple users, so this information is utilized multiple times by different users. Initially, all the users of the same tenant should be routed to the same instance to take benefit from caching, but if we have an increasing number of users on some specific tenant we could have a load issue. The proposed solution was to distribute the load to a subset of the instances. The hashing algorithm maps the key to a subset of nodes, and the distribution in the subset is random. The key can be a URL, a header, or any other information in the HTTP request header.
All the Kubernetes applications live in a private network inside the cluster. By default, applications are not exposed to the world. If you want to make them reachable, you have two options:
Usually, there is not a single pod behind an external endpoint or URL in your cluster. Instead, you deploy a “Deployment” object with multiple replicas or instances of the same application. Internally, when you build this scenario in Kubernetes, Nginx is configured dynamically to route the traffic to multiple upstream servers. The load balancer algorithm works here, deciding how to distribute the traffic. You have multiple options: round robin, hashing, etc.
The Nginx controller was shipped with the consistent hash algorithm which maps a client to a specific upstream server based on some configurable criteria, for example, an argument in the URL. So all the HTTP requests with the argument value are going to be routed to the same pod. The subset option is slightly different, because instead of mapping to a single instance, it maps to a subset of the instances. The traffic is distributed inside the subset at random.
The behavior of the Ingress Controller is defined in the Ingress object using annotations. The following example shows how to enable the consistent hash subset with a subset size of 3. The key is the URL argument predictorid.
apiVersion: extensions/v1beta1 kind: Ingress metadata: annotations: nginx.ingress.kubernetes.io/upstream-hash-by: "$arg_predictorid" nginx.ingress.kubernetes.io/upstream-hash-by-subset: "true" nginx.ingress.kubernetes.io/upstream-hash-by-subset-size: "3" name: nginxhello-ingress namespace: default spec: backend: serviceName: nginxhello servicePort: 80
There were some options to satisfy this requirement, such as hacks or creating a custom version of this controller, but getting it into upstream was definitely a better option. The flexibility of open source and a friendly community like Kubernetes allow getting new features without being blocked by a vendor.