Building Kubernetes Operators part 1: Operator Pattern and Concepts

Posted on 14 Mar 2019, By Miguel García under Infrastructure as Code, Kubernetes

Get Started

Kubernetes operators were introduced as an implementation of the Infrastructure as software concept. Using them you can abstract the deployment of applications and services in a Kubernetes cluster. This is the first article of a series explaining how operators work, and how they can be implemented in different languages.

Introduction

In this article series we are going to dive into the implementation of Kubernetes Operators to orchestrate and manage resources in a Kubernetes cluster. This first article gives an overview about what Kubernetes Operators are, what problem they solve and how they work.

Subsequent articles will be more technical and require some specific knowledge in Kubernetes and programming. But you don’t need to be a Kubernetes guru or even an exceptional programmer to understand this series. There are plenty of online resources to fill the knowledge gaps, so let’s get started.

What problem does a Kubernetes Operator solve?

Suppose, for example, that you want to serve some WordPress sites on a Kubernetes cluster. You would need to create and manage various objects (deployment, service, ingress, etc) for each site. As you might imagine, the maintenance of such a solution could be difficult if you are serving many sites.

Instead of dealing with so many objects, you could install the “wordpress-operator”. This will extend the Kubernetes API, exposing a new object kind called “WordPress” that allows you to specify all the details of a site using a single object. Kubectl can be used to manage such objects just as it manages any other Kubernetes API object.

The operator uses the information in the “WordPress” objects to automatically create and manage all the objects required to serve the sites. Since the wordpress operator knows how wordpress works and how to deploy it on Kubernetes, it can hide a lot of the complexity of sites setup. Each “WordPress” object needs parameters specific only to one site but needs no information about hosting WordPress on Kubernetes.

Example WordPress object for one site (from wordpress-operator docs):

apiVersion: wordpress.presslabs.org/v1alpha1
kind: WordPress
metadata:
  name: example
spec:
  replicas: 3
  domains:
    - example.com
  code: # where to find the code
    git:
      repository: https://github.com/example.com
  media: # where to find the media files
    # by default, code get's an empty dir. Can be one of the following:
    # persistentVolumeClaim: {}
    # hostPath: {}
    # emptyDir: {}
  # extra volumes for the WordPress container
  volumes: []
  # extra volume mounts for the WordPress container
  volumeMounts: []
  # extra env variables for the WordPress container
  env:
    - name: WORDPRESS_DB_HOST
      value: mysite-mysql
    - name: WORDPRESS_DB_PASSWORD
      valueFrom:
        secretKeyRef: mysite-mysql
        key: PASSWORD
  envFrom: []

  # secret containing HTTPS certificate
  tlsSecretRef: mysite-tls
  # extra ingress annotations
  ingressAnnotations: {}

So, using the operator and kubectl you can host multiple WordPress sites in a simpler way. Letting the operator take care of the details saves you a lot of time and headaches.

In general, operators extend the Kubernetes API, providing users new ways to manage and orchestrate objects, abstracting its complexity. In that sense, an operator provides you with an API to specify the desired state – that is to say, which sites to serve – while taking care of the details to reach that state – getting the sites served. They can achieve that because, as we’ve just seen, they have domain and application-specific knowledge that allows them to automate some tasks.

How Operators Work:

Kubernetes operators leverage two concepts to abstract complex orchestration: Custom Resources to extend the API, adding new kinds of objects, and Custom Controllers to manage custom and non-custom objects according to the information stored in the custom resource.

Custom Resources

A resource is an endpoint in the Kubernetes API that stores a collection of API objects of a certain kind. Probably you have already used some built-in resources, like pods, a collection of objects of kind Pod.

A custom resource is a resource not shipped with Kubernetes, but installed on a cluster after its creation. Once installed, users can access its objects using kubectl, the same way they do for built-in objects. Custom resources are installed on a cluster, creating a Custom Resource Definitions (CRDs) object or using API Aggregation (more on that on the next article).

Custom Controllers

Custom Resources alone are just “dummy” storage for structured data, but by joining them with Custom Controllers we can create a declarative API. Such an API allows users to specify the desired state using our resources and tries to keep the actual state in sync with the desired state. This combination of Custom Resources and Custom Controllers is known as the “Operator Pattern”.

Custom Controllers are programs that use Kubernetes API to read the desired state from the custom resource, check the actual state and make the changes needed to transform the current state into the desired state. This way of working is called the “Reconciler Pattern” and you can read more about it in “Cloud Native Infrastructure” by Kris Nova and Justin Garrison (https://www.oreilly.com/library/view/cloud-native-infrastructure/9781491984291/ch04.html)

Note: This pattern, in which controllers read a desired state from resources and try to reconcile the desired and actual states is used not only in operators but also by the Kubernetes core itself. That is how deployments are implemented, for example.

In Conclusion

We have just seen what operators are, what problem they solve, and how they solve it using custom resources and controllers. In the following articles we are going to dive deeper into the implementation of custom operators in Golang and Python following the Operator Pattern and the Reconciler Pattern for the Controller.

References

https://coreos.com/blog/introducing-operators.html
https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources
https://www.oreilly.com/library/view/cloud-native-infrastructure/9781491984291/ch04.html
https://github.com/kubernetes/kubernetes/blob/master/pkg/controller/deployment/deployment_controller.go



Prepare Your Infrastructure for the Challenges Ahead

Your clients, investors and business partners will notice the difference, from now on, you can progress with confidence.

Get Started