Kubernetes operators were introduced as an implementation of the Infrastructure as software concept. Using them you can abstract the deployment of applications and services in a Kubernetes cluster. This is the sixth and last of a series of articles explaining how operators work and how they can be implemented in different languages.
In the previous articles in this series we saw how operators can be implemented in different programming languages. We also saw that, regardless of the chosen language, all operator controllers work the same way, executing the following actions:
Metacontroller is an add-on for Kubernetes that handles the tasks common to every operator controller, (events watching, objects’ states querying, and actions execution). The decoupling of these tasks from the functionality specific to each controller allows the developer to focus on the latter: calculation of the correct state.
In the next sections of this article we are going to see how Metacontroller works and how to implement a simple operator using it.
This article assumes you have Python (version at least 3.6) installed in your computer. You will also need access to a Kubernetes cluster to try the operator. You can use minikube to create a development cluster.
The complete source code for the operator described in this article can be found here https://github.com/flugel-it/k8s-metacontroller-operator
A controller implementation using the Metacontroller has two components: the Metacontroller and one or more lambda controllers.
The Metacontroller is a server that extends Kubernetes API and encapsulates the common parts of writing custom controllers: events watching, states querying and actions execution.
Each lambda controller contains the business logic specific to a custom controller, that is, the functionality determining what the state should be.
The Metacontroller watches multiple resources in order to detect changes in the actual or desired state. Detection of a change invokes the relevant lambda controllers. Using the information from the lambda controller about the desired state, the Metacontroller applies the necessary changes, executing actions calling the Kubernetes API.
The Metacontroller communicates with lambda controllers using webhooks. This greatly simplifies the design and implementation of the lamda controllers, eliminating the need for direct use of Kubernetes API and allowing implementation of controllers in any language that can understand HTTP and JSON.
With Metacontroller, developers have only to create and register their lamda controllers. Metacontroller provides :
Currently, Metacontroller supports the implementation of two kind of controllers:
We are focusing on CompositeControllers in this article. When registering a composite controller, the developer specifies the parent and child resources and the URL of the lambda controller webhook. Metacontroller watches the specified resources and calls the webhook when a change occurs.
When invoked, the webhook receives the state of the parent and its children. It must return the desired state of the parent and a list of all the children that should exist. The Metacontroller then takes care of the actions necessary to move from the current state to the desired one.
In the next sections we are going to use Metacontroller to implement the example operator, immortal containers.
As we said in previous articles, the purpose of the immortal containers operator is to enable users to define containers that should run forever — that is, whenever such containers terminate for any reason, they will be restarted.
Keep in mind that the operator demonstrated in this article is just a toy example which serves only to illustrate the steps involved in the implementation of an operator. The functionality it provides can be achieved with already existing Kubernetes features, such as deployments.
This operator defines a new object kind named ImmortalContainer. Users create objects of this kind to specify containers that must run forever. In each object the user specifies the image he wants to run.
For each ImmortalContainer object the operator’s controller creates a pod to run the container and then recreates the pod whenever it terminates or is deleted. In the same object the operator exposes the name of the created pod. — In previous implementations we added a field to count the number of times the pod has been created; we will omit that step in the current implementation to make it simpler.
Each ImmortalContainer object has the following structure:
ImmortalContainer
- Spec
- Image
- Status
- CurrentPod
Let’s say the operator has been installed and the user wants to create an immortal container to run the image nginx:latest. To do so, he can use kubectl to create an ImmortalContainer object.
# example.yaml
apiVersion: immortalcontainer.flugel.it/v1alpha1
kind: ImmortalContainer
metadata:
name: example-immortalcontainer
spec:
image: nginx:latest
$ kubectl apply -f example.yaml
The controller will detect the new immortal container and respond by creating a pod to run the image nginx:latest. The user can then view the running pod using the following command:
$ kubectl get pods NAME READY STATUS RESTARTS AGE example-immortalcontainer-immortalpod 1/1 unning 0 25m
If someone deletes the pod, it will be recreated.
$ kubectl delete pods example-immortalcontainer-immortalpod pod "example-immortalcontainer-immortalpod" deleted $ kubectl get pods NAME READY STATUS RESTARTS AGE example-immortalcontainer-immortalpod 0/1 ContainerCreating 0 3s
Finally, the user can edit the ImmortalContainer object he has created to see the CurrentPod field.
$ kubectl edit immortalcontainer example-immortalcontainer
apiVersion: immortalcontainer.flugel.it/v1alpha1
kind: ImmortalContainer
metadata:
…
spec:
image: nginx:latest
status:
currentPod: example-immortalcontainer-immortalpod
The custom resources are used to expose the desired and actual states. They define endpoints that give access to collections of objects.
The operator we implemented uses a custom resource to expose a collection of objects belonging to the ImmortalContainer object kind. Users create objects of this kind to specify containers that need to run forever.
As we said previously, each ImmortalContainer object has the following structure:
ImmortalContainer
- Spec
- Image
- Status
- CurrentPod
We used a Custom Resource Definition to create the operator’s custom resource. Again, as in the previous article, we had to write the CRD yaml file. This file defines the new object kind, ImmortalContainer, with its fields and validations.
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: immortalcontainers.immortalcontainers.flugel.it
spec:
group: immortalcontainers.flugel.it
names:
kind: ImmortalContainer
listKind: ImmortalContainerList
plural: immortalcontainers
singular: immortalcontainer
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
properties:
image:
minLength: 1
type: string
required:
- image
type: object
status:
properties:
currentPod:
type: string
type: object
version: v1alpha1
Note that we’ve indicated that our API group is immortalcontainers.flugel.it, the API version is v1alpha1, and the name of the new object kindis ImmortalContainer.
Our operator’s lambda controller provides a webhook. This webhook receives a JSON object containing the current state of an ImmortalContainer object and its Pod child (if any), and returns information specifying the desired state.
The following block shows the complete source code of this lambda controller:
# sync.py
import json
from http.server import BaseHTTPRequestHandler, HTTPServer
def sync(parent, children):
# Compute status based on observed state.
desired_status = {}
if len(children['Pod.v1']) == 1:
desired_status['currentPod'] = list(children['Pod.v1'])[0]
else:
desired_status['currentPod'] = ''
# Generate the desired child object(s).
desired_pods = [
{
'apiVersion': 'v1',
'kind': 'Pod',
'metadata': {
'name': parent['metadata']['name']+'-immortalpod',
'namespace': parent['metadata']['namespace'],
},
'spec': {
'restartPolicy': 'OnFailure',
'containers': [
{
'name': 'acontainer',
'image': parent['spec']['image'],
}
]
}
}
]
return {'status': desired_status, 'children': desired_pods}
class Controller(BaseHTTPRequestHandler):
def do_POST(self):
# Serve the sync() function as a JSON webhook.
self.headers
observed = json.loads(self.rfile.read(int(self.headers.get('content-length'))))
desired = sync(observed['parent'], observed['children'])
self.send_response(200)
self.send_header('Content-type', 'application/json')
self.end_headers()
self.wfile.write(json.dumps(desired).encode())
if __name__ == '__main__':
print("server starting...")
HTTPServer(('', 80), Controller).serve_forever()
You might have noticed that the code is much shorter and simpler than the ones from previous implementations in Go and in Python without Metacontroller. This is one of the main advantages of working with Metacontroller.
The code above uses the following rules to compute desired state for an ImmortalContainer object.
The lambda controller must be registered with the Metacontroller. Registration involves creation of a CompositeController object specifying parent and child resources and the webhook URL. We will see how to do this later.
In order to use the operator we must deploy it to the cluster using the following steps:
Metacontroller can be installed in any Kubernetes cluster following its installation guide. Here we are reproducing the instructions:
Check the cluster availability running
$ kubectl get nodes
Execute these commands to install Metacontroller to the cluster.
# Create metacontroller namespace. $ kubectl create namespace metacontroller # Create metacontroller service account and role/binding. $ kubectl apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/metacontroller/master/manifests/metacontroller-rbac.yaml # Create CRDs for Metacontroller APIs, and the Metacontroller StatefulSet. $ kubectl apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/metacontroller/master/manifests/metacontroller.yaml
Note that, in order for Metacontroller to function properly, the Kubernetes DNS Service must be enabled in the cluster. This is the default on most clusters, but on some development clusters it’s disabled.
After installing Metacontroller you are ready to deploy any existing lambda controller or create your own.
Since the code of our controller is just one file with no extra dependencies, we run it using the default python image instead of building a custom image. To do this we created a ConfigMap to store the code, using kubectl.
$ kubectl -n immortalcontainers create configmap immortalcontainers-controller --from-file=sync.py --dry-run -o yaml | kubectl apply -f -
sync.py is the file containing the webhook code. Note that we created the ConfigMap inside the immortalcontainers namespace (using the -n option).
Then, we created a deployment that runs sync.py using the official Python image. It mounts the code from the ConfigMap using a volume.
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: immortalcontainers-controller
spec:
replicas: 1
selector:
matchLabels:
app: immortalcontainers-controller
template:
metadata:
labels:
app: immortalcontainers-controller
spec:
containers:
- name: controller
image: python:3.7
command: ["python", "-u", "/hooks/sync.py"]
volumeMounts:
- name: hooks
mountPath: /hooks
volumes:
- name: hooks
configMap:
name: immortalcontainers-controller
Since the Metacontroller must be able to resolve and connect to the webhook URL, we created a service to make the webhook callable from outside its pod.
apiVersion: v1
kind: Service
metadata:
name: immortalcontainers-controller
spec:
selector:
app: immortalcontainers-controller
ports:
- port: 80
The lambda controller must be registered with the Metacontroller. Registration requires creation of a CompositeController object containing the names of parent and child resources and the webhook URL.
apiVersion: metacontroller.k8s.io/v1alpha1
kind: CompositeController
metadata:
name: immortalcontainers-controller
spec:
generateSelector: true
parentResource:
apiVersion: immortalcontainers.flugel.it/v1alpha1
resource: immortalcontainers
childResources:
- apiVersion: v1
resource: pods
updateStrategy:
method: Recreate
hooks:
sync:
webhook:
url: http://immortalcontainers-controller.immortalcontainers/sync
Full reference about CompositeController objects can be found here.
this information tells the Metacontroller what resources to watch and what webhook to invoke when a change occurs.
Now that the operator is installed in the cluster, we can try it. We are going to create an ImmortalContainer object to run the nginx:latest image. To do this we need to edit the file config/example-use.yaml to make it look like this:
apiVersion: immortalcontainer.flugel.it/v1alpha1
kind: ImmortalContainer
metadata:
name: example-immortal-container
spec:
image: nginx:latest
We then use kubectl to create the ImmortalContainer object in the cluster.
$ kubectl apply -f config/example-use.yam
The controller will detect the new immortal container and create a pod to run its image. Let’s verify this.
$ kubectl get pods NAME READY STATUS RESTARTS AGE example-immortal-container-immortalpod 1/1 Running 0 2m
Next, let’s see that the pod is recreated if we delete it.
$ kubectl delete pods example-immortal-container-immortalpod pod "example-immortal-container-immortalpod" deleted $ kubectl get pods NAME READY STATUS RESTARTS AGE example-immortal-container-immortalpod 0/1 ContainerCreating 0 3s
Finally, we can edit the ImmortalContainer object to see its status.
$ kubectl edit immortalcontainer example-immortalcontainer
apiVersion: immortalcontainer.flugel.it/v1alpha1
kind: ImmortalContainer
metadata:
…
spec:
image: nginx:latest
status:
currentPod: example-immortal-container-immortalpod
As you can see, the operator works as expected.
Finally, if you’ve been following and reproducing these steps in your cluster, you can use these commands to remove all the changes that have been made.
kubectl delete -f config/crds.yaml kubectl delete -f https://raw.githubusercontent.com/GoogleCloudPlatform/metacontroller/master/manifests/metacontroller.yaml kubectl delete -f https://raw.githubusercontent.com/GoogleCloudPlatform/metacontroller/master/manifests/metacontroller-rbac.yaml kubectl delete namespace immortalcontainers kubectl delete namespace metacontroller
In this article we have seen how, by using Metacontroller, developers can avoid much of the complexity related to operators development. Metacontroller decouples and abstracts the tasks of events watching, state querying, and API calls, allowing developers to focus on the operators’ logic.
Another advantage of Metacontroller, as we have already mentioned, is that its architecture enables developers to use almost any programming language. Also, since lambda controllers don’t depend on any service, they are easy to test.
Finally, we think it’s worth noting that we have used only a subset of the Metacontroller features. You can learn more about the Metacontroller in the official documentation, which also provides examples in various programming languages.
2018, Cryptoland Theme by Artureanec - Ninetheme