Operator SDK: Beginner Tutorial

Kubernetes is in boom now and many corporations are moving to Kubernetes due its feature set and large support of libraries like horizontal auto-scale, databases, CI/CD etc. To provide all these features Kubernetes provides ways to extend itself. All the these extensions are written as operators which automate stuff inside Kubernetes and also sometimes outside. Before we begin let us define few terms that one should know before starting out with operators.

Operators, Controllers and Custom Resources

Operators are software extensions to Kubernetes that make use of custom resources to manage applications and their components.

A resource is an endpoint in the Kubernetes API that stores a collection of API objects of a certain kind; for example, the built-in pods resource contains a collection of Pod objects.

A custom resource is an extension of the Kubernetes API that is not necessarily available in a default Kubernetes installation. It represents a customization of a particular Kubernetes installation.

Operators use controllers to manage custom resources. Those custom resources have a control loop.

A control loop is a non-terminating loop that regulates the state of a system.

A controller tracks at least one Kubernetes resource type. These objects have a spec field that represents the desired state. The controller(s) for that resource are responsible for making the current state come closer to that desired state.

So, at end operator consist of controllers and custom resources, where custom resources are objects that are not defined in default Kubernetes installation and works as extension of Kubernetes API and controllers are loops that maintains defined spec or state of a custom resources.

How to build one?

There are multiple ways to build a operator for Kubernetes, kubebuilder is most prevalent one but I like to Operator SDK that makes scaffolding, building and deploying easy.

In this tutorial we will build operator that does basic math functions, add, subtract, multiply and divide. These are irrelevant for Kubernetes but exercise lays basic foundation of how a operator works.

Pre-requisite

1. Kubernetes cluster, use kind or docker for desktop provides one with kubectl configured.
2. Basic Golang.
3. Installed Operator SDK.

After installation, create folder of any name, I created operator-tutorial inside that folder run scaffolding for your operator

operator-sdk init --domain calc.k8s --repo github.com/ratnadeep007/operator-tutorial

You can give your own github repo. --domain refers to namespace where all your custom resources will belong. In default deployment manifest, apiVersion: apps/v1 that apps is what we define in domain. As best practice we define custom resources as domain it should not collide with other default domains in Kubernetes.

This command will create multiple folders and files which we will look later. Before that we need to define our api and custom resources.

Requirements:

1. Spec will contain input of 2 numbers
2. One output of operation.

operator-sdk create api --group basic --version v1alpha1 --kind Add --resource --controller

In above command, we are telling operator-sdk to scaffold api which belongs to group basic of calc.k8s with version v1alpha1 of kind Add with resources which means it will add Custom Resource Definition and Controller. After running this command you will find, folder add api/v1alpha1 and controllers. api/v1alpha1 will contain Custom Resource Definition generation go code and controllers contain controller for that custom resource.

In add_type.go main 2 structs are AppSpec and AppStatus.

// AddSpec defines the desired state of Add
type AddSpec struct {
    Num1 int `json:"num1"`
    Num2 int `json:"num2"`
}

// AddStatus defines the observed state of Add
type AddStatus struct {
    Output int `json:"result"`
}

Num1 and Num2 will be our input and Output is our result in AddStatus. So, with following our manifest for add will be like:

apiVersion: basic.calc.k8s/v1alpha1
kind: Add
metadata:
  name: basic-add
spec:
  num1: 1
  num2: 2

By default kubectl will only show name and age of resource. For showing our input and output fields we need to add following just below //+kubebuilder:subresource:status:

//+kubebuilder:printcolumn:JSONPath=".spec.num1",name=Number 1,type=integer
//+kubebuilder:printcolumn:JSONPath=".spec.num2",name=Number 2,type=integer
//+kubebuilder:printcolumn:JSONPath=".status.result",name=Result,type=intege

JSONPath is path of variable we want to display. For input we used .spec.num and .spec.num2. For output, we used .status.result cause in AppStatus we gave Output intjson:"result"` which make output field as result field when creating json.

But before applying manifest we need to generate custom resource definitions and also code for *_types.go files.

For generating code:

make generate

For generating Custom Resource Definition(CRD):

make manifest

Now we need to deploy these to our Kubernetes cluster. For that I took easiest path of running it on my system and connecting to my cluster by kubeconfig. So, if you can get nodes using kubectl get nodes then you can run

make install run

to deploy CRD and run controller.

Create following manifest file for add:

apiVersion: basic.calc.k8s/v1alpha1
kind: Add
metadata:
  name: basic-add
spec:
  num1: 1
  num2: 2

and kubectl apply -f add.yml

After this you can see you add custom resource as

$ kubectl get add
NAME            NUMBER 1         NUMBER 2           RESULT
basic-add      1                         2

You can see Result because we didn't write any controller code for our custom resource. Open controllers/add_controller.go and in that file we need to concentrate on Reconcile function which runs on change of our resource be it create, update or delete. For addition we need to update that like following:

func (r *AddReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
    _ = log.FromContext(ctx)

    instance := &v1alpha1.Add{}

    err := r.Client.Get(ctx, req.NamespacedName, instance)
    if err != nil {
        fmt.Printf("Error getting instance: %s", err)
    }

    instance.Status.Output = instance.Spec.Num1 + instance.Spec.Num2
    err = r.Client.Status().Update(ctx, instance)
    if err != nil {
        fmt.Printf("Error updating instance: %s", err)
    }

    return ctrl.Result{}, nil
}

In above code we are doing following:

1. Getting a instance of add.
2. Checking if it exists on our cluster.
3. Updating output field of status with result.

Now, delete and creating our Add Custom Resource and you see output in result field.

$ kubectl get add
NAME            NUMBER 1         NUMBER 2           RESULT
basic-add      1                         2                           3

So, this very basic of operator in Kubernetes. You can call external apis, modifying already present resources like Pods, etc in this controller.

I added all other functions (substract, multiply, divide) in my project which is available in this link: github

For questions, comment.

Keep learning.