Taints and Tolerations vs Node Affinity in Kubernetes

When running workloads on Kubernetes, it's often necessary to control which nodes your pods can be scheduled on. Kubernetes provides two main ways to accomplish this: taints and tolerations, and node affinity.

Taints allow you to repel pods from specific nodes. By applying a taint to a node, you indicate that no pods can be scheduled there unless they have a matching toleration. This allows you to dedicate certain nodes to particular pods. For example, you may want to reserve a set of nodes for a batch workload.

Node affinity gives pods the ability to specify rules about which nodes they can run on, based on node labels. There are two types of node affinity: required and preferred. With required node affinity, pods can only be placed on nodes matching certain rules. With preferred node affinity, scheduling tries to match the rules but is not obligated to. For instance, you may want pods placed on nodes with SSD disks or large amounts of RAM.

The main difference is that taints and tolerations dedicate nodes to specific pods, whereas node affinity allows targeting nodes based on labels in a more flexible way. Taints and tolerations are useful when you need to reserve nodes. Node affinity is better when you want to target certain node characteristics like hardware specs.

Here's an example of how to taint a node:

kubectl taint nodes node1 key1=value1:NoSchedule        

This will add a taint to node1 that has key "key1" and value "value1". The ":NoSchedule" indicates that pods will not be scheduled on this node unless they have a matching toleration.

To add a toleration to a pod:

spec:
  tolerations:
  - key: "key1"
    operator: "Equal"
    value: "value1"
    effect: "NoSchedule"        

Now this pod will be able to schedule onto node1 despite the taint because it has a matching toleration.

Taints and tolerations allow you to dedicate nodes to specific pods. For example, you may want to dedicate a set of nodes to a batch workload 

Node Affinity

Node affinity allows pods to specify rules for which nodes they can be scheduled on. There are two types of node affinity:

Required node affinity - the pod can only be placed on nodes matching the specified rules

Preferred node affinity - the scheduler will try to place the pod on nodes matching the rules, but is not required to.

Here is an example of required node affinity:

spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: size
            operator: In
            values:
            - Large        

This pod can only be placed on nodes that have size=Large.

And here is an example of preferred node affinity:

spec:
  affinity:
    nodeAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 1
        preference:
          matchExpressions:
          - key: disk
            operator: In
            values:
            - ssd        

This pod prefers to be placed on nodes with disk=ssd, but is not required to be.

Node affinity is useful when you want to target specific nodes by their labels, rather than dedicating nodes. For example, you may want pods placed on nodes with fast disks or large amounts of memory.

Conclusion

Taints and tolerations provide a way to dedicate certain nodes to specific pods by tainting nodes and adding tolerations to pods that need to run on those nodes. This ensures those pods will be scheduled on the tainted nodes. Node affinity allows more flexible targeting of nodes based on labels and node characteristics. With node affinity, pods can specify scheduling preferences and requirements using node labels rather than strictly dedicating nodes.

If you need to reserve nodes for certain pods, taints and tolerations are the best approach. Taint nodes and add matching tolerations to the pods that must run on them. If you want to target nodes based on characteristics like hardware specs, node affinity is more flexible and allows pods to specify scheduling rules using node labels. The choice depends on whether you need to dedicate nodes or target them more dynamically. Both methods have their place in controlling pod scheduling in Kubernetes.