Kubernetes Scheduling

Target specific Kubernetes nodes for Connectware components and Connectware agents using node selectors, tolerations, and pod anti-affinity.

Control which Kubernetes nodes run your Connectware components and Connectware agents using node selectors, tolerations, and pod anti-affinity. Use these features, for example, to place agents on specific nodes close to your shop floor, allow workloads to run on tainted nodes, or distribute pods across different nodes for resilience.

Configuration Values

The following Helm values control Kubernetes scheduling. Each value is also available per component (connectware Helm chart) or per agent (connectware-agent Helm chart) to override the chart-wide defaults shown below.

Value

connectware Helm chart

connectware-agent Helm chart

Description

nodeSelector

global.nodeSelector

protocolMapperAgentDefaults.nodeSelector

Node labels that must be present on a Kubernetes node for it to be eligible to run a pod.

tolerations

global.tolerations

protocolMapperAgentDefaults.tolerations

List of tolerations that allow a pod to be scheduled on tainted nodes.

podAntiAffinity

Per component only (no global key).

protocolMapperAgentDefaults.podAntiAffinity

Mode that controls how pods of the same component or agent are distributed across nodes (soft, hard, or none).

Node Selectors

A node selector is a set of labels that a Kubernetes node must have to be eligible to run the pod.

Set global.nodeSelector to apply node selector labels to all Connectware components, or set nodeSelector on a specific component to override the global value for that component only.

Example: Applying node selector labels to all Connectware components

values.yaml

global:
  nodeSelector:
    mycompany.tld/node-location: shopfloor
    mycompany.tld/building: 1a

Example: Overriding the global node selector for a specific component

values.yaml

global:
  nodeSelector:
    mycompany.tld/node-location: shopfloor
authServer:
  nodeSelector:
    mycompany.tld/node-location: control-plane

Set nodeSelector in protocolMapperAgentDefaults to apply node selector labels to all agents, or set it on a specific agent in protocolMapperAgents[] to override the default for that agent only.

Example: Applying node selector labels to all agents

values.yaml

protocolMapperAgentDefaults:
  nodeSelector:
    mycompany.tld/node-location: shopfloor
    mycompany.tld/building: 1a
protocolMapperAgents:
  - name: bender-robots
  - name: welder-robots

Example: Unsetting node selector labels for a specific agent

To remove the default node selector for a specific agent, set nodeSelector to {}:

values.yaml

protocolMapperAgentDefaults:
  nodeSelector:
    mycompany.tld/node-location: shopfloor
protocolMapperAgents:
  - name: bender-robots # uses default node selector labels
  - name: welder-robots # no node selector labels
    nodeSelector: {}

Overriding defaults with the connectware-agent Helm chart

Values set on a specific agent in protocolMapperAgents[] completely replace the corresponding values from protocolMapperAgentDefaults. See Configuration Principles for the connectware-agent Helm Chart.

Tolerations

Kubernetes taints prevent pods from being scheduled on certain nodes unless the pod has a matching toleration.

Set global.tolerations to allow all Connectware components to run on tainted nodes, or set tolerations on a specific component to override the global value for that component only.

Example: Tolerating a taint on all Connectware components

values.yaml

global:
  tolerations:
    - key: workload-type
      operator: Equal
      value: industrial
      effect: NoSchedule

Set tolerations in protocolMapperAgentDefaults to allow all agents to run on tainted nodes, or set it on a specific agent in protocolMapperAgents[] to override the default for that agent only.

Example: Scheduling all agents on nodes tainted for dedicated industrial workloads

values.yaml

protocolMapperAgentDefaults:
  tolerations:
    - key: workload-type
      operator: Equal
      value: industrial
      effect: NoSchedule
protocolMapperAgents:
  - name: bender-robots
  - name: welder-robots

Pod Anti-Affinity

Pod anti-affinity distributes pods of the same component or agent across different Kubernetes nodes. Choose a mode by setting the podAntiAffinity value:

Mode

Effect

soft (default)

Tries to schedule pods on different nodes, but schedules them on the same node if not enough nodes are available.

hard

Schedules pods only on different nodes. If not enough matching nodes are available, pods are not scheduled.

none

Does not add anti-affinity rules.

Pod anti-affinity is configured per component. Not every component supports anti-affinity. Supported components include authServer, nats, resourceStatusTracking, and topicExplorer.

Example: Setting pod anti-affinity for the auth-server component

values.yaml

authServer:
  podAntiAffinity: hard

Example: Changing the topology key

Anti-affinity is evaluated per node by default. To distribute pods across availability zones, set podAntiAffinityTopologyKey:

values.yaml

authServer:
  podAntiAffinity: hard
  podAntiAffinityTopologyKey: topology.kubernetes.io/zone

Set podAntiAffinity in protocolMapperAgentDefaults to apply the same mode to all agents, or set it on a specific agent in protocolMapperAgents[] to override the default for that agent only.

Example: Setting pod anti-affinity for all agents

values.yaml

protocolMapperAgentDefaults:
  podAntiAffinity: hard
protocolMapperAgents:
  - name: bender-robots
  - name: welder-robots

Overriding Pod Anti-Affinity Options (Advanced)

Customize the anti-affinity rules to match your cluster setup using podAntiAffinityOptions.

Example: Changing the topology key

Anti-affinity is evaluated per node by default. To distribute agents across availability zones, set podAntiAffinityOptions.topologyKey:

values.yaml

protocolMapperAgentDefaults:
  podAntiAffinity: hard
  podAntiAffinityOptions:
    topologyKey: topology.kubernetes.io/zone

Example: Changing the match expression

To change which pods are considered for anti-affinity, configure podAntiAffinityOptions.key, podAntiAffinityOptions.operator, and podAntiAffinityOptions.value. Use this, for example, to prevent agents from being scheduled on the same node as agents from other Helm installations:

values.yaml

protocolMapperAgentDefaults:
  podAntiAffinity: hard
  podAntiAffinityOptions:
    key: app.kubernetes.io/component
    operator: In
    value: protocol-mapper-agent

Verification

After applying your changes, verify that pods are scheduled on the expected nodes:

# List pods and the nodes they are scheduled on
kubectl get pods -n ${NAMESPACE} -o wide

# Describe a pod to see its node selector, tolerations, and affinity rules
kubectl describe pod -n ${NAMESPACE} ${POD_NAME}

Replace ${NAMESPACE} with your Connectware namespace and ${POD_NAME} with the name of the pod to inspect.

Result

Pods are scheduled only on nodes that match the configured node selector labels and tolerations. With pod anti-affinity enabled, pods of the same component or agent are spread across different nodes according to the chosen mode.

PreviousEnvironment Variables NextCompute Resources

Last updated 5 days ago

Was this helpful?