> For the complete documentation index, see [llms.txt](https://docs.cybus.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.cybus.io/2-4-1/cybus-helm-charts/working-with-cybus-helm-charts/kubernetes-scheduling.md).

# Kubernetes Scheduling

Control which Kubernetes nodes run your Connectware components and Connectware agents using node selectors, tolerations, and pod anti-affinity. Use these features, for example, to place agents on specific nodes close to your shop floor, allow workloads to run on tainted nodes, or distribute pods across different nodes for resilience.

## Configuration Values

The following Helm values control Kubernetes scheduling. Each value is also available per component (`connectware` Helm chart) or per agent (`connectware-agent` Helm chart) to override the chart-wide defaults shown below.

| Value             | `connectware` Helm chart            | `connectware-agent` Helm chart                | Description                                                                                                          |
| ----------------- | ----------------------------------- | --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| `nodeSelector`    | `global.nodeSelector`               | `protocolMapperAgentDefaults.nodeSelector`    | Node labels that must be present on a Kubernetes node for it to be eligible to run a pod.                            |
| `tolerations`     | `global.tolerations`                | `protocolMapperAgentDefaults.tolerations`     | List of tolerations that allow a pod to be scheduled on tainted nodes.                                               |
| `podAntiAffinity` | Per component only (no global key). | `protocolMapperAgentDefaults.podAntiAffinity` | Mode that controls how pods of the same component or agent are distributed across nodes (`soft`, `hard`, or `none`). |

## Node Selectors

A node selector is a set of labels that a Kubernetes node must have to be eligible to run the pod.

{% tabs %}
{% tab title="connectware Helm Chart" %}
Set `global.nodeSelector` to apply node selector labels to all Connectware components, or set `nodeSelector` on a specific component to override the global value for that component only.

**Example: Applying node selector labels to all Connectware components**

{% code title="values.yaml" lineNumbers="true" %}

```yaml
global:
  nodeSelector:
    mycompany.tld/node-location: shopfloor
    mycompany.tld/building: 1a
```

{% endcode %}

**Example: Overriding the global node selector for a specific component**

{% code title="values.yaml" lineNumbers="true" %}

```yaml
global:
  nodeSelector:
    mycompany.tld/node-location: shopfloor
authServer:
  nodeSelector:
    mycompany.tld/node-location: control-plane
```

{% endcode %}
{% endtab %}

{% tab title="connectware-agent Helm Chart" %}
Set `nodeSelector` in `protocolMapperAgentDefaults` to apply node selector labels to all agents, or set it on a specific agent in `protocolMapperAgents[]` to override the default for that agent only.

**Example: Applying node selector labels to all agents**

{% code title="values.yaml" lineNumbers="true" %}

```yaml
protocolMapperAgentDefaults:
  nodeSelector:
    mycompany.tld/node-location: shopfloor
    mycompany.tld/building: 1a
protocolMapperAgents:
  - name: bender-robots
  - name: welder-robots
```

{% endcode %}

**Example: Unsetting node selector labels for a specific agent**

To remove the default node selector for a specific agent, set `nodeSelector` to `{}`:

{% code title="values.yaml" lineNumbers="true" %}

```yaml
protocolMapperAgentDefaults:
  nodeSelector:
    mycompany.tld/node-location: shopfloor
protocolMapperAgents:
  - name: bender-robots # uses default node selector labels
  - name: welder-robots # no node selector labels
    nodeSelector: {}
```

{% endcode %}

{% hint style="info" %}

## Overriding defaults with the connectware-agent Helm chart

Values set on a specific agent in `protocolMapperAgents[]` completely replace the corresponding values from `protocolMapperAgentDefaults`. See [Configuration Principles for the connectware-agent Helm Chart](/2-4-1/cybus-helm-charts/connectware-agent-helm-chart.md#configuration-principles-for-the-connectware-agent-helm-chart).
{% endhint %}
{% endtab %}
{% endtabs %}

## Tolerations

Kubernetes [taints](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) prevent pods from being scheduled on certain nodes unless the pod has a matching toleration.

{% tabs %}
{% tab title="connectware Helm Chart" %}
Set `global.tolerations` to allow all Connectware components to run on tainted nodes, or set `tolerations` on a specific component to override the global value for that component only.

**Example: Tolerating a taint on all Connectware components**

{% code title="values.yaml" lineNumbers="true" %}

```yaml
global:
  tolerations:
    - key: workload-type
      operator: Equal
      value: industrial
      effect: NoSchedule
```

{% endcode %}
{% endtab %}

{% tab title="connectware-agent Helm Chart" %}
Set `tolerations` in `protocolMapperAgentDefaults` to allow all agents to run on tainted nodes, or set it on a specific agent in `protocolMapperAgents[]` to override the default for that agent only.

**Example: Scheduling all agents on nodes tainted for dedicated industrial workloads**

{% code title="values.yaml" lineNumbers="true" %}

```yaml
protocolMapperAgentDefaults:
  tolerations:
    - key: workload-type
      operator: Equal
      value: industrial
      effect: NoSchedule
protocolMapperAgents:
  - name: bender-robots
  - name: welder-robots
```

{% endcode %}
{% endtab %}
{% endtabs %}

## Pod Anti-Affinity

Pod anti-affinity distributes pods of the same component or agent across different Kubernetes nodes. Choose a mode by setting the `podAntiAffinity` value:

| Mode             | Effect                                                                                                            |
| ---------------- | ----------------------------------------------------------------------------------------------------------------- |
| `soft` (default) | Tries to schedule pods on different nodes, but schedules them on the same node if not enough nodes are available. |
| `hard`           | Schedules pods only on different nodes. If not enough matching nodes are available, pods are not scheduled.       |
| `none`           | Does not add anti-affinity rules.                                                                                 |

{% tabs %}
{% tab title="connectware Helm Chart" %}
Pod anti-affinity is configured per component. Not every component supports anti-affinity. Supported components include `authServer`, `nats`, `resourceStatusTracking`, and `topicExplorer`.

**Example: Setting pod anti-affinity for the auth-server component**

{% code title="values.yaml" lineNumbers="true" %}

```yaml
authServer:
  podAntiAffinity: hard
```

{% endcode %}

**Example: Changing the topology key**

Anti-affinity is evaluated per node by default. To distribute pods across availability zones, set `podAntiAffinityTopologyKey`:

{% code title="values.yaml" lineNumbers="true" %}

```yaml
authServer:
  podAntiAffinity: hard
  podAntiAffinityTopologyKey: topology.kubernetes.io/zone
```

{% endcode %}
{% endtab %}

{% tab title="connectware-agent Helm Chart" %}
Set `podAntiAffinity` in `protocolMapperAgentDefaults` to apply the same mode to all agents, or set it on a specific agent in `protocolMapperAgents[]` to override the default for that agent only.

**Example: Setting pod anti-affinity for all agents**

{% code title="values.yaml" lineNumbers="true" %}

```yaml
protocolMapperAgentDefaults:
  podAntiAffinity: hard
protocolMapperAgents:
  - name: bender-robots
  - name: welder-robots
```

{% endcode %}

### Overriding Pod Anti-Affinity Options (Advanced)

Customize the anti-affinity rules to match your cluster setup using `podAntiAffinityOptions`.

**Example: Changing the topology key**

Anti-affinity is evaluated per node by default. To distribute agents across availability zones, set `podAntiAffinityOptions.topologyKey`:

{% code title="values.yaml" lineNumbers="true" %}

```yaml
protocolMapperAgentDefaults:
  podAntiAffinity: hard
  podAntiAffinityOptions:
    topologyKey: topology.kubernetes.io/zone
```

{% endcode %}

**Example: Changing the match expression**

To change which pods are considered for anti-affinity, configure `podAntiAffinityOptions.key`, `podAntiAffinityOptions.operator`, and `podAntiAffinityOptions.value`. Use this, for example, to prevent agents from being scheduled on the same node as agents from other Helm installations:

{% code title="values.yaml" lineNumbers="true" %}

```yaml
protocolMapperAgentDefaults:
  podAntiAffinity: hard
  podAntiAffinityOptions:
    key: app.kubernetes.io/component
    operator: In
    value: protocol-mapper-agent
```

{% endcode %}
{% endtab %}
{% endtabs %}

## Verification

After applying your changes, verify that pods are scheduled on the expected nodes:

{% code lineNumbers="true" %}

```bash
# List pods and the nodes they are scheduled on
kubectl get pods -n ${NAMESPACE} -o wide

# Describe a pod to see its node selector, tolerations, and affinity rules
kubectl describe pod -n ${NAMESPACE} ${POD_NAME}
```

{% endcode %}

{% hint style="info" %}
Replace `${NAMESPACE}` with your Connectware namespace and `${POD_NAME}` with the name of the pod to inspect.
{% endhint %}

**Result**

Pods are scheduled only on nodes that match the configured node selector labels and tolerations. With pod anti-affinity enabled, pods of the same component or agent are spread across different nodes according to the chosen mode.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.cybus.io/2-4-1/cybus-helm-charts/working-with-cybus-helm-charts/kubernetes-scheduling.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.