> For the complete documentation index, see [llms.txt](https://docs.cybus.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.cybus.io/data-flows/service-commissioning-files/resources/cybus-mapping.md).

# Cybus::Mapping

Route data between MQTT topics and endpoints, and transform it on the way with Rule Engine rules.

The **Cybus::Mapping** resource subscribes to one or more MQTT topics or endpoints ([subscribe](#subscribe)) and publishes the data to another topic or endpoint ([publish](#publish)). [Rules](#rules) can transform the data on the way, directly in Connectware.

By default, a mapping subscribes and publishes on the internal MQTT broker. Through an MQTT connection, it can also subscribe to or publish on another broker.

## Mapping Properties

| Property                          | Type       | Required     | Default             |
| --------------------------------- | ---------- | ------------ | ------------------- |
| [mappings](#mappings)             | `object[]` | **Required** |                     |
| [targetState](#targetstate)       | `enum`     | Optional     | `"enabled"`         |
| [agentName](#agentname)           | `string`   | Optional     | `"protocol-mapper"` |
| [inputBuffering](#inputbuffering) | `object`   | Optional     |                     |

### mappings

A list of mappings. Each mapping routes data from its `subscribe` source to its `publish` target. A mapping has the following properties:

* **Required**
* Type: `object[]`

| Property                | Type   | Required     |
| ----------------------- | ------ | ------------ |
| [subscribe](#subscribe) | object | **Required** |
| [publish](#publish)     | object | **Required** |
| [rules](#rules)         | array  | Optional     |

#### subscribe

The source of the mapping: one endpoint or topic, or a list of them. To combine the values of a list into one message, add the [collect](https://docs.cybus.io/data-flows/rule-engine/data-processing-rules.md#collect) rule to the [rules](#rules) of the mapping.

* **Required**
* Type: `object` or `object[]`

| Property   | Type    | Required                         |
| ---------- | ------- | -------------------------------- |
| endpoint   | string  | One of endpoint, topic, or asset |
| topic      | string  | One of endpoint, topic, or asset |
| asset      | string  | One of endpoint, topic, or asset |
| qos        | integer | Optional                         |
| connection | string  | Optional                         |
| label      | string  | Optional                         |

**endpoint**

A [Cybus::Endpoint](https://docs.cybus.io/data-flows/service-commissioning-files/resources/cybus-endpoint.md), usually written as `!ref <endpoint resource ID>`. The operation of the endpoint decides which of its topics the mapping subscribes to:

* `subscribe`: The topic of the endpoint, with the values that it reads.
* `read` or `write`: The `/res` topic, with the [results](https://docs.cybus.io/data-flows/service-commissioning-files/resources/cybus-endpoint.md#operation-results) of its operations.
* **Required**, one of `endpoint`, `topic`, or `asset`
* Type: `string`

**asset**

A reference to a [Cybus::Asset](https://docs.cybus.io/data-flows/service-commissioning-files/resources/cybus-asset.md), replaced by topics when you install the service:

* `<asset>.<property>`, for example `pump01.temperature`, subscribes to one data property.
* `<asset>`, for example `pump01`, subscribes to every data property of the asset, without its `$meta`, `/set`, and `/res` topics.

The asset must be declared in this service commissioning file or in an installed service, and it must be mounted. See [Topics of an Asset](https://docs.cybus.io/data-flows/service-commissioning-files/resources/cybus-asset.md#topics-of-an-asset).

* **Required**, one of `endpoint`, `topic`, or `asset`
* Type: `string`

**topic**

An MQTT topic. The topic can contain [wildcards](#wildcards).

{% hint style="warning" %}
Connectware uses this topic as written and does not add the topic prefix of the service. To subscribe below the prefix, write `!sub '${Cybus::MqttRoot}/<topic>'`.
{% endhint %}

* **Required**, one of `endpoint`, `topic`, or `asset`
* Type: `string`

**qos**

The MQTT Quality of Service (QoS) level of this subscription.

This level applies **only** between the broker and this mapping. For QoS `1` or `2` from end to end, every MQTT connection along the way must use that level.

* **Optional**
* Type: `integer`, one of `0`, `1`, `2`
* Default: `0`

**connection**

A [Cybus::Connection](https://docs.cybus.io/data-flows/service-commissioning-files/resources/cybus-connection.md) with the protocol `Mqtt`. The mapping then subscribes through this connection instead of the internal broker, for example to the broker of another system.

{% hint style="warning" %}
The connection must use the protocol `Mqtt`. A mapping that references a connection with another protocol fails.
{% endhint %}

* **Optional**
* Type: `string`

The following mapping subscribes to topics on an external broker and publishes them on the internal broker:

{% code lineNumbers="true" %}

```yaml
resources:
  externalMqtt:
    type: Cybus::Connection
    properties:
      protocol: Mqtt
      connection:
        host: broker.example.com
        port: 1883

  mapFromExternal:
    type: Cybus::Mapping
    properties:
      mappings:
        - subscribe:
            topic: 'shopfloor/in/#topic'
            connection: !ref externalMqtt
          publish:
            topic: 'internal/out/$topic' # published on the internal broker
```

{% endcode %}

**label**

A name for the messages of this subscription. The [collect](https://docs.cybus.io/data-flows/rule-engine/data-processing-rules.md#collect) rule uses the label as the key of the value in its output object. In rules, the label is available as `$context.label`. For an example, see [Routing Messages by Label](#routing-messages-by-label). Without a label, collect uses the topic as the key, and `$context.label` is undefined.

With [wildcards](#wildcards) in the topic, you can build the label from the wildcard values. Write the name of a wildcard in curly braces. With dynamic labels, you **must** put the whole label in quotes, because YAML reads unquoted curly braces as an object.

{% code lineNumbers="true" %}

```yaml
testMapping:
  type: Cybus::Mapping
  properties:
    mappings:
      - subscribe:
          - topic: factory/+machineName/+sensorName
            label: 'Machine: {machineName} Sensor: {sensorName}'
        publish:
          topic: factory/out
        rules:
          - collect: {}
```

{% endcode %}

A message on `factory/Robot001/Temperature` gets the label `Machine: Robot001 Sensor: Temperature`. A message on `factory/Robot001/Current` gets the label `Machine: Robot001 Sensor: Current`.

Connectware matches the wildcards by name, so their order in the label is up to you. For example, `'Sensor: {sensorName} Machine: {machineName}'` gives the label `Sensor: Temperature Machine: Robot001`.

Dynamic labels keep collect mappings short, because one subscription with wildcards covers many sources.

* **Optional**
* Type: `string`

#### publish

The target of the mapping: one endpoint or topic.

* **Required**
* Type: `object`

| Property   | Type    | Required                         |
| ---------- | ------- | -------------------------------- |
| endpoint   | string  | One of endpoint, topic, or asset |
| topic      | string  | One of endpoint, topic, or asset |
| asset      | string  | One of endpoint, topic, or asset |
| qos        | integer | Optional                         |
| retain     | boolean | Optional                         |
| connection | string  | Optional                         |

**endpoint**

A [Cybus::Endpoint](https://docs.cybus.io/data-flows/service-commissioning-files/resources/cybus-endpoint.md), usually written as `!ref <endpoint resource ID>`. The operation of the endpoint decides what the mapping does with the message:

* `subscribe`: The mapping publishes it on the topic of the endpoint.
* `read`: The mapping triggers a read, through the `/req` topic.
* `write`: The endpoint writes the message to the device, through the `/set` topic.
* **Required**, one of `endpoint`, `topic`, or `asset`
* Type: `string`

**asset**

A reference to a data property of a [Cybus::Asset](https://docs.cybus.io/data-flows/service-commissioning-files/resources/cybus-asset.md) in the form `<asset>.<property>`, for example `pump01.temperature`. Connectware replaces the reference with the topic of that property when you install the service. The asset must be declared in this service commissioning file or in an installed service, and it must be mounted. See [Topics of an Asset](https://docs.cybus.io/data-flows/service-commissioning-files/resources/cybus-asset.md#topics-of-an-asset).

* **Required**, one of `endpoint`, `topic`, or `asset`
* Type: `string`

**topic**

An MQTT topic. The topic can contain the values of named [wildcards](#wildcards).

{% hint style="warning" %}
Connectware uses this topic as written and does not add the topic prefix of the service. To publish below the prefix, write `!sub '${Cybus::MqttRoot}/<topic>'`.
{% endhint %}

* **Required**, one of `endpoint`, `topic`, or `asset`
* Type: `string`

**qos**

The MQTT Quality of Service (QoS) level for publishing.

This level applies **only** between this mapping and the broker. For QoS `1` or `2` from end to end, every MQTT connection along the way must use that level.

* **Optional**
* Type: `integer`, one of `0`, `1`, `2`
* Default: `0`

**retain**

Whether the broker keeps the last message as a retained message, as a last-value cache. This applies to the internal broker, and to an external broker when you set `connection`.

* **Optional**
* Type: `boolean`
* Default: `false`

**connection**

A [Cybus::Connection](https://docs.cybus.io/data-flows/service-commissioning-files/resources/cybus-connection.md) with the protocol `Mqtt`. The mapping then publishes through this connection instead of the internal broker, for example to a cloud broker.

{% hint style="warning" %}
The connection must use the protocol `Mqtt`. A mapping that references a connection with another protocol fails.
{% endhint %}

* **Optional**
* Type: `string`

The following mapping forwards messages from the internal broker to an external broker. The connection signs in with credentials from parameters, so they are not part of the service commissioning file. The password is a [secret parameter](https://docs.cybus.io/data-flows/service-commissioning-files/parameters.md#secret-parameters), so the API and the Admin UI do not show its value:

{% code lineNumbers="true" %}

```yaml
parameters:
  brokerUser:
    type: string
  brokerPassword:
    type: string
    secret: true

resources:
  externalMqttOut:
    type: Cybus::Connection
    properties:
      protocol: Mqtt
      connection:
        host: cloud.example.com
        port: 1883
        username: !ref brokerUser
        password: !ref brokerPassword

  mapToExternal:
    type: Cybus::Mapping
    properties:
      mappings:
        - subscribe:
            topic: 'internal/in/#topic' # read from the internal broker
          publish:
            topic: 'cloud/$topic'
            connection: !ref externalMqttOut
            qos: 1
            retain: false
```

{% endcode %}

#### rules

[Rule Engine](https://docs.cybus.io/data-flows/rule-engine.md) rules that transform the data before the mapping publishes it. Rules can also change the publish topic.

* **Optional**
* Type: `array` of rules. See [Data Processing Rules](https://docs.cybus.io/data-flows/rule-engine/data-processing-rules.md).

### targetState

The state of the mapping after the service starts.

* **Optional**
* Type: `enum`, one of `enabled`, `disabled`
* Default: `"enabled"`

### agentName

The [agent](https://docs.cybus.io/data-flows/agents.md) that runs the mapping. By default, the mapping runs in Connectware.

To spread mappings with many rules across several agents, set the name of an agent here. The name must match the agent name exactly. Otherwise, Connectware reports an error when it enables the mapping.

If the mapping subscribes or publishes through a `connection`, it must run on the same agent as that connection. Otherwise, the installation fails.

A mapping on an agent still subscribes and publishes on the central MQTT broker. Keep in mind that each message that the mapping processes therefore travels from the broker to the agent and back. This causes two extra MQTT message transmissions.

* **Optional**
* Type: `string`
* Default: `"protocol-mapper"`

### inputBuffering

Gives the subscriptions of the mapping their own input queue. It holds the messages that arrive faster than the mapping can process or publish them. Turn it on for mappings with high data rates.

Input buffering is off by default, and the mapping uses a queue that it shares with other resources. This works as long as Connectware processes messages as fast as they arrive. If processing or forwarding is slow, the queue grows until Connectware can run out of memory.

* **Optional**
* Type: `object`

With input buffering on, the mapping processes 2 messages at a time and queues up to 5000 messages. If messages arrive too fast for a longer time, the queue fills up. The mapping then discards new messages, so that it does not run out of memory. Resources are finite, so this cannot be avoided entirely.

The `inputBuffering` object takes the following properties. Only `enabled` takes effect:

| Property                  | Type      | Default | Description                                                       |
| ------------------------- | --------- | ------- | ----------------------------------------------------------------- |
| `enabled`                 | `boolean` | `false` | Turns input buffering on.                                         |
| `maxInputBufferSize`      | `integer` | `5000`  | Has no effect. The mapping always queues up to 5000 messages.     |
| `maxConcurrentMessages`   | `integer` | `2`     | Has no effect. The mapping always processes 2 messages at a time. |
| `waitingTimeOnEmptyQueue` | `integer` | `10`    | Has no effect.                                                    |

## Mapping Topics and Endpoints

A mapping connects topics and endpoints in any combination: topic to topic, endpoint to topic, topic to endpoint, or endpoint to endpoint directly.

The following mapping maps the topic `test/topic/from` to the topic `test/topic/to`. It also maps the data of the endpoint `testEndpoint1` to the topic `test/endpoint/to/topic`.

{% code lineNumbers="true" %}

```yaml
testMapping:
  type: Cybus::Mapping
  properties:
    mappings:
      - subscribe:
          topic: test/topic/from
        publish:
          topic: test/topic/to
      - subscribe:
          endpoint: !ref testEndpoint1
        publish:
          topic: test/endpoint/to/topic
```

{% endcode %}

To map several topics and endpoints to one topic, give `subscribe` a list. The mapping publishes every message it receives on the publish topic:

{% code lineNumbers="true" %}

```yaml
testMapping:
  type: Cybus::Mapping
  properties:
    mappings:
      - subscribe:
          - topic: test/topic/from
            label: test_topic_subs
          - endpoint: !ref testEndpoint1
            label: test_endpoint_1
          - topic: no/labels/are/ok/too
        publish:
          topic: test/endpoint/to/topic
      - subscribe:
          topic: regular/subscribe
        publish:
          topic: output/topic
```

{% endcode %}

The second item shows that one mapping can combine a list subscription with a single subscription.

With the [collect](https://docs.cybus.io/data-flows/rule-engine/data-processing-rules.md#collect) rule, the mapping keeps the last value of each source. It publishes all of them together as one object. The labels are the keys of the object, or the topics where you set no label.

{% code lineNumbers="true" %}

```yaml
testMapping:
  type: Cybus::Mapping
  properties:
    mappings:
      - subscribe:
          - topic: test/topic/from
            label: test_topic_subs
          - endpoint: !ref testEndpoint1
            label: test_endpoint_1
          - topic: no/labels/are/ok/too
        publish:
          topic: test/endpoint/to/topic
        rules:
          - collect: {}
```

{% endcode %}

## Wildcards

Mappings support the MQTT wildcards `+` and `#` in subscribe topics. In most cases, you want to use the value at a wildcard position in the publish topic or in the payload. Both are possible.

To use the value, name the wildcard: write a name right after it, such as `+machine` or `#path`. You can then use the value in the following places:

* In the publish topic, as `$name`.
* In the `transform`, `filter`, and [setContextVars](https://docs.cybus.io/data-flows/rule-engine/data-processing-rules.md#setcontextvars) rules of the [Rule Engine](https://docs.cybus.io/data-flows/rule-engine.md), in the `$context.vars` object.

The following mapping swaps the last two topic levels:

{% code lineNumbers="true" %}

```yaml
wildcardMapping:
  type: Cybus::Mapping
  properties:
    mappings:
      - subscribe:
          topic: source/+a/+b
        publish:
          topic: target/$b/$a
```

{% endcode %}

The following mapping stores a value from the message with the [setContextVars](https://docs.cybus.io/data-flows/rule-engine/data-processing-rules.md#setcontextvars) rule, and uses it in the publish topic. The message must be a JSON object with at least a `value` property, such as `{"value": "abc"}`. It can contain further properties, and this mapping forwards the whole message unchanged.

{% code lineNumbers="true" %}

```yaml
wildcardMapping:
  type: Cybus::Mapping
  properties:
    mappings:
      - subscribe:
          topic: source/+x
        rules:
          - setContextVars:
              vars:
                fromPayload: 'value'
        publish:
          topic: target/$x/$fromPayload
```

{% endcode %}

With the `#` wildcard, a mapping maps a whole topic tree:

{% code lineNumbers="true" %}

```yaml
wildcardMapping:
  type: Cybus::Mapping
  properties:
    mappings:
      - subscribe:
          topic: in/#topic
        publish:
          topic: out/$topic
```

{% endcode %}

This mapping maps messages from `in` to `out` as follows:

* `in/hall2` → `out/hall2`
* `in/hall2/machine1` → `out/hall2/machine1`

{% hint style="info" %}
In MQTT, the named `#` wildcard also matches the parent level. A message on `in` itself therefore goes to `out`.
{% endhint %}

### Duplicated Messages Due to Wildcards

Overlapping subscriptions with wildcards (`#` or `+`) can deliver the same message twice. You notice this in the [Data Explorer](https://docs.cybus.io/monitoring/data-explorer.md) or in an application that receives each message twice. If you see duplicates, check the wildcards in your mappings first.

For example, the following mapping receives every message on `test/from/topic1` twice, because both subscriptions match it:

{% code lineNumbers="true" %}

```yaml
mapping:
  type: Cybus::Mapping
  properties:
    mappings:
      - subscribe:
          - topic: test/from/topic1
          - topic: test/from/#
        publish:
          topic: test/to
```

{% endcode %}

The overlap does not have to be in one mapping. By default, the resources of all services share one MQTT client. A wildcard can therefore also overlap with a subscription in another service, including one that someone adds later. This makes wildcards risky, because a later subscription can overlap unintentionally.

CybusMQ delivers a message once for every matching subscription of a client. The MQTT specification 3.1.1 leaves this case open, so this is not necessarily a bug.

Within one mapping, remove the overlap, for example by subscribing only to `test/from/#` in the example above. To prevent duplicates from overlaps with other mappings or services, give wildcard subscriptions their own MQTT connection:

{% code lineNumbers="true" %}

```yaml
resources:
  mqttConnection:
    type: Cybus::Connection
    properties:
      protocol: Mqtt
      connection:
        host: !ref Cybus::MqttHost
        port: !ref Cybus::MqttPort
        username: !ref Cybus::MqttUser
        password: !ref Cybus::MqttPassword

  mapping:
    type: Cybus::Mapping
    properties:
      mappings:
        - subscribe:
            - topic: services/+serviceId/connections/+connection
              connection: !ref mqttConnection
          publish:
            topic: !sub '${Cybus::ServiceId}/connections/state'

  defaultRole:
    type: Cybus::Role
    properties:
      permissions:
        - resource: services/#
          operation: read
          context: mqtt
```

{% endcode %}

All other mappings keep working as usual without their own connection, and receive no duplicates.

{% hint style="info" %}
You may need to extend the permissions of the default role (`defaultRole`) with the topics that the mapping subscribes to. By default, the role grants access only below the topic prefix of the service.
{% endhint %}

## Routing Messages by Label

In rules, the `label` of the subscription that received a message is available as `$context.label`. With the [setContextVars](https://docs.cybus.io/data-flows/rule-engine/data-processing-rules.md#setcontextvars) rule, you can use the label in the publish topic. This gives sources with technical topics readable names in the published topics:

{% code lineNumbers="true" %}

```yaml
sensorMapping:
  type: Cybus::Mapping
  properties:
    mappings:
      - subscribe:
          - topic: plc1/db10/real4
            label: Temperature
          - topic: plc1/db10/real8
            label: Pressure
        publish:
          topic: machine/values/$sensor
        rules:
          - setContextVars:
              vars:
                sensor: $context.label
```

{% endcode %}

A message on `plc1/db10/real4` is published on `machine/values/Temperature`, and a message on `plc1/db10/real8` on `machine/values/Pressure`.

## Examples

The following mapping publishes the data of an endpoint on a topic below the topic prefix of the service:

{% code lineNumbers="true" %}

```yaml
myMapping:
  type: Cybus::Mapping
  properties:
    mappings:
      - subscribe:
          endpoint: !ref myEndpoint
        publish:
          topic: !sub '${Cybus::MqttRoot}/topic-in-my-container'
```

{% endcode %}

The following mapping forwards messages from the internal broker to an external broker:

{% code lineNumbers="true" %}

```yaml
myMapping:
  type: Cybus::Mapping
  properties:
    mappings:
      - subscribe:
          topic: 'internal-scope/#topic'
        publish:
          topic: external-scope/$topic
          connection: !ref myMqttConnection
```

{% endcode %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.cybus.io/data-flows/service-commissioning-files/resources/cybus-mapping.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
