Duplicate Message Delivery

Configure duplicate message delivery behavior in CybusMQ.

When a client subscribes to multiple topic filters that can match the same published message (e.g., using wildcards), CybusMQ can be configured to either:

• Send a single message (deduplication enabled, default behavior)

• Send multiple messages (deduplication disabled)

MQTT Specification Context

The MQTT specification does not clearly define whether the same message should be sent multiple times when multiple subscriptions match a published topic. VerneMQ chose to send duplicate messages for each matching subscription. CybusMQ makes this behavior configurable to suit different application requirements.

Use Cases

• Enabled deduplication: Ensures that each client processes a message only once, regardless of subscription patterns.

• Disabled deduplication: When your application specifically requires receiving multiple copies of messages for overlapping subscriptions, such as certain protocol-mapping scenarios.

Example

A client subscribes to +/bar and foo/bar:

• The wildcard subscription +/bar matches any topic with a single-level prefix followed by /bar.

• The specific subscription foo/bar matches only the exact topic foo/bar.

When a message is published to foo/bar, both subscriptions match this topic.

• With deduplication enabled:

Client receives: 1 message
{
  "topic": "foo/bar",
  "payload": "Hello World"
}

• With deduplication disabled:

Client receives: 2 messages
{
  "topic": "foo/bar",  // From first subscription match
  "payload": "Hello World"
}
{
  "topic": "foo/bar",  // From second subscription match
  "payload": "Hello World"
}

Enabling/Disabling Deduplication

By default, CybusMQ deduplicates messages for overlapping subscriptions (enabled). This means a client will receive only one copy of a message even if multiple subscriptions match the published topic.

To allow clients to receive multiple copies of the same message when they have overlapping subscriptions, you can disable deduplication.

Kubernetes

Enabling/Disabling Deduplication on Kubernetes

Prerequisite

• Your Helm chart is version 1.5.0 or later.

Procedure

1. Open your values.yaml file.

2. Set the DOCKER_VERNEMQ_DEDUPLICATE_OVERLAPPING_SUBS to on (enable) or off (disable).

global:
  broker:
    env:
      - name: DOCKER_VERNEMQ_DEDUPLICATE_OVERLAPPING_SUBS
        value: on # Set to "off" to disable deduplication

Docker

Enabling/Disabling Deduplication on Docker

1. Create or modify a docker-compose.override.yml file.

2. In the docker-compose.override.yml file, set DOCKER_VERNEMQ_DEDUPLICATE_OVERLAPPING_SUBS to on (enable) or off (disable).

services:
  broker:
    environment:
      DOCKER_VERNEMQ_DEDUPLICATE_OVERLAPPING_SUBS: on # Set to "off" to disable deduplication