LogoLogo
Contact Uscybus.io
Connectware 1.10.0
Connectware 1.10.0
  • Getting Started
    • Introduction
    • Installing Connectware
      • System Requirements
      • Acquiring your License Key
      • Installing Connectware on Docker
      • Installing Connectware on Kubernetes
    • Connectware Admin UI
    • Basic Components of Connectware
    • Connecting your First Machine
      • Your First Service Commissioning File
  • Documentation
    • Services
      • Service Overview
      • Service Resources View
        • Service Links View
        • Servers View
        • Containers View
        • Volumes View
        • Connections View
        • Endpoints View
        • Mappings View
      • Service Details View
      • Service Commissioning Files
        • Structure of Service Commissioning Files
          • Version
          • Description
          • Metadata
          • Parameters
          • Definitions
          • Resources
            • Cybus::Connection
            • Cybus::Endpoint
            • Cybus:Mapping
            • Cybus::Container
              • Docker problem with network changes
            • Cybus::Link
            • Cybus::IngressRoute
            • Cybus::User
            • Cybus::Role
            • Cybus::Volume
            • Cybus::File
            • Cybus::Server
            • Cybus::Node
      • Setting Up and Configuring Services
        • Installing Services
        • Enabling Services
        • Updating Services
        • Disabling Services
        • Deleting Services
      • FlowSync
        • Example 1 - Node with Transaction Mode (HTTP)
        • Example 2 - Node Responds (HTTP)
        • Example 3 - Node with Error (HTTP)
        • Example 4 - Node with Timeout Error Code and Error Message (HTTP)
        • Example 5 - Full Transactional Data Flow (HTTP)
        • Example 6 - Full Transactional Data Flow (OPC UA)
      • ServiceID
      • Inter-Service Referencing
      • Deviation
      • Service Logs
        • Logs of Individual Services
        • Logs of All Services
      • Rule Engine
        • Data Processing Rules
        • Rule Sandbox
      • Shared Subscriptions
        • Setting Up Shared Subscriptions
    • User Management
      • Users and Roles View
      • Users
      • Roles
      • Permissions
      • Password Policy Rules
      • Default Admin User
      • MQTT Users
      • Adding a MQTT Publish Prefix for Users
      • Multi-Factor Authentication
      • Single Sign-On (SS0)
        • Single Sign-On with Microsoft Entra ID
        • Single Sign-On with LDAP
      • Long lived JSON Web Tokens
      • Access Permissions for Admin-UI
        • UI Access
        • Minimum Access Role Pages
    • Client Registry
      • Implicit Flow
      • Explicit Flow
      • Granting Access
    • Certificates
    • Monitoring
      • Data Explorer
      • Live Data
    • Workbench
      • Flows in Git Repositories
    • System Status
      • Info
      • Metrics
      • Status
      • Retrieving More System Information
      • System Health
    • Backup and Restore
      • Volumes
      • User Database
    • Environment Variables
    • Agents
      • Agents View
      • Installing Agents
        • Installing Agents via Docker
        • Installing Agents via Docker Compose
        • Installing Agents via Kubernetes
        • Using Mutual TLS for Agents
      • Registering Agents in Connectware
      • Using Agents
      • Monitoring Agents
      • Troubleshooting Agents
    • Industry Protocol Details
      • ADS
        • ADS Connection Properties
        • ADS Endpoint Properties
      • BACnet
        • BACnet Connection Properties
        • BACnet Endpoint Properties
      • EtherNet/IP
        • EtherNet/Ip Connection Properties
        • EtherNet/Ip Endpoint Properties
      • FOCAS
        • FOCAS Connection Properties
        • FOCAS Endpoint Properties
      • Generic VRPC
        • Generic VRPC Connection Properties
        • Generic VRPC Endpoint Properties
      • Hottinger Baldwin Messtechnik (HBM)
        • HBM Connection Properties
        • HBM Endpoint Properties
      • Heidenhain DNC
        • Heidenhain DNC Connection Properties
        • Heidenhain DNC Endpoint Properties
      • HTTP/REST
        • HTTP/REST Connection Properties
        • HTTP/REST Endpoint Properties
      • HTTP Server/Node
        • HTTP Server Properties
        • HTTP Node Properties
      • InfluxDB
        • InfluxDB Connection Properties
        • InfluxDB Endpoint Properties
      • Kafka
        • Kafka Connection Properties
        • Kafka Endpoint Properties
      • Modbus/TCP
        • Modbus/TCP Connection Properties
        • Modbus/TCP Endpoint Properties
      • MQTT
        • MQTT Connection Properties
        • MQTT Endpoint Properties
      • MSSQL
        • Mssql Connection Properties
        • Mssql Endpoint Properties
      • OPC DA
        • OPC DA Connection Properties
        • OPC DA Endpoint Properties
      • OPC UA
        • OPC UA Client
          • OPC UA Client Connection Properties
          • OPC UA Client Endpoint Properties
        • OPC UA Server
          • OPC UA Server Properties
          • OPC UA Node Properties
        • OPC UA Object Types
        • OPC UA Server References
          • OPC UA Reference Node
          • OPC UA Object Node
      • Siemens SIMATIC S7
        • Siemens S7 Connection Properties
        • Siemens S7 Endpoint Properties
      • Shdr
        • Shdr Connection Properties
        • Shdr Endpoint Properties
      • SINUMERIK
        • SINUMERIK Connection Properties
        • SINUMERIK Endpoint Properties
      • SOPAS
        • SOPAS Connection Properties
        • SOPAS Endpoint Properties
      • SQL
        • SQL Connection Properties
        • SQL Endpoint Properties
      • Werma WIN Ethernet
        • Werma WIN Ethernet Connection Properties
        • Werma WIN Ethernet Endpoint Properties
      • Systemstate
        • Systemstate Connection Properties
        • Systemstate Endpoint Properties
    • Connectware Licensing
    • API Reference
      • User Management (API)
      • Client Registry (API)
      • Services (API)
      • Resources (API)
      • System Status (API)
      • Industry Protocol Details (API)
    • Changelog
      • General changes from 0.x to 1.0
        • Upgrading from 0.x to 1.0
    • Upgrade Guide
      • Upgrading from 1.x to 1.7.0
      • Upgrading from 1.x to 1.5.0
Powered by GitBook
LogoLogo

Cybus

  • Terms and Condition
  • Imprint
  • Data Privacy

© Copyright 2025, Cybus GmbH

On this page
  • Message assembly
  • Commissioning File Specifics
  • Service Commissioning File Example

Was this helpful?

  1. Documentation
  2. Industry Protocol Details

Kafka

Apache Kafka is a software framework for stream-processing of large amounts of data. It is an open-source software platform developed by the Apache Software Foundation, written in Scala and Java. The project aims to provide a unified, high-throughput, low-latency platform for handling real-time data feeds.

This protocol implementation allows to send the data from the Connectware to an Apache Kafka broker for message processing or to subscribe to a particular topic to receive data.

Currently, the protocol supports authentification by means of SASL and compression of produced messages by using GZIP.

Message assembly

The endpoints in this protocol are available for the write and subscribe operations only. To transmit messages to the Kafka broker, a JSON message with the payload as value needs to be sent to the respective endpoint. The message may look as follows when producing messages to the broker:

{
    'id': 123321,
    'topic': 'some_test_topic',
    'acks': 1,
    'timeout': 3000,
    'compression': 'None',
    'value':
        [
            { 'key': 'sample-key 1', 'value': 'sample message #1', 'partition': 0, 'headers': { 'foo': 'bar' }, 'timestamp': 1633425301000 },
            { 'key': 'sample-key 2', 'value': 'sample message #2', 'partition': 1, 'headers': { 'foo': 'bar' }, 'timestamp': 1633425301001 },
        ],
}

Which upon successful delivery, the following message will be generated on the /res topic:

{
    'id': 123321,
    'timestamp': 1633425302878,
    'result':
        {
            'value':
                [
                    { 'topicName': 'some_test_topic', 'partition': 0, 'errorCode': 0, 'baseOffset': '0', 'logAppendTime': '-1', 'logStartOffset': '0' },
                    { 'topicName': 'some_test_topic', 'partition': 1, 'errorCode': 0, 'baseOffset': '0', 'logAppendTime': '-1', 'logStartOffset': '0' },
                ],
        },
}

And when delivery is unsuccessful, you will get the following message explaining what happened:

{ 'id': 123321, 'timestamp': 1633425495764, 'error': { 'code': -1, 'message': 'Connection error: connect ECONNREFUSED 192.168.0.1:9092' } }

Upon subscription you will get messages like this:

{
    'timestamp': 1649046618574,
    'value': { 'topic': 'some_test_topic', 'partition': 0, 'key': 'totally optional msg key', 'headers': {}, 'message': 'test 123 test msg body' },
}

Commissioning File Specifics

The Endpoints for the Kafka connections may only be of the type write or subscribe.

Special care must be taken when configuring the connection properties since connections can only take all write endpoints or all subscribe endpoints. But not both at the same time.

To choose the type of connection you are planning to use, you will need to configure the property clientType on the connection resource.

Further configuration options are listed below:

Service Commissioning File Example

---
# ----------------------------------------------------------------------------#
# Commissioning File
# ----------------------------------------------------------------------------#
# Copyright: Cybus GmbH (2020)
# Contact: support@cybus.io
# ----------------------------------------------------------------------------#
# Source Interface Definition - Kafka broker
# ----------------------------------------------------------------------------#

description: >
    Sample commissioning file for communicating with Kafka broker (producer)

metadata:
    name: Apache Kafka Connectivity
    icon: https://www.cybus.io/wp-content/uploads/2017/10/for-whom1.svg
    provider: cybus
    homepage: https://www.cybus.io
    version: 0.0.1

parameters:
    param_brokers:
        description: Hosts or IP addresses of the Apache Kafka broker
        type: string
        default: 192.168.0.1:9092

resources:
    kafkaConnection:
        type: Cybus::Connection
        properties:
            protocol: Kafka
            connection:
                brokers: [!ref param_brokers]

    kafkaWriteExample:
        type: Cybus::Endpoint
        properties:
            protocol: Kafka
            connection: !ref kafkaConnection
            write:
                topic: some_test_topic
kafka-example-consumer.yml
---
# ----------------------------------------------------------------------------#
# Commissioning File
# ----------------------------------------------------------------------------#
# Copyright: Cybus GmbH (2020)
# Contact: support@cybus.io
# ----------------------------------------------------------------------------#
# Source Interface Definition - Kafka broker
# ----------------------------------------------------------------------------#

description: >
    Sample commissioning file for communicating with Kafka broker (consumer)

metadata:
    name: Apache Kafka Connectivity
    icon: https://www.cybus.io/wp-content/uploads/2017/10/for-whom1.svg
    provider: cybus
    homepage: https://www.cybus.io
    version: 0.0.1

parameters:
    param_brokers:
        description: Hosts or IP addresses of the Apache Kafka broker
        type: string
        default: 192.168.0.1:9092

resources:
    kafkaConnection:
        type: Cybus::Connection
        properties:
            protocol: Kafka
            connection:
                clientType: 'consumer'
                brokers: [!ref param_brokers]

    kafkaSubscribeExample:
        type: Cybus::Endpoint
        properties:
            protocol: Kafka
            connection: !ref kafkaConnection
            subscribe:
                fromBeginning: false
                topic: some_test_topic
PreviousInfluxDB Endpoint PropertiesNextKafka Connection Properties

Last updated 4 months ago

Was this helpful?

Connection Properties
Endpoint Properties
1KB
kafka-example-producer.yml
1KB
kafka-example-consumer.yml