AWS IoT Greengrass Integration

How to integrate Connectware with AWS IoT Core and Greengrass, including setup, configuration, and data flow between Connectware and AWS IoT services.

This guide walks you step by step through integrating Connectware with AWS IoT services.

You’ll start with an overview of AWS IoT services, including the key differences between AWS IoT Core and IoT Greengrass. If you are already familiar with these concepts, feel free to skip ahead.

Next, you will learn how to connect Connectware with AWS IoT using practical, hands-on examples.

Finally, the guide introduces helpful tools to support your MQTT use case—so you can quickly prototype, review, and monitor your integration setup.

Prerequisites

To follow this guide, you will need the following:

A running instance of Cybus Connectware.
At least one AWS IoT Device set up.
If you are working at the edge: An AWS IoT Greengrass Core instance.
Access to the Admin UI with sufficient user permissions.
Basic knowledge of MQTT and the Connectware services concept (e.g. service commissioning files, connections, and endpoints).
It helps to be familiar with Connecting an MQTT Client to Publish and Subscribe Data.

AWS IoT

AWS IoT is a managed cloud platform that lets connected devices interact easily and securely with cloud applications and other devices. AWS IoT supports a nearly unlimited number of devices and messages, and can process and route those messages to AWS endpoints and to other devices reliably and securely.

For AWS IoT, Amazon offers a software development kit available for most popular programming languages and platforms.

AWS IoT Core

AWS IoT Core is the main component for managing devices, their certificates, shadows, Greengrass resources, and integration rules to subsequent AWS resources like IoT Analytics. It also offers ways to audit and test your IoT use cases.

AWS IoT Greengrass

AWS IoT Greengrass extends AWS Cloud resources to edge devices, so they can act locally on generated data, while still using the cloud for management, analytics, and durable storage. Connected devices can interact with AWS Lambda functions and Docker containers, execute predictions based on machine learning models, keep device data in sync, and communicate with other devices even when not connected to the Internet.

Greengrass offers several advantages:

Reduces latency and enables near real-time responses to local events.
Decreases the cost and amount of data exchanged with the cloud.
Enables offline operation, even with interrupted connectivity to the cloud.
Provides secure communication by authenticating devices and encrypting device data for both local and cloud communications, ensuring data is never exchanged without proven identity.
Simplifies device programming with support for AWS Lambda and Docker containers.

However, there are some drawbacks to consider:

Leveraging these advantages comes with vendor lock-in to AWS resources.
The initial Greengrass setup is comparatively complex and heavyweight.
While costs may decrease compared to using AWS IoT without an edge gateway, Greengrass customers pay per device, for traffic to the cloud, and for any additional AWS resources used within Greengrass.
The learning curve can be steep for users new to AWS resources. In such cases, integrating machines in an IIoT edge solution like Connectware with its endpoints, data mapping, transformation rules, and multiple distribution targets (even different cloud providers) might be easier and more efficient.

Connectware & AWS IoT Integration

Before proceeding, set up AWS IoT Core (and AWS IoT Greengrass for edge deployments) by following the respective instructions:

To integrate AWS IoT with Connectware, the built-in MQTT connector with TLS support is the simplest, most reliable, and secure way to communicate. For a successful AWS IoT integration, Connectware does not require more than that. Additionally, the Connectware MQTT connector has built-in data buffering, so data is stored locally if there is a temporary connection failure with AWS IoT Core or Greengrass Core.

There are two integration scenarios:

Connectware connects directly to the AWS cloud:

Connectware connects to Greengrass Core, which acts as a gateway to the AWS cloud next to the Connectware IIoT Edge Gateway:

Connectware Service for AWS IoT

To connect AWS IoT with Connectware, you will need:

The AWS IoT ATS endpoint address for your Cybus MQTT connection resource
Device certificates generated when creating an AWS IoT Core device
The Amazon root certificate

For details on how to obtain this information, see How to connect AWS IoT and Greengrass.

Example Service Setup

The example below shows how to implement a simple AWS IoT service that forwards any data structure published on a selected MQTT topic.

In the definitions section, you must provide PEM-formatted certificates:

caCert: Amazon’s root certificate (AmazonRootCA1.pem)
clientCert: The device certificate
clientPrivateKey: The device's private key

After this, configure your endpoint and mapping resources as described in Resources.

How the Example Works

The service commissioning file in this example uses a simple transformation rule to forward all messages from:

Connectware topics: ${Cybus::MqttRoot}/test/#topic
To AWS IoT topics: TestDevice/$topic

Make sure your Connectware broker is publishing data on the expected topic.

${Cybus::MqttRoot} is automatically set as the root topic (services/<serviceId>) when the service starts.
#topic/$topic acts as a wildcard mapping: any topic you subscribe to is mirrored when publishing, creating an MQTT bridge with transformation rules applied.

For further details on MQTT topic transformations, see Connecting an MQTT Client to Publish & Subscribe Data.

description: >
  Cybus Connectware to AWS IoT Core
metadata:
  name: AWS IoT Core Test
  version: 1.0.0
  provider: cybus
  homepage: https://www.cybus.io

parameters:
  Aws_IoT_Endpoint_Address:
    type: string
    description: The ATS endpoint to reach your AWS account's AWS IoT Core
    default: <your-aws-account-endpoint-id>-ats.iot.eu-central-1.amazonaws.com

definitions:
  # The root CA certificate as PEM format (AmazonRootCA1.pem)
  caCert: |
    -----BEGIN CERTIFICATE-----
    -----END CERTIFICATE-----
  # The device certificate in PEM CRT format
  clientCert: |
    -----BEGIN CERTIFICATE-----
    -----END CERTIFICATE-----
  # The device private key in PEM format
  clientPrivateKey: |
    -----BEGIN RSA PRIVATE KEY-----
    -----END RSA PRIVATE KEY-----

resources:
  awsMqttConnection:
    type: Cybus::Connection
    properties:
      protocol: Mqtt
      connection:
        host: !ref Aws_IoT_Endpoint_Address
        port: 8883
        scheme: mqtts
        clientId: !sub '${Cybus::ServiceId}-awsMqttConnection'
        mutualAuthentication: true
        caCert: !ref caCert
        clientCert: !ref clientCert
        clientPrivateKey: !ref clientPrivateKey

  sourceTargetMapping:
    type: Cybus::Mapping
    properties:
      mappings:
        - subscribe:
            topic: !sub '${Cybus::MqttRoot}/test/#topic'
          publish:
            connection: !ref awsMqttConnection
            topic: TestDevice/$topic
          rules:
            - transform:
                expression: |
                  (
                    {
                      "deviceId": "TestDevice",
                      "payload": $
                    }
                  )

Changes for AWS IoT Greengrass

To connect to a Greengrass Core, the example service commissioning file needs several changes:

Use the hostname/IP of the Greengrass Core instead of the ATS endpoint.
Use the Greengrass Group Certificate Authority instead of the Amazon Root CA.
Configure the MQTT clientId, which must match the device name for which the certificates are configured.

For more information on connecting AWS IoT and Greengrass, see Cybus Github to learn how to obtain the Greengrass Group Certificate Authority.

parameters:
---
awsGreengrassClientId:
  type: string
  default: TestDeviceEdge
---
resources:
  greengrassTestDeviceEdgeMqttConnection:
    type: Cybus::Connection
    properties:
      protocol: Mqtt
      connection:
        host: !ref Greengrass_Core_Endpoint_Address
        port: 8883
        scheme: mqtts
        clientId: !ref awsGreengrassClientId
        mutualAuthentication: true
        caCert: !ref caCert
        clientCert: !ref clientCert
        clientPrivateKey: !ref clientPrivateKey

Tools

To implement or maintain a new IIoT Edge integration use case as quickly and reliably as possible, there are suitable tools for working with MQTT, Connectware, and AWS IoT.

AWS Command Line Interface (CLI)

The AWS CLI helps with any task on AWS. In this case, at least two tasks are most efficiently completed using the CLI:

Find out the AWS IoT ATS endpoint defined for your AWS account:

aws iot describe-endpoint --endpoint-type iot:Data-ATS

The response contains the AWS account-specific ATS (Amazon Trust Services) endpoint address to be used as the MQTT hostname:

{
  "endpointAddress": "a7t9...1pi-ats.iot.eu-central-1.amazonaws.com"
}

Get the Greengrass Group Certificate Authority certificate if using AWS IoT Greengrass. You then need the following for the caCert setting in the service commissioning file instead of the Amazon Root CA:

aws greengrass list-groups
aws greengrass list-group-certificate-authorities --group-id "4824ea5c-f042-42be-addc-fcbde34587e7"
aws greengrass get-group-certificate-authority --group-id "4824ea5c-f042-42be-addc-fcbde34587e7"
--certificate-authority-id "3e60c373ee3ab10b039ea4a99eaf667746849e3fd87940cb3afd3e1c8de054af"

The JSON output of the latter call has a field PemEncodedCertificate containing the requested information, which needs to be set as the caCert parameter, similar to this:

Node-RED Workbench

The Workbench is essentially a Node-RED application running securely on Connectware as a service. This allows you to install any Node-RED nodes within the service container for quick prototyping. If your use case cannot be achieved with the above service commissioning file, using the Workbench will give you flexibility and additional tools to prototype your solution using Node-RED modules.

For AWS IoT, an MQTT connection is sufficient for most integration scenarios. You may use simple injection nodes and a random value generator to implement and test northbound data flow to AWS IoT:

If you have other requirements, such as working with shadow devices or other AWS resources (e.g., as part of the IoT Greengrass Core deployment), you may want to use additional Node-RED modules supporting AWS.

We do not recommend using Node-RED in production instances. This should only be considered as a rapid-prototyping tool.

AWS IoT SDK

For more complex data management and handling, you may want to use the AWS IoT Device SDK to create a specific connector service for Connectware to meet your requirements.

In most cases, it is sufficient to process device data and apply rules on Connectware as the edge gateway. Similar capabilities can be used on the near-cloud gateway AWS IoT Greengrass or AWS IoT Core itself to manage rules and transformations near the shadow device definitions.

What works best depends on your business strategy and technical constraints.

AWS IoT Core, Analytics, and Other AWS Resources

Now that you are successfully sending data to IoT Core, you can monitor the transmitted data using various AWS resources.

The primary tool is the AWS IoT Core MQTT Client offered on the AWS IoT console. With this tool, you can subscribe to your topic defined in the service commissioning file for outgoing data:

To make use of AWS resources, define AWS IoT rules and actions appropriately, e.g., transmission to IoT Analytics and a DynamoDB table:

The AWS IoT Console helps you quickly implement data transfer to these endpoints.

For example, you may want to enhance the transformation mentioned above to better meet requirements using the fast and easy mapping support of Connectware. Given a requirement to flatten an original data object injected into the internal topic, you can easily transform that data using a Connectware transformation rule with JSONata:

Given a structured object:

"DeviceData": {
    "Temperature": <decimal>,
    "Position": {
        "X": <decimal>,
        "Y": <decimal>,
        "Z": <decimal>
    }
}

For example, the mapping could be enhanced for flattening the elements and adding a timestamp:

sourceTargetMapping:
---
rules:
  - transform:
      expression: |
        (
          {
            "deviceId": "TestDeviceEdge",
            "payload": $
          }
        )
  - transform:
      expression: |
        (
          {
            "deviceId": "TestDeviceEdge",
            "timestamp": $now(),
            "temperature": $.payload.DeviceData.Temperature,
            "position_x": $.payload.DeviceData.Position.X,
            "position_y": $.payload.DeviceData.Position.Y,
            "position_z": $.payload.DeviceData.Position.Z
          }
        )

After implementing the use case, you may see options to simplify things further. Connectware excels at fast integration processes near connected devices, where most data pre-processing can be performed with low latency and lower costs before transmitting to the cloud.

The enhanced transformation rule within Connectware mentioned above may be inspired by a requirement to write the data in a well-structured database:

Or you may want to create a graph with Amazon Quicksight:

In the AWS Cloud, there is a vast array of resources to help you build your IoT application. You should especially consider Lambda functions that can be deployed to your IoT Greengrass Core instance.

Other new tools like AWS IoT SiteWise or AWS IoT Things Graph may also help you build your IoT applications faster with easier management and monitoring.

Summary

This guide provided a brief introduction to AWS IoT and its components available for integration with other services. It explained how to send data from the Connectware MQTT Broker to AWS IoT Core or Greengrass Core with a simple service commissioning file using the built-in MQTT connector of Connectware.

Additionally, the Cybus Workbench service for prototyping more advanced scenarios was presented. The guide concluded with a description of some basic and advanced tools used to monitor data flow between AWS IoT and Connectware.

PreviousAzure IoT Hub Integration NextInfluxDB Service Integration

Last updated 2 days ago

Was this helpful?