Agents

Agents are components of the Connectware that are not running with the central installation but can be deployed and started individually. They solve the requirement for “computing on the edge”, i.e. they are bringing code close to the hardware and allow data-processing at a very early stage of the pipeline.

For agents to work only a single pre-condition must be met: agents must be in a network that allows them to reach the (central) Connectware.

Note

Agents solely use MQTT for communication and hence act as network clients. They do not require any inbound traffic and no special network configuration is needed.

Depending on the use-case the agent itself can be deployed as docker-container or “natively” in form of a Windows or Linux system service.

As “native” deployments are always highly customized, the following documentation will focus on the docker-based agent setup.

Installation

To run the agent, only a single docker container has to be started. The container image to be used is identical to the Connectware’s protocol-mapper image. The image tag (i.e. version) of that container must be the exact same version as the Connectware, otherwise the system may run into unspecified behaviour. In the examples below the image tag (version) is stored as IMAGE_TAG= 1.0.66

Starting the container can be accomplished by executing a docker run command in a terminal or by using docker-compose with a corresponding docker-compose.yml file or by creating a deployment in a Kubernetes cluster, see examples below.

In either case the agent must be configured through environment variables made available to the container.

The following environment variables are required:

  • CYBUS_AGENT_MODE=distributed (switches protocol-mapper to run in agent mode)

  • CYBUS_AGENT_NAME=<agentName> (unique name identifying this agent instance)

  • CYBUS_MQTT_HOST=<connectwareHost> (the IP/hostname of the connectwareHost)

The following environment variables are optional:

  • CYBUS_MQTT_USERNAME=<userName> Username for authentication in the Connectware (default is <agentName>)

  • CYBUS_MQTT_SCHEME=<scheme> (mqtts or mqtt, default: mqtt)

  • CYBUS_MQTT_PORT=<port> (default: 1883)

Additionally, it is recommended to persist the data (credentials and resources) of the agent, guaranteeing unsupervised and automatic availability, even after power cycles. To achieve this persistency, a docker volume mount must be added for the container-internal /data directory. The agent will automatically store the credentials on that volume and re-use them on subsequent start-up.

Note: If multiple agents will be run on the same computer, each agent must use its own volume mount, as otherwise agents would overwrite each other’s data, including username and password credentials.

As an alternative to locally generated credentials, it is also possible to specify a password for this protocol-mapper agent directly by an additional environment variable CYBUS_PROTOCOL_MAPPER_PASSWORD. This might be useful in scenarios where the agent deployment is controlled by the same orchestration tool as the central Connectware instance. In most other cases it is recommended to let the agent generate local credentials, stored in a volume, and authorized to the Connectware through the client registration workflow.

It may also be useful to set the hostname property of the docker container to the local host’s hostname, because this name will be displayed in the Admin-UI overview page. If it is not set, the local docker container ID will be displayed instead (e.g. d172c8c3667b), which might look rather confusing in this overview page. Depending on the operating system, the suitable value for the hostname may be available as an environment variable ${HOSTNAME} or it must be specified manually, as noted in the example below.

Example 1 (docker-compose)

version: '2.0'
services:
  protocol-mapper-agent:
    image: registry.cybus.io/cybus/protocol-mapper:${IMAGE_TAG}
    environment:
      CYBUS_AGENT_MODE: distributed
      CYBUS_AGENT_NAME: myAgent
      CYBUS_MQTT_HOST: 10.11.12.13
    volumes:
      - protocol-mapper-agent:/data
    restart: unless-stopped
    network_mode: host
    hostname: <some-suitable-hostname>
volumes:
  protocol-mapper-agent:

To start, create a new directory (e.g. myAgent). Inside, create a new file called docker-compose.yml with the above content and finally run:

docker-compose up -d

Again it is noted that if multiple agents should run on the same computer, each agent must have defined its own docker volume, otherwise different agents would overwrite each other’s persisted data, including username and password credentials.

Example 2 (docker run)

Using the environment variable IMAGE_TAG= 1.0.66, the following command will run the agent directly (but with the comment lines removed):

# The docker run command
docker run --rm \
    # Attach a volume so the agent can store data across restarts
  -v /tmp/cybus:/data \
    # Set the name of the agent. This must match the agentName used in the commissioning file.
  -e CYBUS_AGENT_NAME=myAgent \
    # The IP address where the Connectware is running.
  -e CYBUS_MQTT_HOST=10.11.12.13 \
    # Make it run as an agent
  -e CYBUS_AGENT_MODE=distributed \
    # Some suitable hostname of the container.
  --hostname=${HOSTNAME} \
    # Run it in host mode networking
  --net=host \
    # The image to use
  registry.cybus.io/cybus/protocol-mapper:${IMAGE_TAG}

This will start an agent named myAgent that persists data to /tmp/cybus on the host OS.

Example 3 (Kubernetes)

By using the following yaml manifest file, you can run an agent on a Kubernetes cluster. You only need to configure the same parameters used by the others types of deployments.

Additionally you will need to create a Kubernetes secret to store the Cybus registry credentials needed to fetch the Connectware docker images.

You can create a secrets like this, using the secret name cybus-docker-registry:

kubectl create secret docker-registry cybus-docker-registry \
        --docker-server=registry.cybus.io \
        --docker-username=john.doe \
        --docker-password=abcd123 \
        --docker-email=john.doe@example.org \
        --namespace=default

The username and password can either be your Cybus Portal username and password, or when using the Connectware license key, the username is license and the password is your Connectware license key.

The manifest file can be created from this example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
apiVersion: apps/v1
kind: Deployment
metadata:
  # The id for this kubernetes object
  name: connectware-agent-deployment
  labels:
    # Attach labels for easy group selection
    app: protocol-mapper-agent
spec:
  replicas: 1
  selector:
    matchLabels:
      app: protocol-mapper-agent
  template:
    metadata:
      labels:
        app: protocol-mapper-agent
    spec:
      # Configure volumes used by the agent to store data across restarts
      volumes:
      - name: agent-data-volume
        emptyDir: {}
      containers:
        - name: protocol-mapper-agent
          # Container image. Watch out: Must use identical version tag to current Connectware version!
          image: registry.cybus.io/cybus/protocol-mapper:1.0.32
          env:
            # Set the name of the agent. This must match the agentName used in the commissioning file.
          - name: CYBUS_AGENT_NAME
            value: "cybus-sample-agent"
            # The IP address or hostname where the Connectware is running
          - name: CYBUS_MQTT_HOST
            value: "10.11.12.13"
            # Make the protocol-mapper runs as an agent
          - name: CYBUS_AGENT_MODE
            value: "distributed"
          ports:
          # Expose the ports used by the agent.
          - containerPort: 443
          # Configure mount points
          volumeMounts:
          - mountPath: /data
            name: agent-data-volume
      # Kubernetes secret used to pull the images from the Cybus registry
      imagePullSecrets:
        - name: cybus-docker-registry

Manifest file for download: agent-deployment.yaml

Registration

Registration describes the process of authorizing an agent to connect to the Connectware.

In the recommended setup where the agent has persistency enabled (see above), this step is needed only once upon initial deployment. To be precise, this registration is needed only in one of these cases:

  • the agent is started the very first time, or

  • the persisted credentials got deleted, or

  • the agent could not login to the Connectware with its cached credentials.

In all other cases the registration process is not needed and the agent will seamlessly start working with the persisted credential from the storage volume. (Again note: If multiple agents should run on the same computer, each agent must use its own volume mount, as otherwise agents would overwrite each other’s credentials.)

In the initial deployment, the registration is performed as follows: First the agent container must be started. Then, the console log output of the container should look as follows:

{"level":30,"time":1590665714502,"msg":"Starting in distributed (edge) mode","pid":6,"hostname":"someHost","v":1}
{"level":30,"time":1590665714568,"msg":"Asking for Connectware registration at 1.2.3.4 with newly generated credentials...","pid":6,"hostname":"someHost","v":1}
{"level":30,"time":1590665714568,"msg":"The registration process must be finished using the web-interface.","pid":6,"hostname":"someHost","v":1}
{"level":30,"time":1590665714568,"msg":"Navigate to \"User Management -> Client Registry\", click unlock and authorize this client.","pid":6,"hostname":"someHost","v":1}
{"level":30,"time":1590665714568,"msg":"Username: cybus-sample-agent, pairing ID (should match in UI): a82123","pid":6,"hostname":"someHost","v":1}

To finalize the registration process, log into the Connectware Admin-UI and authorize the client as described in the chapter Client Registry.

Note

Besides the agent name you can use the Pairing ID to verify that you are granting access to the correct agent instance.

Usage

Agents are able to provide three resource types:

The endpoint resource can also use the rules property to enable direct data-manipulation and pre-processing on the edge using the rule engine (edge processing).

In order to start such resources on the agent, you must set the agentName property at each Connection in your commissioning file. All Endpoint resources using that connection will automatically also run on the the same agent as well without setting any further properties.

Important

The value of the agentName property (as available for Cybus::Connection resources) must match exactly the value of the CYBUS_AGENT_NAME environment variable used during agent startup.

You can find an example commissioning file here.

Monitoring

At any time you can monitor the connectivity status of your agents by navigating to System > Agents in the UI.

../_images/agents.png