Automated Connectware Deployment Using GitLab CI/CD & Ansible

How to automate Connectware deployment using GitLab CI/CD and Ansible, including pipeline setup, playbook configuration, and service orchestration.

In this guide, we will set up a CI/CD pipeline for Connectware using Ansible and GitLab. Our Connectware infrastructure will be composed of:

• 10 Siemens S7 connection services

• 10 Modbus connection services

• 1 data aggregation and transformation service

• 1 SAP Cloud connection service

GitLab CI/CD is a tool for software development that uses the concepts of Continuous Integration (CI) and Continuous Deployment/Delivery (CD), both fundamental parts of modern DevOps.

Continuous Integration is the practice where developers frequently collaborate and merge code changes into a central repository where automated builds, tasks, and tests run. The build process helps ensure that the application is not broken whenever there are new changes in the commits. Continuous Deployment is an extension of CI that deploys the latest changes to a test or production environment.

CI/CD helps deliver applications faster, with higher quality and fewer integration problems. It also removes development pressure on decisions for small changes and promotes faster iterations.

Prerequisites

To follow this guide, you will need the following:

• A running instance of Cybus Connectware.

• A valid Connectware license.

• A Git environment (cloud-hosted or on-premises). In this guide, we will use GitLab.

• Docker and Docker Compose installed on your system.

• Access to the Admin UI with sufficient user permissions.

• Basic knowledge of MQTT, Docker, and Ansible. If you want to refresh your knowledge, we recommend reading ansible.com - Getting started with Ansible.

1. Project Structure

To deploy Connectware using Ansible and the CI/CD pipeline, we will set up a GitLab project with the following structure:

├── README.md
├── .gitlab-ci.yml
├── ansible
│   ├── ansible.cfg
│   ├── hosts.yaml
│   └── playbook.yaml
└── services
    ├── data-aggregation.yml
    ├── modbus.yml
    ├── s7.yml
    └── sap.yml

Under services, we place the service commissioning files. Under ansible, we place our playbook and host configuration (more on this later). Finally, .gitlab-ci.yml is our pipeline configuration file.

2. Pipeline Configuration

To provide GitLab with all the information required to run the CI/CD pipeline, we need to write a configuration file .gitlab-ci.yml.

Here we describe:

• The structure and order of jobs that we want to be executed (e.g., tasks, tests)

• The decisions the pipeline should make when specific conditions are encountered

You need to ensure that you have runners available. If you're using gitlab.com, you can skip this step as gitlab.com provides shared runners for you.

For self-hosted runners, see Registering runners | GitLab.

variables:
  CONNECTWARE_VERSION: '1.1.3'

stages:
  - deploy

deploy-connectware:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "web"' # Trigger from Web UI
      when: always # Always run
    - when: never # Else never run
  image:
    name: registry.cybus.io/cybus/cybus-connectware-ansible:latest
    entrypoint: ['']
  stage: deploy
  script:
    - 'chmod o-w ansible'
    - 'chmod 600 $SSH_KEY'
    - 'cd ansible && ansible-playbook playbook.yaml'

The file uses the YAML text format and follows the GitLab keyword reference. Inside .gitlab-ci.yml, we define variables, stages (i.e., tasks), and the pipeline's trigger.

• Variables: Connectware version to install

• Stages: deploy is the only stage in our pipeline. In this stage, we run the deploy-connectware job

• Jobs: deploy-connectware consists of rules, an image, and a script

• Rules: When the pipeline is triggered

• Script: Makes Ansible executable and executes the ansible/playbook.yml (which deploys Connectware and its services, see "Using Ansible for Connectware orchestration")

• Image: Latest Ansible image from Cybus' registry

• Pipeline trigger: In this case, through the GitLab Web UI (Run Pipeline button). This can also be configured to be triggered by a new commit or merge event

3. Ansible Playbook and Host Configuration

To deploy Connectware and its services, we need to write a playbook with the same structure as seen in Orchestrating Connectware with Ansible. One key difference might be the host. For on-premises, self-hosted Git environments, the hostname should be the IP address where you want Connectware to run. For example, localhost.

However, if you are using a cloud solution such as gitlab.com, the host needs to point to a defined resource, such as a cloud environment. In our example, we will use an AWS EC2 instance to deploy Connectware.

You can choose between two deployment approaches depending on your CI/CD pipeline requirements:

• Sequential Deployment: Services deploy one after another. Simpler to implement and debug.

• Asynchronous Deployment: Services deploy in parallel. Significantly faster for large-scale rollouts.

Sequential Deployment

For straightforward deployments where services are installed one at a time, use the standard cybus.connectware.service module.

• Create an ansible/playbook.yml with the following content:

- name: Connectware Infrastructure Playbook
  hosts: AWS
  vars:
    connectwareVersion: "{{ lookup('env','CONNECTWARE_VERSION') }}"
    adminPassword: "{{ lookup('env','CONNECTWARE_ADMIN_PASSWORD') }}"
    license: "{{ lookup('env','CYBUS_CONNECTWARE_LICENSE') }}"

  tasks:
    - name: 'Deploy Connectware'
      become: yes
      cybus.connectware.instance:
        version: '{{ connectwareVersion }}'
        license: '{{ license }}'
        admin_password: '{{ adminPassword }}'

    - name: Install Services (S7)
      loop:
        - host: dev01
        - host: dev02
        - host: dev03
        - host: dev04
        - host: dev05
        - host: dev06
        - host: dev07
        - host: dev08
        - host: dev09
        - host: dev10
      loop_control:
        loop_var: s7
      cybus.connectware.service:
        admin_password: '{{ adminPassword }}'
        id: '{{ s7.host }}_S7'
        commissioning_file: ../services/s7.yml
        parameters:
          ipAddress: '{{ s7.host }}'

    - name: Install Services (Modbus)
      loop:
        - host: dev11
        - host: dev12
        - host: dev13
        - host: dev14
        - host: dev15
        - host: dev16
        - host: dev17
        - host: dev18
        - host: dev19
        - host: dev20
      loop_control:
        loop_var: modbus
      cybus.connectware.service:
        admin_password: '{{ adminPassword }}'
        id: '{{ modbus.host }}_Modbus'
        commissioning_file: ../services/modbus.yml
        parameters:
          ipAddress: '{{ modbus.host }}'

    - name: Install SAP Connector Service
      cybus.connectware.service:
        admin_password: '{{ adminPassword }}'
        id: 'demo_sapconnector'
        commissioning_file: ../services/sap.yml

    - name: Install Data Mapping Service
      cybus.connectware.service:
        admin_password: '{{ adminPassword }}'
        id: 'demo_datamapping'
        commissioning_file: ../services/data-aggregation.yml

How It Works:

• The playbook uses loops to deploy multiple similar services (S7 and Modbus connections).

• Each service deployment waits for completion before starting the next.

• The host points to AWS, defined in our hosts.yaml file (see below).

Asynchronous Deployment

For CI/CD pipelines where speed is critical, use the cybus.connectware.service_async module to deploy multiple services in parallel. This approach is especially beneficial when deploying many services (in this example, 22 services).

• Create an optimized ansible/playbook.yml with the following content:

- name: Connectware Infrastructure Playbook (Optimized)
  hosts: AWS
  vars:
    connectwareVersion: "{{ lookup('env','CONNECTWARE_VERSION') }}"
    adminPassword: "{{ lookup('env','CONNECTWARE_ADMIN_PASSWORD') }}"
    license: "{{ lookup('env','CYBUS_CONNECTWARE_LICENSE') }}"

  tasks:
    - name: 'Deploy Connectware'
      become: yes
      cybus.connectware.instance:
        version: '{{ connectwareVersion }}'
        license: '{{ license }}'
        admin_password: '{{ adminPassword }}'

    # Deploy S7 services in parallel
    - name: Deploy S7 Services
      loop:
        - host: dev01
        - host: dev02
        - host: dev03
        - host: dev04
        - host: dev05
        - host: dev06
        - host: dev07
        - host: dev08
        - host: dev09
        - host: dev10
      loop_control:
        loop_var: s7
      cybus.connectware.service_async:
        admin_password: '{{ adminPassword }}'
        id: '{{ s7.host }}_S7'
        commissioning_file_b64: "{{ lookup('file', '../services/s7.yml') | b64encode }}"
        parameters:
          ipAddress: '{{ s7.host }}'
      async: 600
      poll: 0
      register: s7_jobs

    # Deploy Modbus services in parallel
    - name: Deploy Modbus Services
      loop:
        - host: dev11
        - host: dev12
        - host: dev13
        - host: dev14
        - host: dev15
        - host: dev16
        - host: dev17
        - host: dev18
        - host: dev19
        - host: dev20
      loop_control:
        loop_var: modbus
      cybus.connectware.service_async:
        admin_password: '{{ adminPassword }}'
        id: '{{ modbus.host }}_Modbus'
        commissioning_file_b64: "{{ lookup('file', '../services/modbus.yml') | b64encode }}"
        parameters:
          ipAddress: '{{ modbus.host }}'
      async: 600
      poll: 0
      register: modbus_jobs

    # Deploy remaining services in parallel
    - name: Deploy SAP and Data Mapping Services
      loop:
        - { id: 'demo_sapconnector', file: '../services/sap.yml' }
        - { id: 'demo_datamapping', file: '../services/data-aggregation.yml' }
      cybus.connectware.service_async:
        admin_password: '{{ adminPassword }}'
        id: '{{ item.id }}'
        commissioning_file_b64: "{{ lookup('file', item.file) | b64encode }}"
      async: 600
      poll: 0
      register: other_jobs

    # Wait for all services to complete
    - name: Wait for S7 services
      async_status:
        jid: '{{ item.ansible_job_id }}'
      loop: '{{ s7_jobs.results }}'
      register: s7_results
      until: s7_results.finished
      retries: 60
      delay: 10

    - name: Wait for Modbus services
      async_status:
        jid: '{{ item.ansible_job_id }}'
      loop: '{{ modbus_jobs.results }}'
      register: modbus_results
      until: modbus_results.finished
      retries: 60
      delay: 10

    - name: Wait for other services
      async_status:
        jid: '{{ item.ansible_job_id }}'
      loop: '{{ other_jobs.results }}'
      register: other_results
      until: other_results.finished
      retries: 60
      delay: 10

Key Differences:

• Uses cybus.connectware.service_async instead of cybus.connectware.service.

• Requires commissioning_file_b64 (base64-encoded file content) instead of commissioning_file.

• Includes async: 600 and poll: 0 to start tasks without waiting.

• Uses async_status tasks to wait for all deployments to complete.

Benefits:

• Faster pipeline execution: All services within a group deploy simultaneously.

• Better resource utilization: Parallel deployments make better use of Connectware's capabilities.

• Time savings: Can reduce deployment time from several minutes to under a minute.

For CI/CD pipelines with many services, the asynchronous approach is recommended to minimize pipeline runtime and improve developer productivity.

Host Configuration

In both playbook approaches, we are using environment variables such as:

"{{ lookup('env','CYBUS_CONNECTWARE_LICENSE') }}"

• To set up the values of these variables, go to GitLab and select Settings > CI/CD > Variables.

It is best practice to place sensitive information as environment variables and NOT in the codebase.

• Create another file called hosts.yaml with the following content:

---
all:
  hosts:
    AWS:
      ansible_connection: ssh
      ansible_host: demo-devops.cybus.io
      ansible_port: 22
      ansible_user: ubuntu
      ansible_ssh_private_key_file: "{{ lookup('env','SSH_KEY') }}"

Result: This will help Ansible deploy the Connectware infrastructure to our cloud endpoint (demo-devops.cybus.io).

4. Running the Pipeline

Make sure your project has:

• Ansible playbook and host configuration under /ansible

• Connectware service commissioning files under /services

• CI/CD configuration file .gitlab-ci.yml

• Necessary environment variables

• Valid deployment endpoint (host)

• Available runners (either cloud-based or self-hosted)

Then on the left panel, select CI/CD > Pipelines > Run Pipeline.

Result: Your pipeline should be running soon.

After a few minutes, the build will finish and GitLab will update the status to Passed. You should be able to open your host endpoint and log in to Connectware.

In case the pipeline fails, the status will be updated to Failed.

• To see more information, click the status button. This will display the logs from the execution.