Developing Custom Connectors

Configuration, naming, and implementation details for developing custom connectors.

Configuration Conventions

When developing custom connectors, follow the general custom connector conventions.

Directory Naming Conventions

• When defining the Dockerfile, ensure that the destination path for the copied source files ends in a protocol-specific directory name written in lowercase.

Example

# protocol directory must be lowercase!
COPY ./src ./src/protocols/foobar

Schema Naming Conventions

• The schema $id must match the file name (without the .json).

• The schema must start with a capital letter, e.g. Foobar.

Example

• In FoobarConnection.json, the class must be like:

{
  ...
  "$id": "FoobarConnection"
  ...
}

• In FoobarEndpoint.json, the class must be like:

{
  ...
  "$id": "FoobarEndpoint"
  ...
}

Schema Versioning

Schemas support versioning through an additional property called version, which must be a positive integer greater than zero. If this property is omitted, the default value is 1.

Versioning ensures that only the latest version of a schema is considered active and valid. This means that even though all custom connector instances should run the same version of schemas, the latest version will overwrite any previous version in the CW control plane.

Example

• FoobarConnection.json supporting versioning.

{
  ...
  "$id": "FoobarConnection",
  "version": 3
  ...
}

Source Directory Naming Conventions

Follow the case-sensitive naming conventions based on the protocol name.

• File names must start with an uppercase protocol name (e.g., Foobar).

• Connection and endpoint suffixes are mandatory.

• JS files define classes.

• JSON files define schemas.

Example

src/
├── FoobarConnection.js
├── FoobarConnection.json
├── FoobarEndpoint.js
└── FoobarEndpoint.json

Class Naming Conventions

• The class name must match the file name, excluding the .js extension.

• The class name must start with a capital letter, such as Foobar.

Example

• In FoobarConnection.js, the class must be:

class FoobarConnection extends Connection { ... }

• In FoobarEndpoint.js, the class must be:

class FoobarEndpoint extends Endpoint { ... }

Class Constructors

Unless you need a specific constructor, there is no need to specify one because it is inherited from the parent class. However, if you need to implement a custom constructor for the Connection or Endpoint classes, preserve the following format:

Example

• In FoobarConnection.js, the class constructor must be like:

class FoobarConnection extends Connection {
    // other methods and properties

    constructor(params) {
        super(params)
        // other code
    }

    // other methods
}

• In FoobarEndpoint.js, the class constructor must be like:

class FoobarEndpoint extends Endpoint {
    // other methods and properties

    constructor(params, dataPlaneConnectionInstance, parentConnectionInstance) {
        super(params, dataPlaneConnectionInstance, parentConnectionInstance)
        // other code
    }

    // other methods
}

Do Not Overwrite _topic Property

It is not allowed to overwrite the _topic property. The following code is invalid:

// this is invalid, please remove it
this._topic = 'this/is/a/topic'

ES Modules Not Supported

The standard JavaScript environment of custom connectors is based on CommonJS modules. ES modules are not supported.

TypeScript Configuration

TypeScript is not officially supported in development workflows. However, if you want to use TypeScript and compile it to JavaScript, make sure to configure your tsconfig.json file as follows:

{
  "compilerOptions": {
    ....
    "target": "es2022",    /* Set the JavaScript language version for emitted JavaScript and include compatible library declarations. */
    "lib": ["es6"],        /* Specify a set of bundled library declaration files that describe the target runtime environment. */
    "module": "commonjs",  /* Specify what module code is generated. */
    ....
  },
  "include": ["src/**/*.ts", "src/**/*.json", "src/**/*.js", "src/**/*.d.ts", "test/**/*"]
}

Additionally, the compiled JavaScript output must include an exports.default assignment and the exported class itself. This ensures interoperability with our CommonJS-based module system. The compiled .js file should result in:

class FoobarConnection { ... }
exports.default = FoobarConnection;

Custom Connector Development Example

For the following custom connector development example, we provide a sample base repository on GitHub. We recommend that you clone the base repository and follow the custom connector development guide below.

Sample base repository:

https://github.com/cybusio/custom-connector-tcp-example

This base repository has the following structure:

├── Dockerfile
├── examples
│   └── service.yaml
├── package.json
├── package-lock.json
├── README.md
└── src
├── CustomConnection.js
├── CustomConnection.json
├── CustomEndpoint.js
├── CustomEndpoint.json
└── index.js

• service.yaml is the sample service commissioning file.

• CustomConnection.js and CustomEndpoint.js contain the basic connection and data handling methods.

• CustomConnection.json and CustomEndpoint.json are the JSON Schemas that contain the configuration parameters.

Methods for Custom Connectors

To implement a custom connector, you need to implement certain methods. The most common methods are Connection and DataHandle.

Connection Methods

Connection methods establish connections to machines and handle disconnect and reconnect scenarios. Your protocol connection must typically implement the following Connection methods:

• handleConnect

• handleDisconnect

• handleReconnect

DataHandle Methods

To handle the data, implement the following DataHandle methods:

• handleRead

• handleWrite

• handleSubscribe/handleUnsubscribe (optional, otherwise a cron-based polling of read() is provided)

Implementing Connection Methods

To update the process with the correct state, the Connection methods handleConnect, handleDisconnect, and handleReconnect must call the appropriate transition methods of the base class.

The image below shows the different state machines and how they relate to the handle methods:

Example: Your handleConnect method will probably start a network operation that receives a callback method that is called when the operation completes successfully or unsuccessfully. In this callback, you must call this.connectDone or this.connectFailed to trigger the correct state transition so that all other hooks and event listeners follow up correctly.

Connection Methods With and Without a Disconnect Method

You need to consider the following Connection methods:

• Connection methods that require both a connect and a disconnect method.

• Connection methods that require only a connect method, even when reconnecting.

Connect and Disconnect

• To create variables and objects on connection, add them to your handleConnect method.

• To delete variables and objects on disconnection or before a new connection attempt, add them to your handleDisconnect method.

• To delete variables and objects first and then recreate them when reconnecting, add them to your handleReconnect method.

• It is recommended that you add a private method to your class that creates the variables and objects, another private method that deletes them, and call them via the handleConnect, handleDisconnect, and handleReconnect methods.

Connect without Disconnect

• To create variables and objects that are static, add them to the constructor of your derived implementation class.

Reconnecting

Depending on what your client protocol library does, it can be sufficient to listen to all error events from your client, pass them to the connection state machine by calling e.g. this.connectLost, and implement the handleReconnect method with correct deletion and recreation of your connection objects. However, pay attention to what your client library is doing. The reconnect mechanism of the connection state machine will just start calling handleReconnect with an increasing delay (as configured by the connectionStrategy section in the user parameters).

Code Example for Implementing Connection Methods

The following code example uses the host and port parameters to establish a TCP connection in the CustomConnection.js file. First, you specify the host and port, then you define your handleConnect method, which calls the createConnection method and opens a TCP connection.

You can call the different states with the following methods:

• connectFailed

• reconnectFailed

• connectDone

class CustomConnection extends Connection {
    constructor(params) {
        super(params)
        this._host = params.connection.host
        this._port = params.connection.port
        this._client = new Client()
        this._client
            .on('error', err => { console.log('err:', err.message) })
            .on('close', this._onClose.bind(this))
    }

    // Protocol implementation interface method
    async handleConnect() {
        await this._createConnection();
    }

    async _createConnection() {
        try {
            await this._client.connect(this._host, this._port)
        } catch (err) {
            switch (this.getState()) {
                case 'connecting':
                    this.connectFailed(err.message)
                    break
                case 'reconnecting':
                    this.reconnectFailed(err.message)
                    break
            }
            return
        }

        this.connectDone()
    }

Implementing DataHandle Methods

There are two types of DataHandle methods:

• handleRead/handleWrite for single-message data (similar to POST and GET behavior)

• handleSubscribe/handleUnsubscribe for data subscriptions (similar to MQTT behavior)

If you implement data subscription in your client library, you must implement the handleSubscribe/handleUnsubscribe method pair. Again, this is optional because we can achieve similar behavior by calling the handleRead on schedule. Those receive the address object from the configuration file as first parameter, which in turn was defined in the JSON Schema (CustomEndpoint.json).

Your actual protocol addresses are in the properties of this object according to your JSON Schema. The second argument is the JavaScript callback method which must be called on each data reception, i.e. just like an event listener method. In order to properly unregister that method, your handleUnsubscribe method will receive the same callback method argument so that you can properly unregister it if needed.

Code Example for Implementing DataHandling Methods (1)

async handleSubscribe (address, onData) {
this._log.info(address, 'handleSubscribe')
this._client.on(address, onData)
return this._client.subscribe(address)
}

async handleUnsubscribe (address, onData) {
this._log.info(address, 'handleUnsubscribe')
this._client.removeListener(address, onData)
return this._client.unsubscribe(address)
}

For single-message data read and write, you need to implement handleRead/handleWrite, which similarly receives the address as defined in the JSON Schema (CustomEndpoint.json) and the callback that needs to be called with the result.

Code Example for Implementing DataHandling Methods (2)

// Protocol implementation interface method (called for READ and SUBSCRIBE Endpoints)
async handleRead(address, requestData = {}) {
const data = await this._client.read(address.address)
return data
}

// Protocol implementation interface method (called for WRITE Endpoints)
async handleWrite(address, writeData) {
const data = await this._client.write(address.address, writeData)
return data
}

Specifying Custom Configuration Parameters

The set of all available parameters for each class is described using a JSON Schema. The following code example contains JSON Schemas for all properties (connection parameters) that you can configure in all Connectware resources.

The JSON Schema descriptions are stored in files next to the JavaScript files but with a .json suffix. For example, for Connection.js, the properties are described in the file Connection.json file right next to the JavaScript file. For each derived protocol connection, there must also be JSON Schema files for connection and endpoint.

Related Links

• Creating a JSON Schema (json-schema.org)

Code Example for Specifying Custom Configuration Parameters

For the derived connection classes, you must implement a static getCustomSchema method, as shown in the following code example:

const schema = require('./CustomConnection.json');

class CustomConnection extends Connection {
constructor(params) {
....
}

// Protocol implementation interface method
static getCustomSchema() {
return { ...schema };
}

The JSON Schemas of the derived classes will contain the properties that are specific to that particular connection.

We recommend to add descriptions to all properties.

The object with all properties from the JSON Schema will be passed into the constructor in the base classes, which is then available as this._connection member variable (or this._address in the endpoint) during runtime.