# Using Mutual Transport Layer Security (mTLS) for agents with the connectware-agent Helm chart By default, agents use a password for authentication. As an alternative to password-based authentication you can use mutual TLS (mTLS) as the authentication mechanism for agents. mTLS is an X.509 certificate-based authentication and provides better performance compared to password-based authentication. We recommend using mTLS when handling a large number of agents. **See also** * [Client Certificates with mTLS](https://docs.cybus.io/2-1-0/documentation/security/tls-certificates/client-certificates) ## Prerequisites * mTLS is enabled for Connectware. * You are familiar with Public Key Infrastructure (PKI) and openssl. ## Procedure To configure agents for mTLS, do the following: 1. [Extract Certificate Authority key pairs](#extracting-certificate-authority-key-pairs). 2. [Sign certificate key pairs for your agents](#signing-certificate-key-pairs-for-your-agents). 3. [Configure agents for key pairs](#key-pairs-for-agents). 4. [Configure your agent to use the Certificate Authority](#certificate-authority-for-agents). 5. [Enable mTLS for the agent](#enabling-mtls-for-the-agent). ## Extracting Certificate Authority Key Pairs To use mTLS authentication, you must extract the Certificate Authority (CA) that Connectware uses to sign certificates that are created for your agents. You can extract the certificate from Connectware or replace it with a CA certificate that you already have. In both cases, you must extract the truststore that Connectware uses, as well as the affiliated private key. The steps in this section are executed in your Connectware installation, not your connectware-agent installation. {% hint style="info" %} For production setups, we recommend that you replace the Public Key Infrastructure (PKI) that is generated during the Connectware installation with a PKI that is managed and approved by your company. {% endhint %} ### Extracting the CA Key Pair of Connectware To extract the existing CA key pair, use `kubectl cp` to copy the certificate from the running auth-server pod via the following commands. Make sure to specify the Kubernetes namespace that contains the Connectware installation. {% code lineNumbers="true" %} ```bash pod=$(kubectl -n ${NAMESPACE} get pod -o name -lapp.kubernetes.io/name=auth-server | head -1 | sed 's/pod\///g'); kubectl -n ${NAMESPACE} cp -c auth-server $pod:/connectware_certs/cybus_ca.key cybus_ca.key kubectl -n ${NAMESPACE} cp -c auth-server $pod:/connectware_certs/cybus_ca.crt cybus_ca.crt ``` {% endcode %} **Result** The files `cybus_ca.crt` and `cybus_ca.key` are created in your current directory. ### Using a Custom Certificate Authority (Optional) In production environments, we recommend that you use a Certificate Authority (CA) certificate that your organization manages and has authorized. You can add your CA's certificate or a valid intermediate CA certificate via the `certs` volume. For more information, see [Adding Custom CA Certificates via the Certs Volume](https://docs.cybus.io/2-1-0/security/tls-certificates/ca-certificates#adding-custom-ca-certificates-via-the-certs-volume). ## Signing Certificate Key Pairs for Your Agents Every agent needs a certificate key pair that is signed by the Certificate Authority (CA) that you want to use. We will assume that the CA certificate files are named `cybus_ca.crt` and `cybus_ca.key` and are in your current directory. {% hint style="info" %} If you have already signed certificates for your agents, skip this task and continue with configuring the agent to use your key pair. {% endhint %} The exact parameters for the key pair are subject to your preferences and security requirements. The commands used here are meant as an example. 1. To generate a new key for your agent, enter the following command (Do not set a password for the key): {% code lineNumbers="true" %} ```bash openssl genrsa -out tls.key 4096 ``` {% endcode %} 2. To generate a Certificate Signing Request (CSR), enter the following command: {% code lineNumbers="true" %} ```bash openssl req -new -key tls.key -out tls.csr ``` {% endcode %} 3. Fill out the details for the certificate. Make sure to set `Common Name (e.g. server FQDN or YOUR name)` to the exact name of the agent that you are generating a certificate for. 4. To sign the CSR, use the following command: {% code lineNumbers="true" %} ```bash openssl x509 -req -in tls.csr -CA cybus_ca.crt -CAkey cybus_ca.key -CAcreateserial -out tls.crt -days 365 -sha256 ``` {% endcode %} If you are using other file names than the ones that we are using in this documentation, make sure to use them in the command. **Result** The certificate is created. The certificate is valid for one year. Make sure to create new certificates before the old certificates expire to avoid impact on the operation of the corresponding agents. ## Key Pairs for Agents You can configure key pairs for agents in your `values.yaml` file or you can create a Kubernetes Secret before you deploy your agent. Each method has its advantages. However, the most important difference is that a key is considered private data, like a password. If you do not want to store this information in your unencrypted `values.yaml` file, we recommend that you use the Kubernetes Secret. ### Configuring the Agent to Use Your Key Pair You can configure the key pair in the `mTLS.keyPair` section inside the agents entry in the `protocolMapperAgents` context of your `values.yaml` file. Alternatively, you can create a Kubernetes Secret before you deploy your agent. {% hint style="warning" %} A key is considered private data, like a password. If you do not want to store this information in your unencrypted `values.yaml` file, you can create a Kubernetes Secret before you deploy your agent instead. {% endhint %} #### Configuring Key Pair via Helm Values To add the key pair to your Helm values, add the respective files as literal block scalars to these Helm values inside the agents entry in `protocolMapperAgents` context of your `values.yaml` file: | Value | Content | | ------------------- | ------------------------------------------------------------------------------- | | `mTLS.keyPair.cert` | The certificate you generated for the agent (tls.crt) as a literal block scalar | | `mTLS.keyPair.key` | The key you generated for the agent (tls.key) as a literal block scalar | Follow the YAML indentation rules. Indent the certificate and key by two spaces relative to `cert`/`key`, see the example below. **Example** {% code lineNumbers="true" %} ```yaml protocolMapperAgents: - name: bender-robots mTLS: keyPair: cert: | -----BEGIN CERTIFICATE----- IIEgTCCAmkCFCN+Wi9RpeajIunZnxdIhvdZep6ZMA0GCSqGSIb3DQEBCwUAMIGN [skipped for brevity - include whole certificate] sD9hY3o= -----END CERTIFICATE----- key: | -----BEGIN PRIVATE KEY----- IIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCg+mC1iGmz+qCO [skipped for brevity - include whole private key] nJn5oNH9lcodhXcuOYVg3kQ= -----END PRIVATE KEY----- ``` {% endcode %} #### Configuring Key Pair via Kubernetes Secret **Creating the Kubernetes Secret** If you want to manually manage the certificate and key as a Kubernetes Secret you will need to create it before configuring your agent. You can use any process you like, as long as the result is a Kubernetes Secret that: * Is of the type `tls`. * Contains a file called `tls.crt` containing the signed certificate for the agent. * Contains a file called `tls.key` containing the matching key for the agent. * Is in the same namespace as your `connectware-agent` installation. We will demonstrate how to create this secret through kubectl. Ensure that your agent certificate is stored in a file named `tls.crt` and your key is stored in a file named `tls.key`. Define a name for your secret. You will need this name later to configure the agent, so keep it at hand. We recommend choosing a name following the scheme `---mtls`. Some characters, most prominently the “.” character will be replaced by “-” characters in the agent name. For example, if your agent is named bender-robots and you named your installation `connectware-agent` as used in our docs, the name should be `connectware-agent-bender-robots-mtls`. If you follow this naming convention you will need to include less configuration in your `values.yaml`. If you are unsure how to name the secret, you can first deploy your agent without the mTLS configuration, and check the name of its StatefulSet using `kubectl get sts`. The secret should have the same name with the suffix `-mtls`. **Example** {% code lineNumbers="true" %} ```bash kubectl -n ${NAMESPACE} create secret tls --key="./tls.key" --cert="./tls.crt" ``` {% endcode %} **Configuring the Agent Through Helm Values** If you followed the naming convention of “---mtls” you don’t need to configure the name of the secret, as this name will be assumed. If you have chosen a different name you need to specify it in the Helm value `mTLS.keyPair.existingSecret` inside the agents entry in `protocolMapperAgents` context of your `values.yaml` file. **Example** {% code lineNumbers="true" %} ```yaml protocolMapperAgents: - name: bender-robots mTLS: keyPair: existingSecret: ``` {% endcode %} ### Certificate Authority for Agents There are two ways you can configure your agent to use the Certificate Authority (CA): * Directly through including it in your `values.yaml` file * By creating a Kubernetes ConfigMap before deploying your agent #### Configuring Certificate Authority for Agents via Helm Values To add the CA certificate to your Helm values, add the file as literal block scalar to the Helm value `tls.ca.certChain` inside the agents entry in `protocolMapperAgents` section of your `values.yaml` file. Pay attention to the indentation of your CA certificate. It needs to be indented once relative to `certChain` and keep this indentation (see example below). If you configure more than one agent, it is recommended to provide the CA certificate through `protocolMapperAgentDefaults` instead of `protocolMapperAgents`, as it should be the same for all agents. **Example** {% code lineNumbers="true" %} ```yaml protocolMapperAgents: - name: bender-robots tls: ca: certChain: | -----BEGIN CERTIFICATE----- MIIFpTCCA40CFGFL86145m7JIg2RaKkAVCOV1H71MA0GCSqGSIb3DQEBCwUAMIGN [skipped for brevity - include whole certificate] SKnBS1Y1Dn2e -----END CERTIFICATE----- ``` {% endcode %} #### Configuring Certificate Authority for Agents via Manual Kubernetes ConfigMap Alternatively, you can provide the CA certificate as a Kubernetes ConfigMap. **Creating the Kubernetes ConfigMap** If you want to manually manage the CA certificate as a Kubernetes ConfigMap you will need to create it before configuring your agent. You can use any process you like, as long as the result is a Kubernetes ConfigMap that: * Contains a file called ca-chain.pem containing the CA certificate * Is in the same namespace as your `connectware-agent` installation We will demonstrate how to create this ConfigMap through kubectl. Ensure that your CA certificate is stored in a file named `ca-chain.pem`. Because the CA certificate extracted from Connectware is named `cybus_ca.crt`, we will create a copy in our example. Define a name for your ConfigMap. You will need this name later to configure the agent, so keep it at hand. **Example** {% code lineNumbers="true" %} ```bash cp cybus_ca.crt ca-chain.pem kubectl create configmap --from-file ca-chain.pem ``` {% endcode %} **Configuring the Agent Through Helm Values** You must specify the name of your ConfigMap in the Helm value `tls.ca.existingConfigMap` inside the agents entry in the `protocolMapperAgents` context of your `values.yaml` file. If you configure more than one agent, we recommend to provide the CA certificate through `protocolMapperAgentDefaults` instead of `protocolMapperAgents`, as it should be the same for all agents. **Example** {% code lineNumbers="true" %} ```yaml protocolMapperAgents: - name: bender-robots mTLS: caChain: existingConfigMap: ``` {% endcode %} ## Enabling mTLS for the Agent Finally, you must enable mTLS for the agent. To do this, set the Helm value `mTLS.enabled` to `true` inside the agents entry in the `protocolMapperAgents` section of your `values.yaml` file. **Example** {% code lineNumbers="true" %} ```yaml protocolMapperAgents: - name: bender-robots mTLS: enabled: true ``` {% endcode %} To apply this configuration to your agent, you must use Helm upgrade on your `connectware-agent` installation with the same parameters you originally used. **Example** {% code lineNumbers="true" %} ```bash helm upgrade connectware-agent cybus/connectware-agent -f values.yaml -n ${NAMESPACE} ``` {% endcode %} ## Replacing mTLS Certificates and Keys for the connectware-agent Helm Chart If you want to replace certificates or keys, follow the same steps as when adding them. However, the agents will not automatically start using the new certificates. You must manually restart the Kubernetes StatefulSets associated with the agents for which you replaced certificates. This StatefulSet is named `--`. Some characters, most prominently the `.` character, will be replaced by `-` characters in the agent name. If you followed the recommendations in this documentation, the first two parts are abbreviated to `connectware-agent`. An agent named `bender-robots`, for example, would then be deployed as a StatefulSet named `connectware-agent-bender-robots`. {% code lineNumbers="true" %} ```bash kubectl -n ${NAMESPACE} rollout restart sts ``` {% endcode %} This will restart the agent and apply the new certificate/key. If you want to restart all agents from your installation, you can use this command in combination with the name you gave to your agent deployment: {% code lineNumbers="true" %} ```bash kubectl -n ${NAMESPACE} rollout restart sts -l app.kubernetes.io/instance= ``` {% endcode %} If you followed the recommendations in this documentation, `` is `connectware-agent`. ### Full mTLS Examples for the connectware-agent Helm Chart #### Two Agents with Manually Created Kubernetes Secrets/Configmap with Default Names This example assumes: * You created a ConfigMap with the name `connectware-agent-tls-ca-cert` containing the CA certificate as a file called `ca-chain.pem`. * You created a secret named `connectware-agent-bender-robots-mtls` containing a certificate for the Common Name (CN) `bender-robots`, signed by the CA, as a file called `tls.crt` and the matching key as a file called `tls.key`. * You created a secret named `connectware-agent-welder-robots-mtls` containing a certificate for the Common Name (CN) `welder-robots`, signed by the CA, as a file called `tls.crt` and the matching key as a file called `tls.key`. * You named your connectware-agent Helm installation `connectware-agent`. * Your Connectware installation is located in the namespace `cybus`. {% code lineNumbers="true" %} ```yaml licenseKey: protocolMapperAgentDefaults: connectwareHost: connectware.cybus mTLS: enabled: true tls: ca: existingConfigMap: connectware-agent-tls-ca-cert protocolMapperAgents: - name: bender-robots - name: welder-robots ``` {% endcode %} #### Two Agents with Manually Created Kubernetes Secrets/Configmap with Custom Names This example assumes: * You created a ConfigMap with the name `my-ca-cert` containing the CA certificate as a file called `ca-chain.pem`. * You created a secret named `mtls-keypair-1` containing a certificate for the Common Name (CN) `bender-robots`, signed by the CA, as a file called `tls.crt` and the matching key as a file called `tls.key`. * You created a secret named `mtls-keypair-2` containing a certificate for the Common Name (CN) `welder-robots`, signed by the CA, as a file called `tls.crt` and the matching key as a file called `tls.key`. * You named your connectware-agent Helm installation `connectware-agent`. * Your Connectware installation is located in the namespace `cybus`. {% code lineNumbers="true" %} ```yaml licenseKey: protocolMapperAgentDefaults: connectwareHost: connectware.cybus mTLS: enabled: true tls: ca: existingConfigMap: my-ca-cert protocolMapperAgents: - name: bender-robots mTLS: keypair: existingSecret: mtls-keypair-1 - name: welder-robots mTLS: keypair: existingSecret: mtls-keypair-2 ``` {% endcode %} #### Two Agents with Manually Created CA Certificate Configmap but Key Pair in Helm Values This example assumes: * You created a ConfigMap with the name `connectware-agent-tls-ca-cert` containing the CA certificate as a file called `ca-chain.pem`. * You named your `connectware-agent` Helm installation `connectware-agent`. * Your Connectware installation is located in the namespace `cybus`. {% code lineNumbers="true" %} ```yaml licenseKey: protocolMapperAgentDefaults: connectwareHost: connectware.cybus mTLS: enabled: true tls: ca: existingConfigMap: connectware-agent-tls-ca-cert protocolMapperAgents: - name: bender-robots mTLS: keyPair: cert: | -----BEGIN CERTIFICATE----- MIIEgTCCAmkCFCN+Wi9RpeajIunZnxdIhvdZep6ZMA0GCSqGSIb3DQEBCwUAMIGN [skipped for brevity - include whole certificate] sD9hY3o= -----END CERTIFICATE----- key: | -----BEGIN PRIVATE KEY----- MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCg+mC1iGmz+qCO [skipped for brevity - include whole private key] nJn5oNH9lcodhXcuOYVg3kQ= -----END PRIVATE KEY----- - name: welder-robots mTLS: keyPair: cert: | -----BEGIN CERTIFICATE----- MIIFcjCCA1oCFFgO7SgdLBuU6YBOuZxhQg0eW5f+MA0GCSqGSIb3DQEBCwUAMIGN [skipped for brevity - include whole certificate] VM6E0Lqy -----END CERTIFICATE----- key: | -----BEGIN PRIVATE KEY----- MIIJRAIBADANBgkqhkiG9w0BAQEFAASCCS4wggkqAgEAAoICAQDvmp+v3+x1am6m [skipped for brevity - include whole private key] Y6vWPIuRCwum9DxjrdIva6Z6Pqkdyed9 -----END PRIVATE KEY----- ``` {% endcode %} #### Two Agents with Manually Created CA Certificate and Key Pair in Helm Values This example assumes: * You named your `connectware-agent` Helm installation `connectware-agent`. * Your Connectware installation is located in the namespace `cybus`. {% code lineNumbers="true" %} ```yaml licenseKey: protocolMapperAgentDefaults: connectwareHost: connectware.cybus mTLS: enabled: true tls: ca: certChain: | -----BEGIN CERTIFICATE----- MIIFpTCCA40CFGFL86145m7JIg2RaKkAVCOV1H71MA0GCSqGSIb3DQEBCwUAMIGN [skipped for brevity - include whole certificate] SKnBS1Y1Dn2e -----END CERTIFICATE----- protocolMapperAgents: - name: bender-robots mTLS: keyPair: cert: | -----BEGIN CERTIFICATE----- MIIEgTCCAmkCFCN+Wi9RpeajIunZnxdIhvdZep6ZMA0GCSqGSIb3DQEBCwUAMIGN [skipped for brevity - include whole certificate] sD9hY3o= -----END CERTIFICATE----- key: | -----BEGIN PRIVATE KEY----- MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCg+mC1iGmz+qCO [skipped for brevity - include whole private key] nJn5oNH9lcodhXcuOYVg3kQ= -----END PRIVATE KEY----- - name: welder-robots mTLS: keyPair: cert: | -----BEGIN CERTIFICATE----- MIIFcjCCA1oCFFgO7SgdLBuU6YBOuZxhQg0eW5f+MA0GCSqGSIb3DQEBCwUAMIGN [skipped for brevity - include whole certificate] VM6E0Lqy -----END CERTIFICATE----- key: | -----BEGIN PRIVATE KEY----- MIIJRAIBADANBgkqhkiG9w0BAQEFAASCCS4wggkqAgEAAoICAQDvmp+v3+x1am6m [skipped for brevity - include whole private key] Y6vWPIuRCwum9DxjrdIva6Z6Pqkdyed9 -----END PRIVATE KEY----- ``` {% endcode %}