Setting Up connectors

The first step when working with ZSA is to define how your data will be gathered. This is done in the Messaging tab of the ZSA module's Configuration page.

Defining Your First Connector

Creating a New Connector

To create a new connector, do the following:

1. From the Messaging Connectors List, select "Create Connector" in the top bar.

   

2. We must select which kind of connector, based on the data source

   1. Mainframe: data will be collected from ZSA Mainframe agent

   2. FTP: data will be collected from a FTP server (uncrypted)

   3. SFTP: data will be collected from a FTP server using crypted connexion

   4. FTPS: data will be collected using FTP over SSH protocol

   5. S3: data will be collected from a AWS S3 bucket

3. An "Create new connector" dialog box appears.

1. In the dialog box, do the following:

   1. CONFIGURATION

      1. In the Label field type the intended name for connector.

      2. You can choose your parsing type

         1. SMF Legacy: SMF data already parsed by the MainFrame Zetaly Agent

         2. SMF RAW: SMF data in binary format

         3. CSV (only for non Mainframe sources): data in CSV. A related parser must be defined

      3. In the Host field type the host address of the data source to which you want to connect.

      4. In the Port field type the port number at which the datasource is accessible. Typically this is 9999, however you must confirm the port used by the mainframe.

      5. Specific configurations

         1. FTP/SFTP/FTPS

            1. Username and Password used for connexion. Data are encoded in database

            2. Path pattern: you can defined a specific directory and files pattern to select specific files to import

         2. S3

            1. You must select the AWS region of your bucket

            2. Bucketname to use

            3. Your AWS access and secret keys. Data are encoded in database

            4. Path pattern: you can defined a specific directory and files pattern to select specific files to import

   2. TAGS section

      1. In the Tags section you can link or create personnalized tags. (also read this article)

   3.  PROPERTIES section (only for Mainframe sources): see advices in Configuring your Connectors

      1. In the Number of instances field, type the number of separate connections that will be made to the datasource. 

      2. The Autoscale instances option automatically creates a new instance when the Usage Value reaches 90%. (read more here)

      3. In the field "Number of records" you can modify the default maximum number of messages requested/processed per execution cycle. 

   4. Click Apply.
      The connector details you specified appear in a new row within the Connector List, and several actions are available (see Overview of the Connector Actions).

Information

Only one connector per type and per host can be created

Overview of the Connector Actions

Once saved, the connector details appear in a new row in the Connector List. In the example below, one connector called ZDEV exists.

Buttons are available for a connector within the table, as summarized below:

• Start/Stop button : starts or stops the connector to the datasource and thus the collection of data.

• Edit connector button : displays the same "Add new connector" dialog box as when creating a new connector.

• Duplicate connector button : creates a copy of this connector. You are prompted to enter a new name for the new duplicate connector. The duplicated connector appears within an additional row in the Connector List.

• Delete connector button : deletes the connector.

• Connector logs section (expanded row): displays the logs related to that connector.

• Files (only for FTP/FTPS/SFTP/S3 connectors): View file import logs and retry all failed file processing.

  • 

Other buttons are present in the top bar of the interface, as summarized below:

• Start all the selected connectors. All the selected connectors will be started.

• Stop all the selected  connectors. All the selected connectors will be stopped. 

• Assign tags to all the selected connectors. Assign one or more tag(s) to all the selected connectors. Read more

• Delete all the selected connectors. All the selected connectors will be deleted. 

• Add a new connector. Opens an Add Server dialog box, letting you create a new connector as described in Creating a New Connector.

Every connector displays its status in a dedicated column of the table:

• STARTING: The messaging initialized everything it needs before starting to collect data.

• RUNNING: The messaging is running and collecting data.

• STOPPING: The messaging is stopping.

• STOPPED: The messaging is not running and did not terminate with an error.

• READY: The messaging is launched but there is nothing to collect yet.

• DEGRADED: At least one instance of the connector is in "error", but not all the instances.

• ERROR: The messaging is not running and terminated with an error. You can find the detail in the logs by clicking the logs button.

To start the connector click on "start". The collection of data starts if your datasource is accessible, and your data is being gathered by ZSA.

The default configuration of your connectors might be enough for your datasources, but you can find yourself needing to optimize it to meet your needs. This usually happens when the datasource is producing a lot of data that is not fetched fast enough by ZSA. You can optimize your connector by doing one or both of the following:

• increasing the number of connector instances (the easiest/quickest way to optimize) (see Increasing the Number of Connector Instances)

• modifying the message size and connection sleep time (see Configuring the Datasource)

Increasing the Number of Connector Instances

To increase the number of connector instances, do the following in the connector properties section:

1. Click on edit (2).
   An Update Server dialog box appears.

2. From the Update Server dialog box that appears, do the following:

   1. In the Quantity of instances field, specify a larger number (e.g. 3).

   2. Click Apply.
      The configuration change you made takes effect immediately.

Each instance is a copy of the same connector configuration. The number of connector instances is displayed in the connector list.

Configuring the Datasource

If you want to further optimize your connector configuration, you can determine how the datasource is fetched by each of the connector's instances. Do the following in the connector list: 

1. Click on edit (2).

2. From the Properties section of the dialog box that appears, do the following:

   1. In the Message size field, specify a different message size. The message size defines the number of individual data entries to fetch at once. The default message size is 5000.
      If your network is slow, you may want to reduce the message size.
      If you network is fast, you may want to increase the message size.

   2. In the Connection sleep time (s) field, specify a different sleep time. The connection sleep time defines the delay (in seconds) between two calls to the datasource, if the last call did not find any data. The default connection sleep time is 5 seconds.

   3. Click Apply.
      The configuration change you made takes effect immediately, and is applied the same way on all of the connector's instances.