Concepts

Back-ends

Uniloader supports three different back-ends: HTTP/HTTPS, MYSQL and FILE.

Each back end is functionally similar and can be thought of as a black box; it can be selected simpy by entering a proper URI for the server.

HTTP/HTTPS back-end

If your URI looks like:

http://myserver/queuemetrics

Then you are using HTTP. In this case, the value of the "token" parameter is either a server in a cluster, or you can leave it blank to denote the default server, and user/password are for a valid QueueMetrics HTTP user.

The HTTP back-end also supports HTTPS URLs and will, by default, retrieve actions to be performed on the PBX.

Please note that some appliances do not support HTTPS, so running HTTP might be mandatory.

HTTPS CA certificate issues

On some systems (especially appliances) it is possible that when running HTTPS requests, they all fail with an error like:

x509: failed to load system roots and no roots provided

In this case, you have to manually tell the Uniloader where to find the correct CA .pem files for your system, by using the "--cacert" parameter.

E.g.

./uniloader --cacert=/etc/certs/default.pem upload ....

Will force Uniloader to use the supplied root certificates. In case they are totally missing, we suggest copying a recent certificate file from a working Linux distribution and point to that.

As a temporary workaround, you can completely disable certificate validation by issuing --cacert insecure. When running in this mode, a log line is on command start-up to remind you that you are not verifying certificates.

MySQL back-end

If your URI looks like:

mysql:127.0.0.1/queuemetrics

the loader will connect to a MySQL database called "queuemetrics" on "127.0.0.1", using the supplied login and password; the token in this case is the partition that we want to upload data to.

If your MySQL is running on a remote system, it might be advisable to use a MySQL URI of the format:

mysql:tcp(1.2.3.4:3306)/uniloader?allowOldPasswords=1

This will connect to a database called "uniloader" on 1.2.3.4 and will set the parameter "allowOldPasswords" to 1, as it is sometimes needed to use old versions of MySQL.

A complete reference of all allowed DSN (Data Source Name) formats and connection parameters is available at https://github.com/go-sql-driver/mysql

If upload is stuck at some point and cannot progress because of an error that looks like Data too long for column 'agent' at row 1, this means that some fields on your queue_log file are larger than the allowed space on the server’s database. In our experience, this only happens if you have a corruped file or you manually logged a value that is exceedingly long. At this point, you can either:

  • Edit the queue_log file to remove the extraneous value and restart Uniloader, or

  • Restart Uniloader with a connection string like mysql:tcp(1.2.3.4:3306)/uniloader?allowOldPasswords=1&sql_mode='' that will prevent the database from complaining by silently trimming any long strings. Please note that this may cause synchonization issues when trying to find the current high water mark and in any case causes data loss, so should be used only sparingly to override a temporary stumbling block.

File back-end

If your URI looks like:

file:/my/file/path

The loader will try and append to a local file. This module is meant for quick testing of splitting rules and does not currently check the state of the local file before writing to it.

It can also be used as a quick way to "throw away" a log file, by using 'file:/dev/null' on Unix systems.

This back-end is only meant for testing and experimentation; the file is rebuilt on every run, so no guarantee about data integrity is implied.