Installation
Uniloader is installed from a .tar.gz package, using the install.sh script it contains. This works on any system
running Systemd, including CentOS-based and RPM-based PBX systems (FreePBX, PBXact, etc.), Debian and Ubuntu.
If your system does not run Systemd, refer to the manual install below.
The Loway yum repository is no longer available, so yum install uniloader and yum update cannot be used anymore.
If you installed Uniloader with yum in the past, see Migrating from the yum (RPM) installation.
|
Standard installation (all systems)
Installing Uniloader is very easy. First find the package you want to download under https://downloads.loway.ch/software/uniloader/
mkdir -p /opt/loway/uniloader cd /opt/loway/uniloader wget https://downloads.loway.ch/software/uniloader/uniloader-26.10.1.tar.gz tar zxvf uniloader-26.10.1.tar.gz cd uniloader-26.10.1 ./install.sh
The automated install script will:
-
install Uniloader system-wide (so you can call it from any shell)
-
create configuration and init scripts for all services below
-
start the service
uniloader(though it won’t work until you edit its credentials and restart it) and make it startable on reboots -
install a script called
./check.shthat you can use to check the settings for each of the services, by using the configuration you specified for each unit. This makes it very easy to make sure it is correct.
For each service, we keep a separate configuration file under /etc, that is the one you are generally supposed to edit. If you make a change to the configuration files, make sure you issue a systemctl restart uniloader so that the new configuration can be picked up.
If you choose to enable any other service, it’s up to you to make sure that it is correctly restarted on boot.
To update, just download the new version of Uniloader as you did, and run the install script again. When updating, configuration files are not changed, while unit files are replaced with the current version.
|
A quick recap of Systemd commands might help:
|
Uniloader
-
Service name:
uniloader -
Configuration file:
/etc/uniloader -
Testing:
./check.sh uploadthat will use the current configuration and make sure the destination QM server is reachable.
We strongly suggest always using the HTTP driver to connect to QM - this makes it easy to
send data to a local instance, a remote instance you control, or a QueueMetrics Live instance.
Remember to enable the user webqloader on QM, and set a password for it.
URI=http://127.0.0.1:8080/queuemetrics LOGIN=webqloader UPASSWD=Secret1234 TOKEN=
|
To see which one applies to you, run |
Make sure you restart the service after making any changes.
You can also turn-on the splitter for Uniloader by uncommenting the right line,
and the editing the example splitter file and /etc/uniloader_splitter.json
Unitracker
-
Service name:
unitracker -
Configuration file:
/etc/unitracker- you must set up the correct AMI credentials -
Testing:
./check.sh unitrackerwill make sure that the AMI credentials used by Unitracker are working -
This service must be enabled manually if needed.
By default, only MOH tracking is enabled. You may want to enable parkings or, more commonly, outbound tracking. In this case, do not forget to set up the "hidden channels" mask to decide which calls you don’t want to track.
AudioVault
-
Service name:
audiovault -
Configuration file:
/etc/audiovault -
Testing:
./check.sh audiovaultthat will try requesting a (non-existent) file as if it were an external client. You are expected to see no errors there. -
This service must be enabled manually if needed
-
In order to run this effectively in production, it’s strongly advisable to put an HTTPS proxy in front, as per its documentation.
Freeswitch
-
Service name:
uniloader-freeswitch -
Configuration file:
/etc/uniloader-freeswitch -
Testing:
./check.sh freeswitchthat will check ESL credentials -
This service must be enabled manually if needed
-
When enabiling this service, you must make sure that the queue_log file that this service generates and the one being imported by Uniloader match - so do not forget to check
/etc/uniloaderas well. -
This service is meant to run all of the time and never being restarted, as it will lose events if it is restarted during its operation.
Migrating from the yum (RPM) installation
If you installed Uniloader with yum install uniloader, you can no longer update it with yum.
To upgrade, remove the RPM package and install the .tar.gz package as described in Standard installation (all systems).
From then on, every update is done by running install.sh again. You need root access and a system running Systemd;
plan a short maintenance window, as data is not uploaded while Uniloader is stopped.
The two installations place their files in different locations:
| yum (RPM) installation | tar.gz with install.sh |
|
|---|---|---|
Updates |
|
download the new package and run |
Program files |
|
installed system-wide by |
Uniloader configuration |
|
|
Unitracker configuration |
|
|
Service control |
|
|
Follow these steps:
-
Back up your configuration files. Save every file you use, and write down the installed version with
uniloader --version:tar czf /root/uniloader-config-backup-$(date +%F).tar.gz \ $(ls -d /etc/sysconfig/uniloader* /etc/sysconfig/unitracker* \ /etc/uniloader* /etc/unitracker* /etc/audiovault* \ /etc/defaults/uniloader* 2>/dev/null)If you start Uniloader in other ways (for example from
/etc/rc.localor a cron job), save those files too. -
Remove the RPM package.
service uniloader stop yum remove uniloader rpm -qa | grep -i uniloader
The last command should print nothing. If you do not use the Loway repository for any other package, you can also delete
/etc/yum.repos.d/loway.repo. -
Install the new package with
install.sh, as described in Standard installation (all systems). -
Restore your settings in the new configuration files under
/etc, using your backup as a reference. Copy the values one by one (URI,LOGIN, the password,TOKEN) instead of overwriting the new file with the old one: the new files contain comments that explain each setting. Pay attention to the name of the password variable: see thePASSorUPASSWDtip above. Settings that only served the old init script, such as the log, lock and pid file paths, are not needed anymore; logs are now available withjournalctl -u uniloader. -
Start and check. Restart the service and test the connection:
systemctl restart uniloader ./check.sh upload
Then place a test call and make sure that it appears in QueueMetrics.
Only the uniloader service is started by install.sh. If you used other services, such as the tracking service, set up the
configuration file under /etc, test it with ./check.sh <service> and enable it with systemctl enable --now <service>.
If you were using a direct MySQL connection (a URI starting with mysql:), you can keep it, copying URI, LOGIN,
the password and TOKEN (the partition) exactly as they were. We suggest instead to switch to the HTTP driver with the webqloader user:
see Switching from the MySQL connection to the webqloader user.
|
Switching from the MySQL connection to the webqloader user
Uniloader can upload data to QueueMetrics in two ways:
-
MySQL (the old way): Uniloader writes directly to the QueueMetrics database (
URIstarting withmysql:). This is the method used by older installations, and it is still supported. -
HTTP/S (the new way): Uniloader sends data to QueueMetrics over HTTP/S, using the
webqloaderuser of QueueMetrics (URIstarting withhttp:orhttps:). This is now the recommended method.
The HTTP/S connection is better because:
-
the PBX does not need to reach the MySQL server, and does not need to know the database credentials;
-
it works in the same way with a local instance, a remote instance or QueueMetrics Live;
-
when running over HTTP, Uniloader can act as a proxy for the actions QueueMetrics sends to the PBX (log agents on and off, pause them, listen to calls, etc.);
-
data deduplication is available (it requires QueueMetrics 19 or later and the HTTP interface).
To switch:
-
Enable the
webqloaderuser in QueueMetrics. On QueueMetrics Live it is already enabled; on an on-premise installation, go to Users, findwebqloader, tick Enabled, set a strong password and save. The user must hold the keysWQLOADER,USERandROBOT. -
Edit
/etc/uniloader. Comment out the MySQL settings and set the HTTP ones:#URI="mysql:tcp(127.0.0.1:3306)/queuemetrics?allowOldPasswords=1" #LOGIN=queuemetrics URI=http://127.0.0.1:8080/queuemetrics LOGIN=webqloader UPASSWD=Secret1234 TOKEN=
The
URIis the base address of QueueMetrics, as you would use it in a browser, withhttps://if QueueMetrics is behind SSL. For the name of the password variable, see thePASSorUPASSWDtip above. -
Mind the token. In MySQL mode the token holds the partition (such as
P001); in HTTP/S mode it is usually left blank. In a cluster, it holds the name of the cluster member instead. If you were using a partition, check your cluster configuration before switching. -
Restart and check.
systemctl restart uniloader ./check.sh upload
Then place a test call and make sure it appears in QueueMetrics. If the login fails, check that
webqloaderis enabled and that the password variable has the right name.
Once everything works, the MySQL account that Uniloader used is no longer needed: if it was dedicated to the PBX, you can remove it and close access to the MySQL port from the PBX.
Manual installation
The Uniloader can be downloaded from https://downloads.loway.ch/software/uniloader/
The package contains:
-
Uniloader binaries for all supported architectures (i386, amd64, arm7, arm64),
-
A sample extensions_queuemetrics file,
-
A sample splitter file.
Just copy the file "uniloader_xxx" for your architecture (Intel 32 / 64 bit, ARM 7, ARM64) into your computer and make it executable:
cp ./bin/uniloader_arm7 ./uniloader chmod a+x ./uniloader
To test it, run:
./uniloader -?
It should output a result like:
NAME:
uniloader - the data upload companion for QueueMetrics and QueueMetrics Live.
USAGE:
uniloader [global options] command [command options] [arguments...]
VERSION:
22.11.3 - build: 1234-20230511.0849/8300761 - OS: darwin/arm64 - RT: go1.17.5
COMMANDS:
help, h Shows a list of commands or help for one command
Diagnostics:
pbxinfo Loads PBX information
test Runs environment tests
Services:
upload, u Uploads a source file to a QueueMetrics or QueueMetrics Live instance
track, t Tracks Asterisk events and creates relevant queue_log entries.
fsw Parses FreeSwitch mod_callcenter events
av Runs AudioVault - a media storage server for QM (experimental)
Tools:
cfgfile Reads and writes a configuration file
user Creates or alters users
qmdb Performs direct operations on QueueMetrics' database
asterisk Performs direct operations on Asterisk queues
GLOBAL OPTIONS:
--src value, -s value The source queue_log file to be uploaded (default: "/var/log/asterisk/queue_log")
--cacert value An optional CA Cert file in .pem format, or 'insecure' to skip any certificate validation
--verbose-back-end Enables verbose back-end logging. Default: false.
--read-pipe The source file is a Unix pipe
--help, -h show help
--version, -v print the version
If it does, it is working.
Running in production
Uniloader produces a verbose log on STDOUT that should be redirected to a file and periodically rotated.
You should also make sure that Uniloader is started when the PBX boots and that in case it should crash it is automatically restarted.
We advise running Uniloader using 'nice' so that it has reduced access to scarce CPU resources in case of high load / contention with the PBX - while the PBX voice quality quickly degrades on a resource-starved system, Uniloader does not really care about small delays in data uploading.
We also advise adding Uniloader to the command path, so that its many debugging functions can be accessed easily.
Example: Uploading data to a local QueueMetrics system
This is the most common scenario when using a locally installed QueueMetrics system.
nohup nice \
./uniloader -s /var/log/asterisk/queue_log \
upload --uri "mysql:tcp(1.2.3.4:3306)/queuemetrics?allowOldPasswords=1" \
--login queuemetrics --pass javadude --token P001 \
>> /var/log/uniloader.log &
Will upload the queue_log file located at /var/log/asterisk/queue_log to the
remote "queuemetrics" database on server 1.2.3.4 with login "queuemetrics" and password "javadude", using the default partition "P001".
After you start it, check the file /var/log/uniloader.log to make sure there are no errors. The most common
error is that you did not create the correct grants for your MySQL user to upload data remotely.
If everything seems to work, log in into QueueMetrics, select "Edit system parameters" and make sure that the default partition is P001.
# This is the default queue log file. default.queue_log_file=sql:P001
At this point, log off; log on again and click on "Mysql storage information"; select partition P001 and select "Autoconfigure queues". Now the default queue "00 All" will include all of your queues and you can see your historical and real-time status.
Example: Uploading data to a QueueMetrics Live system
You received your QueueMetrics Live access information; and in the received email you find that your instance is called "ABCD" and the password is "1234".
nohup nice \
./uniloader --src=/var/log/asterisk/queue_log \
upload --uri https://my.queuemetrics-live.com/ABCD \
--login webqloader --pass 1234 \
>> /var/log/uniloader.log &
At this point Uniloader will start feeding the remote database. You can login at any time by visting the address https://my.queuemetrics-live.com/ABCD and logging in as "demoadmin" and the password you were given.
The first time you log in, click on "System diagnostic tools" and then "Live DB inspector" to see data being uploaded. When your database is complete (this may take a few minutes, depending on how much data is on your PBX), go back to the home page and click on "Mysql storage information"; select partition P001 and select "Autoconfigure queues". Now the default queue "00 All" will include all of your queues and you can see your historical and real-time status.
See also One Asterisk instance, one hosted QueueMetrics Live instance. If you run a Yeastar system, see the chapter dedicated to Yeastar MyPBX.