docs.loway.ch :: Loway Documentation Center

FreeSwitch / FusionPBX support

Uniloader can work with FreeSwitch to act as an adaptor between its 'mod_callcenter' features and QueueMetrics. In particular, when using FusionPBX, events are divided by 'domain' as to be able to feed them to multiple QueueMetrics Live systems, one for each tenant who subscribed the service.

General overview

Uniloader is able to build a synthesized queue_log file out of a stream of events flowing from a FreeSwitch system. This queue_log file can then be uploaded by a regular Uniloader process to a single QueueMetrics instance, or can be split by tenant and pushed to multiple, separate QueueMetrics Live instances.

Agent state is controlled by QueueMetrics by sending ESL commands to implement agent actions (eg. log-on or log-off).

In order to enable this feature, you need to run two copies of Uniloader in parallel, as different services:

one runs uniloader fsw: it connects to FreeSwitch over ESL, reads events from 'mod_callcenter' and generates a queue_log file
another one runs uniloader upload: it reads the generated queue_log file and uploads it to one or more QueueMetrics systems, splitting data as appropriate between different tenants

The system was designed to be run as two different services because log generation is based on events streamed in real-time, so the idea is that the service is run at all times when FreeSwitch is active. Upload also happens in real-time, but it can be restarted with no data loss if you need to change its configuration - e.g. adding or removing a tenant.

Agent and queue translation

With FusionPBX, uniloader fsw can be given access to the FusionPBX’s Postgres database, in order to decode queue and agent names (that appear as UUIDs) into their own tenant and extension.

For example, a queue with id 75082016-6394-4738-b896-b9121c060612 that belongs to domain (tenant) 'abc.example.com' where it is reachable under extension 200 will be logged as abc.example.com-200. An agent working from extension 300 under the same tenant will appear as Agent/abc.example.com-300.

On current versions of FusionPBX (5.x), some queue events are logged with their UUID and some use the format agent@domain. Starting from Uniloader 21.04.5 onwards, such events are recognized and translated trasparently. If you get a crash that says invalid input syntax for type uuid: "1600@tenant.server.com", you have an older Uniloader that needs to be upgraded.

When uploading data, you can then use the splitter feature to send only data for domain 'abc.example.com' to a QueueMetrics Live instance named (for example) 'my.queuemetrics-live.com/customer-abc', where the data mentioned above appears as queue code '200' and agent code 'Agent/300'.

While the settings work for FusionPBX, it is possible to override the database and the queries used by setting sql-agent and sql-queue. For example, if we run:

uniloader fsw --queuelog my_log.txt --ps-uri 127.0.0.1/fusionpbx
    --shorten-domain 1  --sql-agent 'SELECT 1000 as TENANT, $1 as AGENT'

we will always get back the same agent and a tenant "1000". If your logic is on MySQL, we could get the same result by issuing:

uniloader fsw --queuelog my_log.txt --ps-uri "10.10.5.10/sugarcrm"
        --ps-database "mysql" --ps-login queuemetrics --ps-pwd javadude
        --shorten-domain 1
        --sql-agent 'SELECT "abcd" as TENANT, ? as AGENT'

This will return the same agent with a tenant called "abcd".

Any query that receives a single input value and returns two columns will work. The first column is the tenant, while the second one is the decoded queue or agent id.

Disabling database translation

Database translation is needed from FusionPBX 4.4 onwards. On earlier versions, you may want to switch it off by explicitly setting the ps-uri option to blank or a dash, as in --ps-uri "-".

Writing custom queries

MySQL and Postgres use a different format for allowed SQL strings:

The placeholder used in Postgres is $1 while Mysql uses ?.
The placeholder in Postgres can be used multiple times. On MySQL you have only one, so you need to write a Common Table Expression instead, as shown in the examples below.
Literal strings must be quoted with a single quote in Postgres ' while MySQL also allows a double-quote ".

The names of fileds don’t really matter, as Uniloder will take the first one as the tenant and the second one as the queue or agent.

Examples

Split an entry like 1600@tenant.server.com into queue and tenant (Postgres):

SELECT split_part($1, '@', 2) as TENANT, split_part($1, '@', 1) as QUEUE

Split an entry like 1600@tenant.server.com into queue and tenant (MySQL). As we cannot reuse the placeholder, we sport the Commont Table Expression syntax, where we create a literal table cte with a single columns named v and only one row of data that contain our value:

WITH cte (v) as (values ( ? ))
SELECT substring_index(v, '@', -1) as TENANT, substring_index(v, '@', 1) as QUEUE
FROM cte

Given an agent code like 1000, add a fixed tenant "ABCD" (MySQL):

SELECT "abcd" as TENANT, ? as AGENT

ADDMEMBER mode

FusionPBX does not allow the selection of which queues an agent is supposed to work on; an agent becomes available (or unavailable) on all of the queues they are configured on.

In terms of reporting, this means that QueueMetrics will display an agent as available on a fake queue called * ALL *, as we have no information on which queues an agent is actually working on.

If you enable ADDMEMBER mode by passing --use-addmember "1" to uniloader fsw (requires Uniloader 21.04.6), then each time an agent logs on or off, their current set of queues is queried on the FusionPBX database, and separate log-on records are emitted for each queue. This gives QueueMetrics the information needed to display the current set of queues an agent is working on.

This causes a database access on each agent log-on or log-off; the query run can be overridden though sql-qs-for-agent, that given the agent’s UUID, should return the UUIDs of all queues this agent is supposed to be working on. The UUIDs for agent and queues will then be translated through the normal mechanism described above.

This does not change the fact that logging on, logging off or pausing happen at once on all queues, and never separately. A possible workaround is explained in our FusionPBX Integration Guide.

Setting up

You can install Uniloader normally; make sure you enable both services uniloader and uniloader-fsw.

You can also run it manually to test it.

$ ./uniloader fsw -?

NAME:
   uniloader fsw - Parses FreeSwitch mod_callcenter events

USAGE:
   uniloader fsw [command options] [arguments...]

DESCRIPTION:
   This command listens on FreeSwitch's Event Socket.

It reacts to mod_callcenter events and attempts to coerce them
to queue_log format, to make them compatible
with QueueMetrics.
It only generates a queue_log file; it should then be
uploaded by a separate instance of 'uniloader upload'.

OPTIONS:
   --host "127.0.0.1"     Your FreeSwitch server
   --port "8021"      The ESL port on FreeSwitch
   --auth "ClueCon"     The ESL auth secret [$AUTH]
   --queuelog         The queue_log file to write
   --events         A debug file to dump mod_callcenter events to
   --ps-database "postgres"   The kind of database we  can connect to.
   --ps-uri "localhost/fusionpbx" A FusionPbx database to connect to
   --ps-login "fusionpbx"   A FusionPbx database user
   --ps-pwd         A FusionPbx database password [$FUSIONPWD]
   --sql-agent        The query used to extract (tenat,agent) from agentId. Blank for default.
   --sql-queue        The query used to extract (tenat,queue) from queueId.  Blank for default.
   --sql-qs-for-agent value  The query to extract queue UUIDs that agentId works on.  Blank for default.
   --shorten-domain value    If 1, the domain will be shortened (default: 0)
   --use-addmember value     If 1, agent log-ons will be logged queue-by-queue. (default: 0)
   --pid        The PID file to write. If already present, won't start.

When setting up:

The 'queuelog' option should create a queue log file. It can be anywhere - you must make sure that it is the same location that will be read by the uniloader service
The file 'events' is optional, but we suggest creating it so anomalies can be tracked
If you use FusionPBX, credentials to the database can be entered in 'ps-uri', 'ps-login' and 'ps-pwd'. If you don’t want to use it, set 'ps-uri' to a single dash. The option 'ps-database' can be set to postgres or mysql.
The 'shorten-domain' option will try shortening the domain name to the first element in it, e.g. 'abc.example.com' will be shortened to 'abc'.
The 'use-addmember' option will generate ADDMEMBER records instead of AGENTLOGIN - see ADDMEMBER.

The database connection and the ESL connection can be checked using uniloader test postgres and uniloader test fsw-esl.

Setting up mult-tenant systems with QueueMetrics Live

When using QueueMetrics Live with multiple instances on a multi-tenat system, you need to run the uniloader service as:

./uniloader -s qlog.txt  u  -x splitter_rules.json

Where 'splitter_rules.json' is a file that contains multiple tenants, defined as:

[
    {
	    "clientname": "Acme Company Ltd",
	    "uri": "https://my.queuemetrics-live.com/acmeco",
	    "login": "webqloader",
	    "pass": "itsasecret",
	    "token": "",
	    "matcher": ["acmeco-"],
	    "match": "any",
	    "removematch": true,
	    "disabled": false,
	    "noactions": false
    }
]

Note that:

'uri', 'login' and 'pass' are the ones that you are given for your QueueMetrics Live instance
'matcher' contains the domain (tenant) and a trailing slash.
'clientname' is not needed in this scenario, but we suggest setting it for readability

You can safely restart the service when you make changes to the rules, as data is queued on the 'queue_log' file.

You do not need to have all tenants configured; only the ones that match will be fed, and other data will be ignored. If you create a new tenant, and there is existing data for it on the log file, it will be uploaded on the first run.

A complete explanation of the splitter logic is available at Splitter.

Enabling user actions

If you want, your QueueMetrics system can send login/logoff actions back to your FreeSwitch server. An explanation of how this works at AMI Feedback.

In QueueMetrics you need to set:

callfile.dir=fsw:ClueCon@127.0.0.1
default.webloaderpbx=true
platform.pbx=FREESWITCH

You do not need to include any dial-plan, as actions work directly.

In QueueMetrics, you also need to enter the "External Reference ID" identifier in the Agent (and possibly Queue) page, as this code will be used to generate ESL login/logoff commands. The external reference for queues and agents is easily found by running uniloader pbxinfo fusionpbx, as explained in PbxInfo for FusionPBX.