xHTTP_PROM Module

Vicente Hernando

   <vhernando@sonoc.io>

Edited by

Vicente Hernando

   <vhernando@sonoc.io>

Javier Gallart

   <jgallart@sonoc.io>

   Copyright © 2019 www.sonoc.io
     __________________________________________________________________

   Table of Contents

   1. Admin Guide

        1. Overview
        2. Dependencies

              2.1. Kamailio Modules
              2.2. External Libraries or Applications

        3. Parameters

              3.1. xhttp_prom_buf_size (integer)
              3.2. xhttp_prom_timeout (integer)
              3.3. xhttp_prom_stats (str)
              3.4. prom_counter (str)
              3.5. prom_gauge (str)

        4. Functions

              4.1. prom_counter_reset(name, l0, l1, l2)
              4.2. prom_gauge_reset(name, l0, l1, l2)
              4.3. prom_counter_inc(name, number, l0, l1, l2)
              4.4. prom_gauge_set(name, number, l0, l1, l2)
              4.5. prom_dispatch()
              4.6. prom_check_uri()

        5. RPC Commands

              5.1. xhttp_prom.counter_reset
              5.2. xhttp_prom.counter_inc
              5.3. xhttp_prom.gauge_reset
              5.4. xhttp_prom.gauge_set
              5.5. xhttp_prom.metric_list_print

   List of Examples

   1.1. Set xhttp_prom_buf_size parameter
   1.2. Set xhttp_prom_timeout parameter
   1.3. Set xhttp_prom_stats parameter
   1.4. prom_counter label example
   1.5. Set prom_counter parameter
   1.6. prom_gauge label example
   1.7. Set prom_gauge parameter
   1.8. prom_counter_reset usage
   1.9. prom_gauge_reset usage
   1.10. prom_counter_inc usage
   1.11. prom_gauge_set usage
   1.12. prom_dispatch usage
   1.13. prom_dispatch usage (more complete)
   1.14. prom_check_uri usage
   1.15. xhttp_prom.counter_reset usage
   1.16. xhttp_prom.counter_inc usage
   1.17. xhttp_prom.gauge_reset usage
   1.18. xhttp_prom.gauge_set usage
   1.19. xhttp_prom.metric_list_print usage

Chapter 1. Admin Guide

   Table of Contents

   1. Overview
   2. Dependencies

        2.1. Kamailio Modules
        2.2. External Libraries or Applications

   3. Parameters

        3.1. xhttp_prom_buf_size (integer)
        3.2. xhttp_prom_timeout (integer)
        3.3. xhttp_prom_stats (str)
        3.4. prom_counter (str)
        3.5. prom_gauge (str)

   4. Functions

        4.1. prom_counter_reset(name, l0, l1, l2)
        4.2. prom_gauge_reset(name, l0, l1, l2)
        4.3. prom_counter_inc(name, number, l0, l1, l2)
        4.4. prom_gauge_set(name, number, l0, l1, l2)
        4.5. prom_dispatch()
        4.6. prom_check_uri()

   5. RPC Commands

        5.1. xhttp_prom.counter_reset
        5.2. xhttp_prom.counter_inc
        5.3. xhttp_prom.gauge_reset
        5.4. xhttp_prom.gauge_set
        5.5. xhttp_prom.metric_list_print

1. Overview

   This module generates suitable metrics for a Prometheus monitoring
   platform.

   It answers Prometheus pull requests (HTTP requests to /metrics URL).

   The module generates metrics based on Kamailio statistics, and also the
   user can create his own metrics (currently counters and gauges).

   The xHTTP_PROM module uses the xHTTP module to handle HTTP requests.
   Read the documentation of the xHTTP module for more details.

   NOTE: This module is based on xHTTP_RPC one.

   IMPORTANT: This module uses private memory to generate HTTP responses,
   and shared memory to store all the metrics. Remember to increase size
   of private and shared memory if you use a huge amount of metrics.

   Prometheus URLs:
     * https://prometheus.io/
     * https://prometheus.io/docs/concepts/data_model/#metric-names-and-la
       bels
     * https://prometheus.io/docs/instrumenting/exposition_formats/

2. Dependencies

   2.1. Kamailio Modules
   2.2. External Libraries or Applications

2.1. Kamailio Modules

   The following modules must be loaded before this module:
     * xhttp -- xHTTP.

2.2. External Libraries or Applications

   The following libraries or applications must be installed before
   running Kamailio with this module loaded:
     * None

3. Parameters

   3.1. xhttp_prom_buf_size (integer)
   3.2. xhttp_prom_timeout (integer)
   3.3. xhttp_prom_stats (str)
   3.4. prom_counter (str)
   3.5. prom_gauge (str)

3.1. xhttp_prom_buf_size (integer)

   Specifies the maximum length of the buffer (in bytes) used to write the
   metric reply information in order to build the HTML response.

   Default value is 0 (auto set to 1/3 of the size of the configured pkg
   mem).

   Example 1.1. Set xhttp_prom_buf_size parameter
...
modparam("xhttp_prom", "xhttp_prom_buf_size", 1024)
...

3.2. xhttp_prom_timeout (integer)

   Specifies a timeout in minutes. A metric not used during this timeout
   is automatically deleted. Listing metrics does not count as using them.

   Default value is 60 minutes.

   Example 1.2. Set xhttp_prom_timeout parameter
...
# Set timeout to 10 hours
modparam("xhttp_prom", "xhttp_prom_timeout", 600)
...

3.3. xhttp_prom_stats (str)

   Specifies which internal statistics from Kamailio to show. Possible
   values:
     * all - Show whole Kamailio statistics
     * group_name: - Show all statistics for a group
     * statistic_name - Show a specific statistic. It automatically finds
       the group.

   Default value is "", meaning do not display any Kamailio statistics.

   Example 1.3. Set xhttp_prom_stats parameter
...
# show all kamailio statistics.
modparam("xhttp_prom", "xhttp_prom_stats", "all")

# show statistics for sl group.
modparam("xhttp_prom", "xhttp_prom_stats", "sl:")

# Show statistic for 200_replies in sl group.
modparam("xhttp_prom", "xhttp_prom_stats", "200_replies")

# Do not display internal Kamailio statistics. This is the default option.
modparam("xhttp_prom", "xhttp_prom_stats", "")
...

3.4. prom_counter (str)

   Create a counter metric.

   This function declares a counter but the actual counter is only created
   when using it (by adding to or resetting it)

   It takes a list of attribute=value separated by semicolon, the
   attributes can be name and label.
     * name - name of the counter. This attribute is mandatory. It is used
       to generate the metric name. Each name is unique, no metric shall
       repeat a name.
     * label - names of labels in the counter. Optional. Only one label
       parameter at most allowed in counters. Each label name is separated
       by : without spaces. At most only three label names allowed in each
       label parameter.
       Example 1.4. prom_counter label example
# Create two labels called method and handler
label = method:handler
This would generate  {method="whatever", handler="whatever2"} when building
the metric.

   Example 1.5. Set prom_counter parameter
...

# Create cnt_first counter with no labels.
modparam("xhttp_prom", "prom_counter", "name=cnt_first;");

# Create cnt_second counter with no labels.
modparam("xhttp_prom", "prom_counter", "name=cnt_second;");


# Create cnt_third counter with label method
modparam("xhttp_prom", "prom_counter", "name=cnt_third; label=method");

These lines declare the counter but the actual metric will be created when
using it by prom_counter_inc or prom_counter_reset functions.

...

3.5. prom_gauge (str)

   Create a gauge metric.

   This function declares the gauge but the actual gauge is only created
   when using it (by setting or resetting it)

   It takes a list of attribute=value separated by semicolon, the
   attributes can be name and value.
     * name - name of the gauge. This attribute is mandatory. It is used
       to generate the metric name. Each name is unique, no metric shall
       repeat a name.
     * label - names of labels in the gauge. Optional. Only one label
       parameter at most allowed in gauges. Each label name is separated
       by : without spaces. At most only three label names allowed inside
       each label parameter.
       Example 1.6. prom_gauge label example
# Create two labels called method and handler
label = method:handler
This would generate  {method="whatever", handler="whatever2"} when building
the metric.

   Example 1.7. Set prom_gauge parameter
...

# Create gg_first gauge with no labels
modparam("xhttp_prom", "prom_gauge", "name=gg_first;");

# Create gg_second gauge with no labels
modparam("xhttp_prom", "prom_gauge", "name=gg_second;");


# Create gg_third gauge with two labels method and handler:
modparam("xhttp_prom", "prom_gauge", "name=gg_third; label=method:handler;");

...

4. Functions

   4.1. prom_counter_reset(name, l0, l1, l2)
   4.2. prom_gauge_reset(name, l0, l1, l2)
   4.3. prom_counter_inc(name, number, l0, l1, l2)
   4.4. prom_gauge_set(name, number, l0, l1, l2)
   4.5. prom_dispatch()
   4.6. prom_check_uri()

4.1.  prom_counter_reset(name, l0, l1, l2)

   Get a counter based on its name and labels and reset its value to 0.
   Name parameter is mandatory. Values of labels are optional (from none
   up to three). Name in prom_counter_reset has to match same name in
   prom_counter parameter. Number of labels in prom_counter_reset has to
   match number of labels in prom_counter parameter. First time a counter
   is used with this reset function the counter is created if it does not
   exist already.

   This function accepts pseudovariables on its parameters.

   Available via KEMI framework as counter_reset_l0, counter_reset_l1,
   counter_reset_l2 and counter_reset_l3.

   Example 1.8. prom_counter_reset usage
...
# Definition of counter with prom_counter with labels method and IP
modparam("xhttp_prom", "prom_counter", "name=cnt01; label=method:IP;");
...
# Reset cnt01 counter with two values "push" and "192.168.0.1" in labels to zero
.
# First time we execute this function the counter will be created.
prom_counter_reset("cnt01", "push", "192.168.0.1");
...
# A metric like this will appear when listing this counter:
kamailio_cnt01 {method="push", IP="192.168.0.1"} 0 1234567890
...

4.2.  prom_gauge_reset(name, l0, l1, l2)

   Get a gauge based on its name and labels and reset its value to 0. Name
   parameter is mandatory. Values of labels are optional (from none up to
   three). Name in prom_gauge_reset has to match same name in prom_gauge
   parameter. Number of labels in prom_gauge_reset has to match number of
   labels in prom_gauge parameter. First time a gauge is used with this
   reset function the gauge is created if it does not exist already.

   This function accepts pseudovariables on its parameters.

   Available via KEMI framework as gauge_reset_l0, gauge_reset_l1,
   gauge_reset_l2 and gauge_reset_l3.

   Example 1.9. prom_gauge_reset usage
...
# Definition of gauge with prom_gauge with labels method and IP
modparam("xhttp_prom", "prom_gauge", "name=cnt01; label=method:IP;");
...
# Reset cnt01 gauge with two values "push" and "192.168.0.1" in labels to zero.
# First time we execute this function the gauge will be created.
prom_gauge_reset("cnt01", "push", "192.168.0.1");
...
# A metric like this will appear when listing this gauge:
kamailio_cnt01 {method="push", IP="192.168.0.1"} 0 1234567890
...

4.3.  prom_counter_inc(name, number, l0, l1, l2)

   Get a counter identified by its name and labels and increase its value
   by a number. If counter does not exist it creates the counter,
   initializes it to zero and adds the number.

   Name is mandatory, number is mandatory. Number has to be positive or
   zero (integer). l0, l1, l2 are values of labels and are optional.

   name value and number of labels have to match a previous counter
   definition with prom_counter.

   This function accepts pseudovariables on its parameters.

   Available via KEMI framework as counter_inc_l0, counter_inc_l1,
   counter_inc_l2 and counter_inc_l3.

   Example 1.10. prom_counter_inc usage
...
# Definition of cnt01 counter with no labels.
modparam("xhttp_prom", "prom_counter", "name=cnt01;");
...
# Add 10 to value of cnt01 counter (with no labels) If counter does not exist it
 gets created.
prom_counter_inc("cnt01", "10");
...

# Definition of cnt02 counter with two labels method and IP
modparam("xhttp_prom", "prom_counter", "name=cnt02; label=method:IP;");
...
# Add 15 to value of cnt02 counter with labels method and IP. It creates the cou
nter if it does not exist.
prom_counter_inc("cnt02", "15", "push", "192.168.0.1");
# When listed the metric it will show a line like this:
kamailio_cnt02 {method="push", IP="192.168.0.1"} 15 1234567890
...

4.4.  prom_gauge_set(name, number, l0, l1, l2)

   Get a gauge identified by its name and labels and set its value to a
   number. If gauge does not exist it creates the gauge and assigns the
   value to it.

   Name is mandatory, number is mandatory. Number is a string that will be
   parsed as a float. l0, l1, l2 are values of labels and are optional.

   name value and number of labels have to match a previous gauge
   definition with prom_gauge.

   This function accepts pseudovariables on its parameters.

   Available via KEMI framework as gauge_set_l0, gauge_set_l1,
   gauge_set_l2 and gauge_set_l3.

   Example 1.11. prom_gauge_set usage
...
# Definition of gg01 gauge with no labels.
modparam("xhttp_prom", "prom_gauge", "name=gg01;");
...
# Assign -12.5 to value of gg01 gauge (with no labels) If gauge does not exist i
t gets created
prom_gauge_set("gg01", "-12.5");
...

# Definition of gg02 gauge with two labels method and IP
modparam("xhttp_prom", "prom_gauge", "name=cnt02; label=method:IP;");
...
# Assign 2.8 to value of gg02 gauge with labels method and IP. It creates the ga
uge if it does not exist.
prom_gauge_set("gg02", "2.8", "push", "192.168.0.1");
# When listed the metric it will show a line like this:
kamailio_gg02 {method="push", IP="192.168.0.1"} 2.8 1234567890
...

4.5.  prom_dispatch()

   Handle the HTTP request and generate a response.

   Available via KEMI framework as xhttp_prom.dispatch

   Example 1.12. prom_dispatch usage
...
# Needed to use SIP frames as HTTP ones.
tcp_accept_no_cl=yes
...
# xhttp module depends on sl one.
loadmodule "sl.so"
loadmodule "xhttp.so"
loadmodule "xhttp_prom.so"
...
# show all kamailio statistics.
modparam("xhttp_prom", "xhttp_prom_stats", "all")
...
event_route[xhttp:request] {
        $var(xhttp_prom_root) = $(hu{s.substr,0,8});
        if ($var(xhttp_prom_root) == "/metrics")
                prom_dispatch();
        else
                xhttp_reply("200", "OK", "text/html",
                        "<html><body>Wrong URL $hu</body></html>");
}
...

   Example 1.13. prom_dispatch usage (more complete)

   Another example with counters and gauge:
...
# Needed to use SIP frames as HTTP ones.
tcp_accept_no_cl=yes

# xhttp module depends on sl one.
loadmodule "sl.so"
loadmodule "xhttp.so"
loadmodule "xhttp_prom.so"

# show Kamailio statistics for sl group
modparam("xhttp_prom", "xhttp_prom_stats", "sl:")

# Define two counters and a gauge
modparam("xhttp_prom", "prom_counter", "name=cnt_first;");
modparam("xhttp_prom", "prom_counter", "name=cnt_second; label=method:IP");
modparam("xhttp_prom", "prom_gauge", "name=gg_first; label=handler");

event_route[xhttp:request] {
        $var(xhttp_prom_root) = $(hu{s.substr,0,8});
        if ($var(xhttp_prom_root) == "/metrics") {
            prom_counter_reset("cnt_first");
                prom_counter_inc("cnt_second", "10", "push", "192.168.0.1");
                prom_gauge_set("gg_first", "5.2", "my_handler");
                prom_dispatch();
        } else
                xhttp_reply("200", "OK", "text/html",
                        "<html><body>Wrong URL $hu</body></html>");
}
...

We can manually check the result with a web browser:
We assume Kamailio runs in localhost and port is set to default (same as SIP: 50
60)
http://localhost:5060
...

# User defined metrics
kamailio_cnt_first 0 1554839325427
kamailio_cnt_second {method="push", IP="192.168.0.1"} 10 1554839325427
kamailio_gg_first{handler="my_handler"} 5.2 1554839325427

# Kamailio internal statistics
kamailio_sl_1xx_replies 0 1554839325427
kamailio_sl_200_replies 15 1554839325427
kamailio_sl_202_replies 0 1554839325427
kamailio_sl_2xx_replies 0 1554839325427
kamailio_sl_300_replies 0 1554839325427
kamailio_sl_301_replies 0 1554839325427
kamailio_sl_302_replies 0 1554839325427
kamailio_sl_3xx_replies 0 1554839325427
kamailio_sl_400_replies 0 1554839325427
kamailio_sl_401_replies 0 1554839325427
kamailio_sl_403_replies 0 1554839325427
kamailio_sl_404_replies 0 1554839325427
kamailio_sl_407_replies 0 1554839325427
kamailio_sl_408_replies 0 1554839325427
kamailio_sl_483_replies 0 1554839325427
kamailio_sl_4xx_replies 0 1554839325427
kamailio_sl_500_replies 0 1554839325427
kamailio_sl_5xx_replies 0 1554839325427
kamailio_sl_6xx_replies 0 1554839325427
kamailio_sl_failures 0 1554839325427
kamailio_sl_received_ACKs 0 1554839325427
kamailio_sl_sent_err_replies 0 1554839325427
kamailio_sl_sent_replies 15 1554839325427
kamailio_sl_xxx_replies 0 1554839325461
...

4.6.  prom_check_uri()

   Check if path of HTTP URL equals /metrics. This avoids us to check hu
   variable from xHTTP module.

   NOTE: Remember not to block /metrics URL in xHTTP module

   Available via KEMI framework as xhttp_prom.check_uri

   Example 1.14. prom_check_uri usage
...
# Needed to use SIP frames as HTTP ones.
tcp_accept_no_cl=yes
...
# xhttp module depends on sl one.
loadmodule "sl.so"
loadmodule "xhttp.so"
loadmodule "xhttp_prom.so"
...
# show all kamailio statistics.
modparam("xhttp_prom", "xhttp_prom_stats", "all")
...
event_route[xhttp:request] {
        if (prom_check_uri())
                prom_dispatch();
        else
                xhttp_reply("200", "OK", "text/html",
                        "<html><body>Wrong URL $hu</body></html>");
}
...

5. RPC Commands

   5.1. xhttp_prom.counter_reset
   5.2. xhttp_prom.counter_inc
   5.3. xhttp_prom.gauge_reset
   5.4. xhttp_prom.gauge_set
   5.5. xhttp_prom.metric_list_print

5.1. xhttp_prom.counter_reset

   Set a counter to zero.

   Name: xhttp_prom.counter_reset

   Parameters:
     * name: name of the counter (mandatory)
     * l0: value of the first label (optional)
     * l1: value of second label (optional)
     * l2: value of the third label (optional)

   Example 1.15. xhttp_prom.counter_reset usage
                  ...
                  kamcmd xhttp_prom.counter_reset "cnt01" "push" "192.168.0.1"
                  ...

5.2. xhttp_prom.counter_inc

   Add a number to a counter based on its name and labels.

   Name: xhttp_prom.counter_inc

   Parameters:
     * name: name of the counter (mandatory)
     * number: integer to add to counter value. Negative values not
       allowed.
     * l0: value of the first label (optional)
     * l1: value of second label (optional)
     * l2: value of the third label (optional)

   Example 1.16. xhttp_prom.counter_inc usage
                  ...
                  kamcmd xhttp_prom.counter_inc "cnt01" 15 "push" "192.168.0.1"
                  ...

5.3. xhttp_prom.gauge_reset

   Set gauge value to zero. Select gauge based on its name and labels.

   Name: xhttp_prom.gauge_reset

   Parameters:
     * name: name of the gauge (mandatory)
     * l0: value of the first label (optional)
     * l1: value of second label (optional)
     * l2: value of the third label (optional)

   Example 1.17. xhttp_prom.gauge_reset usage
                  ...
                  kamcmd xhttp_prom.gauge_reset "gg01" "push" "192.168.0.1"
                  ...

5.4. xhttp_prom.gauge_set

   Set a gauge to a number. Select the gauge by its name and labels.

   Name: xhttp_prom.gauge_set

   Parameters:
     * name: name of the gauge (mandatory)
     * number: float value to set the gauge to (mandatory)
     * l0: value of the first label (optional)
     * l1: value of second label (optional)
     * l2: value of the third label (optional)

   Example 1.18. xhttp_prom.gauge_set usage
                  ...
                  kamcmd xhttp_prom.gauge_set "gg01" -- -5.2
                  ...

5.5. xhttp_prom.metric_list_print

   List of all user defined metrics.

   Name: xhttp_prom.metric_list_print

   Parameters:none

   Example 1.19. xhttp_prom.metric_list_print usage
                  ...
                  kamcmd xhttp_prom.metric_list_print
                  ...