Differences

This shows you the differences between two versions of the page.

--- en:dpi:dpi_components:platform:mon_stat:start [2024/09/26 08:41] – elena.krasnobryzh
+++ en:dpi:dpi_components:platform:mon_stat:start [Unknown date] (current) – removed - external edit (Unknown date) 127.0.0.1
@@ Line 1: / Line 1: @@
-====== FastDPI monitoring and logs ======
-{{indexmenu_n>12}}
-System logs are presented as text files that are located in the /var/log/dpi directory for DPI and PCRF modules.
-Types of messages in the log:
-  - [CRITICAL] - critical error, system operation is impossible without troubleshooting
-  - [WARNING] - warning, the system does not stop, but it is worth eliminating this malfunction
-  - [TRACE] - messages when the diagnostic trace mode is enabled
-  - [INFO] - notification of system actions
-  - [ERROR] - error when connecting services and policies, incorrect configuration
-The FastDPI process by default logs all system actions to the following debug and statistics log files:
-  - [[en:dpi:dpi_components:platform:mon_stat:slave_log:start|/var/log/dpi/fastdpi_slave.log]] - a log of traffic processing processes((For each handler, its own fastdpi_slave log is created, other log files are created in a single copy.))
-  - [[en:dpi:dpi_components:platform:mon_stat:stat_log:start| /var/log/dpi/fastdpi_stat.log]] - traffic processing statistics log
-  - [[en:dpi:dpi_components:platform:mon_stat:alert_log:start| /var/log/dpi/fastdpi_alert.log]] - common fastDPI functions log
-[[en:dpi:dpi_components:platform:mon_stat:admin_stat_log_block:start| Blocking counters that are saved in the statistics log]]
-===== Log file rotation =====
-File rotation provides a daily backup of the daily log. By default, this process is performed during the hours with the lowest system load. The log storage depth is defined in the configuration of ''/etc/logrotate.d/fastdpi'' by the parameter ''maxage'', the value is specified **in days**.
-=====Monitoring SNMPD=====
-====Description====
-SNMP is a protocol that allows retrieving information about the DPI server for centralized monitoring of critical parameters.\\
-**How it works:** an SNMP request is sent to the server, it retrieves the necessary information and sends it to the monitoring server. The monitoring server receives and processes this information, allowing to understand the server's status—tracking at which levels various parameters and data are located.
-The ''bngsnmp'' package contains scripts needed to form the VENDOR branch of OIDs for SNMP polling.\\
-It is based on the ''snmp_passpersist'' library. Vendor branch OID: ''.1.3.6.1.4.1.43823''
-Contents of the vendor branch:
-  * CPU statistics, including core utilization by BNG processes;
-  * number of illegitimate drops;
-  * drop statistics on the dispatcher;
-  * availability list of RADIUS servers (**when using a proxy or load balancer, this proxy will be shown in the statistics**);
-  * number of DHCP subscribers — **when the function is active**;
-  * list of VRFs and active subscribers in each — **when the function is active**;
-  * DPDK interface statistics (number of packets, errors, signal level, etc.);
-  * NAT statistics (list of profiles, list of pools, and number of translations for each public address) — **when the function is active**;
-  * list of hardware modules installed on the motherboard — **if supported by the BMC controller**;
-  * metrics from hardware sensors (voltage, power consumption, FAN RPMs) — **if supported by the BMC controller**;
-  * utilization of DHCP server pools (supported __only by KEA-DHCP__ server) — **when the function is active**.
-This script also overrides standard branches by adding statistics from DPDK interfaces:
-  * ''.1.3.6.1.2.1.2.2.1'' — 32-bit counters
-  * ''.1.3.6.1.2.1.31.1.1.1'' — 64-bit counters
-If necessary, you can disable the override by commenting out or deleting the following lines from ''/etc/snmp/snmpd.conf'':
-<code bash>pass_persist    .1.3.6.1.2.1.2.2.1      /usr/local/bin/bng_snmp/vas_pp.py
-pass_persist    .1.3.6.1.2.1.31.1.1.1   /usr/local/bin/bng_snmp/vas_pp_hc.py</code>
-<note>{{ :dpi:dpi_components:platform:mon_stat:vasexperts-mib.mib |MIB File}}</note>
-====Installation====
-  - Installation: <code bash>yum install bngsnmp</code>
-  - Configuration: <code bash>/bin/cp -f /etc/snmp/snmpd.conf.example /etc/snmp/snmpd.conf</code>
-  - Restart the net-snmp service: <code bash>systemctl restart snmpd</code>
-  - Disable selinux (in case of error when starting SNMP): <code bash>vi /etc/selinux/config
-...
-SELINUX=disabled
-...</code>
-====Checking SNMPD and vendor branch functionality====
-The snmpwalk utility allows you to check whether data is being received via SNMP:
-  - Install snmpwalk: <code bash>dnf install net-snmp-utils</code>
-  - Command to check SNMP functionality: <code bash>snmpwalk -v 2c -c nokiamon localhost -On .1.3.6.1.4.1.43823</code>
-====How to read the MIB file====
-If you need to see the MIB file in a graphical representation, you can open it through the MIB Browser. It will describe the entire vendor branch and its OIDs.\\
-The example below shows:
-  - Vendor branch file
-  - Its OID
-{{:dpi:dpi_components:platform:mon_stat:mib_browser.png?nolink&800|}}
-<note>You can also create your own walk in the MIB Browser by specifying the server address</note>
-====Known limitations====
-On-stick device support is available starting from version 13.2-beta4.2.
-===== Monitoring via SNMP agent (Zabbix-agent) =====
-We offer you the following set of parameters that can be taken from the SSG DPI:
-  * Errors in fastDPI process log ''/var/log/dpi/fastdpi_alert.log''
-  * Errors in the ''/var/log/messages'' system log
-  * Losses (Drop) on dna interfaces
-  * Traffic volume on interfaces
-  * Availability of control interfaces
-  * Number of HTTP and HTTPS requests processed
-  * Number of blocked resources by HTTP, HTTPS, IP
-  * Number of PPPoE sessions
-You can use Zabbix Agent for monitoring.\\
-Current and final supported version of agent and server is 6.0, Zabbix agent 1 should be used. For newer versions of Zabbix, monitoring will be done via SNMP.
-====Agent Setup====
-  - Install Zabbix agent 1 on the DPI server according to the [[https://www.zabbix.com/download?zabbix=6.0&os_distribution=centos&os_version=8&components=agent&db=&ws=|instructions on the Zabbix website]].\\ In the first step, select the following values:
-    * Zabbix Packages
-    * Zabbix version: 6.0+
-    * OS distribution: CentOS
-    * OS version: 8 STREAM
-    * Zabbix component: AGENT
-  - Edit the configuration file ''/etc/zabbix/zabbix_agentd.conf'': change the parameters ''Server='' and ''ServerActive='' to your server address, ''hostname='' to the server hostname.
-  - Change the context of the ''/var/log/dpi/fastdpi_stat.log'' file: <code bash>chcon unconfined_u:object_r:zabbix_log_t:s0 /var/log/dpi/fastdpi_stat.log</code>
-  - Open tcp/udp ports 10050 and 10051 in firewall
-  - Upload the {{ :dpi:dpi_components:platform:mon_stat:ssg_userparams.conf |}} file to the ''/etc/zabbix/zabbix_agent.d/'' directory
-  - Edit the ''ssg_userparams.conf'' file by replacing the interface number in ''UserParameter''\\ **''02-00.0'' should be replaced with the interface names of your server!\\ The name must match the DPI config. If you have more than 2 interfaces in use, you must add a line similar to the existing parameters.** <code bash>UserParameter=dpi.02-00.0.drops,tac /var/log/dpi/fastdpi_stat.log | sed /'IF 02-00.0'/q | tac | sed -e 1,/'Actual   Stats'/d | sed '6!D' | awk '{print $1}' | sed 's/^.//'</code>
-  - Restart the agent: ''systemctl restart Zabbix-agent''
-====Server setup====
-  - Install and configure Zabbix servers according to the  [[https://www.zabbix.com/ru/download?zabbix=6.0&os_distribution=centos&os_version=8&components=agent&db=&ws=|instructions]]  on the official website.
-  - Add template {{ :dpi:dpi_components:platform:mon_stat:zbx_export_templates.xml |}}\\ 1) Go to Configuration section\\ 2) Templates section\\ 3) Click “Import”\\ 4) Import template file\\ 5) Save changes\\ {{:dpi:dpi_components:platform:mon_stat:import_tmp.png?nolink&1200|}}
-  - Add a DPI server as a host\\ 1) Go to Monitoring section\\ 2) Hosts section\\ 3) Click “Create host”\\ 4) Set the required parameters, hostname, group and the previously added template\\ 5) Save changes\\ {{:dpi:dpi_components:platform:mon_stat:create_host.png?nolink&1200|}}
-  - Edit the template: change the names of the interfaces and keys so that they match the ''UserParameter''.
-===== Monitoring traffic distribution by class =====
-SSG allows traffic distribution by class to be monitored.
-. Enable traffic prioritization. For the example, we will use the following prioritization rules:
-<code bash>
-dns cs0
-http cs0
-https cs0
-Bittorrent cs7
-ICMP cs0
-TCP Unknown cs7
-GOOGLEVIDEO cs1
-default cs2
-</code>
-. In the ''/etc/dpi/fastdpi.conf'' configuration, set the parameter:
-<code bash>dbg_log_mask=0x4</code>
-. Enable common channel polysync (the example shown is polysync with full channel width restriction):
-<code bash>
-htb_inbound_root=rate 1300mbit
-htb_inbound_class0=rate 8bit ceil 1300mbit
-htb_inbound_class1=rate 8bit ceil 1300mbit
-htb_inbound_class2=rate 8bit ceil 1300mbit
-htb_inbound_class3=rate 8bit ceil 1300mbit
-htb_inbound_class4=rate 8bit ceil 1300mbit
-htb_inbound_class5=rate 8bit ceil 1300mbit
-htb_inbound_class6=rate 8bit  ceil 1300mbit
-htb_inbound_class7=rate 8bit  ceil 1300mbit
-htb_root=rate 1300mbit
-htb_class0=rate 8bit ceil 1300mbit
-htb_class1=rate 8bit ceil 1300mbit
-htb_class2=rate 8bit ceil 1300mbit
-htb_class3=rate 8bit ceil 1300mbit
-htb_class4=rate 8bit ceil 1300mbit
-htb_class5=rate 8bit ceil 1300mbit
-htb_class6=rate 8bit  ceil 1300mbit
-htb_class7=rate 8bit  ceil 1300mbit
-</code>
-. Update the configuration:
-<code bash>
-service fastdpi reload
-</code>
-<note important>
-If polyscing for a shared channel is applied for the first time, you must restart the service:
-<code bash>service fastdpi restart</code>
-</note>
-. Use the following custom settings for the zabbix agent installed on the SSG:
-{{ :dpi:dpi_components:platform:mon_stat:ssg_userparams.conf |}}
-. Import the template to the Zabbix server as described in the section [[en:dpi:dpi_components:platform:mon_stat:start#server_setup|"Monitoring via SNMP agent"]]:
-{{ :dpi:dpi_components:platform:mon_stat:zbx_export_templates.xml |}}
-<note important>If necessary, change the interface names in the template and in the custom parameter file</note>
-==== View flow and protocol statistics ====
-=== By flow ===
-  - IPv4/IPv6
-  - protocol type: 0 - IPv4, 1 - IPv6
-  - total allocated records
-  - a queue with a short lifespan:\\
-    - occupied records\\
-    - reusable\\
-    - difference 3.1 - 3.2 (number of active flows)\\
-  - also for the long line\\
-  - also total\\
-//Example://\\
-<code ini>fdpi_ctrl stat --flow
-IPv4 0 6784000 834 814 20 0 0 0 834 814 20</code>
-=== By protocols ===
-  - internal index of protocol statistics\\
-  - protocol name\\
-  - protocol port number\\ //direction subs --> inet//\\
-  - number of packages\\
-  - volume in bytes ip total\\
-  - dropped packages\\
-  - dropped byte\\ //direction inet --> subs number of packages etc.//\\
-//Example://\\
-<code ini>fdpi_ctrl stat --proto
-Autodetected fastdpi params : dev='em1', port=29001
-connecting 94.140.198.68:29001 ...
-================================
-'ntp' 123 0 0 0 0 91 23569 0 0
-'sip' 5060 0 0 0 0 2479 1170579 0 0
-'Bittorrent' 49165 0 0 0 0 0 0 3 495
-'ICMP' 65025 0 0 0 0 225 18900 0 0
-'TCP Unknown' 65030 0 0 0 0 41034 3448836 0 0
-'UDP Unknown' 65041 3900 4227600 0 0 277 24825 0 0
-'ARP' 65282 30 2520 0 0 30 2520 0 0
-'CHAMELEON' 49236 0 0 0 0 589 72475 0 0</code>

Profile

Settings

Differences