Configuring IPFIX receivers [Документация VAS Experts]

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
en:dpi:dpi_components:qoestor:configuration:ipfix_receivers:start [2023/10/10 08:25] – external edit 127.0.0.1en:dpi:dpi_components:qoestor:configuration:ipfix_receivers:start [Unknown date] (current) – removed - external edit (Unknown date) 127.0.0.1
Line 1: Line 1:
-====== Configuring IPFIX receivers ====== 
-{{indexmenu_n>3}} 
- 
-Configuring ipfix receivers via the .env file  \\ <code>/var/qoestor/backend/.env</code> 
- 
-The standard configuration looks like this \\ <code> 
-#Ipfix form DPI 0 
-IPFIX_FULLFLOW_PORT_TYPE[0]=tcp 
-IPFIX_FULLFLOW_PORT[0]=1500 
-#IPFIX_FULLFLOW_ROTATE_MINUTES[0]=10 
-#IPFIX_FULLFLOW_ROTATE_DELAY_SECONDS[0]=0 
-#IPFIX_FULLFLOW_FW_MAX_QUEUE_SIZE[0]=10 
-#IPFIX_FULLFLOW_DUMP_INSERT_PROCESSES[0]=0 
-#IPFIX_FULLFLOW_EXPORT[0]=10.0.0.2/9920/tcp,10.0.0.3/3440/udp 
- 
-IPFIX_CLICKSTREAM_PORT_TYPE[0]=tcp 
-IPFIX_CLICKSTREAM_PORT[0]=1501 
-#IPFIX_CLICKSTREAM_ROTATE_MINUTES[0]=12 
-#IPFIX_CLICKSTREAM_ROTATE_DELAY_SECONDS[0]=400 
-#IPFIX_CLICKSTREAM_FW_MAX_QUEUE_SIZE[0]=10 
-#IPFIX_CLICKSTREAM_DUMP_INSERT_PROCESSES[0]=0 
-#IPFIX_CLICKSTREAM_EXPORT[0]=10.0.0.2/9921/tcp,10.0.0.3/3441/udp 
- 
-IPFIX_GTPFLOW_PORT_TYPE[0]=tcp 
-IPFIX_GTPFLOW_PORT[0]=1502 
-#IPFIX_GTPFLOW_ROTATE_MINUTES[0]=10 
-#IPFIX_GTPFLOW_ROTATE_DELAY_SECONDS[0]=0 
-#IPFIX_GTPFLOW_FW_MAX_QUEUE_SIZE[0]=10 
-#IPFIX_GTPFLOW_DUMP_INSERT_PROCESSES[0]=0 
-#IPFIX_GTPFLOW_EXPORT[0]=10.0.0.2/9921/tcp,10.0.0.3/3441/udp 
- 
-IPFIX_NATFLOW_PORT_TYPE[0]=tcp 
-IPFIX_NATFLOW_PORT[0]=1503 
-#IPFIX_NATFLOW_ROTATE_MINUTES[0]=10 
-#IPFIX_NATFLOW_ROTATE_DELAY_SECONDS[0]=0 
-#IPFIX_NATFLOW_FW_MAX_QUEUE_SIZE[0]=10 
-#IPFIX_NATFLOW_DUMP_INSERT_PROCESSES[0]=0 
-#IPFIX_NATFLOW_EXPORT[0]=10.0.0.2/9921/tcp,10.0.0.3/3441/udp 
- 
-#Traffic direction definition 
-# 0 - as is 
-# 1 - by AS (for fullflow only) 
-# 2 - by CIDR (for fullflow and clickstream) 
-# 3 - by both: AS and CIDR 
-# 4 - any: AS or CIDR 
-TRAFFIC_DIR_DEF_MODE=0 
- 
-#Subscriber filter 
-# 0 - no filter 
-# 1 - by AS (for fullflow only) 
-# 2 - by CIDR (for fullflow and clickstream) 
-# 3 - by both: AS and CIDR 
-# 4 - any: AS or CIDR 
-SUBSCRIBER_FILTER_MODE=0 
- 
-#Subscriber exclude 
-# 0 - no exclude 
-# 1 - by AS (for fullflow only) 
-# 2 - by CIDR (for fullflow and clickstream) 
-# 3 - by both: AS and CIDR 
-# 4 - any: AS or CIDR 
-SUBSCRIBER_EXCLUDE_MODE=0 
- 
-#Enable host (url) categories dics autoload 
-URLS_CATEGORIES_DIC_AUTOLOAD_ENABLED=1 
- 
-#Enable asnum dic autoload 
-ASNUM_DIC_AUTOLOAD_ENABLED=1 
- 
-#Enable auto replacing Login with vchannel on insert 
-# 0 - Disabled 
-# 1 - Enabled 
-# 2 - Enabled if Login is empty 
-ULR_REPLACE_LOGIN_WITH_VCHANNEL=0 
- 
-# Use dictionary when replacing login 
-ULR_USE_DIC_WHEN_REPLACING_LOGIN=0 
- 
-# Enable autoload of vchannel_name_dic 
-ULR_VCHANNEL_NAME_DIC_AUTOLOAD_ENABLED=0 
- 
-# vchannel_name_dic remote url 
-ULR_VCHANNEL_NAME_DIC_URL= 
- 
-#Import NAT events from fullflow 
-NAT_IMPORT_FROM_FULLFLOW 
-# 0 - Disabled 
-# 1 - Enabled 
- 
-#Fields to save when aggregating NAT log (bitmask) 
-# 0x1 - Save protocol ID 
-# 0x2 - Save event type, 
-# 0x4 - Save source ipv4, 
-# 0x8 - Save source port, 
-# 0x10 - Save destination ipv4, 
-# 0x20 - Save destination port, 
-# 0x40 - Save post NAT source ipv4, 
-# 0x80 - Save post NAT source_port, 
-# 0x100 - Save session ID, 
-# 0x200 - Save login, 
-# 0x400 - Save DPI ID 
-NAT_AGG_LOG_FIELDS_TO_SAVE_BITMASK=0 
- 
-#Time interval for aggregating NAT logs 
-NAT_AGG_LOG_GROUP_TIME_INTERVAL 
-# 1 - 1 minute 
-# 5 - 5 minutes 
-# 10 - 10 minutes 
-# 15 - 15 minutes 
-# 30 - 30 minutes 
-# 60 - 60 minutes 
-</code> 
- 
-In the presented configuration, the launch of fullflow and clickstream receivers is configured on tcp sockets 1500 and 1501, respectively. <<0>> in the array index means that reception is coming from DPI 0. 
- 
-<note tip>It is better to use tcp, because for udp packets can be lost when the MTU is exceeded. </note> 
- 
-**Parameter list** 
- 
-  * IPFIX_FULLFLOW_PORT_TYPE[i] и IPFIX_CLICKSTREAM_PORT_TYPE[i] determine the type of traffic received on the port: tcp or udp. It is recommended to install tcp.  
- 
-  * IPFIX_FULLFLOW_PORT[i] и IPFIX_CLICKSTREAM_PORT[i] determine the port number.  
-   
-  * TRAFFIC_DIR_DEF_MODE и SUBSCRIBER_FILTER_MODE defines the subscriber filtering mode according to the asnum_local_dic and subnets_local_dic directories. TRAFFIC_DIR_DEF_MODE = 0 and SUBSCRIBER_FILTER_MODE = 0 mean that there is no need to calculate traffic direction and filter subscribers.  
-   
-  * SUBSCRIBER_EXCLUDE_MODE defines the subscriber filtering mode according to the asnum_exclude_diс and subnets_exclude_dic directories. SUBSCRIBER_EXCLUDE_MODE = 0 means no filtering is required.  
-   
-  * IPFIX_FULLFLOW_EXPORT[i] and IPFIX_CLICKSTREAM_EXPORT[i] make it possible to configure export to third-party receivers. Format ip/port/proto[,ip/port/proto]. 
-   
-  * IPFIX_FULLFLOW_ROTATE_MINUTES[i] и IPFIX_CLICKSTREAM_ROTATE_MINUTES[i] make it possible to configure the period of rotation of dumps and write them to the database. By default, this is 10 minutes for fullflow and 12 minutes for clickstream.  
-   
-  * IPFIX_FULLFLOW_ROTATE_DELAY_SECONDS[i] и IPFIX_CLICKSTREAM_ROTATE_DELAY_SECONDS[i] make it possible to configure the delay for inserting data for a certain number of seconds. The default for fullflow is 0 seconds, for clickstream it is 400 seconds. The latency for clickstream relative to fullflow is needed to ensure that the fullflow and clickstream logs are connected to enrich statistical reports.  
-   
-  * IPFIX_FULLFLOW_FW_MAX_QUEUE_SIZE[i] и IPFIX_CLICKSTREAM_FW_MAX_QUEUE_SIZE[i] determine the maximum queue size on receivers. Better not to touch.  
- 
- 
-<note important>If the configuration has changed, you need to run <code>fastor-restart</code></note> 
- 
-**The following configuration example allows you to configure reception from multiple DPI** \\ <code> 
-#Ipfix form DPI 0 
-IPFIX_FULLFLOW_PORT_TYPE[0]=tcp 
-IPFIX_FULLFLOW_PORT[0]=1500 
- 
-IPFIX_CLICKSTREAM_PORT_TYPE[0]=tcp 
-IPFIX_CLICKSTREAM_PORT[0]=1501 
- 
-#Ipfix form DPI 1 
-IPFIX_FULLFLOW_PORT_TYPE[1]=tcp 
-IPFIX_FULLFLOW_PORT[1]=1510 
- 
-IPFIX_CLICKSTREAM_PORT_TYPE[1]=tcp 
-IPFIX_CLICKSTREAM_PORT[1]=1511 
- 
-#Ipfix form DPI 2 
-IPFIX_FULLFLOW_PORT_TYPE[2]=tcp 
-IPFIX_FULLFLOW_PORT[2]=1520 
- 
-IPFIX_CLICKSTREAM_PORT_TYPE[2]=tcp 
-IPFIX_CLICKSTREAM_PORT[2]=1521 
-</code> 
- 
-**An example of a configuration when subscriber identification by CIDR is required** 
- 
-This configuration is relevant in cases when the DPI is installed on the mirror. 
- 
-<code> 
-TRAFFIC_DIR_DEF_MODE=2 
-SUBSCRIBER_FILTER_MODE=2 
-</code> 
- 
-Don't forget to set up the subnets_local_dic reference for this configuration example!  
- 
-**Configuration example when export to third-party receivers is configured** 
- 
-<code> 
-IPFIX_FULLFLOW_PORT_TYPE[0]=tcp 
-IPFIX_FULLFLOW_PORT[0]=1500 
-IPFIX_FULLFLOW_EXPORT[0]=10.0.0.2/1600/tcp 
- 
-IPFIX_CLICKSTREAM_PORT_TYPE[0]=tcp 
-IPFIX_CLICKSTREAM_PORT[0]=1501 
-IPFIX_CLICKSTREAM_EXPORT[0]=10.0.0.2/1601/tcp 
-</code> 
- 
-== Restarting receivers  == 
- 
-Restarting all receivers can be done with the command  <code>fastor-restart</code> 
- 
-If you need to restart the receivers separately, this can be done by restarting the services, for example 
- 
-  * For CentOS 7 <code>systemctl restart qoestor_fullflow_0.service 
-systemctl restart qoestor_clickstream_0.service</code> 
-  * For CentOS 6 <code>service qoestor_fullflow_0 stop 
-service qoestor_clickstream_0 stop 
-service qoestor_fullflow_0 start 
-service qoestor_clickstream_0 start</code> 
- 
-== Stopping receivers  == 
-  * For CentOS 7 <code>systemctl stop qoestor_fullflow_0.service 
-systemctl stop qoestor_clickstream_0.service</code> 
-  * For CentOS 6 <code>service qoestor_clickstream_0 stop 
-service qoestor_fullflow_0 stop</code> 
- 
-== Stopping and starting the clickhouse database == 
-  * Stopping <code>fastor-db-stop</code> 
-  * Starting <code>fastor-db-restart</code>