Set up NetFlow export by protocols, directions and billing [Документация VAS Experts]
Документация VAS Experts Документация VAS Experts-mobile
in

Settings

  • Show page
  • Old revisions
  • Export to PDF
  • Fold/unfold all
  • Backlinks
  • Show page
  • Old revisions
  • Export to PDF
  • Fold/unfold all
  • Backlinks

    • Differences

    This shows you the differences between two versions of the page.

    Link to this comparison view

    Both sides previous revisionPrevious revision
    Next revision    		Previous revision
    en:dpi:dpi_options:opt_statistics:statistics_settings [2025/05/26 13:59] – elena.krasnobryzhen:dpi:dpi_options:opt_statistics:statistics_settings [2026/02/03 14:01] (current) – elena.krasnobryzh
    Line 1: Line 1:
    -====== Configuring service ====== +====== Set up NetFlow export by protocols, directions and billing ====== 
    -{{indexmenu_n>2}} +{{indexmenu_n>3}} 
    -The option is configured or disabled by parameters in configuration file ''/etc/dpi/fastdpi.conf''.+Changing the settings or disabling the option is done by editing 
     +The configuration file is ''/etc/dpi/fastdpi.conf''. 
     +<note warning>NetFlow options are cold and you need to restart the service.</note>
      
    -<note warning>NetFlow parameters are cold, so changes require a service restart.</note> +===== NetFlow on protocols ===== 
    - +IP address and NetFlow collector port number with statistics **by protocols**:
    -===== Statistics export configuration ===== +
    - +
    -To switch on the export of statistics: +
    - +
    -<code>netflow=1</code> +
    - +
    -  * 0 or no parameter - export disabled; +
    -  * 1 - export statistics by protocols (port numbers); +
    -  * 2 - export statistics by directions (independent systems' numbers); +
    -  * 4 - export statistics for billing; +
    -  * 8 - export the complete statistical information on sessions; +
    - +
    -<note important>3 = 1 + 2 - simultaneous export of statistics by protocols and by directions. Other values are combined similarly.</note> +
    - +
    -The network interface name to export netflow with statistics: +
    -<code>netflow_dev=eth2</code> +
    - +
    -Data export period in seconds: +
    -<code>netflow_timeout=10</code> +
    - +
    -===== Configuration of each type NetFlow ===== +
    -IP address and port number of netflow collector for statistics **by protocols**:+
     <code>netflow_collector=192.168.0.1:9997</code> <code>netflow_collector=192.168.0.1:9997</code>
    - +===== NetFlow in directions ===== 
    -<note warning>It is necessary to specify a separate collector for each type so that the data does not mix!</note> +IP address and NetFlow collector port number with statistics **by direction**:
    - +
    -IP address and port number of netflow collector with statistics **by directions**: +
     <code>netflow_as_collector=192.168.0.1:9998</code> <code>netflow_as_collector=192.168.0.1:9998</code>
      
    -Directions to collect and aggregate data: +Directions for which statistics and aggregation are collected:
    - +
    -  * 1 for external independent systems only (OK for household operators as there are no other independent systems rather than the operator itself on one side); +
    -  * 2 for internal independent systems only; +
    -  * 3 = 1 + 2 for both (OK for transit operators. However, as the aggregation by AS is made separately, data would be counted twice in the exported statistics - for each of AS participating in data transmission). +
     <code>netflow_as_direction=1</code> <code>netflow_as_direction=1</code>
    - +  * 1 - only for external autonomous systems (suitable for home operators, since there are no other autonomous systems on one side apart from the operator himself) 
    -IP address and port number of NetFlow collector with** statistics for billing.** One has to specify the separate collector to avoid data messing with other statistics: +  * 2 - only for internal autonomous systems 
    - +  * 3 = 1 + 2 - suitable for transit operators, but since AS is an independent aggregation, the exported statistics data will get 2 times - for each of the AS, participants in the transfer 
    -<code>netflow_bill_collector=192.168.0.1:9995</code> +===== NetFlow for billing ===== 
    - +IP address and NetFlow collector port number with **statistics for billing**, you need to allocate a separate collector so that the data is not mixed with other statistics: 
    -<note important>The billing statistics is formed only for those subscribers that have [[en:dpi:dpi_components:platform:dpi_billing|service 9]] enabled.\\ +<code>netflow_bill_collector=192. 168.0.1:9995</code> 
    -IPFIX does not pass information about the host IP:port with which the subscriber is communicating.</note> +<note important>Billing statistics are transmitted only by subscribers who are connected to [[en:dpi:dpi_components:platform:dpi_billing|service 9]].\\ 
    - +IPFIX does not transmit information about the host IP:port with which the subscriber exchanges information. </note> 
    -Setting up the statistics format:+Determination of the format of the transmitted information:
     <code>netflow_bill_collector_type=2</code> <code>netflow_bill_collector_type=2</code>
    -  * 0 - netflow_v5 ( default )+  * 0 - netflow_v5 (default)
       * 1 - ipfix udp   * 1 - ipfix udp
       * 2 - ipfix tcp   * 2 - ipfix tcp
      
    -The whole traffic volume is counted by default. This includes packet headers as well. In order to count the useful traffic only((The traffic with no packet headers may be 3.5 times smaller than the total one. For example, the minimal torrent UDP packet may be 64 bytes. The UDP header is 28 bytes and Ethernet frame size is 18 bytes. The resulting useful information is 18 bytes only of 64 total.)) please specify: +By default, the full amount of information transmitted, including packet headers L1-L7, is considered to be considered payload and only L3-L7 headers must specify the parameter: 
    - +<code>netflow_bill_method=1</code> 
    -<code>netflow_bill_method=1</code>  +In netflow, the TOS field of billing statistics transmits [[en:dpi:dpi_options:opt_priority:priority_config|the traffic class assigned by DPI]], which can be used to create flexible pricing plans.
    - +
    -[[en:dpi:dpi_options:opt_priority:priority_config|The traffic class assigned by DPI]] is specified in TOS field of netflow with billing statistics. This information can be used to create attractive billing plans. +
    - +
    -IP address and port number of netflow collector with full statistics. One has to specify the separate collector to avoid data messing with other statistics: +
    - +
    -<code>netflow_full_collector=192.168.0.1:9996 +
    -netflow_passive_timeout=30 +
    -netflow_active_timeout=300 +
    -</code>  +
    - +
    -Here +
    -  * netflow_passive_timeout=30 - is the time to wait for session activity. If no activity in this time, the session is treated as closed and its information is transmitted +
    -  * netflow_active_timeout=300 - is the time to report on the long sessions: in fact, such sessions are split into fragments of this duration. +
    - +
    -The complete statistics preserved original port numbers. The information on detected protocols is sent in normally empty bytes 46-47. In case one needs to analyse the protocols in use, the system can be configured to send the protocol information in the port number: +
    -<code>netflow_full_port_swap=1</code> +
    -  +
    -<note>NetFlow protocol does not ensure the delivery of packets as it works over UDP. If the collector has not enough capacity to receive the data, some packets are just lost. The collector has to receive data at rates no less than 60 Mb/s to handle full statistics for 10Gb channel. Please check your collector's capabilities before sending the netflow traffic. +
    - +
    -At the same time short peaks up to 100 Mb/s may happen when sending netflow from DPI. Few collectors are able to receive such data flow with no losses, for example nfsen/nfdump.</note> +
    - +
    -In order to smooth such peaks and load the collector in a uniform way one can set the parameter: +
    - +
    -<code>netflow_rate_limit=60</code>  +
    - +
    -Here 60 is the maximal netflow rate in Mb/s.  +
    - +
    -<note important>This value has to be set according to 6 Mb/s for each 1Gb of the external channel. Insufficient rate setting leads to data losses already on DPI side. This event is reflected in log file **/var/log/dpi/fastdpi_alert.log.**</note> +
    - +
    -=====Sending Template in IPFIX===== +
    -  - Transport protocol TCP.\\ The Template is sent once after the TCP session is established. +
    -  - Transport protocol UDP.\\ The Template is sent by default every 20 seconds. This is controlled by the ''ipfix_udp_template_timer'' parameter.+
      
    -  +===== Sending template to IPFIX ===== 
    -===== Configuration Example ===== +  - The TCP.\\ Template transport protocol is sent once after the TCP session has been established. 
    -An example is described in the section [[en:dpi:dpi_components:qoestor:configuration:dpi|QoE Stor: DPI Configuration]]+  - The UDP.\\ Template transport protocol is sent by default every 20 seconds. Adjusted by ''ipfix_udp_template_timer''.
      
     +===== Example of configuration =====
     +[[en:dpi:qoe_analytics:implementation_administration:configuration_setup:dpi|An example of configuration is described in the section QoE Stor: DPI Configuration.]]