Full NetFlow export format [Документация VAS Experts]
Документация VAS Experts Документация VAS Experts-mobile
in

Settings

  • Show pagesource
  • Old revisions
  • Export to PDF
  • Fold/unfold all
  • Backlinks
  • Show pagesource
  • Old revisions
  • Export to PDF
  • Fold/unfold all
  • Backlinks
    •   Stingray Service Gateway by VAS ExpertsDPIStatistics gathering and analysis on protocols and directionsFull NetFlow export format

    Table of Contents

    Full NetFlow export format

    General Settings

    Enabling statistics collection and export:

    netflow=1
    • 0 or not set - the option is disabled
    • 1 - export statistics by protocols (port numbers)
    • 2 - export statistics by directions (autonomous system numbers)
    • 4 - export statistics for billing
    • 8 - export full session statistics
    3 = 1 + 2 — simultaneous export of protocol-based and direction-based statistics (the same logic applies to other combinations)
    12 = 8 + 4 — simultaneous export of full NetFlow and billing statistics. In particular, this is used for RADIUS Accounting

    Network interface name through which NetFlow statistics will be sent: 

    netflow_dev=eth2

    Data export interval (in seconds): 

    netflow_timeout=10

    Session timeout:

    • netflow_passive_timeout — inactivity timeout after which, if there is no activity, the session is considered finished and its data is exported
    • netflow_active_timeout — interval after which information about long sessions is reported (i.e., long sessions are effectively split into fragments of this duration)

    To smooth peaks and distribute the load on the collector more evenly, set the tuning parameter 

    netflow_rate_limit=60

    , where 60 is the maximum NetFlow rate in Mbit/s.

    The parameter value should be set based on the following calculation: 6 Mbit/s per each 1G of external link bandwidth.
    Setting an insufficient value will result in data being dropped on the DPI side.
    Information about this event will be recorded in the log /var/log/dpi/fastdpi_alert.log.
    A separate collector must be allocated for each type so that data is not mixed!
    IPFIX/NetFlow parameters can be changed without restarting fastDPI.
    The configuration parameter ipfix_reserved allows reserving the required amount of memory to enable/change IPFIX/NetFlow parameters.
    If IPFIX/NetFlow parameters are specified in the configuration file, memory reservation for IPFIX/NetFlow is enabled automatically, and parameters as well as new IPFIX/NetFlow exporter types can be changed without restarting fastDPI.

    Full NetFlow

    IP address and port number of the NetFlow collector with full statistics; a separate collector must be allocated so that data is not mixed with other statistics: 

    netflow_full_collector=192.168.0.1:9996

    In NetFlow5 format, the full statistics retain the original port numbers, and information about detected protocols is transmitted in the usually unused bytes 46–47. If it is necessary to analyze the protocols in use, you can enable a setting in which protocol information is transmitted in the port number: 

    netflow_full_port_swap=1

    For compatibility with older collectors, this setting also applies to the IPFIX format, but it is not recommended to use it together with IPFIX, since protocol information in IPFIX is transmitted in a separate dedicated field.

    It is also necessary to define the full NetFlow export format: 

    netflow_full_collector_type=2

    Possible values:

    • 0 - export in NetFlow5 format (default)
    • 1 - export IPFIX to a UDP collector
    • 2 - export IPFIX to a TCP collector
    We recommend using full NetFlow export in IPFIX format over TCP (parameter value 2).
    The NetFlow protocol does not guarantee packet delivery (as it runs over UDP), and if the collector cannot handle the incoming data rate, some packets will simply be lost. Exporting full NetFlow statistics for a 10G link requires the collector to be able to receive data at a rate of at least 60 Mbit/s.
    Check the capabilities of your collector before sending NetFlow traffic to it. At the same time, short-term bursts of up to 100 Mbit/s may occur when exporting NetFlow from DPI. Only a few collectors can handle such a data stream without losses, for example, nfsen/nfdump.

    The netflow_tos_format parameter defines the data format of the TOS field in IPFIX. Possible values:

    • 0 - 3 bits are transmitted (default)
    • 1 - 6 bits are transmitted (full DSCP)

    The netflow_plc_stat parameter defines the set of transmitted statistics data for dropped packets according to policing or drop rules. The parameter is a bit mask.
    By default, the mask value is 0x07 — statistics are transmitted for dropped data due to session + subscriber + virtual channel policing.
    :!: Affects the formation of the DROPPED_BYTES and DROPPED_PACKETS counters.
    Mask values:

    • 0xff - count any drop
    • 0 - do not count
    • 1 - count for session policing
    • 2 - count for subscriber policing
    • 4 - count for virtual channel policing
    • 8 - count drops by protocol
    • 16 - count in all other cases

    The ipfix_mtu_limit parameter sets the maximum UDP packet size when sending IPFIX. By default, it is equal to the minimum MTU size of the interfaces used for transmission.

    The tethering_ttl_allowed = 128:64 parameter specifies the list of allowed TTL values for subscriber traffic that are not considered tethering. Values are separated by ':'. Up to 256 values (0–255) are supported.

    For receiving, processing, and storing IPFIX data, it is recommended to use the QoE Store statistics collection software and the DPIUI2 graphical interface.

    For collecting data in IPFIX format, any universal IPFIX collector that supports templates, or the IPFIX Receiver utility, can be used.

    Export Template in IPFIX Format (Netflow v10) for IPv4 Protocol

    Export Template for IPv4
    № Bytes Data Type IANA Description Notes Used in QoEStor
    1 8 int64 0 OCTET_DELTA_COUNT Analog in NetFlow v9 IN_BYTES Used
    2 8 int64 0 PACKET_DELTA_COUNT Analog in NetFlow v9 IN_PKTS Used
    4 1 int8 0 PROTOCOL_IDENTIFIER Analog in NetFlow v9 PROTOCOL Used
    5 1 int8 0 IP_CLASS_OF_SERVICE Analog in NetFlow v9 TOS Used
    7 2 int16 0 SOURCE_TRANSPORT_PORT Analog in NetFlow v9 L4_SRC_PORT Used
    8 4 int32 0 SOURCE_IPV4_ADDRESS Analog in NetFlow v9 IPV4_SRC_ADDR Used
    11 2 int16 0 DESTINATION_TRANSPORT_PORT Analog in NetFlow v9 L4_DST_PORT Used
    12 4 int32 0 DESTINATION_IPV4_ADDRESS Analog in NetFlow v9 IPV4_DST_ADDR Used
    16 4 int32 0 BGP_SOURCE_AS_NUMBER Analog in NetFlow v9 SRC_AS Used
    17 4 int32 0 BGP_DESTINATION_AS_NUMBER Analog in NetFlow v9 DST_AS Used
    152 8 int64 0 FLOW_START_MILLISECOND Used
    153 8 int64 0 FLOW_END_MILLISECOND Used
    10 2 int16 0 INPUT_SNMP Analog in NetFlow v9 IngressInterface Used
    14 2 int16 0 OUTPUT_SNMP Analog in NetFlow v9 EgressInterface Used
    60 1 int8 0 IP_VERSION Analog in NetFlow v9 IP_PROTOCOL_VERSION Used
    2000 8 int64 43823 SESSION_ID Used
    2001 - string 43823 HTTP_HOST or CN_HTTPS Used
    2002 2 int16 43823 DPI_PROTOCOL Used
    2003 - string 43823 LOGIN Analog in Radius User-Name Used
    225 4 int32 0 POST_NAT_SOURCE_IPV4_ADDRESS Used
    227 2 int16 0 POST_NAPT_SOURCE_TRANSPORT_PORT Used
    2010 2 int16 43823 FRGMT_DELTA_PACKS Delta of fragmented packets. Used
    2011 2 int16 43823 REPEAT_DELTA_PACK Delta of retransmissions. Used
    2012 4 int32 43823 PACKET_DELIVER_TIME Delay (RTT/2) in ms (RTT=round-trip time). Used
    2016 2 int16 43823 BRIDGE_CHANNEL_NUM Channel number (vchannel) or bridge.
    If vchannels are configured in DPI,
    the channel number will be transmitted, otherwise the bridge number.     		Used
    6 2 int16 0 TCP_FLAGS TCP control bits Used
    58 2 int16 0 SRC_VLAN VLAN ID Used
    59 2 int16 0 DST_VLAN Post VLAN ID Used
    56 6 mac_address 0 SRC_MAC Source MAC address Used
    57 6 mac_address 0 DST_MAC Destination MAC address Used
    2017 - raw 43823 MPLS Lables Used
    132 8 int64 0 DROPPED_BYTES Delta count of dropped octets.
    For example: data is dumped at minute T1 and T2. The delta will show the difference in the number of octets between minute T1 and T2.     		Used
    133 8 int64 0 DROPPED_PACKETS Delta count of dropped packets.
    For example: data is dumped at minute T1 and T2. The delta will show the difference in the number of packets between minute T1 and T2.     		Used
    2019 1 int8 43823 originalTOS Original TOS value from IP header Used
    192 1 int8 0 IP_TTL TTL packets
    2020 2 int16 43823 RATING_GROUP Rating group number

    Export Template in IPFIX Format (Netflow v10) for IPv6 Protocol

    The template is similar to IPv4 except that the following fields are absent: SOURCE_IPV4_ADDRESS, DESTINATION_IPV4_ADDRESSs, POST_NAT_SOURCE_IPV4_ADDRESS, POST_NAT_SOURCE_TRANSPORT_PORT, – and the following are present:

    Export Template for IPv6
    № Bytes Data Type IANA Description Notes
    27 16 int128 0 SOURCE_IPV6_ADDRESS Analog in NetFlow v9 IPV6_SRC_ADDR
    28 16 int128 0 DESTINATION_IPV6_ADDRESS Analog in NetFlow v9 IPV6_DST_ADDR

    Was this information helpful?