Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| en:dpi:dpi_options:opt_statistics:opt_li [2026/07/07 13:36] – removed - external edit (Unknown date) 127.0.0.1 | en:dpi:dpi_options:opt_statistics:opt_li [2026/07/07 13:40] (current) – elena.krasnobryzh | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| + | ====== PCAP, IPFIX Export (Clickstream, | ||
| + | {{indexmenu_n> | ||
| + | The option allows real-time recording of network traffic and can be used to monitor traffic for diagnostic purposes, analyze security threats and conduct market research. Traffic for diagnostic purposes, security threat analysis and market research. | ||
| + | |||
| + | <note important> | ||
| + | |||
| + | For Clickstream data analisys (subscribers' | ||
| + | |||
| + | A list of the correspondence between the Protocol and the port number in netfow5 can be found __[[en: | ||
| + | |||
| + | Any universal IPFIX collector that accepts templates or the __[[en: | ||
| + | |||
| + | To receive, process and store ClickStream, | ||
| + | |||
| + | If the link quality between SSG and NetFlow/ | ||
| + | |||
| + | < | ||
| + | [NFLW] very long operation …. | ||
| + | </ | ||
| + | |||
| + | Starting from version 12.0, the statistics for sending NetFlow/ | ||
| + | |||
| + | < | ||
| + | [STAT ][2022/ | ||
| + | |||
| + | a - number of sending cycles | ||
| + | b - number of sending cycles, when the time spent on sending exceeded the cycle execution period | ||
| + | c - percentage of exceeding the number of sending cycles: 100 * b/a | ||
| + | d - time of maximum sending cycle duration, microseconds | ||
| + | e - time of the period of sending statistics, microseconds ('' | ||
| + | |||
| + | Example: | ||
| + | [STAT ][2022/ | ||
| + | </ | ||
| + | ===== ClickStream export Setup ===== | ||
| + | Clickstream experts is configured by following parameters: | ||
| + | |||
| + | <code bash> | ||
| + | ipfix_dev=em1 | ||
| + | ipfix_udp_collectors=1.2.3.4: | ||
| + | ipfix_tcp_collectors=1.2.3.6: | ||
| + | dbg_log_mask=0x80 | ||
| + | </ | ||
| + | |||
| + | here | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | |||
| + | The '' | ||
| + | ==== IPFIX format template for Clickstream ==== | ||
| + | |||
| + | The format of IPFIX templates for IPV6 differs only in the **IP_SOURCE** and **IP_DESTINATION** fields. | ||
| + | |||
| + | ^ № ^ Size in bytes ^ | ||
| + | | 1003 | 16 | IPv6 | 43823 |IP_SOURCE |Sender address| | ||
| + | | 1004 | 16 | IPv6 | 43823 |IP_DESTINATION |Recipient address| | ||
| + | |||
| + | ^ IPFIX format template for Clickstream | ||
| + | ^ № ^ Size in bytes ^ Type | ||
| + | | 1001 | ||
| + | | 1002 | ||
| + | | 1003 | ||
| + | | 1004 | ||
| + | | 1005 | ||
| + | | 1006 | ||
| + | | 1007 | ||
| + | | 1008 | ||
| + | | 1009 | ||
| + | | 2000 | ||
| + | | 1010 | ||
| + | | 1011 | ||
| + | | 1012 | ||
| + | | 1013 | ||
| + | | 1014 | ||
| + | | 2016 | ||
| + | | 1024 | ||
| + | | 1025 | ||
| + | | 58 | ||
| + | | 59 | ||
| + | | 56 | ||
| + | | 57 | ||
| + | | 2017 | ||
| + | | 2018 | ||
| + | |||
| + | **ND: | ||
| + | * LOCKED = 1 — blocked by HTTPS, 2 — HTTP redirect, 3 — blocked by HTTP (transmitted by bitmask) | ||
| + | * HOST TYPE = 1 in case of HTTP, 2 — CNAME, 3 — SNI, 4 — QUIC | ||
| + | * METHOD = 1 — GET, 2 — POST, 3 — PUT, 4 — DELETE | ||
| + | |||
| + | If the configuration parameter '' | ||
| + | |||
| + | ^ Clickstream export template IPFIX format for HTTP responses((for the IPv6 variant see difference above)) | ||
| + | ^ № ^ Size in bytes ^ Type ^ IANA ^ Description | ||
| + | | 1001 | 4 | int32 | 43823 |TIME_STAMP| | ||
| + | | 1002 | - | string | ||
| + | | 1003 | 4 | IPv4 | 43823 |IP_SOURCE| | ||
| + | | 1004 | 4 | IPv4 | 43823 |IP_DESTINATION| | ||
| + | | 1020 | 4 | int32 | 43823 |RESULT_CODE| | ||
| + | | 1021 | 8 | int64 | 43823 |CONTENT_LENGTH| | ||
| + | | 1022 | - | string | ||
| + | | 2000 | 8 | int64 | 43823 |SESSION_ID| | ||
| + | | 1023 | - | string | ||
| + | | 2016 | 2 | int16 | 43823 |BRIDGE_CHANNEL_NUM| Channel (vchannel) or bridge number. If vchannel is set in the DPI configuration, | ||
| + | | 58 | 2 | int16 | - |VlanId|VLAN| | ||
| + | | 59 | 2 | int16 | - |postVlanID|POST VLAN| | ||
| + | | 56 | 6 | mac_address | ||
| + | | 57 | 6 | mac_adress | ||
| + | | 2017 | - | raw | 43823 |MPLS Labels| | ||
| + | |||
| + | If the configuration parameter '' | ||
| + | |||
| + | ^ Clickstream export template IPFIX format for responses over SSL/TLS, HTTPS((for the IPv6 variant, see difference above)) | ||
| + | ^ № ^ Size in bytes ^ Type ^ IANA ^ Description | ||
| + | | 1001 | 4 | int32 | 43823 |TIME_STAMP| | ||
| + | | 1002 | - | string | ||
| + | | 1003 | 4 | IPv4 | 43823 |IP_SOURCE| | ||
| + | | 1004 | 4 | IPv4 | 43823 |IP_DESTINATION| | ||
| + | | 2000 | 8 | int64 | 43823 |SESSION_ID| | ||
| + | | 1030 | 2 | int16 | 43823 |SSL_VERSION| | ||
| + | | 1031 | 2 | int16 | 43823 |CIPHER_SUITE| | ||
| + | | 1032 | 1 | int8 | 43823 |COMPRESSION_METHOD| | ||
| + | | 2016 | 2 | ||
| + | | 58 | 2 | ||
| + | | 59 | 2 | ||
| + | | 56 | 6 | ||
| + | | 57 | 6 | ||
| + | | 2017 | - | ||
| + | | 1011 | 1 | int8 | 43823 | type_host | | ||
| + | | 1005 | - | string | ||
| + | |||
| + | |||
| + | ===== Metadata Export Setting ===== | ||
| + | Export of metadata of other protocols for SORM is configured by the following parameters | ||
| + | |||
| + | <code bash> | ||
| + | ipfix_dev=em1 | ||
| + | ipfix_meta_udp_collectors=1.2.3.4: | ||
| + | ipfix_meta_tcp_collectors=1.2.3.6: | ||
| + | dbg_log_mask=0x80 | ||
| + | </ | ||
| + | |||
| + | where | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | |||
| + | ==== IPFIX metadata export template formats ==== | ||
| + | |||
| + | ^ SIP metadata export template IPFIX format | ||
| + | ^ № ^ Size in bytes ^ Type ^ IANA ^ Description | ||
| + | | 1001 | 4 | int32 | 43823 | TIME_STAMP | | ||
| + | | 1002 | - | string | ||
| + | | 1003 | 4 | IPv4 | 43823 | IP_SRC | Sender' | ||
| + | | 1004 | 4 | IPv4 | 43823 | IP_DST | Recipient' | ||
| + | | 2000 | 8 | int64 | 43823 | SESSION_ID | | ||
| + | | 3000 | - | string | ||
| + | | 3001 | 2 | int16 | 43823 | STATUS_CODE | | ||
| + | | 3002 | - | string | ||
| + | | 3003 | - | string | ||
| + | | 3004 | - | string | ||
| + | | 3005 | - | string | ||
| + | | 3006 | - | string | ||
| + | | 3007 | - | string | ||
| + | | 3008 | - | string | ||
| + | | 58 | 2 | int16 | - |VlanId|VLAN| | ||
| + | | 59 | 2 | int16 | - |postVlanID|POST VLAN| | ||
| + | | 56 | 6 | mac_address | ||
| + | | 57 | 6 | mac_adress | ||
| + | | 2017 | - | raw | 43823 |MPLS Labels| | ||
| + | **Notes:** \\ | ||
| + | **IP_SRC** — IP SOURCE\\ | ||
| + | **IP_DST** — IP DESTINATION\\ | ||
| + | **GATEWAYS** — comma separated list of gateways (IP or hostname) | ||
| + | |||
| + | ^ FTP Metadata Export Template IPFIX Format | ||
| + | ^ № ^ Size in bytes ^ Type ^ IANA ^ Description | ||
| + | | 1001 | 4 | int32 | 43823 | TIME_STAMP | | ||
| + | | 1002 | - | string | ||
| + | | 1003 | 4 | IPv4 | 43823 | IP_SRC | Sender' | ||
| + | | 1004 | 4 | IPv4 | 43823 | IP_DST | Recipient' | ||
| + | | 2000 | 8 | int64 | 43823 | SESSION_ID | | ||
| + | | 3050 | - | string | ||
| + | | 3051 | - | string | ||
| + | | 3052 | - | string | ||
| + | | 3053 | 1 | int8 | 43823 | MODE | | ||
| + | | 1020 | 4 | int32 | 43823 |RESULT_CODE| | ||
| + | | 58 | 2 | int16 | - |VlanId|VLAN| | ||
| + | | 59 | 2 | int16 | - |postVlanID|POST VLAN| | ||
| + | | 56 | 6 | mac_address | ||
| + | | 57 | 6 | mac_adress | ||
| + | | 2017 | - | raw | 43823 |MPLS Labels| | ||
| + | |||
| + | **Note:** the MODE field contains the FTP connection type 0 — active, 1 — passive | ||
| + | |||
| + | ^ Messenger Metadata Export Template IPFIX Format (XMPP) | ||
| + | ^ № ^ Size in bytes ^ Type ^ IANA ^ Description | ||
| + | | 1001 | 4 | int32 | 43823 | TIME_STAMP | | ||
| + | | 1002 | - | string | ||
| + | | 1003 | 4 | IPv4 | 43823 | IP_SRC | Sender' | ||
| + | | 1004 | 4 | IPv4 | 43823 | IP_DST | Recipient' | ||
| + | | 2000 | 8 | int64 | 43823 | SESSION_ID | | ||
| + | | 3100 | - | string | ||
| + | | 3101 | - | string | ||
| + | | 3102 | - | string | ||
| + | | 3103 | - | string | ||
| + | | 3104 | 1 | int8 | 43823 | IM_PROTOCOL | Type of protocol used | | ||
| + | | 3105 | - | string | ||
| + | | 1020 | 4 | int32 | 43823 | RESULT_CODE | | ||
| + | | 58 | 2 | int16 | - |VlanId|VLAN| | ||
| + | | 59 | 2 | int16 | - |postVlanID|POST VLAN| | ||
| + | | 56 | 6 | mac_address | ||
| + | | 57 | 6 | mac_adress | ||
| + | | 2017 | - | raw | 43823 |MPLS Labels| | ||
| + | |||
| + | **Note:** the IM_PROTOCOL field contains the type of protocol used: 0 — ICQ, 7 — XMPP, 106 — ZELLO | ||
| + | |||
| + | ^ IPFIX format of mail protocol metadata export template (POP, IMAP, SMTP) ^^^^^^ | ||
| + | ^ № ^ Size in bytes ^ Type ^ IANA ^ Description | ||
| + | | 1001 | 4 | int32 | 43823 | TIME_STAMP | | ||
| + | | 1002 | - | string | ||
| + | | 1003 | 4 | IPv4 | 43823 | IP_SRC | Sender' | ||
| + | | 1004 | 4 | IPv4 | 43823 | IP_DST | Recipient' | ||
| + | | 2000 | 8 | int64 | 43823 | SESSION_ID | | ||
| + | | 3150 | - | string | ||
| + | | 3151 | - | string | ||
| + | | 3152 | - | string | ||
| + | | 3153 | - | string | ||
| + | | 3154 | - | string | ||
| + | | 3155 | - | string | ||
| + | | 3156 | 1 | int8 | 43823 | EVENT | Event type | | ||
| + | | 3157 | 1 | int8 | 43823 | ATTACHMENT | Indication of attachment | | ||
| + | | 3158 | 1 | int8 | 43823 | MAIL_PROTOCOL | | ||
| + | | 1020 | 4 | int32 | 43823 | RESULT_CODE | | ||
| + | | 58 | 2 | int16 | - |VlanId| VLAN | | ||
| + | | 59 | 2 | int16 | - |postVlanID| POST VLAN | | ||
| + | | 56 | 6 | mac_address | ||
| + | | 57 | 6 | mac_adress | ||
| + | | 2017 | - | raw | 43823 | MPLS Labels | | ||
| + | |||
| + | **Note:** the EVENT field indicates the event type 1 — send, 2 — receive, \\ | ||
| + | ATTACHMENT sign of an attachment, mail_protocol = 0 — smtp, 1 — pop3, 2 — imap | ||
| + | |||
| + | ^ The raw unparsed metadata export template IPFIX format | ||
| + | ^ № ^ Size in bytes ^ Type ^ IANA ^ Description | ||
| + | | 1001 | 4 | int32 | 43823 | TIME_STAMP | | ||
| + | | 1002 | - | string | ||
| + | | 1003 | 4 | IPv4 | ||
| + | | 1004 | 4 | IPv4 | ||
| + | | 2000 | 8 | int64 | 43823 | SESSION_ID | | ||
| + | | 2013 | 1 | int8 | ||
| + | | 2014 | 1 | int8 | ||
| + | | 2015 | 2 | int16 | 43823 | VDPI_PROTO | The protocol that determined the DPI | | ||
| + | | 2900 | 2 | int16 | 43823 | META_PROTO | Internal protocol identifier | | ||
| + | | 2901 | - | string | ||
| + | | 4 | ||
| + | | 7 | ||
| + | | | ||
| + | | 6 | ||
| + | | 2018 | 4 | int32 | - | TCP Sequence | | ||
| + | | 58 | 2 | int16 | - |VlanId|VLAN| | ||
| + | | 59 | 2 | int16 | - |postVlanID|POST VLAN| | ||
| + | | 56 | 6 | mac_address | ||
| + | | 57 | 6 | mac_adress | ||
| + | | 2017 | - | raw | 43823 |MPLS Labels| | ||
| + | **Note:** | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | |||
| + | Aggregating '' | ||
| + | |||
| + | =====Configuring the export of DNS responses or DNS queries===== | ||
| + | DNS export is configured with the following settings: | ||
| + | <code bash> | ||
| + | ipfix_dev=em1 | ||
| + | ipfix_dns_udp_collectors=1.2.3.4: | ||
| + | ipfix_dns_tcp_collectors=1.2.3.6: | ||
| + | </ | ||
| + | where | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | |||
| + | The format of IPFIX templates for IPV6 differs in the format of the '' | ||
| + | |||
| + | ^ № | ||
| + | | 1103 | 16 | ||
| + | | 1104 | 16 | ||
| + | |||
| + | |||
| + | ^ DNS Export Template IPFIX Format | ||
| + | ^ № | ||
| + | | 1001 | 4 | int32 | 43823 | TIME_STAMP | ||
| + | | 1002 | - | string | ||
| + | | 1003 | 4 | IPv4 | ||
| + | | 1004 | 4 | IPv4 | ||
| + | | 1013 | 2 | int16 | 43823 | SOURCE PORT | | | ||
| + | | 1014 | 2 | int16 | 43823 | DESTINATION PORT | ||
| + | | 2000 | 8 | int64 | 43823 | SESSION_ID | ||
| + | | 3200 | 1 | int8 | ||
| + | | 3201 | - | string | ||
| + | | 3202 | 2 | int16 | 43823 | RRCLASS | ||
| + | | 3203 | 2 | int16 | 43823 | RRTYPE | ||
| + | | 3204 | 4 | int32 | 43823 | TTL | | | ||
| + | | 3205 | - | raw | 43823 | RDATA | | | ||
| + | | 58 | 2 | int16 | - | VlanId | ||
| + | | 59 | 2 | int16 | - | postVlanID | ||
| + | | 56 | 6 | mac_address | ||
| + | | 57 | 6 | mac_adress | ||
| + | | 2017 | - | raw | 43823 | MPLS Labels | ||
| + | | 2016 | 2 | int16 | 43823 | BRIDGE_CHANNEL_NUM | ||
| + | | 224 | ||
| + | | 3206 | 2 | uint16 | ||
| + | |||
| + | An alternative is to save the data in a local text log. Parameters: | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | |||
| + | <note tip>The filter for DNS **request** types to be sent via IPFIX is specified by the '' | ||
| + | The filter for DNS **response** types to be sent via IPFIX is specified by the '' | ||
| + | |||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | |||
| + | Switching to the next file occurs when the file size reaches '' | ||
| + | |||
| + | '' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | |||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | * **'' | ||
| + | |||
| + | Default: '' | ||
| + | |||
| + | =====Sending Template in IPFIX===== | ||
| + | - Transport protocol TCP.\\ The Template is sent once after the TCP session is established. | ||
| + | - Transport protocol UDP.\\ The Template is sent by default every 20 seconds. This is controlled by the '' | ||