Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
en:dpi:dpi_options:opt_li:li_ipfix [2020/02/05 17:36] – ↷ Page moved from en:dpi:dpi_options:base_functionality:opt_li:li_ipfix to en:dpi:dpi_options:opt_li:li_ipfix lexx26 | en:dpi:dpi_options:opt_li:li_ipfix [2025/06/26 07:26] (current) – [IPFIX metadata export template formats] elena.krasnobryzh | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== | + | ====== |
+ | {{indexmenu_n> | ||
+ | For Clickstream data analisys (subscribers' | ||
+ | |||
+ | A list of the correspondence between the Protocol and the port number in netfow5 can be found __[[en: | ||
+ | |||
+ | Any universal IPFIX collector that accepts templates or the __[[en: | ||
+ | |||
+ | To receive, process and store ClickStream, | ||
+ | |||
+ | If the link quality between SSG and NetFlow/ | ||
+ | |||
+ | < | ||
+ | [NFLW] very long operation …. | ||
+ | </ | ||
+ | |||
+ | Starting from version 12.0, the statistics for sending NetFlow/ | ||
+ | |||
+ | < | ||
+ | [STAT ][2022/ | ||
+ | |||
+ | a - number of sending cycles | ||
+ | b - number of sending cycles, when the time spent on sending exceeded the cycle execution period | ||
+ | c - percentage of exceeding the number of sending cycles: 100 * b/a | ||
+ | d - time of maximum sending cycle duration, microseconds | ||
+ | e - time of the period of sending statistics, microseconds ('' | ||
+ | |||
+ | Example: | ||
+ | [STAT ][2022/ | ||
+ | </ | ||
+ | ===== ClickStream export Setup ===== | ||
+ | Clickstream experts is configured by following parameters: | ||
+ | |||
+ | <code bash> | ||
+ | ipfix_dev=em1 | ||
+ | ipfix_udp_collectors=1.2.3.4: | ||
+ | ipfix_tcp_collectors=1.2.3.6: | ||
+ | dbg_log_mask=0x80 | ||
+ | </ | ||
+ | |||
+ | here | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | |||
+ | ==== IPFIX format template for Clickstream ==== | ||
+ | |||
+ | The format of IPFIX templates for IPV6 differs only in the **IP_SOURCE** and **IP_DESTINATION** fields. | ||
+ | |||
+ | ^ № ^ Size in bytes ^ | ||
+ | | 1003 | 16 | IPv6 | 43823 |IP_SOURCE |Sender address| | ||
+ | | 1004 | 16 | IPv6 | 43823 |IP_DESTINATION |Recipient address| | ||
+ | |||
+ | ^ IPFIX format template for Clickstream | ||
+ | ^ № ^ Size in bytes ^ | ||
+ | | 1001 | 4 | int32 | | ||
+ | | 1002 | - | string | | ||
+ | | 1003 | 4 | IPv4 | | ||
+ | | 1004 | 4 | IPv4 | | ||
+ | | 1005 | - | string | | ||
+ | | 1006 | - | string | | ||
+ | | 1007 | - | string | | ||
+ | | 1008 | - | string | | ||
+ | | 1009 | - | string | | ||
+ | | 2000 | 8 | int64 | | ||
+ | | 1010 | 8 | int64 | | ||
+ | | 1011 | 1 | int8 | | ||
+ | | 1012 | 1 | int8 | | ||
+ | | 1013 | 2 | int16 | | ||
+ | | 1014 | 2 | int16 | | ||
+ | | 2016 | 2 | ||
+ | | 1024 | 2 | ||
+ | | 1025 | - | ||
+ | | 58 | 2 | ||
+ | | 59 | 2 | ||
+ | | 56 | 6 | ||
+ | | 57 | 6 | ||
+ | | 2017 | - | ||
+ | | 2018 | 4 | ||
+ | |||
+ | **ND: | ||
+ | * LOCKED = 1 — blocked by HTTPS, 2 — HTTP redirect, 3 — blocked by HTTP (transmitted by bitmask) | ||
+ | * HOST TYPE = 1 in case of HTTP, 2 — CNAME, 3 — SNI, 4 — QUIC | ||
+ | * METHOD = 1 — GET, 2 — POST, 3 — PUT, 4 — DELETE | ||
+ | |||
+ | If the configuration parameter '' | ||
+ | |||
+ | ^ Clickstream export template IPFIX format for HTTP responses((for the IPv6 variant see difference above)) | ||
+ | ^ № ^ Size in bytes ^ Type ^ IANA ^ Description | ||
+ | | 1001 | 4 | int32 | 43823 |TIME_STAMP| | ||
+ | | 1002 | - | string | ||
+ | | 1003 | 4 | IPv4 | 43823 |IP_SOURCE| | ||
+ | | 1004 | 4 | IPv4 | 43823 |IP_DESTINATION| | ||
+ | | 1020 | 4 | int32 | 43823 |RESULT_CODE| | ||
+ | | 1021 | 8 | int64 | 43823 |CONTENT_LENGTH| | ||
+ | | 1022 | - | string | ||
+ | | 2000 | 8 | int64 | 43823 |SESSION_ID| | ||
+ | | 1023 | - | string | ||
+ | | 2016 | 2 | int16 | 43823 |BRIDGE_CHANNEL_NUM| Channel (vchannel) or bridge number. If vchannel is set in the DPI configuration, | ||
+ | | 58 | 2 | int16 | - |VlanId|VLAN| | ||
+ | | 59 | 2 | int16 | - |postVlanID|POST VLAN| | ||
+ | | 56 | 6 | mac_address | ||
+ | | 57 | 6 | mac_adress | ||
+ | | 2017 | - | raw | 43823 |MPLS Labels| | ||
+ | |||
+ | If the configuration parameter '' | ||
+ | |||
+ | ^ Clickstream export template IPFIX format for responses over SSL/TLS, HTTPS((for the IPv6 variant, see difference above)) | ||
+ | ^ № ^ Size in bytes ^ Type ^ IANA ^ Description | ||
+ | | 1001 | 4 | int32 | 43823 |TIME_STAMP| | ||
+ | | 1002 | - | string | ||
+ | | 1003 | 4 | IPv4 | 43823 |IP_SOURCE| | ||
+ | | 1004 | 4 | IPv4 | 43823 |IP_DESTINATION| | ||
+ | | 2000 | 8 | int64 | 43823 |SESSION_ID| | ||
+ | | 1030 | 2 | int16 | 43823 |SSL_VERSION| | ||
+ | | 1031 | 2 | int16 | 43823 |CIPHER_SUITE| | ||
+ | | 1032 | 1 | int8 | 43823 |COMPRESSION_METHOD| | ||
+ | | 2016 | 2 | ||
+ | | 58 | 2 | ||
+ | | 59 | 2 | ||
+ | | 56 | 6 | ||
+ | | 57 | 6 | ||
+ | | 2017 | - | ||
+ | | 1011 | 1 | int8 | 43823 | type_host | | ||
+ | | 1005 | - | string | ||
+ | |||
+ | |||
+ | ===== Metadata Export Setting ===== | ||
+ | Export of metadata of other protocols for SORM is configured by the following parameters | ||
+ | |||
+ | <code bash> | ||
+ | ipfix_dev=em1 | ||
+ | ipfix_meta_udp_collectors=1.2.3.4: | ||
+ | ipfix_meta_tcp_collectors=1.2.3.6: | ||
+ | dbg_log_mask=0x80 | ||
+ | </ | ||
+ | |||
+ | where | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | |||
+ | ==== IPFIX metadata export template formats ==== | ||
+ | |||
+ | ^ SIP metadata export template IPFIX format | ||
+ | ^ № ^ Size in bytes ^ Type ^ IANA ^ Description | ||
+ | | 1001 | 4 | int32 | 43823 | TIME_STAMP | | ||
+ | | 1002 | - | string | ||
+ | | 1003 | 4 | IPv4 | 43823 | IP_SRC | Sender' | ||
+ | | 1004 | 4 | IPv4 | 43823 | IP_DST | Recipient' | ||
+ | | 2000 | 8 | int64 | 43823 | SESSION_ID | | ||
+ | | 3000 | - | string | ||
+ | | 3001 | 2 | int16 | 43823 | STATUS_CODE | | ||
+ | | 3002 | - | string | ||
+ | | 3003 | - | string | ||
+ | | 3004 | - | string | ||
+ | | 3005 | - | string | ||
+ | | 3006 | - | string | ||
+ | | 3007 | - | string | ||
+ | | 3008 | - | string | ||
+ | | 58 | 2 | int16 | - |VlanId|VLAN| | ||
+ | | 59 | 2 | int16 | - |postVlanID|POST VLAN| | ||
+ | | 56 | 6 | mac_address | ||
+ | | 57 | 6 | mac_adress | ||
+ | | 2017 | - | raw | 43823 |MPLS Labels| | ||
+ | **Notes:** \\ | ||
+ | **IP_SRC** — IP SOURCE\\ | ||
+ | **IP_DST** — IP DESTINATION\\ | ||
+ | **GATEWAYS** — comma separated list of gateways (IP or hostname) | ||
+ | |||
+ | ^ FTP Metadata Export Template IPFIX Format | ||
+ | ^ № ^ Size in bytes ^ Type ^ IANA ^ Description | ||
+ | | 1001 | 4 | int32 | 43823 | TIME_STAMP | | ||
+ | | 1002 | - | string | ||
+ | | 1003 | 4 | IPv4 | 43823 | IP_SRC | Sender' | ||
+ | | 1004 | 4 | IPv4 | 43823 | IP_DST | Recipient' | ||
+ | | 2000 | 8 | int64 | 43823 | SESSION_ID | | ||
+ | | 3050 | - | string | ||
+ | | 3051 | - | string | ||
+ | | 3052 | - | string | ||
+ | | 3053 | 1 | int8 | 43823 | MODE | | ||
+ | | 1020 | 4 | int32 | 43823 |RESULT_CODE| | ||
+ | | 58 | 2 | int16 | - |VlanId|VLAN| | ||
+ | | 59 | 2 | int16 | - |postVlanID|POST VLAN| | ||
+ | | 56 | 6 | mac_address | ||
+ | | 57 | 6 | mac_adress | ||
+ | | 2017 | - | raw | 43823 |MPLS Labels| | ||
+ | |||
+ | **Note:** the MODE field contains the FTP connection type 0 — active, 1 — passive | ||
+ | |||
+ | ^ Messenger Metadata Export Template IPFIX Format (XMPP) | ||
+ | ^ № ^ Size in bytes ^ Type ^ IANA ^ Description | ||
+ | | 1001 | 4 | int32 | 43823 | TIME_STAMP | | ||
+ | | 1002 | - | string | ||
+ | | 1003 | 4 | IPv4 | 43823 | IP_SRC | Sender' | ||
+ | | 1004 | 4 | IPv4 | 43823 | IP_DST | Recipient' | ||
+ | | 2000 | 8 | int64 | 43823 | SESSION_ID | | ||
+ | | 3100 | - | string | ||
+ | | 3101 | - | string | ||
+ | | 3102 | - | string | ||
+ | | 3103 | - | string | ||
+ | | 3104 | 1 | int8 | 43823 | IM_PROTOCOL | Type of protocol used | | ||
+ | | 3105 | - | string | ||
+ | | 1020 | 4 | int32 | 43823 | RESULT_CODE | | ||
+ | | 58 | 2 | int16 | - |VlanId|VLAN| | ||
+ | | 59 | 2 | int16 | - |postVlanID|POST VLAN| | ||
+ | | 56 | 6 | mac_address | ||
+ | | 57 | 6 | mac_adress | ||
+ | | 2017 | - | raw | 43823 |MPLS Labels| | ||
+ | |||
+ | **Note:** the IM_PROTOCOL field contains the type of protocol used: 0 — ICQ, 7 — XMPP, 106 — ZELLO | ||
+ | |||
+ | ^ IPFIX format of mail protocol metadata export template (POP, IMAP, SMTP) ^^^^^^ | ||
+ | ^ № ^ Size in bytes ^ Type ^ IANA ^ Description | ||
+ | | 1001 | 4 | int32 | 43823 | TIME_STAMP | | ||
+ | | 1002 | - | string | ||
+ | | 1003 | 4 | IPv4 | 43823 | IP_SRC | Sender' | ||
+ | | 1004 | 4 | IPv4 | 43823 | IP_DST | Recipient' | ||
+ | | 2000 | 8 | int64 | 43823 | SESSION_ID | | ||
+ | | 3150 | - | string | ||
+ | | 3151 | - | string | ||
+ | | 3152 | - | string | ||
+ | | 3153 | - | string | ||
+ | | 3154 | - | string | ||
+ | | 3155 | - | string | ||
+ | | 3156 | 1 | int8 | 43823 | EVENT | Event type | | ||
+ | | 3157 | 1 | int8 | 43823 | ATTACHMENT | Indication of attachment | | ||
+ | | 3158 | 1 | int8 | 43823 | MAIL_PROTOCOL | | ||
+ | | 1020 | 4 | int32 | 43823 | RESULT_CODE | | ||
+ | | 58 | 2 | int16 | - |VlanId| VLAN | | ||
+ | | 59 | 2 | int16 | - |postVlanID| POST VLAN | | ||
+ | | 56 | 6 | mac_address | ||
+ | | 57 | 6 | mac_adress | ||
+ | | 2017 | - | raw | 43823 | MPLS Labels | | ||
+ | |||
+ | **Note:** the EVENT field indicates the event type 1 — send, 2 — receive, \\ | ||
+ | ATTACHMENT sign of an attachment, mail_protocol = 0 — smtp, 1 — pop3, 2 — imap | ||
+ | |||
+ | ^ The raw unparsed metadata export template IPFIX format | ||
+ | ^ № ^ Size in bytes ^ Type ^ IANA ^ Description | ||
+ | | 1001 | 4 | int32 | 43823 | TIME_STAMP | | ||
+ | | 1002 | - | string | ||
+ | | 1003 | 4 | IPv4 | ||
+ | | 1004 | 4 | IPv4 | ||
+ | | 2000 | 8 | int64 | 43823 | SESSION_ID | | ||
+ | | 2013 | 1 | int8 | ||
+ | | 2014 | 1 | int8 | ||
+ | | 2015 | 2 | int16 | 43823 | VDPI_PROTO | The protocol that determined the DPI | | ||
+ | | 2900 | 2 | int16 | 43823 | META_PROTO | Internal protocol identifier | | ||
+ | | 2901 | - | string | ||
+ | | 4 | ||
+ | | 7 | ||
+ | | | ||
+ | | 6 | ||
+ | | 2018 | 4 | int32 | - | TCP Sequence | | ||
+ | | 58 | 2 | int16 | - |VlanId|VLAN| | ||
+ | | 59 | 2 | int16 | - |postVlanID|POST VLAN| | ||
+ | | 56 | 6 | mac_address | ||
+ | | 57 | 6 | mac_adress | ||
+ | | 2017 | - | raw | 43823 |MPLS Labels| | ||
+ | **Note:** | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | |||
+ | Aggregating '' | ||
+ | |||
+ | =====DNS===== | ||
+ | DNS export is configured with the following settings: | ||
+ | <code bash> | ||
+ | ipfix_dev=em1 | ||
+ | ipfix_dns_udp_collectors=1.2.3.4: | ||
+ | ipfix_dns_tcp_collectors=1.2.3.6: | ||
+ | </ | ||
+ | where | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | |||
+ | The format of IPFIX templates for IPV6 differs in the format of the '' | ||
+ | |||
+ | ^ № | ||
+ | | 1103 | 16 | ||
+ | | 1104 | 16 | ||
+ | |||
+ | |||
+ | ^ DNS Export Template IPFIX Format | ||
+ | ^ № | ||
+ | | 1001 | 4 | int32 | 43823 | TIME_STAMP | ||
+ | | 1002 | - | string | ||
+ | | 1003 | 4 | IPv4 | ||
+ | | 1004 | 4 | IPv4 | ||
+ | | 1013 | 2 | int16 | 43823 | SOURCE PORT | | | ||
+ | | 1014 | 2 | int16 | 43823 | DESTINATION PORT | ||
+ | | 2000 | 8 | int64 | 43823 | SESSION_ID | ||
+ | | 3200 | 1 | int8 | ||
+ | | 3201 | - | string | ||
+ | | 3202 | 2 | int16 | 43823 | RRCLASS | ||
+ | | 3203 | 2 | int16 | 43823 | RRTYPE | ||
+ | | 3204 | 4 | int32 | 43823 | TTL | | | ||
+ | | 3205 | - | raw | 43823 | RDATA | | | ||
+ | | 58 | 2 | int16 | - | VlanId | ||
+ | | 59 | 2 | int16 | - | postVlanID | ||
+ | | 56 | 6 | mac_address | ||
+ | | 57 | 6 | mac_adress | ||
+ | | 2017 | - | raw | 43823 | MPLS Labels | ||
+ | | 2016 | 2 | int16 | 43823 | BRIDGE_CHANNEL_NUM | ||
+ | |||
+ | An alternative is to save the data in a local text log. Parameters: | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | |||
+ | Switching to the next file occurs when the file size reaches '' | ||
+ | |||
+ | ajb_save_dns_format : format for writing to a text file | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | |||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | * **'' | ||
+ | |||
+ | Default: '' | ||
+ | |||
+ | =====Sending Template in IPFIX===== | ||
+ | - Transport protocol TCP.\\ The Template is sent once after the TCP session is established. | ||
+ | - Transport protocol UDP.\\ The Template is sent by default every 20 seconds. This is controlled by the '' |