Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revision | |||
| en:dpi:dpi_components:utilities:oldutility:ipfixreceiver [2023/09/01 08:00] – elena.krasnobryzh | en:dpi:dpi_components:utilities:oldutility:ipfixreceiver [2024/09/26 15:29] (current) – external edit 127.0.0.1 | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== ipfixreceiver ====== | + | ====== |
| + | ===== Introduction ===== | ||
| + | |||
| + | The utility is designed to receive data stream from devices using the IPFIX protocol and save the data as a file for subsequent processing by other means. | ||
| + | |||
| + | ===== Installation and Update ===== | ||
| + | ==== CentOS6 ==== | ||
| + | |||
| + | - add the VAS Experts repository < | ||
| + | rpm -Uvh http:// | ||
| + | - instal the ipfixreceiver: | ||
| + | - check for the changes in the configuration files so they to be consistent with ipfixreceiver current version, see the " | ||
| + | |||
| + | ==== CentOS7 ==== | ||
| + | - add the VAS Experts repository < | ||
| + | rpm -Uvh http:// | ||
| + | - install the epel repository < | ||
| + | - install the forencis repository: < | ||
| + | rpm -Uvh https:// | ||
| + | - install the ipfixreceiver: | ||
| + | yum -y install | ||
| + | yum -y install ipfixreceiver --disablerepo=forensics</ | ||
| + | - check for the changes in the configuration files so they to be consistent with ipfixreceiver current version, see the " | ||
| + | |||
| + | ===== Important changes in version 1.0.3 relative to 1.0.2 one ===== | ||
| + | - the configuration file has been changed with respect to IP address translation, | ||
| + | - the process of information saving has been allocated to a separate process; remember that when dealing with a large number of sessions (> 25k sessions per second) the process will completely load 2 processor cores. In order to check that the process has time to process the entire data stream the following messages are added in the DEBUG mode:\\ (a)cnt=NNNNN - the buffer has been sent with the given number\\ (b)cnt=YYYYY - the buffer with the given number is saved. | ||
| + | - a new '' | ||
| + | |||
| + | ===== The files supplied with the ipfixreceiver ===== | ||
| + | - configuration examples:\\ < | ||
| + | / | ||
| + | / | ||
| + | - program files are located under the:\\ < | ||
| + | - auxiliary files:\\ < | ||
| + | it is used by the utility to get the protocol text-based name by its identifier</ | ||
| + | - links to the executables: | ||
| + | |||
| + | ===== Additional OS settings ===== | ||
| + | - configure iptables to accept external data\\ For ipfixreceiver to work properly, you should open the ports that will also be used in the [connect] section of the configuration. For example, you use the TCP protocol, port 1500 and IP=212.12.11.10\\ < | ||
| + | protocol=tcp | ||
| + | host=212.12.11.10 | ||
| + | port=1500</ | ||
| + | - configure log rotation\\ An example of rotation for the / | ||
| + | / | ||
| + | rotate 5 | ||
| + | missingok | ||
| + | notifempty | ||
| + | compress | ||
| + | size 10M | ||
| + | daily | ||
| + | copytruncate | ||
| + | nocreate | ||
| + | postrotate | ||
| + | endscript | ||
| + | }</ | ||
| + | - Configure the deleting of old files. For example, deleting old archives (more than 31 days) containing sessions records packed with gzip:\\ < | ||
| + | |||
| + | ===== Ipfixreceiver startup options ===== | ||
| + | The ipfixreceiver utility has the following startup options: | ||
| + | < | ||
| + | где | ||
| + | start - start as a service | ||
| + | stop - service stop | ||
| + | state - get the service state | ||
| + | restart - service restart | ||
| + | -v - show version info | ||
| + | -f <config file> - specify the configuration file for the service to start | ||
| + | |||
| + | Example: | ||
| + | ipfixreceiver start -f / | ||
| + | </ | ||
| + | ===== Configuration ===== | ||
| + | |||
| + | The default configuration file is / | ||
| + | :!:More information on configuring logging can be found here: [[https:// | ||
| + | |||
| + | ==== Service sections ==== | ||
| + | - loggers - specifies the log identifiers used | ||
| + | - handlers - specifies the handlers used to save the log | ||
| + | - formatters - specifies the formats used for the log | ||
| + | |||
| + | ==== logger_root ==== | ||
| + | - level - specifies the logging level (upper level)\\ Possible values are: < | ||
| + | CRITICAL | ||
| + | ERROR - including errors | ||
| + | WARNING | ||
| + | INFO - including information | ||
| + | DEBUG - including debug messages | ||
| + | NOTSET | ||
| + | </ | ||
| + | - handlers - message handlers used\\ Example: < | ||
| + | ==== handler_ipfixreceiverlogger ==== | ||
| + | - class - handler class\\ Example: < | ||
| + | - level - message level < | ||
| + | - formatter - message format name< | ||
| + | - args - handler parameters < | ||
| + | ==== formatter_ipfixreceiverlogger ==== | ||
| + | - format - message format description\\ Example: < | ||
| + | here | ||
| + | %(name)s | ||
| + | %(levelname)s - message level (' | ||
| + | %(asctime)s | ||
| + | %(message)s | ||
| + | </ | ||
| + | - datefmt - date format description\\ Example: < | ||
| + | ==== connect ==== | ||
| + | - protocol - protocl (tcp or udp). < | ||
| + | - host - server IP or its name. < | ||
| + | - port - port number. < | ||
| + | ==== dump ==== | ||
| + | - rotate_minutes is the period in minutes, after which the temporary file in dumpfiledir/< | ||
| + | < | ||
| + | - processcmd is the command that will be launched at the end of the file rotation, the file name parameter with the path to it.< | ||
| + | - dumpfiledir is a directory to store the files with data received. < | ||
| + | - buffer_size is the size of the i/o buffer between the process of receiving and writing to a file, it is used in the [dump] section, the default value of the parameter is 100000 records (it is focused on 20 Gbit traffic or 25 000 sessions per second). If the number of sessions per second is considerably less than the mentioned value, then you should to change this parameter proportionally. | ||
| + | |||
| + | ==== InfoModel ==== | ||
| + | The block specifies the data received via the IPFIX protocol. | ||
| + | - InfoElements - parameter describing the information model elements for IPFIX< | ||
| + | packetDeltaCount, | ||
| + | protocolIdentifier, | ||
| + | session_id, | ||
| + | here, | ||
| + | session_id - is the name of the field from the IPFIX description, | ||
| + | 43823 - unique organization number (enterprise number) | ||
| + | 1 - unique field number | ||
| + | UINT64 - field type | ||
| + | True - use reverse byte order (endian). Possible values are: True or empty. | ||
| + | </ | ||
| + | Field types:\\ | ||
| + | ^ Type ^ Length | ||
| + | | OCTET_ARRAY | ||
| + | | UINT8 | 1 | unsigned8 | ||
| + | | UINT16 | ||
| + | | UINT32 | ||
| + | | UINT64 | ||
| + | | INT8 | 1 | signed8 | ||
| + | | INT16 | 2 | signed16 | ||
| + | | INT32 | 4 | signed32 | ||
| + | | INT64 | 8 | signed64 | ||
| + | | FLOAT32 | ||
| + | | FLOAT64 | ||
| + | | BOOL | 1 | boolean | ||
| + | | MAC_ADDR | ||
| + | | STRING | ||
| + | | SECONDS | ||
| + | | MILLISECONDS | ||
| + | | MICROSECONDS | ||
| + | | NANOSECONDS | ||
| + | | IP4ADDR | ||
| + | | IP6ADDR | ||
| + | |||
| + | The field names and their description can be accessed from the following links:\\ | ||
| + | - [[en: | ||
| + | - [[en: | ||
| + | - [[en: | ||
| + | |||
| + | |||
| + | Additional information: | ||
| + | [[https:// | ||
| + | ==== ExportModel ==== | ||
| + | specifies the model parameters used for export, is reserved for future use. | ||
| + | - Mode - the type of export used< | ||
| + | ==== ExportModelFile ==== | ||
| + | Description of the File export model. | ||
| + | - Delimiter ( \t - tabulation, more examples - |,;) < | ||
| + | - ExportElements - description of the fields that will be saved to the file.< | ||
| + | | ||
| + | | ||
| + | | ||
| + | host, decodehost | ||
| + | path, decodepath | ||
| + | | ||
| + | | ||
| + | where the fields in each row are the following: | ||
| + | name - the field name from the information model [InfoModel] (login, session_id and etc.) | ||
| + | handler - field processing procedure before output | ||
| + | | ||
| + | | ||
| + | | ||
| + | | ||
| + | | ||
| + | | ||
| + | format - format description for seconds, milliseconds. | ||
| + | | ||
| + | | ||
| + | </ | ||
| + | |||
| + | ==== Creation of systemd ipfixreceiver service in Centos7 ==== | ||
| + | Step-by-step creation of service in Centos 7, here the service name is ** ipfix1 **, its configuration is in the **/ | ||
| + | Create the / | ||
| + | < | ||
| + | [Unit] | ||
| + | Description=ipfix test restart | ||
| + | After=network.target | ||
| + | After=syslog.target | ||
| + | |||
| + | [Service] | ||
| + | Type=forking | ||
| + | PIDFile=/ | ||
| + | ExecStart=/ | ||
| + | ExecStop=/ | ||
| + | ExecReload=/ | ||
| + | Restart=always | ||
| + | RestartSec=10s | ||
| + | |||
| + | [Install] | ||
| + | WantedBy=multi-user.target | ||
| + | </ | ||
| + | Issue the following commands: | ||
| + | systemctl enable ipfix1.service | ||
| + | systemctl start ipfix1.service | ||
| + | systemctl daemon-reload | ||
| + | |||
| + | Check whether the service is running: | ||
| + | systemctl status ipfix1.service -l | ||
| + | :!: **Do not forget to check the service status after rebooting!** | ||
| + | |||
| + | ===== Troubleshooting ===== | ||
| + | - how to get utility version?\\ You should use the following commands:\\ < | ||
| + | - Is it allowed to send IPFIX flows from different DPI to one port?\\ Yes, it is. The only thing is that they can not be distinguished in the recorded flow. | ||
| + | - How can I understand that the utility works properly?\\ a) check that the port specified in the configuration file is listened on by the utility, for example 1500:< | ||
| + | - everything is checked, but the messages are not received?\\ a) it seems you have forgotten to open port in iptables\\ b) it seems you have initialized ipfixreceiver with the wrong server IP. | ||
| + | - a huge number of sessions (more than 2 million sessions/ | ||