Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| en:dpi:dpi_components:utilities:ipfixreceiver [2018/05/15 13:59] – [InfoModel] kvazikrav | en:dpi:dpi_components:utilities:ipfixreceiver [2023/09/01 11:09] (current) – removed elena.krasnobryzh | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== IPFIX receiver utility ====== | ||
| - | ===== Introduction ===== | ||
| - | IPFIX receiver is used for receiving an IPFIX (Netflow 10) stream from DPI devices and store the stream to a local file. The stored file can be processed as a text file any unix utilities. | ||
| - | |||
| - | ===== Installation and upgrade ===== | ||
| - | ==== CentOS6 ==== | ||
| - | |||
| - | - use VAS Experts repository according to p.1 of the [[en: | ||
| - | - install ipfixreceiver: | ||
| - | - check changes in configuration files for installed version look at part " | ||
| - | |||
| - | ==== CentOS7 ==== | ||
| - | - use VAS Experts repository according to p.1 of the [[en: | ||
| - | - add the epel repository < | ||
| - | - add the forensics repository < | ||
| - | rpm -Uvh https:// | ||
| - | - install ipfixreceiver: | ||
| - | - check changes in configuration files for installed version look at part " | ||
| - | |||
| - | ===== Important changes in version 1.0.3 vs 1.0.2 ===== | ||
| - | - chenged configuration file in part of IP address transformation, | ||
| - | - saving data now in separate process, important if DPI has more than 25 000 session per second, it can load upto 2 proccesor cores. In DEBUG loging added check records to controll save processing\\ (a)cnt=NNNNN - send NNNNN buffer\\ (b)cnt=YYYYY - saved YYYYY buffer. | ||
| - | - buffer_size parameter added - size of buffer to interchange between receiver and saver processes, use it in [dump] section, default value - 100000 records (for 20Gbe or 25 000 seesion per second). If the buffer size is not reached then 30sec timeout is used to push buffer into saver process. | ||
| - | |||
| - | ===== Supplied files ===== | ||
| - | - configuration expamples: | ||
| - | / | ||
| - | / | ||
| - | - programm files directory: | ||
| - | - additional files:\\ < | ||
| - | - link to executable: | ||
| - | |||
| - | ===== Additional OS settings ===== | ||
| - | - set iptables for receive external data\\ Ipfixreceiver is requred to open ports that will be used to receive IPFIX streams (in configuration see section [connect])\\ For instance you are using TCP protocol, 1500 port and IP=212.12.11.10\\ < | ||
| - | protocol=tcp | ||
| - | host=212.12.11.10 | ||
| - | port=1500</ | ||
| - | - configure logrotate\\ Example for logrotate file / | ||
| - | / | ||
| - | rotate 5 | ||
| - | missingok | ||
| - | notifempty | ||
| - | compress | ||
| - | size 10M | ||
| - | daily | ||
| - | copytruncate | ||
| - | nocreate | ||
| - | postrotate | ||
| - | endscript | ||
| - | }</ | ||
| - | - configure remove old files. Example, removing old archive files (more then 31 days) with session records in gzip:\\ < | ||
| - | |||
| - | ===== Execution parameters ===== | ||
| - | ipfixreceiver utility has next parameters: | ||
| - | < | ||
| - | где | ||
| - | start - start as service | ||
| - | stop - stop service | ||
| - | state - get state of service | ||
| - | restart - restart service | ||
| - | -v - version info | ||
| - | -f <config file> - config file name (required) | ||
| - | |||
| - | Example: | ||
| - | ipfixreceiver start -f / | ||
| - | </ | ||
| - | ===== Configuration ===== | ||
| - | |||
| - | By default config file / | ||
| - | :!: More information about config parameter you can find by link [[https:// | ||
| - | |||
| - | ==== Logging sections ==== | ||
| - | - loggers - define logging identifiers | ||
| - | - handlers - define used logging workers | ||
| - | - formatters - define used logging formats | ||
| - | |||
| - | ==== logger_root ==== | ||
| - | - level - log level\\ Values:< | ||
| - | CRITICAL | ||
| - | ERROR - errors included | ||
| - | WARNING | ||
| - | INFO - information included | ||
| - | DEBUG - debug messages included | ||
| - | NOTSET | ||
| - | </ | ||
| - | - handlers - used message handlers\\ Example: < | ||
| - | ==== handler_ipfixreceiverlogger ==== | ||
| - | - class - class of the message handler\\ Example: < | ||
| - | - level - log level < | ||
| - | - formatter - name of formatter that is used< | ||
| - | - args - handlers' | ||
| - | ==== formatter_ipfixreceiverlogger ==== | ||
| - | - format - log message format description\\ Example: < | ||
| - | here: | ||
| - | %(name)s | ||
| - | %(levelname)s - level (' | ||
| - | %(asctime)s | ||
| - | %(message)s | ||
| - | </ | ||
| - | - datefmt - date format\\ Example: < | ||
| - | ==== connect ==== | ||
| - | - protocol - protocol(tcp or udp). < | ||
| - | - host - IP or server name or 0.0.0.0 (to receive from all devices). < | ||
| - | - port - port number. < | ||
| - | ==== dump ==== | ||
| - | - rotate_minutes - rotation period in minuties, after it temp file will be moved to dumpfiledir/< | ||
| - | - processcmd - command that will be executed to process new data file after rotation, parameter is full file name.< | ||
| - | - dumpfiledir - directory where received data files will be stored. < | ||
| - | - buffer_size - size of buffer to interchange between receiver and saver processes, use it in [dump] section, default value - 100000 records (for 20Gbe or 25 000 seesion per second). If the buffer size is not reached then 30sec timeout is used to push buffer into saver process. | ||
| - | |||
| - | ==== InfoModel ==== | ||
| - | the section describes IPFIX receiveng template. | ||
| - | - InfoElements - parameter with description of information model elements for IPFIX template < | ||
| - | packetDeltaCount, | ||
| - | protocolIdentifier, | ||
| - | session_id, | ||
| - | here, | ||
| - | session_id - field name according to IPFIX description table (see according sections) | ||
| - | 43823 - enterprise number or general IPFIX protocol number | ||
| - | 1 - unique field name | ||
| - | UINT64 - field type | ||
| - | True - endian (True or empty). | ||
| - | </ | ||
| - | Field types:\\ | ||
| - | ^ Type ^ Length | ||
| - | | OCTET_ARRAY | ||
| - | | UINT8 | 1 | unsigned8 | ||
| - | | UINT16 | ||
| - | | UINT32 | ||
| - | | UINT64 | ||
| - | | INT8 | 1 | signed8 | ||
| - | | INT16 | 2 | signed16 | ||
| - | | INT32 | 4 | signed32 | ||
| - | | INT64 | 8 | signed64 | ||
| - | | FLOAT32 | ||
| - | | FLOAT64 | ||
| - | | BOOL | 1 | boolean | ||
| - | | MAC_ADDR | ||
| - | | STRING | ||
| - | | SECONDS | ||
| - | | MILLISECONDS | ||
| - | | MICROSECONDS | ||
| - | | NANOSECONDS | ||
| - | | IP4ADDR | ||
| - | | IP6ADDR | ||
| - | |||
| - | Field names and their description: | ||
| - | - [[en: | ||
| - | - [[en: | ||
| - | - [[en: | ||
| - | |||
| - | |||
| - | additional information: | ||
| - | [[https:// | ||
| - | ==== ExportModel ==== | ||
| - | defines the export model parameters, reserved for future use. | ||
| - | - Mode - type used export (File only) < | ||
| - | ==== ExportModelFile ==== | ||
| - | defines the File export model. | ||
| - | - Delimiter field delimiter ( \t - TAB, examples - |,;) < | ||
| - | - ExportElements - fields description that will be saved to file. < | ||
| - | login | ||
| - | | ||
| - | | ||
| - | host, decodehost | ||
| - | path, decodepath | ||
| - | | ||
| - | | ||
| - | here: | ||
| - | name - field name from infornation model described before [InfoModel] (login, session_id и т.п.) | ||
| - | worker - internal transformation routune | ||
| - | | ||
| - | | ||
| - | | ||
| - | | ||
| - | | ||
| - | | ||
| - | | ||
| - | format - format for seconds, milliseconds. | ||
| - | | ||
| - | | ||
| - | </ | ||
| - | |||
| - | ==== Centos7 service for ipfixreceiver ==== | ||
| - | Centos7 service creation step by step, service name **ipfix1**, config file name **/ | ||
| - | Create the file / | ||
| - | < | ||
| - | [Unit] | ||
| - | Description=ipfix test restart | ||
| - | After=network.target | ||
| - | After=syslog.target | ||
| - | |||
| - | [Service] | ||
| - | Type=forking | ||
| - | PIDFile=/ | ||
| - | ExecStart=/ | ||
| - | ExecStop=/ | ||
| - | ExecReload=/ | ||
| - | Restart=always | ||
| - | RestartSec=10s | ||
| - | |||
| - | [Install] | ||
| - | WantedBy=multi-user.target | ||
| - | </ | ||
| - | Execute to register and run service: | ||
| - | systemctl enable ipfix1.service | ||
| - | systemctl start ipfix1.service | ||
| - | systemctl daemon-reload | ||
| - | |||
| - | Check status: | ||
| - | systemctl status ipfix1.service -l | ||
| - | :!: check service start after reboot | ||
| - | |||
| - | ===== Troubleshooting ===== | ||
| - | - how can I get version?\\ Use:\\ < | ||
| - | - can I send IPFIX streams fom differ DPI deveces to one port?\\ Yes, for UDP. Ipfixreceiver will write it to the same output files. | ||
| - | - How can I learn that utility is working?\\ a) check port is lissening, example 1500:< | ||
| - | - all checked, but no data is received?\\ a) check iptables rules.\\ b) check ipfixreceiver configuration for IP server address. | ||
| - | - DPI sends more then 2 millions session per second, in DEBUG level I see that counter in saver is slower | ||