Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| en:ipfixreceiver [2017/03/09 10:11] – [Important changes in version 1.0.3 vs 1.0.2] aalekseenko | en:ipfixreceiver [2019/02/13 16:32] (current) – removed lexx26 | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== IPFIX receiver utility ====== | ||
| - | ===== Introduction ===== | ||
| - | IPFIX receiver is used for receiving an IPFIX (Netflow 10) stream from DPI devices and store the stream to a local file. The stored file can be processed as a text file any unix utilities. | ||
| - | |||
| - | ===== Installation and upgrade ===== | ||
| - | - use VAS Experts repositore according to p.1 of the [[dpi_10_update|DPI installation instruction]]. | ||
| - | - install ipfixreceiver: | ||
| - | - check changes in configuration files for installed version look at part " | ||
| - | |||
| - | ===== Important changes in version 1.0.3 vs 1.0.2 ===== | ||
| - | - chenged configuration file in part of IP address transformation, | ||
| - | - saving data now in separate process, important if DPI has more than 25 000 session per second, it can load upto 2 proccesor cores. In DEBUG loging added check records to controll save processing\\ (a)cnt=NNNNN - send NNNNN buffer\\ (b)cnt=YYYYY - saved YYYYY buffer. | ||
| - | - buffer_size parameter added - size of buffer to interchange between receiver and saver processes, use it in [dump] section, default value - 100000 records (for 20Gbe or 25 000 seesion per second). If the buffer size is not reached then 30sec timeout is used to push buffer into saver process. | ||
| - | |||
| - | ===== Supplied files ===== | ||
| - | - configuration expamples: | ||
| - | / | ||
| - | / | ||
| - | - programm files directory: | ||
| - | - additional files:\\ < | ||
| - | - link to executable: | ||
| - | |||
| - | ===== Additional OS settings ===== | ||
| - | - set iptables for receive external data\\ Ipfixreceiver is requred to open ports that will be used to receive IPFIX streams (in configuration see section [connect])\\ For instance you are using TCP protocol, 1500 port and IP=212.12.11.10\\ < | ||
| - | protocol=tcp | ||
| - | host=212.12.11.10 | ||
| - | port=1500</ | ||
| - | - configure logrotate\\ Example for logrotate file / | ||
| - | / | ||
| - | rotate 5 | ||
| - | missingok | ||
| - | notifempty | ||
| - | compress | ||
| - | size 10M | ||
| - | daily | ||
| - | copytruncate | ||
| - | nocreate | ||
| - | postrotate | ||
| - | endscript | ||
| - | }</ | ||
| - | - configure remove old files. Example, removing old archive files (more then 31 days) with session records in gzip:\\ < | ||
| - | |||
| - | ===== Execution parameters ===== | ||
| - | ipfixreceiver utility has next parameters: | ||
| - | < | ||
| - | где | ||
| - | start - start as service | ||
| - | stop - stop service | ||
| - | state - get state of service | ||
| - | restart - restart service | ||
| - | -v - version info | ||
| - | -f <config file> - config file name (required) | ||
| - | |||
| - | Example: | ||
| - | ipfixreceiver start -f / | ||
| - | </ | ||
| - | ===== Configuration ===== | ||
| - | |||
| - | By default config file / | ||
| - | :!:More information about config parameter you can find by link [[https:// | ||
| - | |||
| - | ==== Logging sections ==== | ||
| - | - loggers - define logging identifiers | ||
| - | - handlers - define used logging workers | ||
| - | - formatters - define used logging formats | ||
| - | |||
| - | ==== logger_root ==== | ||
| - | - level - log level\\ Values:< | ||
| - | CRITICAL | ||
| - | ERROR - errors included | ||
| - | WARNING | ||
| - | INFO - information included | ||
| - | DEBUG - debug messages included | ||
| - | NOTSET | ||
| - | </ | ||
| - | - handlers - used message handlers\\ Example: < | ||
| - | ==== handler_ipfixreceiverlogger ==== | ||
| - | - class - class of the message handler\\ Example: < | ||
| - | - level - log level < | ||
| - | - formatter - name of formatter that is used< | ||
| - | - args - handlers' | ||
| - | ==== formatter_ipfixreceiverlogger ==== | ||
| - | - format - log message format description\\ Example: < | ||
| - | here: | ||
| - | %(name)s | ||
| - | %(levelname)s - level (' | ||
| - | %(asctime)s | ||
| - | %(message)s | ||
| - | </ | ||
| - | - datefmt - date format\\ Example: < | ||
| - | ==== connect ==== | ||
| - | - protocol - protocol(tcp or udp). < | ||
| - | - host - IP or server name or 0.0.0.0 (to receive from all devices). < | ||
| - | - port - port number. < | ||
| - | ==== dump ==== | ||
| - | - rotate_minutes - rotation period in minuties, after it temp file will be moved to dumpfiledir/< | ||
| - | - processcmd - command that will be executed to process new data file after rotation, parameter is full file name.< | ||
| - | - dumpfiledir - directory where received data files will be stored. < | ||
| - | - buffer_size - size of buffer to interchange between receiver and saver processes, use it in [dump] section, default value - 100000 records (for 20Gbe or 25 000 seesion per second). If the buffer size is not reached then 30sec timeout is used to push buffer into saver process. | ||
| - | |||
| - | ==== InfoModel ==== | ||
| - | Блок описывает получаемые данные по IPFIX протоколу. | ||
| - | - InfoElements - параметр с описанием элементов информационной модели для IPFIX < | ||
| - | packetDeltaCount, | ||
| - | protocolIdentifier, | ||
| - | session_id, | ||
| - | где, | ||
| - | session_id - наименование поля из описания IPFIX см. разделы | ||
| - | 43823 - уникальный номер организации (enterprise number) | ||
| - | 1 - уникальный номер поля | ||
| - | UINT64 - тип поля | ||
| - | True - использовать обратный порядок байт (endian). Значения - True или пусто. | ||
| - | </ | ||
| - | Типы полей: | ||
| - | ^ Type ^ Length | ||
| - | | OCTET_ARRAY | ||
| - | | UINT8 | 1 | unsigned8 | ||
| - | | UINT16 | ||
| - | | UINT32 | ||
| - | | UINT64 | ||
| - | | INT8 | 1 | signed8 | ||
| - | | INT16 | 2 | signed16 | ||
| - | | INT32 | 4 | signed32 | ||
| - | | INT64 | 8 | signed64 | ||
| - | | FLOAT32 | ||
| - | | FLOAT64 | ||
| - | | BOOL | 1 | boolean | ||
| - | | MAC_ADDR | ||
| - | | STRING | ||
| - | | SECONDS | ||
| - | | MILLISECONDS | ||
| - | | MICROSECONDS | ||
| - | | NANOSECONDS | ||
| - | | IP4ADDR | ||
| - | | IP6ADDR | ||
| - | |||
| - | Наименование полей и описание можно взять по ссылкам: | ||
| - | - [[statistics_ipfix|Шаблон экспорта Netflow в формате IPFIX]] | ||
| - | - [[li_ipfix|Шаблоны экспорта clickstream и SIP]] | ||
| - | |||
| - | Дополнительная информация: | ||
| - | [[https:// | ||
| - | ==== ExportModel ==== | ||
| - | определяет параметры модели для экспорта, | ||
| - | - Mode - тип используемого экспорта < | ||
| - | ==== ExportModelFile ==== | ||
| - | Описание модели экспорта File. | ||
| - | - Delimiter разделитель полей в строке ( \t - табуляция, | ||
| - | - ExportElements - описание полей которые будут сохранены в файл. < | ||
| - | login | ||
| - | | ||
| - | | ||
| - | host, decodehost | ||
| - | path, decodepath | ||
| - | | ||
| - | | ||
| - | где поля в каждой строке: | ||
| - | имя - наименование поля из информационной модели [InfoModel] (login, session_id и т.п.) | ||
| - | обработчик - процедура обработки поля перед выводом | ||
| - | | ||
| - | | ||
| - | | ||
| - | | ||
| - | | ||
| - | | ||
| - | формат - описание формата для seconds, milliseconds. | ||
| - | | ||
| - | | ||
| - | </ | ||
| - | ===== Проблемы и решения ===== | ||
| - | - как получить версию утилиты? | ||
| - | - можно ли на один порт отправлять IPFIX потоки с разных DPI?\\ Да. Единственное в записываемом потоке их будет не различить. | ||
| - | - как понять, | ||
| - | - все проверено, | ||
| - | - с DPI идет большое количество сессий (более 2 млн сессий/ | ||