| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| en:dpi:dpi_components:utilities:rcollector [2023/08/28 15:21] – ↷ Links adapted because of a move operation elena.krasnobryzh | en:dpi:dpi_components:utilities:rcollector [2024/10/11 08:31] (current) – removed elena.krasnobryzh |
|---|
| ====== Utility designed to assemble the IPFIX flows ====== | |
| {{indexmenu_n>3}} | |
| ===== Introduction ===== | |
| |
| The utility is designed to complement streams such as clickstream, SIP from the sessions stream (netflow) by auxiliary data. | |
| |
| ===== Installation and upgrade ===== | |
| - add the VAS Experts repository similar to the item 1 of [[en:dpi:update:previous:dpi_update_10:dpi_10_update:start|DPI installation]] manual. | |
| - install the rcollector:\\ <code>yum install -y rcollector</code> | |
| - edit the configuration files in the /etc/rcollector/ directory (see further) | |
| |
| ===== Delivery files ===== | |
| - configuration examples:\\ <code>/etc/rcollector/ipfixreceiver2.conf - clickstream configuration example (http requests) | |
| /etc/rcollector/ipfixreceiverflow2.conf - configuration example for retreiving the information about sessions (netflow analogue) | |
| /etc/rcollector/ipfixreceiversip2.conf - configuration example for retreiving the information about sip sessions | |
| /etc/rcollector/rcflowprocess - example of executable file for session handling (netflow) | |
| /etc/rcollector/rcurlprocess - example of executable file for HTTP sessions handling | |
| /etc/rcollector/rcsipprocess - example of executable file for SIP sessions handling | |
| </code> | |
| - program files are located within the directory:\\ <code>/usr/local/lib/rcollector.d/</code> | |
| - auxiliary files:\\ <code>/etc/dpiui/port_proto.txt - information about the resolution of protocol identifier to its name, it is used by the utility to obtain the protocol text name </code> | |
| - links to the executable module:\\ <code>/usr/local/bin/rcollector -> symlink to the /usr/local/lib/ipfixreceiver.d/rcollector</code> | |
| |
| ===== Configuration ===== | |
| - create the following directories to place the ipfixreceiver and rcollector files <code>example for device 111:<code> | |
| mkdir -p /var/dump/111/ipfixflow | |
| mkdir -p /var/dump/111/ipfixsip | |
| mkdir -p /var/dump/111/ipfixurl | |
| |
| mkdir -p /var/collector/111/email | |
| mkdir -p /var/collector/111/ftp | |
| mkdir -p /var/collector/111/http_requests | |
| mkdir -p /var/collector/111/raw_flow | |
| mkdir -p /var/collector/111/sip | |
| mkdir -p /var/collector/111/ssh | |
| </code> | |
| - copy the /etc/rcollector sample of configuration files to the /etc/rcollector/<NNN> directory, here the <NNN> - the device identifier <code>example for device 111:<code> | |
| mkdir -p /etc/rcollector/111 | |
| cp /etc/rcollector/* /etc/rcollector/111 | |
| chmod a+x /etc/rcollector/111/rc* | |
| </code> | |
| - edit the [[en:dpi:dpi_components:utilities:ipfixreceiver2|ipfixreceiver]] configuration files:<code>In the following files: ipfixreceiver2.conf, ipfixreceiverflow2.conf, ipfixreceiversip2.conf:<code> | |
| 1. specify the configuration of the port used to receive stream data depending on the DPI configuration, for example, for clickstream 1501: | |
| port=1501 | |
| 2. specify the handler for the received file, for example for the clickstream of device 111: | |
| processcmd=/etc/collector/111/rcurlprocess %%s | |
| 3. specify the directory for the received files, for example for clickstream: | |
| dumpfiledir=/var/dump/111/ipfixurl/ | |
| |
| </code> | |
| - edit configuration files rcollector. Example for device 111, local ASN = 47438,57451,56613,65535 specify the following variables values in rcflowprocess, rcurlprocess, rcsipprocess files:<code> | |
| chome="/var/collector/111" | |
| cipfix="/etc/rcollector/111" | |
| localASN="47438,57451,56613,65535" | |
| devuid="111" | |
| </code> here chome - the root directory of the resulting collector files \\ cipfix - root directory of configuration files \\ localASN - local autonomous systems of the communications provider \\ devuid - device number. | |
| - create a file for log rotation<code>cat /etc/logrotate.d/ipfix | |
| /var/log/dpiui*.log | |
| /var/log/rflowcollector.log | |
| { | |
| rotate 5 | |
| missingok | |
| notifempty | |
| compress | |
| size 10M | |
| daily | |
| copytruncate | |
| nocreate | |
| postrotate | |
| endscript | |
| } | |
| </code> | |
| - create jobs that move files to the archive or delete them as in the example: <code> | |
| # dell collector data after 1.5 and 1 days | |
| 15 * * * * /bin/find /var/collector/ -name url_\*gz -cmin +2160 -delete > /dev/null 2>&1 | |
| 05 * * * * /bin/find /var/db/rcollector/ -name \*.val -cmin +120 -delete > /dev/null 2>&1 | |
| 15 * * * * /bin/find /var/dump/ -name url_\*gz -cmin +1440 -delete > /dev/null 2>&1 | |
| </code> | |
