This is an old revision of the document!
Utility designed to assemble the IPFIX flows
Introduction
The utility is designed to complement streams such as clickstream, SIP from the sessions stream (netflow) by auxiliary data.
Installation and upgrade
- add the VAS Experts repository similar to the item 1 of DPI installation manual.
- install the rcollector:
yum install -y rcollector
- edit the configuration files in the /etc/rcollector/ directory (see further)
Delivery files
- configuration examples:
/etc/rcollector/ipfixreceiver2.conf - clickstream configuration example (http requests) /etc/rcollector/ipfixreceiverflow2.conf - configuration example for retreiving the information about sessions (netflow analogue) /etc/rcollector/ipfixreceiversip2.conf - configuration example for retreiving the information about sip sessions /etc/rcollector/rcflowprocess - example of executable file for session handling (netflow) /etc/rcollector/rcurlprocess - example of executable file for HTTP sessions handling /etc/rcollector/rcsipprocess - example of executable file for SIP sessions handling
- program files are located within the directory:
/usr/local/lib/rcollector.d/
- auxiliary files:
/etc/dpiui/port_proto.txt - information about the resolution of protocol identifier to its name, it is used by the utility to obtain the protocol text name
- links to the executable module:
/usr/local/bin/rcollector -> symlink to the /usr/local/lib/ipfixreceiver.d/rcollector
Configuration
- create the following directories to place the ipfixreceiver and rcollector files
example for device 111:<code> mkdir -p /var/dump/111/ipfixflow mkdir -p /var/dump/111/ipfixsip mkdir -p /var/dump/111/ipfixurl mkdir -p /var/collector/111/email mkdir -p /var/collector/111/ftp mkdir -p /var/collector/111/http_requests mkdir -p /var/collector/111/raw_flow mkdir -p /var/collector/111/sip mkdir -p /var/collector/111/ssh
- copy the /etc/rcollector sample of configuration files to the /etc/rcollector/<NNN> directory, here the <NNN> - the device identifier
example for device 111:<code> mkdir -p /etc/rcollector/111 cp /etc/rcollector/* /etc/rcollector/111 chmod a+x /etc/rcollector/111/rc*
- edit the ipfixreceiver configuration files:
In the following files: ipfixreceiver2.conf, ipfixreceiverflow2.conf, ipfixreceiversip2.conf:<code> 1. specify the configuration of the port used to receive stream data depending on the DPI configuration, for example, for clickstream 1501: port=1501 2. specify the handler for the received file, for example for the clickstream of device 111: processcmd=/etc/collector/111/rcurlprocess %%s 3. specify the directory for the received files, for example for clickstream: dumpfiledir=/var/dump/111/ipfixurl/
- edit configuration files rcollector. Example for device 111, local ASN = 47438,57451,56613,65535 specify the following variables values in rcflowprocess, rcurlprocess, rcsipprocess files:
chome="/var/collector/111" cipfix="/etc/rcollector/111" localASN="47438,57451,56613,65535" devuid="111" here chome - the root directory of the resulting collector files cipfix - root directory of configuration files localASN - local autonomous systems of the communications provider devuid - device number
- create a file for log rotation
cat /etc/logrotate.d/ipfix /var/log/dpiui*.log /var/log/rflowcollector.log { rotate 5 missingok notifempty compress size 10M daily copytruncate nocreate postrotate endscript }
- create jobs that move files to the archive or delete them as in the example:
# dell collector data after 1.5 and 1 days 15 * * * * /bin/find /var/collector/ -name url_\*gz -cmin +2160 -delete > /dev/null 2>&1 05 * * * * /bin/find /var/db/rcollector/ -name \*.val -cmin +120 -delete > /dev/null 2>&1 15 * * * * /bin/find /var/dump/ -name url_\*gz -cmin +1440 -delete > /dev/null 2>&1