en:dpi:dpi_components:utilities:rcollector [Документация VAS Experts]
Документация VAS Experts Документация VAS Experts-mobile
in

Settings

  • Show page
  • Old revisions
  • Export to PDF
  • Fold/unfold all
  • Backlinks
  • Show page
  • Old revisions
  • Export to PDF
  • Fold/unfold all
  • Backlinks

    • Differences

    This shows you the differences between two versions of the page.

    Link to this comparison view

    Next revision    		Previous revision
    en:dpi:dpi_components:utilities:rcollector [2018/07/11 22:51] – created kvazikraven:dpi:dpi_components:utilities:rcollector [2024/10/11 08:31] (current) – removed elena.krasnobryzh
    Line 1: Line 1:
    -====== Utility designed to assemble the IPFIX flows ====== 
    -{{indexmenu_n>3}} 
    -===== Introduction ===== 
      
    -The utility is designed to complement streams such as clickstream, SIP from the sessions stream (netflow) by auxiliary data.  
    - 
    -===== Installation and upgrade ===== 
    -  - add the VAS Experts repository similar to the item 1 of [[en:dpi:dpi_components:platform:dpi_update:dpi_update_10:dpi_10_update:start|DPI installation]] manual. 
    -  - install the rcollector:\\ <code>yum install -y rcollector</code> 
    -  - edit the configuration files in the /etc/rcollector/ directory (see further) 
    - 
    -===== Delivery files ===== 
    -  - configuration examples:\\ <code>/etc/rcollector/ipfixreceiver2.conf - clickstream configuration example (http запросы) 
    -/etc/rcollector/ipfixreceiverflow2.conf - configuration example for retreiving the information about sessions (netflow analogue) 
    -/etc/rcollector/ipfixreceiversip2.conf - configuration example for retreiving the information about sip sessions 
    -/etc/rcollector/rcflowprocess - example of executable file for session handling (netflow) 
    -/etc/rcollector/rcurlprocess - example of executable file for HTTP sessions handling 
    -/etc/rcollector/rcsipprocess - example of executable file for SIP sessions handling 
    -</code> 
    -  - program files are located withing the directory:\\ <code>/usr/local/lib/rcollector.d/</code> 
    -  - auxiliary files:\\ <code>/etc/dpiui/port_proto.txt - information about the resolution of protocol identifier to its name, it is used by the utility to obtain the protocol text name </code> 
    -  - links to the executable module:\\ <code>/usr/local/bin/rcollector -> symlink to the /usr/local/lib/ipfixreceiver.d/rcollector</code> 
    - 
    -===== Configuration ===== 
    -  - create the following directories to place the ipfixreceiver and rcollector files <code> example for device 111:<code> 
    -mkdir -p /var/dump/111/ipfixflow 
    -mkdir -p /var/dump/111/ipfixsip 
    -mkdir -p /var/dump/111/ipfixurl 
    - 
    -mkdir -p /var/collector/111/email 
    -mkdir -p /var/collector/111/ftp 
    -mkdir -p /var/collector/111/http_requests 
    -mkdir -p /var/collector/111/raw_flow 
    -mkdir -p /var/collector/111/sip 
    -mkdir -p /var/collector/111/ssh 
    -</code> 
    -  - copy the /etc/rcollector sample of configuration files to the /etc/rcollector/<NNN> directory, here the <NNN>  - the device identifier <code> example for device 111:<code> 
    -mkdir -p /etc/rcollector/111 
    -cp /etc/rcollector/* /etc/rcollector/111 
    -chmod a+x /etc/rcollector/111/rc* 
    -</code> 
    -  - edit the [[ipfixreceiver|ipfixreceiver]] configuration files :<code> In the following files: ipfixreceiver2.conf, ipfixreceiverflow2.conf, ipfixreceiversip2.conf:<code> 
    -1. specify the configuration of the port used to receive stream data depending on the DPI configuration, for example, for clickstream 1501: 
    -port=1501 
    -2. specify the handler for the received file, for example for the clickstream of device 111: 
    -processcmd=/etc/collector/111/rcurlprocess %%s 
    -3. specify the directory for the received files, for example for clickstream: 
    -dumpfiledir=/var/dump/111/ipfixurl/ 
    - 
    -</code> 
    -  - edit configuration files rcollector. Example for device 111, local ASN = 47438,57451,56613,65535 specify the following variables values in rcflowprocess, rcurlprocess, rcsipprocess files:<code> 
    -chome="/var/collector/111" 
    -cipfix="/etc/rcollector/111" 
    -localASN="47438,57451,56613,65535" 
    -devuid="111" 
    -here 
    -chome - the root directory of the resulting collector files 
    -cipfix - root directory of configuration files 
    -localASN - local autonomous systems of the communications provider  
    -devuid - device number 
    -</code> 
    -  - create a file for log rotation<code>cat /etc/logrotate.d/ipfix 
    -/var/log/dpiui*.log 
    -/var/log/rflowcollector.log 
    -{ 
    -    rotate 5 
    -    missingok 
    -    notifempty 
    -    compress 
    -    size 10M 
    -    daily 
    -    copytruncate 
    -    nocreate 
    -    postrotate 
    -    endscript 
    -} 
    -</code> 
    -  - create jobs that move files to the archive or delete them as in the example: <code> 
    -# dell collector data after 1.5 and 1 days 
    -15 * * * * /bin/find /var/collector/ -name url_\*gz -cmin +2160 -delete > /dev/null 2>&1 
    -05 * * * * /bin/find /var/db/rcollector/ -name \*.val -cmin +120 -delete > /dev/null 2>&1 
    -15 * * * * /bin/find /var/dump/ -name url_\*gz -cmin +1440 -delete > /dev/null 2>&1 
    -</code>