IPFIX-balancer

The key task of the module is retransmission (replication and load balancing) of all IPFIX data flows exported from DPI (Fullflow/Clickstream/DNS flow): it receives IPFIX from fastDPI and then retransmits it to other hosts according to the settings. Additionally, a recording mode can be implemented. The IPFIX-balancer operates with data portions (messages) when distributing them between data collection nodes (receivers/collectors). It has two main modes of data distribution:

The "RoundRobin" mode distributes (balances) the IPFIX stream by sending one message to each specified host.
It sends one message to one receiving node → collector → sends the next message to the next available receiving host → collector, and so on in a loop until all messages are processed.

The "All" mode replicates the IPFIX stream.
It copies the received messages from DPI to all receiving nodes — collectors.

The module can work with both UDP and TCP, and it is possible to bind a specific traffic type handler (Fullflow/Clickstream/DNS flow) to a CPU core, ensuring load distribution across CPUs.

The balancer allows the following tasks to be performed:

Traffic distribution between data collection and storage nodes, in case QoE is working in "Cluster" mode, where data storage is performed on different servers.

Data duplication in replication mode, between data collection and storage nodes, to address redundancy or simultaneous export of the stream in different directions, or between different receiving hosts (Fullflow/Clickstream/DNS flow).

Redirecting the flow (Fullflow/Clickstream/DNS flow) from DPI without changing the configuration or requiring a fastdpi process restart.
When the data stream is exported to the balancer at 127.0.0.1, running on the same host as DPI.
In this case, if the destination address of the stream or collector address changes, you need to make these changes in the IPFIX-balancer configuration file and restart it without affecting the DPI-processed traffic.

Management

Installation:

yum install fast_ipfix_balancer

It is recommended to install the IPFIX-balancer module on the same server where DPI is installed.

Stopping and restarting:

ipfix-balancer-stop ipfix-balancer-restart

Viewing logs:

journalctl -xe | grep IPFIXcol2 journalctl -f | grep IPFIXcol2

Determine which CPU the balancing process belongs to. CPU is set by the parameter IPFIX_xx_TASKSET, description in the section Working in CLI → Configuration.

ps -ax -o %cpu,psr,comm | grep ipfix

Working in GUI

IPFIX-balancer log files can be viewed in the "Administrator" section → "IPFIX-balancer Logs"

Adding Equipment

To work with the IPFIX-balancer, you need to set up the appropriate equipment. To do this:

Go to the Administrator section → Equipment;

Click + above the equipment table;

In the window that appears, select the equipment type — IPFIX-balancer Server;

Fill in the remaining fields and save the equipment.

Detailed description of working with the Equipment section — Equipment Management.

Case 1. Replication

In the Administrator section → IPFIX-balancer Configuration, select the IPFIX server and add a receiver (by clicking + above the receiver table);

Select the value "All" in the “Balancer” field.

Specify Port Type, Balancer Subreceivers, Balancer Subreceiver Type.

It is recommended to set the same value for the Port Type and Balancer Subreceiver Type parameters.

Configure the other receiver parameters. Their description is provided in the section Parameter Description.

As a result, we will get two identical streams on two different hosts.

Receiver settings example for replication case:

IPFIX-balancer logs can be viewed in the Administrator section → IPFIX-balancer Logs

Case 2. Load Balancing

In the Administrator section → IPFIX-balancer Configuration, select the IPFIX server and add a receiver (by clicking + above the receiver table);

Select the value "RoundRobin" in the “Balancer” field.

Specify Port Type, Balancer Subreceivers, Balancer Subreceiver Type.
It is recommended to set the same value for the Port Type and Balancer Subreceiver Type parameters.

Configure the other receiver parameters. Their description is provided in the section Parameter Description.

As a result, we will get an even distribution of packets across all hosts.

Receiver settings example for load balancing case:

If one of the balancing modes (All or RoundRobin) is enabled, IPFIXCol2 and the subreceivers to which it sends packets are activated.

Case 3. Recording Mode

In the Administrator section → IPFIX-balancer Configuration, select the IPFIX server and add a receiver (by clicking + above the receiver table);

Select the value "Disabled" in the “Balancer” field;

Configure the other receiver parameters. Their description is provided in the section Parameter Description.

In this case, the IPFIX receiver works, writes data to a file, and can export IPFIX to the IP addresses specified in the "Export" field.

The IPFIX export configuration on DPI is described in the following sections: Netflow Clickstream, DNS, etc.

Receiver settings example for recording case:

Working in CLI

Configuration

The balancer configuration file is located at /var/fast_ipfix_balancer/backend/.env

The configuration consists of parameters in the IPFIX_xx_yy format, where:

xx — stream type:

FULLFLOW — Fullflow

CLICKSTREAM — Clickstream

DNSFLOW — DNS flow

yy — parameter name:

PORT_TYPE[0] — port type.
Possible values:

tcp (recommended)

udp

PORT[0] — any desired port, it must be unique for each stream.
Default values:

1500 for Fullflow

1501 for Clickstream

ROTATE_MINUTES[0] — Receiver dump rotation period or data upload period to the database. The Clickstream rotation period should be greater than or equal to the same period for Fullflow. When the Fullflow dump rotates, a HUP signal is sent to synchronize the streams. Do not change this parameter unnecessarily
Default values:

10 minutes for Fullflow

12 minutes for Clickstream

ROTATE_DELAY_SECONDS[0] — Dump rotation in seconds. Same as rotation in minutes, only in seconds. If this value is enabled, the value in minutes is ignored.

FW_MAX_QUEUE_SIZE[0] — Rotation by flow count. Works simultaneously with rotation in minutes or seconds.

DUMP_INSERT_PROCESSES[0] — Receiver(s) to which data is exported.
Format: 10.0.0.2/9920/tcp,10.0.0.3/3440/udp.

DPI_ID[0] — DPI number.
Default value: -1.
If the value is -1 or not specified, the DPI number is determined by the order of the recipient in the list.

BALANCER[0] — Balancer type.
Possible values:

RoundRobin — distributes the IPFIX stream

All — replicates the IPFIX stream

BALANCER_SUB[0] — Receiver(s) to which data is distributed or replicated.
Format: 10.0.0.2/9920,10.0.0.3/3440

BALANCER_SUB_PROTO[0] — Port type.
Possible values:

tcp (recommended)

udp

TASKSET — CPU number on which the traffic handler will work.

Example of Configuration File

# Fullflow is distributed among three QoE servers IPFIX_FULLFLOW_PORT_TYPE[0]=tcp IPFIX_FULLFLOW_PORT[0]=1500 IPFIX_FULLFLOW_BALANCER[0]=RoundRobin IPFIX_FULLFLOW_BALANCER_SUB[0]=10.19.3.21/1500,10.19.3.23/1600,10.19.3.24/1600 IPFIX_FULLFLOW_BALANCER_SUB_PROTO[0]=tcp IPFIX_FULLFLOW_BALANCER_TASKSET[0]=61 # Clickstream is distributed between two QoE servers: 10.19.3.21/10102 и 10.19.3.22/10102 IPFIX_CLICKSTREAM_PORT_TYPE[0]=tcp IPFIX_CLICKSTREAM_PORT[0]=1501 IPFIX_CLICKSTREAM_BALANCER[0]=RoundRobin IPFIX_CLICKSTREAM_BALANCER_SUB[0]=10.19.3.21/1501,10.19.3.23/1601 IPFIX_CLICKSTREAM_BALANCER_SUB_PROTO[0]=tcp IPFIX_CLICKSTREAM_BALANCER_TASKSET[0]=62 # DNS flow is distributed between two QoE servers: 10.19.3.21/10103 и 10.19.3.22/10103 IPFIX_DNSFLOW_PORT_TYPE[0]=tcp IPFIX_DNSFLOW_PORT[0]=1101 IPFIX_DNSFLOW_BALANCER[0]=RoundRobin IPFIX_DNSFLOW_BALANCER_SUB[0]=10.19.3.21/1101,10.19.3.23/2101 IPFIX_DNSFLOW_BALANCER_SUB_PROTO[0]=tcp IPFIX_DNSFLOW_BALANCER_TASKSET[0]=63

Parameter Description

Parameter name in GUI	Parameter name in CLI	Description	Possible values and format	Notes
Receiver type	FULLFLOW CLICKSTREAM DNSFLOW	Receiver type: NetFlow or Clickstream	Values: - NetFlow - receiver for NetFlow data - Clickstream - receiver for Clickstream data
Port type	PORT_TYPE[0]	Protocol type used by the receiver to transfer data	Values: - TCP - UDP	It is recommended to use TCP
Port	PORT[0]	Port through which the receiver listens to traffic	Default values: - 1500 for NetFlow - 1501 for Clickstream Any port can be specified, as long as it is unique for each flow
Rotate in minutes	ROTATE_MINUTES[0]	Dump rotation period for the receiver or the period for uploading data to the database	Default values: - 10 minutes for NetFlow - 12 minutes for Clickstream The Clickstream dump rotation period should be equal to or longer than that of the NetFlow. When NetFlow dump rotates, an HUP signal is sent to synchronize the flows.	This only applies in recording mode Do not change this parameter unless necessary
Rotate in seconds	ROTATE_DELAY_SECONDS[0]	Dump rotation period for the receiver or the period for uploading data to the database	This only applies in recording mode If this value is enabled, the value in minutes is ignored.
Rotate by flow	FW_MAX_QUEUE_SIZE[0]	Dump rotation period based on the number of flow entries	This only applies in recording mode This works alongside minute or second-based rotation.
Export	DUMP_INSERT_PROCESSES[0]	Receiver addresses to which data is exported	Format: 10.0.0.2/9920/tcp,10.0.0.3/3440/udp	This only applies in recording mode
DPI ID	DPI_ID[0]	Adds the DPI ID to the statistics for easier identification	Default value: -1 If the value is -1 or not specified, the DPI number is determined by the order of the receiver in the list.	This only applies in recording mode
Load balancer	BALANCER[0]	Load balancing type	Values: - Disabled — disables load balancing, enables recording mode - RoundRobin — distributes the IPFIX stream - All — replicates the IPFIX stream
Balancer subreceivers	BALANCER_SUB[0]	Addresses of subreceivers to which distributed or replicated data is sent	Format: 10.0.0.2/9920,10.0.0.3/3440	Applies in replication and load balancing modes
Subreceiver protocol type	BALANCER_SUB_PROTO[0]	Protocol type used by the receiver to transfer data to subreceivers	Values: - TCP - UDP	Applies in replication and load balancing modes It is recommended to use TCP
Balancer auto-creation	Automatically creates and launches subreceivers	Values: - Disabled — subreceivers must be manually specified - Enabled — subreceivers are automatically created and launched	If QoE and IPFIX-balancer are installed on different servers, it is recommended to set this value to "Disabled."