3 DPDK Version Configuration

DPDK (Data Plane Development Kit) allows working with network cards directly without actually using the Linux kernel. This improves the performance of the solution. DPDK supports many more models of network cards than pf_ring, and a much richer interface. So it allows you to implement various working schemes, suitable for 10G, 25G, 40G, 100G traffic, etc.

System Preparation

The first step to work with DPDK is to take the network cards out of the control of the operating system. DPDK works with PCI devices, that can be displayed with the command:

> lspci|grep Eth
41:00.0 Ethernet controller: Intel Corporation Ethernet Controller XXV710 for 25GbE SFP28 (rev 02)
41:00.1 Ethernet controller: Intel Corporation Ethernet Controller XXV710 for 25GbE SFP28 (rev 02)
c6:00.0 Ethernet controller: Broadcom Inc. and subsidiaries BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet Controller (rev 01)
c6:00.1 Ethernet controller: Broadcom Inc. and subsidiaries BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet Controller (rev 01)
>

This command will list all PCI ethernet devices. Each line starts with the system PCI device identifier - these PCI identifiers are the unique for the network card in the DPDK.

Transferring the card to DPDK mode (disconnecting from the system network driver) is carried out by the dpdk-devbind.py utility from the DPDK:

# Example - devices 41:00.0 and 41:00.1 transfer to the DPDK mode
>insmod $RTE/module/igb_uio.ko
# 25G NICs
>$RTE/bin/dpdk-devbind.py --bind igb_uio 0000:41:00.0
>$RTE/bin/dpdk-devbind.py --bind igb_uio 0000:41:00.1

here, igb_uio - is UIO driver. The system uio_pci_generic or igb_uio from the DPDK can act as a uio driver. Usually uio_pci_generic is used for modern cards, and igb_uio for the older ones, for details see DPDK Linux Drivers. Uio-driver is only needed to register interrupts of network cards (e.g. to recognize link down/link up), and is not involved in receiving and sending data packets.

When switching cards to DPDK mode, be careful not to accidentally switch the server's control interface to DPDK mode - the connection with the server will be interrupted immediately!

To see if the card is properly initialized to work with DPDK, use the command

> $RTE/bin/dpdk-devbind.py --status

If the cards are in DPDK mode, you will see them in Network devices using DPDK-compatible driver section:

> $RTE/bin/dpdk-devbind.py --status
Network devices using DPDK-compatible driver
============================================
0000:41:00.0 'Ethernet Controller XXV710 for 25GbE SFP28 158b' drv=igb_uio unused=i40e
0000:41:00.1 'Ethernet Controller XXV710 for 25GbE SFP28 158b' drv=igb_uio unused=i40e
....

Also you have to reserve huge page:

#!/bin/bash
# Reserve 4 1G-pages - 4 GB in total:
HUGEPAGES_NUM=4
HUGEPAGES_PATH=/dev/hugepages
sync && echo 3 > /proc/sys/vm/drop_caches
echo $HUGEPAGES_NUM > /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages
HUGEPAGES_AVAIL=$(grep HugePages_Total /sys/devices/system/node/node0/meminfo | cut -d ':' -f 2|sed 's/ //g')
if [ $HUGEPAGES_AVAIL -ne $HUGEPAGES_NUM ]; then
printf "Warning: %s hugepages available, %s requested\n" "$HUGEPAGES_AVAIL" "$HUGEPAGES_NUM"
fi

Usually 2-4 GB for a huge page is enough for the normal functioning of Stingray SG. If it is not enough, Stingray SG will display a critical error in fastdpi_alert.log and will not start. All the memory necessary for the operation of Stingray SG is allocated when starting from the huge page, so if the SSG has started with the current settings, the system will not need more and more memory from the huge page. In case of startup errors associated with a shortage of huge pages, you need to increase the number of allocated huge pages in the script above and try to run the Stingray SG again.

All these actions - transferring cards into DPDK mode and reserving the huge page - must be performed at OS startup.

Stingray SG Configuration

When the system is configured to work with DPDK, you can start configuring the Stingray SG.