This is an old revision of the document!
3 DPDK Version Configuration
DPDK (Data Plane Development Kit) allows working with network cards directly without actually using the Linux kernel. This improves the performance of the solution. DPDK supports many more models of network cards than pf_ring, and a much richer interface. So it allows you to implement various working schemes, suitable for 10G, 25G, 40G, 100G traffic, etc.
System Preparation
The first step to work with DPDK is to take the network cards out of the control of the operating system. DPDK works with PCI devices, that can be displayed with the command:
> lspci|grep Eth 41:00.0 Ethernet controller: Intel Corporation Ethernet Controller XXV710 for 25GbE SFP28 (rev 02) 41:00.1 Ethernet controller: Intel Corporation Ethernet Controller XXV710 for 25GbE SFP28 (rev 02) c6:00.0 Ethernet controller: Broadcom Inc. and subsidiaries BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet Controller (rev 01) c6:00.1 Ethernet controller: Broadcom Inc. and subsidiaries BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet Controller (rev 01) >
This command will list all PCI ethernet devices. Each line starts with the system PCI device identifier - these PCI identifiers are the unique for the network card in the DPDK.
Transferring the card to DPDK mode (disconnecting from the system network driver) is carried out by the dpdk-devbind.py utility from the DPDK:
# Example - devices 41:00.0 and 41:00.1 transfer to the DPDK mode >insmod $RTE/module/igb_uio.ko # 25G NICs >$RTE/bin/dpdk-devbind.py --bind igb_uio 0000:41:00.0 >$RTE/bin/dpdk-devbind.py --bind igb_uio 0000:41:00.1
here, igb_uio - is UIO driver. The system uio_pci_generic
or igb_uio
from the DPDK can act as a uio driver. Usually uio_pci_generic
is used for modern cards, and igb_uio
for the older ones, for details see DPDK Linux Drivers. Uio-driver is only needed to register interrupts of network cards (e.g. to recognize link down/link up), and is not involved in receiving and sending data packets.
To see if the card is properly initialized to work with DPDK, use the command
> $RTE/bin/dpdk-devbind.py --status
If the cards are in DPDK mode, you will see them in Network devices using DPDK-compatible driver
section:
> $RTE/bin/dpdk-devbind.py --status Network devices using DPDK-compatible driver ============================================ 0000:41:00.0 'Ethernet Controller XXV710 for 25GbE SFP28 158b' drv=igb_uio unused=i40e 0000:41:00.1 'Ethernet Controller XXV710 for 25GbE SFP28 158b' drv=igb_uio unused=i40e ....
Also you have to reserve huge page:
#!/bin/bash # Reserve 4 1G-pages - 4 GB in total: HUGEPAGES_NUM=4 HUGEPAGES_PATH=/dev/hugepages sync && echo 3 > /proc/sys/vm/drop_caches echo $HUGEPAGES_NUM > /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages HUGEPAGES_AVAIL=$(grep HugePages_Total /sys/devices/system/node/node0/meminfo | cut -d ':' -f 2|sed 's/ //g') if [ $HUGEPAGES_AVAIL -ne $HUGEPAGES_NUM ]; then printf "Warning: %s hugepages available, %s requested\n" "$HUGEPAGES_AVAIL" "$HUGEPAGES_NUM" fi
Usually 2-4 GB for a huge page is enough for the normal functioning of Stingray SG. If it is not enough, Stingray SG will display a critical error in fastdpi_alert.log and will not start. All the memory necessary for the operation of Stingray SG is allocated when starting from the huge page, so if the SSG has started with the current settings, the system will not need more and more memory from the huge page. In case of startup errors associated with a shortage of huge pages, you need to increase the number of allocated huge pages in the script above and try to run the Stingray SG again.
Stingray SG Configuration
When the system is configured to work with DPDK, you can start configuring the Stingray SG.