1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
|
Copyright (c) 2014, Linaro Limited
All rights reserved.
SPDX-License-Identifier: BSD-3-Clause
ODP-DPDK:
---------
This effort is to port ODP on top of DPDK and use DPDK as the
accelerator for all intel NIC's. Pre-requisite is DPDK should be cloned and
compiled. DPDK and ODP was compiled and tested on Ubuntu 14.04 LTS
3.13.0-29-generic kernel.
To Clone DPDK
git clone http://92.243.14.124/git/dpdk ./<dpdk-dir>
We support only 1.7.1 of DPDK for now
during upgrade, make sure that PMD constuctors are correctly and fully
referred in refer_constructors()1
git tag -l -- will list all the tags available
git checkout -b 1.7.1 tags/v1.7.1
Please refer to http://dpdk.org/doc for more details on how to build
DPDK. Getting started guide for Linux might be of help.
Best effort is done to provide some help on DPDK cmds below for Ubuntu,
where it was compiled and tested.
Please refer "How to setup and compile DPDK" section in this document
To compile ODP with linux-dpdk
cd <odp-dir>
./bootstrap
./configure --with-platform=linux-dpdk --with-sdk-install-path=<dpdk-dir>/x86_64-native-linuxapp-gcc
make
App commands to test
l2fwding app - sudo ./example/l2fwd/odp_l2fwd -i 0,1 -m 0 -c 2
loopback app - sudo ./example/packet/odp_pktio -i 0,1 -m 0 -c 2
-i 0,1 - interface number
-m 0 - burst mode
-c 2 - number of cpus
How to setup and compile DPDK:
==============================
cd <dpdk-dir>
This has to be done only once.
make config T=x86_64-native-linuxapp-gcc O=x86_64-native-linuxapp-gcc
Set CONFIG_RTE_BUILD_COMBINE_LIBS=y and CONFIG_RTE_BUILD_SHARED_LIB=y in
./x86_64-native-linuxapp-gcc/.config file
Note: if non-intel SFP's are used in IXGBE, then set
CONFIG_RTE_LIBRTE_IXGBE_ALLOW_UNSUPPORTED_SFP=y in .config file
make install T=x86_64-native-linuxapp-gcc EXTRA_CFLAGS="-fPIC"
If "conflicting types for skb_set_hash" error happens during DPDK
build, then please knock-off skb_set_hash function from kcompat.h as
shown below. This was seen in Ubuntu 3.13.0-30-generic.
diff --git a/lib/librte_eal/linuxapp/kni/ethtool/igb/kcompat.h b/lib/librte_eal/linuxapp/kni/ethtool/igb/kcompat.h
index 19df483..78a794a 100644
--- a/lib/librte_eal/linuxapp/kni/ethtool/igb/kcompat.h
+++ b/lib/librte_eal/linuxapp/kni/ethtool/igb/kcompat.h
@@ -3845,11 +3845,6 @@ static inline struct sk_buff *__kc__vlan_hwaccel_put_tag(struct sk_buff *skb,
#if ( LINUX_VERSION_CODE < KERNEL_VERSION(3,14,0) )
#ifdef NETIF_F_RXHASH
#define PKT_HASH_TYPE_L3 0
-static inline void
-skb_set_hash(struct sk_buff *skb, __u32 hash, __always_unused int type)
-{
- skb->rxhash = hash;
-}
#endif /* NETIF_F_RXHASH */
#endif /* < 3.14.0 */
This only ensures building DPDK, but traffic is not tested with this
build yet. It is upto the user to test it.
To reserve huge pages, which is needed for DPDK, execute following command
sudo sh -c 'echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages'
If you are running on a multi-node machine then, hugepages should be reserved on each node
ls /sys/devices/system/node
sudo sh -c 'echo 1024 > /sys/devices/system/node/node*/hugepages/hugepages-2048kB/nr_hugepages'
sudo mkdir /mnt/huge
sudo mount -t hugetlbfs nodev /mnt/huge
To load uio driver
sudo /sbin/modprobe uio
ulimit -Sn 2048
cd <dpdk-dir>
sudo insmod ./build/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.ko
sudo rmmod ixgbe
sudo modprobe ixgbe
If the SFP's used are non-intel, then
sudo modprobe ixgbe allow_unsupported_sfp=1
cd <dpdk-dir>
./tools/igb_uio_bind.py --status
This command produces output that is something similar as given below
Network devices using IGB_UIO driver
====================================
0000:05:00.0 'Ethernet 10G 2P X520 Adapter' drv=igb_uio unused=
0000:05:00.1 'Ethernet 10G 2P X520 Adapter' drv=igb_uio unused=
Network devices using kernel driver
===================================
0000:01:00.0 'NetXtreme II BCM5709 Gigabit Ethernet' if=eth0 drv=bnx2 unused=<none> *Active*
0000:01:00.1 'NetXtreme II BCM5709 Gigabit Ethernet' if=eth1 drv=bnx2 unused=<none>
0000:07:00.0 'T320 10GbE Dual Port Adapter' if=eth2,eth3 drv=cxgb3 unused=<none>
Other network devices
=====================
<none>
Now you should look for pci id listed and give it in the following command
in place of 05:00.X
To give the interfaces to DPDK, use following command
sudo ./tools/igb_uio_bind.py --bind=igb_uio 05:00.0
sudo ./tools/igb_uio_bind.py --bind=igb_uio 05:00.1
To restore it back to kernel, use following command
sudo ./tools/igb_uio_bind.py --bind=ixgbe 05:00.0
sudo ./tools/igb_uio_bind.py --bind=ixgbe 05:00.1
Howto debug DPDK apps on the host
========================================
For example you need to debug some l2fwd application. Then network configuration
might be:
<veth1-2> <-----> <veth2-1> (iface0 DPDK L2FWD APP iface1) <veth2-3> <-----> <veth3-2>
Where:
vethX-Y - virtual devices for host.
Packet sent to veth1-2 goes to chain and appears on veth3-2
Steps:
Recompile with:
CONFIG_RTE_LIBRTE_PMD_PCAP=y
ip link add veth1-2 type veth peer name veth2-1
ip link add veth2-3 type veth peer name veth3-2
ifconfig veth1-2 up -arp
ifconfig veth2-1 up -arp
ifconfig veth2-3 up -arp
ifconfig veth3-2 up -arp
mount -t hugetlbfs none /mnt/huge
Finally give l2fwd fake devices:
./l2fwd -c '0xf' -n 4 --vdev "eth_pcap0,iface=veth2-1" --vdev="eth_pcap1,iface=veth2-3" -- -p 3
|
