VM for dpdk: различия между версиями
Материал из noname.com.ua
Перейти к навигацииПерейти к поискуSirmax (обсуждение | вклад) (→VMs) |
Sirmax (обсуждение | вклад) |
||
(не показаны 2 промежуточные версии этого же участника) | |||
Строка 1: | Строка 1: | ||
+ | [[Категория:Linux]] |
||
+ | [[Категория:KVM]] |
||
+ | [[Категория:Virtualisation]] |
||
+ | [[Категория:HugePages]] |
||
+ | [[Категория:DPDK]] |
||
+ | [[Категория:NUMA]] |
||
=VM Setup for DPDK testing= |
=VM Setup for DPDK testing= |
||
==Host== |
==Host== |
||
Строка 6: | Строка 12: | ||
grep pdpe1gb /proc/cpuinfo | uniq |
grep pdpe1gb /proc/cpuinfo | uniq |
||
flags : [...] pdpe1gb [...] |
flags : [...] pdpe1gb [...] |
||
− | <PRE> |
+ | </PRE> |
Edit /etc/default/grub |
Edit /etc/default/grub |
Текущая версия на 10:52, 4 апреля 2024
VM Setup for DPDK testing
Host
Check that host CPU has 1Gb hugepages flag:
grep pdpe1gb /proc/cpuinfo | uniq flags : [...] pdpe1gb [...]
Edit /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on iommu=pt default_hugepagesz=1G hugepagesz=1GB hugepages=30"
Reboot and check that all params have been applied
# cat /proc/cmdline BOOT_IMAGE=/boot/vmlinuz-4.15.0-43-generic root=UUID=5f20cc6c-ff53-11e8-8f82-ecf4bbc26e30 ro maybe-ubiquity intel_iommu=on iommu=pt default_hugepagesz=1G hugepagesz=1G hugepages=128.0 transparent_hugepage=never isolcpu=2-11 rcu_nocbs=2-11 nohz=off Check hugepages # cat /proc/meminfo | grep Huge AnonHugePages: 0 kB ShmemHugePages: 0 kB HugePages_Total: 100 HugePages_Free: 99 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 1048576 kB
Get NIC NUMA node:
# cat /sys/bus/pci/devices/0000\:d8\:00.1/numa_node 1
where device ID can be found with command
lshw -class network -businfo
Get CPUs with needed NUMA node for VM CPU pinning
# lscpu | grep NUMA NUMA node(s): 2 NUMA node0 CPU(s): 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46 NUMA node1 CPU(s): 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47
VMs
Outside VM
Edit VM settings with virsh using command:
virsh edit <VM NAME>
CPU Pinning (manual configuration, need to add cputune section
<vcpu placement='static' cpuset='1,3,5,7'>4</vcpu> <cputune> <vcpupin vcpu='0' cpuset='1'/> <vcpupin vcpu='1' cpuset='2'/> <vcpupin vcpu='2' cpuset='5'/> <vcpupin vcpu='3' cpuset='7'/> </cputune> Add hugepages configuration <memoryBacking> <hugepages/> </memoryBacking> Enable CPU flags as on host CPU <cpu mode='host-model' check='partial'> <model fallback='allow'/> </cpu>
Video Card MUST be set to virtio (may be configured using virt-manager)
<video> <model type='virtio' heads='1' primary='yes'> <acceleration accel3d='no'/> </model> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </video>
Inside VM
manual module load
modprobe vfio enable_unsafe_noiommu_mode=Y
OR add to configuration in file /etc/modprobe.d/vfio.conf
options vfio enable_unsafe_noiommu_mode=Y
Kernel settings
modprobe vfio enable_unsafe_noiommu_mode=Y modprobe uio
/etc/grub.d/50-curtin-settings.cfg
GRUB_CMDLINE_LINUX_DEFAULT="maybe-ubiquity default_hugepagesz=1G hugepagesz=1G hugepages=128.0 transparent_hugepage=never intel_iommu=on iommu=pt"
Check that hugepages have been applyed
AnonHugePages: 0 kB ShmemHugePages: 0 kB HugePages_Total: 3 HugePages_Free: 2 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 1048576 kB