KVM : GPU Passthrough2023/02/28

	Configure GPU Passthrough for Virtual Machines. By this configuration, it's possible to use GPU on Virtual Machines and run GPU Computing like Machine learning and so on. Before configuration, Enable VT-d (Intel) or AMD IOMMU (AMD) on BIOS Setting first.
[1]	Enable IOMMU feature and [vfio-pci] kernel module on KVM Host.

# add kernel parameter
# for AMD CPU, specify [amd_iommu=on]

[root@dlp ~]#

grubby --update-kernel ALL --args intel_iommu=on --args iommu=pt

# show PCI identification number and [vendor-ID:device-ID] of Graphic card
# PCI number ⇒ it matchs [02:00.*] below
# vendor-ID:device-ID ⇒ it matchs [10de:***] below

[root@dlp ~]#

lspci -nn | grep -i nvidia

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP106 [GeForce GTX 1060 6GB] [10de:1c03] (rev a1)
01:00.1 Audio device [0403]: NVIDIA Corporation GP106 High Definition Audio Controller [10de:10f1] (rev a1)

[root@dlp ~]#

vi /etc/modprobe.d/vfio.conf

# create new : for [ids=***], specify [vendor-ID:device-ID]

options vfio-pci ids=10de:1c03,10de:10f1

[root@dlp ~]#

echo 'vfio-pci' > /etc/modules-load.d/vfio-pci.conf

[root@dlp ~]#

reboot

# confirm IOMMU is enabled

[root@dlp ~]#

dmesg | grep -E "DMAR|IOMMU"

[    0.009814] ACPI: DMAR 0x000000005C6E6B70 0000C4 (v01 ALASKA A M I    00000001 INTL 20091013)
[    0.009837] ACPI: Reserving DMAR table memory at [mem 0x5c6e6b70-0x5c6e6c33]
[    0.021188] DMAR: IOMMU enabled
[    0.092606] DMAR: Host address width 46
[    0.092607] DMAR: DRHD base: 0x000000fbffd000 flags: 0x0
[    0.092612] DMAR: dmar0: reg_base_addr fbffd000 ver 1:0 cap 8d2008c10ef0466 ecap f0205b
[    0.092615] DMAR: DRHD base: 0x000000fbffc000 flags: 0x1
[    0.092618] DMAR: dmar1: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    0.092620] DMAR: RMRR base: 0x0000005ce2b000 end: 0x0000005ce3afff
[    0.092622] DMAR: ATSR flags: 0x0
[    0.092623] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x0
[    0.092625] DMAR-IR: IOAPIC id 1 under DRHD base  0xfbffc000 IOMMU 1
[    0.092627] DMAR-IR: IOAPIC id 2 under DRHD base  0xfbffc000 IOMMU 1
[    0.092627] DMAR-IR: HPET id 0 under DRHD base 0xfbffc000
[    0.092628] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.
[    0.092629] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting.
[    0.093240] DMAR-IR: Enabled IRQ remapping in xapic mode
[    0.563570] DMAR: No SATC found
[    0.563572] DMAR: IOMMU feature sc_support inconsistent
[    0.563573] DMAR: IOMMU feature dev_iotlb_support inconsistent
[    0.563574] DMAR: dmar0: Using Queued invalidation
[    0.563577] DMAR: dmar1: Using Queued invalidation
[    0.570123] DMAR: Intel(R) Virtualization Technology for Directed I/O

# confirm vfio_pci is enabled

[root@dlp ~]#

dmesg | grep -i vfio

[    4.069552] VFIO - User Level meta-driver version: 0.3
[    4.080264] vfio_pci: add [10de:1c03[ffffffff:ffffffff]] class 0x000000/00000000
[    4.091796] vfio_pci: add [10de:10f1[ffffffff:ffffffff]] class 0x000000/00000000

[2]	That's OK. For example, create a AlmaLinux 9 Virtual machine with GPU. Specify PCI identification number of GPU for [--host-device].

[root@dlp ~]#

virt-install \
--name almalinux9 \
--ram 8192 \
--disk path=/var/kvm/images/almalinux9.img,size=30 \
--vcpus 4 \
--os-variant almalinux9 \
--network bridge=br0 \
--graphics none \
--console pty,target_type=serial \
--location /home/CentOS-Stream-9-latest-x86_64-dvd1.iso \
--extra-args 'console=ttyS0,115200n8 serial' \
--host-device 01:00.0 \
--features kvm_hidden=on

[3]	After creating Virtual machine, Confirm GPU is found on it like follows.

[root@localhost ~]#

lspci | grep -i nvidia

05:00.0 VGA compatible controller: NVIDIA Corporation GP106 [GeForce GTX 1060 6GB] (rev a1)

Matched Content