OpenStack Zed : Add Compute Nodes (GPU)2023/01/10 |
|
Add GPU attached Compute Nodes to use GPU on virtual machine instances.
On this example, add [node03.srv.world] like follows.
------------+-----------------------------+-----------------------------+------------
| | |
eth0|10.0.0.30 eth0|10.0.0.50 eth0|10.0.0.51
+-----------+-----------+ +-----------+-----------+ +-----------+-----------+
| [ dlp.srv.world ] | | [ network.srv.world ] | | [ node01.srv.world ] |
| (Control Node) | | (Network Node) | | (Compute Node) |
| | | | | |
| MariaDB RabbitMQ | | Open vSwitch | | Libvirt |
| Memcached Nginx | | Neutron Server | | Nova Compute |
| Keystone httpd | | OVN-Northd | | Open vSwitch |
| Glance Nova API | | Nginx iSCSI Target | | OVN Metadata Agent |
| Cinder API | | Cinder Volume | | OVN-Controller |
| Skyline API/Console | | Heat API/Engine | | |
+-----------------------+ +-----------------------+ +-----------------------+
------------+-----------------------------+-----------------
| |
eth0|10.0.0.52 eth0|10.0.0.53
+-----------+-----------+ +-----------+-----------+
| [ node02.srv.world ] | | [ node03.srv.world ] |
| (Compute Node) | | (Compute Node (GPU)) |
| | | |
| Libvirt | | Libvirt |
| Nova Compute | | Nova Compute |
| Open vSwitch | | Open vSwitch |
| OVN Metadata Agent | | OVN Metadata Agent |
| OVN-Controller | | OVN-Controller |
| | | |
+-----------------------+ +-----------------------+
|
| [1] |
On the new Node with GPU,
Configure GPU Passthrough setting, refer to here for [1].
|
| [2] |
On the new Node with GPU,
add it to Openstack Cluster as a Compute Node, refer to here.
|
| [3] | On the new Node with GPU, Configure additional settings for Nova-Compute. |
|
root@node03:~# lspci -nn | grep -i nvidia 02:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK104 [GeForce GTX 680] [10de:1180] (rev a1) 02:00.1 Audio device [0403]: NVIDIA Corporation GK104 HDMI Audio Controller [10de:0e0a] (rev a1)
root@node03:~#
vi /etc/nova/nova.conf
# add to the end
# add [vendor_id], [product_id] of passthrough device
[pci]
passthrough_whitelist = { "vendor_id": "10de", "product_id": "1180" }
systemctl restart nova-compute
|
| [4] | On Control Node, Change Nova settings. |
|
root@dlp ~(keystone)#
vi /etc/nova/nova.conf
# add to the end
# add [vendor_id], [product_id] that are the same one on compute node
# for the [name], set any name you like
[pci]
alias: { "vendor_id":"10de", "product_id":"1180", "device_type":"type-PCI", "name":"GTX-680" }
[filter_scheduler]
enabled_filters = PciPassthroughFilter
root@dlp ~(keystone)#
systemctl restart nova-api nova-scheduler
# create [flavor] for GPU instance root@dlp ~(keystone)# openstack flavor create --id 2 --vcpus 4 --ram 8192 --disk 20 --property "pci_passthrough:alias"="GTX-680:1" gpu1.small +----------------------------+-----------------------------------+ | Field | Value | +----------------------------+-----------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 0 | | description | None | | disk | 20 | | id | 2 | | name | gpu1.small | | os-flavor-access:is_public | True | | properties | pci_passthrough:alias='GTX-680:1' | | ram | 8192 | | rxtx_factor | 1.0 | | swap | | | vcpus | 4 | +----------------------------+-----------------------------------+root@dlp ~(keystone)# openstack flavor list +----+------------+------+------+-----------+-------+-----------+ | ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public | +----+------------+------+------+-----------+-------+-----------+ | 0 | m1.small | 2048 | 10 | 0 | 1 | True | | 1 | m1.medium | 8192 | 20 | 0 | 4 | True | | 2 | gpu1.small | 8192 | 20 | 0 | 4 | True | +----+------------+------+------+-----------+-------+-----------+ |
| [5] | Verify settings to create a GPU instance with any Openstack user. |
|
ubuntu@dlp ~(keystone)$ openstack network list +--------------------------------------+---------+--------------------------------------+ | ID | Name | Subnets | +--------------------------------------+---------+--------------------------------------+ | 394ec727-7e9a-473e-8068-45aa841a9ac2 | public | ea6cb6cb-74ed-4007-bd25-b3fb84111e84 | | 7336a271-bec9-4d33-8dc5-e9f3a9892ed7 | private | a454cf3e-fef5-46f6-8468-4a26cbd14983 | +--------------------------------------+---------+--------------------------------------+ubuntu@dlp ~(keystone)$ netID=$(openstack network list | grep private | awk '{ print $2 }')
ubuntu@dlp ~(keystone)$
ubuntu@dlp ~(keystone)$ openstack server create --flavor gpu1.small --image Ubuntu2204 --security-group secgroup01 --nic net-id=$netID --key-name mykey Ubuntu-2204GPU
openstack server list +--------------------------------------+----------------+---------+-------------------------------------+------------+------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+----------------+---------+-------------------------------------+------------+------------+ | 4e9bb3fa-cd5d-45c5-a521-78066eae72c5 | Ubuntu-2204GPU | ACTIVE | private=192.168.100.81 | Ubuntu2204 | gpu1.small | | a1bad5a9-f808-4982-9de9-981e5138d523 | Ubuntu-220401 | SHUTOFF | private=192.168.100.20 | Ubuntu2204 | m1.small | | a456ee32-b105-4c5e-8eae-ddccb48f913e | Ubuntu-2204 | SHUTOFF | private=10.0.0.228, 192.168.100.187 | Ubuntu2204 | m1.small | +--------------------------------------+----------------+---------+-------------------------------------+------------+------------+ubuntu@dlp ~(keystone)$ openstack floating ip create public +---------------------+--------------------------------------+ | Field | Value | +---------------------+--------------------------------------+ | created_at | 2023-01-10T03:36:50Z | | description | | | dns_domain | | | dns_name | | | fixed_ip_address | None | | floating_ip_address | 10.0.0.233 | | floating_network_id | 394ec727-7e9a-473e-8068-45aa841a9ac2 | | id | 2bfe580f-c62d-4fde-9855-22c646351504 | | name | 10.0.0.233 | | port_details | None | | port_id | None | | project_id | 4dd1e1f6bac441ff9c77002c3ab4c58a | | qos_policy_id | None | | revision_number | 0 | | router_id | None | | status | DOWN | | subnet_id | None | | tags | [] | | updated_at | 2023-01-10T03:36:50Z | +---------------------+--------------------------------------+ubuntu@dlp ~(keystone)$ openstack server add floating ip Ubuntu-2204GPU 10.0.0.233 ubuntu@dlp ~(keystone)$ ssh ubuntu@10.0.0.233
The authenticity of host '10.0.0.233 (10.0.0.233)' can't be established.
ED25519 key fingerprint is SHA256:eGE+ZLrJlOhL/eujywn2WnKsie9AJi1EW70WGdP2UIw.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '10.0.0.233' (ED25519) to the list of known hosts.
Welcome to Ubuntu 22.04.1 LTS (GNU/Linux 5.15.0-48-generic x86_64)
.....
.....
ubuntu@ubuntu-2204gpu:~$ubuntu@ubuntu-2204gpu:~$ lspci | grep -i nvidia 00:05.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 680] (rev a1) |
| Sponsored Link |