OpenStack Antelope : Add Compute Nodes (GPU)2023/04/18 |
|
Add GPU attached Compute Nodes to use GPU on virtual machine instances.
On this example, add [node02.srv.world] like follows.
------------+--------------------------+--------------------------+------------
| | |
eth0|10.0.0.30 eth0|10.0.0.50 eth0|10.0.0.51
+-----------+-----------+ +-----------+-----------+ +-----------+-----------+
| [ dlp.srv.world ] | | [ network.srv.world ] | | [ node01.srv.world ] |
| (Control Node) | | (Network Node) | | (Compute Node) |
| | | | | |
| MariaDB RabbitMQ | | Open vSwitch | | Libvirt |
| Memcached Nginx | | Neutron Server | | Nova Compute |
| Keystone httpd | | OVN-Northd | | Open vSwitch |
| Glance Nova API | | Nginx | | OVN Metadata Agent |
| | | | | OVN-Controller |
+-----------------------+ +-----------------------+ +-----------------------+
------------+------------
|
eth0|10.0.0.52
+-----------+-----------+
| [ node02.srv.world ] |
| (Compute Node (GPU)) |
| |
| Libvirt |
| Nova Compute |
| Open vSwitch |
| OVN Metadata Agent |
| OVN-Controller |
+-----------------------+
|
| [1] |
On the new Node with GPU,
Configure GPU Passthrough setting, refer to here for [1].
|
| [2] |
On the new Node with GPU,
add it to Openstack Cluster as a Compute Node, refer to here.
|
| [3] | On the new Node with GPU, Configure additional settings for Nova-Compute. |
|
[root@node02 ~]# lspci -nn | grep -i nvidia 81:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP106 [GeForce GTX 1060 6GB] [10de:1c03] (rev a1) 81:00.1 Audio device [0403]: NVIDIA Corporation GP106 High Definition Audio Controller [10de:10f1] (rev a1)
[root@node02 ~]#
vi /etc/nova/nova.conf
# add to the end
# add [vendor_id], [product_id] of passthrough device
[pci]
passthrough_whitelist = { "vendor_id": "10de", "product_id": "1c03" }
systemctl restart openstack-nova-compute
|
| [4] | On Control Node, Change Nova settings. |
|
[root@dlp ~(keystone)]#
vi /etc/nova/nova.conf
# add to the end
# add [vendor_id], [product_id] that are the same one on compute node
# for the [name], set any name you like
[pci]
alias: { "vendor_id":"10de", "product_id":"1c03", "device_type":"type-PCI", "name":"GTX-1060" }
[filter_scheduler]
enabled_filters = PciPassthroughFilter
[root@dlp ~(keystone)]#
systemctl restart openstack-nova-api openstack-nova-scheduler
# create [flavor] for GPU instance [root@dlp ~(keystone)]# openstack flavor create --id 4 --vcpus 4 --ram 8192 --disk 20 --property "pci_passthrough:alias"="GTX-1060:1" gpu1.small +----------------------------+------------------------------------+ | Field | Value | +----------------------------+------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 0 | | description | None | | disk | 20 | | id | 4 | | name | gpu1.small | | os-flavor-access:is_public | True | | properties | pci_passthrough:alias='GTX-1060:1' | | ram | 8192 | | rxtx_factor | 1.0 | | swap | | | vcpus | 4 | +----------------------------+------------------------------------+[root@dlp ~(keystone)]# openstack flavor list +----+------------+------+------+-----------+-------+-----------+ | ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public | +----+------------+------+------+-----------+-------+-----------+ | 1 | m1.small | 2048 | 10 | 0 | 1 | True | | 2 | m1.medium | 4096 | 10 | 0 | 2 | True | | 3 | m1.large | 8192 | 20 | 0 | 4 | True | | 4 | gpu1.small | 8192 | 20 | 0 | 4 | True | +----+------------+------+------+-----------+-------+-----------+ |
| [5] | Verify settings to create a GPU instance with any Openstack user. |
|
[cent@dlp ~(keystone)]$ openstack network list +--------------------------------------+---------+--------------------------------------+ | ID | Name | Subnets | +--------------------------------------+---------+--------------------------------------+ | 913ec733-5a60-4df0-b182-49a393675bca | private | bd8d7602-95e4-4540-971e-a0104dd8b658 | | c1c6a4c6-776e-4c4f-b557-b9efc4a43ee8 | public | 68cec7d2-f12f-4272-ac16-8dba6148888b | +--------------------------------------+---------+--------------------------------------+[cent@dlp ~(keystone)]$ netID=$(openstack network list | grep private | awk '{ print $2 }')
[cent@dlp ~(keystone)]$
[cent@dlp ~(keystone)]$ openstack server create --flavor gpu1.small --image CentOS-Stream9 --security-group secgroup01 --nic net-id=$netID --key-name mykey CentOS-St9GPU
openstack server list +--------------------------------------+---------------+---------+-------------------------------------+----------------+------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+---------------+---------+-------------------------------------+----------------+------------+ | d9fa5803-db7b-45c8-bd7a-12c2bf344373 | CentOS-St9GPU | ACTIVE | private=192.168.100.165 | CentOS-Stream9 | gpu1.small | | 284a5b69-7b76-49c3-8180-c9b22284d429 | CentOS-St9 | SHUTOFF | private=10.0.0.229, 192.168.100.140 | CentOS-Stream9 | m1.medium | +--------------------------------------+---------------+---------+-------------------------------------+----------------+------------+[cent@dlp ~(keystone)]$ openstack floating ip create public +---------------------+--------------------------------------+ | Field | Value | +---------------------+--------------------------------------+ | created_at | 2023-04-18T07:30:38Z | | description | | | dns_domain | | | dns_name | | | fixed_ip_address | None | | floating_ip_address | 10.0.0.244 | | floating_network_id | c1c6a4c6-776e-4c4f-b557-b9efc4a43ee8 | | id | 87109a43-bf51-426b-a232-89055d9d226b | | name | 10.0.0.244 | | port_details | None | | port_id | None | | project_id | 40aa0807595447f0bd8d5b2dbfa1839b | | qos_policy_id | None | | revision_number | 0 | | router_id | None | | status | DOWN | | subnet_id | None | | tags | [] | | updated_at | 2023-04-18T07:30:38Z | +---------------------+--------------------------------------+[cent@dlp ~(keystone)]$ openstack server add floating ip CentOS-St9GPU 10.0.0.244 [cent@dlp ~(keystone)]$ ssh centos@10.0.0.244
The authenticity of host '10.0.0.244 (10.0.0.244)' can't be established.
ED25519 key fingerprint is SHA256:p5H3Xz6u9jawvfqcAr87pj+3WeBf2aYOPz1pAss8+r8.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '10.0.0.244' (ED25519) to the list of known hosts.
[centos@centos-st9gpu ~]$
[centos@centos-st9gpu ~]$ lspci | grep -i nvidia 00:05.0 VGA compatible controller: NVIDIA Corporation GP106 [GeForce GTX 1060 6GB] (rev a1) |
| Sponsored Link |