OpenStack Yoga : How to use Magnum2022/04/12 |
|
Install OpenStack Container Infrastructure Management Service (Magnum).
This example is based on the environment like follows.
------------+---------------------------+---------------------------+------------
| | |
eth0|10.0.0.30 eth0|10.0.0.50 eth0|10.0.0.51
+-----------+-----------+ +-----------+-----------+ +-----------+-----------+
| [ Control Node ] | | [ Storage Node ] | | [ Compute Node ] |
| | | | | |
| MariaDB RabbitMQ | | Open vSwitch | | Libvirt |
| Memcached httpd | | Neutron Server | | Nova Compute |
| Keystone Glance | | OVN-Northd | | Open vSwitch |
| Nova API | | Cinder Volume | | OVN Metadata Agent |
| Cinder API | | iSCSI Target | | OVN-Controller |
| Barbican API | | Heat API/Engine | | |
| | | Magnum API | | |
+-----------------------+ +-----------------------+ +-----------------------+
|
| [1] | On Control Node, Download a VM image for containers (Fedora CoreOS) and add it to Glance. |
|
root@dlp ~(keystone)# wget https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/35.20220313.3.1/x86_64/fedora-coreos-35.20220313.3.1-openstack.x86_64.qcow2.xz root@dlp ~(keystone)# xz -dv fedora-coreos-35.20220313.3.1-openstack.x86_64.qcow2.xz root@dlp ~(keystone)# openstack image create Fedora-CoreOS --file=fedora-coreos-35.20220313.3.1-openstack.x86_64.qcow2 --disk-format=qcow2 --container-format=bare --property os_distro='fedora-coreos' --public |
| [2] | How to use Magnum. For example, Create Kubernetes Cluster with [admin] user. |
|
root@dlp ~(keystone)# openstack flavor list +----+----------+------+------+-----------+-------+-----------+ | ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public | +----+----------+------+------+-----------+-------+-----------+ | 0 | m1.small | 2048 | 10 | 0 | 1 | True | | 1 | m1.large | 8192 | 20 | 0 | 4 | True | +----+----------+------+------+-----------+-------+-----------+root@dlp ~(keystone)# openstack keypair list +--------+-------------------------------------------------+------+ | Name | Fingerprint | Type | +--------+-------------------------------------------------+------+ | my-key | f3:14:05:3e:de:01:95:20:6e:4e:94:11:09:7a:b0:85 | ssh | +--------+-------------------------------------------------+------+root@dlp ~(keystone)# openstack network list +--------------------------------------+---------+--------------------------------------+ | ID | Name | Subnets | +--------------------------------------+---------+--------------------------------------+ | 8fd7472d-9182-4a17-aa35-984fb7fb059d | public | 99f81758-a8a8-4a25-8c20-a8c52f374815 | | b19c6ee5-26f7-4a98-8a8c-58a2e7637610 | private | 5d1edead-636e-4622-942a-bfd2347908e3 | +--------------------------------------+---------+--------------------------------------+root@dlp ~(keystone)# openstack subnet list +--------------------------------------+----------------+--------------------------------------+------------------+ | ID | Name | Network | Subnet | +--------------------------------------+----------------+--------------------------------------+------------------+ | 5d1edead-636e-4622-942a-bfd2347908e3 | private-subnet | b19c6ee5-26f7-4a98-8a8c-58a2e7637610 | 192.168.100.0/24 | | 99f81758-a8a8-4a25-8c20-a8c52f374815 | public-subnet | 8fd7472d-9182-4a17-aa35-984fb7fb059d | 10.0.0.0/24 | +--------------------------------------+----------------+--------------------------------------+------------------+ # create Kubernetes Cluster template root@dlp ~(keystone)# openstack coe cluster template create k8s-cluster-template \ --image Fedora-CoreOS \ --external-network public \ --fixed-network private \ --fixed-subnet private-subnet \ --dns-nameserver 10.0.0.10 \ --network-driver flannel \ --docker-storage-driver overlay2 \ --docker-volume-size 10 \ --master-flavor m1.large \ --flavor m1.large \ --coe kubernetes Request to create cluster template k8s-cluster-template accepted +-----------------------+--------------------------------------+ | Field | Value | +-----------------------+--------------------------------------+ | insecure_registry | - | | labels | {} | | updated_at | - | | floating_ip_enabled | True | | fixed_subnet | private-subnet | | master_flavor_id | m1.large | | uuid | a897802d-c2c4-4b63-a40f-43ea060d6a25 | | no_proxy | - | | https_proxy | - | | tls_disabled | False | | keypair_id | - | | public | False | | http_proxy | - | | docker_volume_size | 10 | | server_type | vm | | external_network_id | public | | cluster_distro | fedora-coreos | | image_id | Fedora-CoreOS | | volume_driver | - | | registry_enabled | False | | docker_storage_driver | overlay2 | | apiserver_port | - | | name | k8s-cluster-template | | created_at | 2022-04-12T07:46:07+00:00 | | network_driver | flannel | | fixed_network | private | | coe | kubernetes | | flavor_id | m1.large | | master_lb_enabled | False | | dns_nameserver | 10.0.0.10 | | hidden | False | | tags | - | +-----------------------+--------------------------------------+ # create Kubernetes Cluster with 2 nodes root@dlp ~(keystone)# openstack coe cluster create k8s-cluster \ --cluster-template k8s-cluster-template \ --master-count 1 \ --node-count 1 \ --keypair my-key Request to create cluster 406ecbcc-ea0f-4ca1-8ed4-2cc3d3d211c4 accepted # verify status # proceed to create cluster during [CREATE_IN_PROGRESS] state root@dlp ~(keystone)# openstack coe cluster list +--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+ | 406ecbcc-ea0f-4ca1-8ed4-2cc3d3d211c4 | k8s-cluster | my-key | 1 | 1 | CREATE_IN_PROGRESS | None | +--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+ # Heat orchestration System is used for creating root@dlp ~(keystone)# openstack stack list +--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+ | ID | Stack Name | Project | Stack Status | Creation Time | Updated Time | +--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+ | 43797700-bdeb-48f5-b771-c1e5224bbac5 | k8s-cluster-bivfkluajpoy | ddb7c08ba73a48eea040270d13a7b0cf | CREATE_IN_PROGRESS | 2022-04-12T07:46:44Z | None | +--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+ # confirm checkpoints for creation root@dlp ~(keystone)# openstack stack list --nested | grep k8s-cluster | 3d953ef1-0fff-4f0d-9dd3-328bb042e711 | k8s-cluster-bivfkluajpoy-kube_minions-3elssevbibu3-0-hacig2iifajl | ddb7c08ba73a48eea040270d13a7b0cf | CREATE_COMPLETE | 2022-04-12T07:50:42Z | None | 9b5475db-48c8-4a2b-8be3-3d36cc3965c9 | | 9b5475db-48c8-4a2b-8be3-3d36cc3965c9 | k8s-cluster-bivfkluajpoy-kube_minions-3elssevbibu3 | ddb7c08ba73a48eea040270d13a7b0cf | CREATE_COMPLETE | 2022-04-12T07:50:41Z | None | 43797700-bdeb-48f5-b771-c1e5224bbac5 | | 52c7ef51-c036-43cd-b3c0-1022877e75a9 | k8s-cluster-bivfkluajpoy-api_address_floating_switch-7odq46hizfcw | ddb7c08ba73a48eea040270d13a7b0cf | CREATE_COMPLETE | 2022-04-12T07:50:41Z | None | 43797700-bdeb-48f5-b771-c1e5224bbac5 | | d42bd31f-bb12-4ca4-a6dd-f205024ffcfd | k8s-cluster-bivfkluajpoy-api_address_lb_switch-utopvqfgzjev | ddb7c08ba73a48eea040270d13a7b0cf | CREATE_COMPLETE | 2022-04-12T07:50:40Z | None | 43797700-bdeb-48f5-b771-c1e5224bbac5 | | 0f379350-f7e0-4e0a-a1bc-4eb3fda2e5a2 | k8s-cluster-bivfkluajpoy-etcd_address_lb_switch-dv53ws5rmjxj | ddb7c08ba73a48eea040270d13a7b0cf | CREATE_COMPLETE | 2022-04-12T07:50:40Z | None | 43797700-bdeb-48f5-b771-c1e5224bbac5 | | f5615292-c6eb-4aa9-9f35-1fb923baf128 | k8s-cluster-bivfkluajpoy-kube_masters-3qzwogjxhlna-0-zr3vjxb4yppj-api_address_switch-ep72qxu2aj7w | ddb7c08ba73a48eea040270d13a7b0cf | CREATE_COMPLETE | 2022-04-12T07:47:27Z | None | 3d181305-a4ce-4bea-a638-423bb167a3c9 | | 3d181305-a4ce-4bea-a638-423bb167a3c9 | k8s-cluster-bivfkluajpoy-kube_masters-3qzwogjxhlna-0-zr3vjxb4yppj | ddb7c08ba73a48eea040270d13a7b0cf | CREATE_COMPLETE | 2022-04-12T07:46:53Z | None | 827f164b-4e06-49c1-b036-c820e4eb4fef | | 827f164b-4e06-49c1-b036-c820e4eb4fef | k8s-cluster-bivfkluajpoy-kube_masters-3qzwogjxhlna | ddb7c08ba73a48eea040270d13a7b0cf | CREATE_COMPLETE | 2022-04-12T07:46:49Z | None | 43797700-bdeb-48f5-b771-c1e5224bbac5 | | fc02f356-d7bf-4e70-bc71-13b754780295 | k8s-cluster-bivfkluajpoy-api_lb-jgbibrk66s4t | ddb7c08ba73a48eea040270d13a7b0cf | CREATE_COMPLETE | 2022-04-12T07:46:47Z | None | 43797700-bdeb-48f5-b771-c1e5224bbac5 | | 8454b7f0-fcc7-4f35-b570-b6f45d182591 | k8s-cluster-bivfkluajpoy-etcd_lb-vfztt37lrsjj | ddb7c08ba73a48eea040270d13a7b0cf | CREATE_COMPLETE | 2022-04-12T07:46:47Z | None | 43797700-bdeb-48f5-b771-c1e5224bbac5 | | d34b40ea-f7d9-4aad-8687-8c8923927b65 | k8s-cluster-bivfkluajpoy-network-i2pdfdvxyrfl | ddb7c08ba73a48eea040270d13a7b0cf | CREATE_COMPLETE | 2022-04-12T07:46:45Z | None | 43797700-bdeb-48f5-b771-c1e5224bbac5 | | 1a9f2fa4-cc07-406a-8c13-a32e7dff5b01 | k8s-cluster-bivfkluajpoy-network-i2pdfdvxyrfl-network_switch-kv7hicybn3zb | ddb7c08ba73a48eea040270d13a7b0cf | CREATE_COMPLETE | 2022-04-12T07:46:45Z | None | d34b40ea-f7d9-4aad-8687-8c8923927b65 | | 43797700-bdeb-48f5-b771-c1e5224bbac5 | k8s-cluster-bivfkluajpoy | ddb7c08ba73a48eea040270d13a7b0cf | CREATE_COMPLETE | 2022-04-12T07:46:44Z | None | None | # if sucessfully finished, state is [CREATE_COMPLETE] + [HEALTHY] root@dlp ~(keystone)# openstack coe cluster list +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | 406ecbcc-ea0f-4ca1-8ed4-2cc3d3d211c4 | k8s-cluster | my-key | 1 | 1 | CREATE_COMPLETE | HEALTHY | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ # instances are running root@dlp ~(keystone)# openstack server list +--------------------------------------+-----------------------------------+--------+-------------------------------------+---------------+----------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-----------------------------------+--------+-------------------------------------+---------------+----------+ | daceaeba-d6d7-40c1-b74d-7e9bdffac5f6 | k8s-cluster-bivfkluajpoy-node-0 | ACTIVE | private=10.0.0.226, 192.168.100.128 | Fedora-CoreOS | m1.large | | 44ce2762-a6cd-4518-bf1d-dd9ad95202bb | k8s-cluster-bivfkluajpoy-master-0 | ACTIVE | private=10.0.0.229, 192.168.100.67 | Fedora-CoreOS | m1.large | +--------------------------------------+-----------------------------------+--------+-------------------------------------+---------------+----------+ |
| [3] | To access to use Kubernetes Cluster, Set like follows. |
|
# install [kubectl] root@dlp ~(keystone)# snap install kubectl --classic kubectl 1.23.5 from Canonical✓ installed openstack coe cluster list +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | 406ecbcc-ea0f-4ca1-8ed4-2cc3d3d211c4 | k8s-cluster | my-key | 1 | 1 | CREATE_COMPLETE | HEALTHY | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+
root@dlp ~(keystone)#
root@dlp ~(keystone)# openstack coe cluster config k8s-cluster export KUBECONFIG=/root/config root@dlp ~(keystone)# export KUBECONFIG=/root/config
kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-cluster-bivfkluajpoy-master-0 Ready master 5m50s v1.23.3 k8s-cluster-bivfkluajpoy-node-0 Ready <none> 3m16s v1.23.3root@dlp ~(keystone)# kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE coredns-56448757b9-76dlr 1/1 Running 0 5m51s coredns-56448757b9-vhqdq 1/1 Running 0 5m51s dashboard-metrics-scraper-67f57ff746-pq2x6 1/1 Running 0 5m48s k8s-keystone-auth-t8zgg 1/1 Running 0 5m47s kube-dns-autoscaler-6d5b5dc777-69v8p 1/1 Running 0 5m50s kube-flannel-ds-5k4ct 1/1 Running 0 3m33s kube-flannel-ds-cx47x 1/1 Running 0 5m49s kubernetes-dashboard-7b88d986b4-7hx6k 1/1 Running 0 5m48s magnum-metrics-server-6c4c77844b-5bg6b 1/1 Running 0 5m42s npd-c9vdz 1/1 Running 0 3m3s # verify cluster to create test pods root@dlp ~(keystone)# kubectl create deployment test-nginx --image=nginx --replicas=2 deployment.apps/test-nginx created kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES test-nginx-976fbbd77-fzdzs 1/1 Running 0 35s 10.100.1.5 k8s-cluster-bivfkluajpoy-node-0 <none> <none> test-nginx-976fbbd77-v9pnh 1/1 Running 0 35s 10.100.1.6 k8s-cluster-bivfkluajpoy-node-0 <none> <none>
root@dlp ~(keystone)#
root@dlp ~(keystone)# kubectl expose deployment test-nginx --type="NodePort" --port 80 service/test-nginx exposed kubectl get services test-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE test-nginx NodePort 10.254.165.205 <none> 80:32042/TCP 5s
root@dlp ~(keystone)#
root@dlp ~(keystone)# kubectl port-forward service/test-nginx --address 0.0.0.0 10443:80 & Forwarding from 0.0.0.0:10443 -> 80 curl localhost:10443
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
.....
.....
|
| [4] | If you'd like to use Magnum with common users, it needs to change some settings. |
|
root@dlp ~(keystone)# openstack role list +----------------------------------+------------------+ | ID | Name | +----------------------------------+------------------+ | 119d976b7118436b86bb6a6eef561909 | heat_stack_owner | | 3eca53d4474f4d81bf2c85c8ea238f7c | reader | | 7eea1df857244828afbf10ac46356b23 | CloudUser | | 99fe824a4d5b40f1abb6efa9f6820dff | heat_stack_user | | dbd8fff588c74deaa4e2192ba1d92933 | member | | f125eaa73aed4ea2ba2b7cba02acd42c | admin | +----------------------------------+------------------+root@dlp ~(keystone)# openstack project list +----------------------------------+------------------------------------------------------------------+ | ID | Name | +----------------------------------+------------------------------------------------------------------+ | 3ef434d0d82846ddb7a257365467697c | ddb7c08ba73a48eea040270d13a7b0cf-43797700-bdeb-48f5-b771-c1e5224 | | c043fb355eff47e69642adfcd7a55620 | service | | d3434f55aa5541cfab5f13916da0697d | hiroshima | | ddb7c08ba73a48eea040270d13a7b0cf | admin | +----------------------------------+------------------------------------------------------------------+root@dlp ~(keystone)# openstack user list +----------------------------------+---------------------+ | ID | Name | +----------------------------------+---------------------+ | a8f07b80a7b34425a7778dee92319ef1 | admin | | 924b949b9fcd440abc7def686443cc21 | glance | | bcb1420b3c5145289f7f99e55a477705 | nova | | 2de5e07505a548829776022cd469ff64 | placement | | 5cd4b73f2cd74402b56be5c672dcfaa7 | neutron | | 95f196a1851c4b93b016871f7d5ded82 | serverworld | | 8eac38f96d014cd9b0c7cc824f3fc95e | cinder | | 0e5574b8cffa4cd1b35d6af22d61aa3c | heat | | 04a871e222474a0e87405012de25af52 | heat_domain_admin | | 15fa68d76ee34c1b9d6fb9dec44b73d2 | barbican | | 5c9a975728494d74b24056f7349cb847 | magnum | | dc7ee73e04a146e8ab806bc577d140a8 | magnum_domain_admin | +----------------------------------+---------------------+ # for example, add [serverworld] user in [hiroshima] project to [heat_stack_owner] role root@network:~# openstack role add --project hiroshima --user serverworld heat_stack_owner
# on the Node Neutron server is running, change settings like follows root@network:~# vi /etc/neutron/policy.json # create new # overwrite some settings
{
"create_port:fixed_ips:subnet_id": "",
"create_port:allowed_address_pairs": "",
"create_port:allowed_address_pairs:ip_address": "",
}
root@network:~# systemctl restart neutron-server
# that's OK, common users can create clusters ubuntu@dlp ~(keystone)$ openstack coe cluster list +--------------------------------------+---------------+---------+------------+--------------+-----------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+---------------+---------+------------+--------------+-----------------+---------------+ | ab81cc22-8a2e-4dca-9e36-8f2a784b1b77 | k8s-cluster-1 | mykey | 1 | 1 | CREATE_COMPLETE | HEALTHY | +--------------------------------------+---------------+---------+------------+--------------+-----------------+---------------+ |
| Sponsored Link |