OpenStack Caracal : Neutron Network (VXLAN)2024/08/06 |
|
Configure virtual networking by OpenStack Network Service (Neutron).
For example, configure VXLAN type of networking on here. Furthermore, this example is based on the environment that Network Node has 2 network interfaces.
------------+--------------------------+--------------------------+------------
| | |
eth0|10.0.0.30 eth0|10.0.0.50 eth0|10.0.0.51
+-----------+-----------+ +-----------+-----------+ +-----------+-----------+
| [ dlp.srv.world ] | | [ network.srv.world ] | | [ node01.srv.world ] |
| (Control Node) | | (Network Node) | | (Compute Node) |
| | | | | |
| MariaDB RabbitMQ | | Neutron L2/L3 Agent | | Libvirt |
| Memcached Nginx | | Neutron Metadata | | Nova Compute |
| Keystone httpd | | Open vSwitch | | Neutron L2 Agent |
| Glance Nova API | | | | Open vSwitch |
| Neutron Server | | | | |
| Neutron Metadata | | | | |
+-----------------------+ +-----------+-----------+ +-----------------------+
eth1|(UP with no IP)
|
| [1] | Change settings on Control Node. |
|
[root@dlp ~(keystone)]#
vi /etc/neutron/plugins/ml2/ml2_conf.ini # add a value to [tenant_network_types] [ml2] type_drivers = flat,vlan,gre,vxlan tenant_network_types = vxlan
# add to last line [ml2_type_flat] flat_networks = physnet1 [ml2_type_vxlan] vni_ranges = 1:1000 systemctl restart neutron-server |
| [2] | Change settings on Network Node. |
|
# add bridge [root@network ~]# ovs-vsctl add-br br-eth1
# add [eth1] to the port of the bridge above # replace the interface name [eth1] to your own environment [root@network ~]# ovs-vsctl add-port br-eth1 eth1
[root@network ~]#
vi /etc/neutron/plugins/ml2/ml2_conf.ini # add a value to [tenant_network_types] [ml2] type_drivers = flat,vlan,gre,vxlan tenant_network_types = vxlan mechanism_drivers = openvswitch extension_drivers = port_security,qos # add to last line [ml2_type_flat] flat_networks = physnet1 [ml2_type_vxlan] vni_ranges = 1:1000
[root@network ~]#
vi /etc/neutron/plugins/ml2/openvswitch_agent.ini # add to last line
[agent]
tunnel_types = vxlan
prevent_arp_spoofing = True
[ovs]
# specify IP address of this host for [local_ip]
local_ip = 10.0.0.50
bridge_mappings = physnet1:br-eth1
[root@network ~]#
systemctl restart neutron-dhcp-agent neutron-l3-agent neutron-metadata-agent neutron-openvswitch-agent
# if Firewalld is running, allow VXLAN port [root@network ~]# firewall-cmd --add-port=4789/udp [root@network ~]# firewall-cmd --runtime-to-permanent |
| [3] | Change settings on Compute Node. |
|
[root@node01 ~]#
vi /etc/neutron/plugins/ml2/ml2_conf.ini # add a value to [tenant_network_types] [ml2] type_drivers = flat,vlan,gre,vxlan tenant_network_types = vxlan mechanism_drivers = openvswitch extension_drivers = port_security,qos # add to last line [ml2_type_flat] flat_networks = physnet1 [ml2_type_vxlan] vni_ranges = 1:1000
[root@node01 ~]#
vi /etc/neutron/plugins/ml2/openvswitch_agent.ini # add to last line
[agent]
tunnel_types = vxlan
prevent_arp_spoofing = True
[ovs]
# specify IP address of this host for [local_ip]
local_ip = 10.0.0.51
[root@node01 ~]#
systemctl restart neutron-openvswitch-agent
# if Firewalld is running, allow VXLAN port [root@node01 ~]# firewall-cmd --add-port=4789/udp [root@node01 ~]# firewall-cmd --runtime-to-permanent |
| [4] | Create a Virtual router. It's OK to work on any node. (This example is on Control Node) |
|
[root@dlp ~(keystone)]# openstack router create router01 +-------------------------+--------------------------------------+ | Field | Value | +-------------------------+--------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | | | created_at | 2024-08-06T06:07:17Z | | description | | | distributed | False | | enable_ndp_proxy | None | | external_gateway_info | null | | flavor_id | None | | ha | False | | id | e048476f-038f-44a0-898b-0d7a946442a0 | | name | router01 | | project_id | b62e60c6d4b7406a90eee2f2d9c45917 | | revision_number | 1 | | routes | | | status | ACTIVE | | tags | | | tenant_id | b62e60c6d4b7406a90eee2f2d9c45917 | | updated_at | 2024-08-06T06:07:17Z | +-------------------------+--------------------------------------+ |
| [5] | Create internal network and associate with the router above. |
|
# create internal network [root@dlp ~(keystone)]# openstack network create private --provider-network-type vxlan +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | | | created_at | 2024-08-06T06:07:38Z | | description | | | dns_domain | None | | id | 06ea0ac0-3e7f-4bd9-ae9a-82215284cd7b | | ipv4_address_scope | None | | ipv6_address_scope | None | | is_default | False | | is_vlan_transparent | None | | mtu | 1450 | | name | private | | port_security_enabled | True | | project_id | b62e60c6d4b7406a90eee2f2d9c45917 | | provider:network_type | vxlan | | provider:physical_network | None | | provider:segmentation_id | 203 | | qos_policy_id | None | | revision_number | 1 | | router:external | Internal | | segments | None | | shared | False | | status | ACTIVE | | subnets | | | tags | | | tenant_id | b62e60c6d4b7406a90eee2f2d9c45917 | | updated_at | 2024-08-06T06:07:38Z | +---------------------------+--------------------------------------+ # create subnet in the internal network [root@dlp ~(keystone)]# openstack subnet create private-subnet --network private \ --subnet-range 192.168.100.0/24 --gateway 192.168.100.1 \ --dns-nameserver 10.0.0.10 +----------------------+--------------------------------------+ | Field | Value | +----------------------+--------------------------------------+ | allocation_pools | 192.168.100.2-192.168.100.254 | | cidr | 192.168.100.0/24 | | created_at | 2024-08-06T06:08:04Z | | description | | | dns_nameservers | 10.0.0.10 | | dns_publish_fixed_ip | None | | enable_dhcp | True | | gateway_ip | 192.168.100.1 | | host_routes | | | id | dddd52f4-a592-44e7-a9ec-889c3d1c0da8 | | ip_version | 4 | | ipv6_address_mode | None | | ipv6_ra_mode | None | | name | private-subnet | | network_id | 06ea0ac0-3e7f-4bd9-ae9a-82215284cd7b | | project_id | b62e60c6d4b7406a90eee2f2d9c45917 | | revision_number | 0 | | segment_id | None | | service_types | | | subnetpool_id | None | | tags | | | updated_at | 2024-08-06T06:08:04Z | +----------------------+--------------------------------------+ # set internal network to the router above [root@dlp ~(keystone)]# openstack router add subnet router01 private-subnet
|
| [6] | Create external network and associate with the router above. |
|
# create external network [root@dlp ~(keystone)]# openstack network create \ --provider-physical-network physnet1 \ --provider-network-type flat --external public +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | | | created_at | 2024-08-06T06:08:36Z | | description | | | dns_domain | None | | id | af151899-ed5a-486d-9b3a-e635f2f3c7af | | ipv4_address_scope | None | | ipv6_address_scope | None | | is_default | False | | is_vlan_transparent | None | | mtu | 1500 | | name | public | | port_security_enabled | True | | project_id | b62e60c6d4b7406a90eee2f2d9c45917 | | provider:network_type | flat | | provider:physical_network | physnet1 | | provider:segmentation_id | None | | qos_policy_id | None | | revision_number | 1 | | router:external | External | | segments | None | | shared | False | | status | ACTIVE | | subnets | | | tags | | | tenant_id | b62e60c6d4b7406a90eee2f2d9c45917 | | updated_at | 2024-08-06T06:08:36Z | +---------------------------+--------------------------------------+ # create subnet in the external network [root@dlp ~(keystone)]# openstack subnet create public-subnet \ --network public --subnet-range 10.0.0.0/24 \ --allocation-pool start=10.0.0.200,end=10.0.0.254 \ --gateway 10.0.0.1 --dns-nameserver 10.0.0.10 --no-dhcp +----------------------+--------------------------------------+ | Field | Value | +----------------------+--------------------------------------+ | allocation_pools | 10.0.0.200-10.0.0.254 | | cidr | 10.0.0.0/24 | | created_at | 2024-08-06T06:08:57Z | | description | | | dns_nameservers | 10.0.0.10 | | dns_publish_fixed_ip | None | | enable_dhcp | False | | gateway_ip | 10.0.0.1 | | host_routes | | | id | 779d2b66-0bde-4919-95e1-49d66106c4d0 | | ip_version | 4 | | ipv6_address_mode | None | | ipv6_ra_mode | None | | name | public-subnet | | network_id | af151899-ed5a-486d-9b3a-e635f2f3c7af | | project_id | b62e60c6d4b7406a90eee2f2d9c45917 | | revision_number | 0 | | segment_id | None | | service_types | | | subnetpool_id | None | | tags | | | updated_at | 2024-08-06T06:08:57Z | +----------------------+--------------------------------------+ # set gateway to the router above [root@dlp ~(keystone)]# openstack router set router01 --external-gateway public |
| [7] | By default, it's possible to access for all projects to external network, however, for internal network, only admin projects can access to it, so grant access permission of internal network to a project you'd like to let users in the project use. |
|
# show network RBAC list [root@dlp ~(keystone)]# openstack network rbac list +--------------------------------------+-------------+--------------------------------------+ | ID | Object Type | Object ID | +--------------------------------------+-------------+--------------------------------------+ | 9e07f62b-833e-4d5a-94c4-e22b7ec188e0 | network | af151899-ed5a-486d-9b3a-e635f2f3c7af | +--------------------------------------+-------------+--------------------------------------+ # RBAC details # all projects can access only to [access_as_external] [root@dlp ~(keystone)]# openstack network rbac show 9e07f62b-833e-4d5a-94c4-e22b7ec188e0 +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | action | access_as_external | | id | 9e07f62b-833e-4d5a-94c4-e22b7ec188e0 | | object_id | af151899-ed5a-486d-9b3a-e635f2f3c7af | | object_type | network | | project_id | b62e60c6d4b7406a90eee2f2d9c45917 | | target_project_id | * | +-------------------+--------------------------------------+ # show network list [root@dlp ~(keystone)]# openstack network list +--------------------------------------+---------+--------------------------------------+ | ID | Name | Subnets | +--------------------------------------+---------+--------------------------------------+ | 06ea0ac0-3e7f-4bd9-ae9a-82215284cd7b | private | dddd52f4-a592-44e7-a9ec-889c3d1c0da8 | | af151899-ed5a-486d-9b3a-e635f2f3c7af | public | 779d2b66-0bde-4919-95e1-49d66106c4d0 | +--------------------------------------+---------+--------------------------------------+ # show project list [root@dlp ~(keystone)]# openstack project list +----------------------------------+-----------+ | ID | Name | +----------------------------------+-----------+ | 03a31c43ede44dcbaacb05c870920e81 | hiroshima | | 0b826f1bf49d489494e94badc32f2cea | service | | b62e60c6d4b7406a90eee2f2d9c45917 | admin | +----------------------------------+-----------+ # grant [access_as_shared] permission for [private] to [hiroshima] project [root@dlp ~(keystone)]# netID=$(openstack network list | grep private | awk '{ print $2 }') [root@dlp ~(keystone)]# prjID=$(openstack project list | grep hiroshima | awk '{ print $2 }') [root@dlp ~(keystone)]# openstack network rbac create --target-project $prjID --type network --action access_as_shared $netID +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | action | access_as_shared | | id | 289983d0-4c98-47db-be81-f95d642604bf | | object_id | 06ea0ac0-3e7f-4bd9-ae9a-82215284cd7b | | object_type | network | | project_id | b62e60c6d4b7406a90eee2f2d9c45917 | | target_project_id | 03a31c43ede44dcbaacb05c870920e81 | +-------------------+--------------------------------------+ |
| [8] | Login as a user who can use Openstack system and Create and start a Virtual machine Instance with the network created above. |
|
# show available [flavor] list [cent@dlp ~(keystone)]$ openstack flavor list +----+-----------+-------+------+-----------+-------+-----------+ | ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public | +----+-----------+-------+------+-----------+-------+-----------+ | 1 | m1.tiny | 2048 | 10 | 0 | 1 | True | | 2 | m1.small | 4096 | 10 | 0 | 2 | True | | 3 | m1.medium | 8192 | 10 | 0 | 4 | True | | 4 | m1.large | 16384 | 10 | 0 | 8 | True | | 5 | m2.large | 16384 | 10 | 10 | 8 | True | +----+-----------+-------+------+-----------+-------+-----------+ # show available image list [cent@dlp ~(keystone)]$ openstack image list +--------------------------------------+----------------+--------+ | ID | Name | Status | +--------------------------------------+----------------+--------+ | cfa9783d-ca91-4e3b-8016-81e3336bd5f0 | CentOS-Stream9 | active | +--------------------------------------+----------------+--------+ # show available network list [cent@dlp ~(keystone)]$ openstack network list +--------------------------------------+---------+--------------------------------------+ | ID | Name | Subnets | +--------------------------------------+---------+--------------------------------------+ | 06ea0ac0-3e7f-4bd9-ae9a-82215284cd7b | private | dddd52f4-a592-44e7-a9ec-889c3d1c0da8 | | af151899-ed5a-486d-9b3a-e635f2f3c7af | public | 779d2b66-0bde-4919-95e1-49d66106c4d0 | +--------------------------------------+---------+--------------------------------------+ # create a security group for instances [cent@dlp ~(keystone)]$ openstack security group create secgroup01 +-----------------+---------------------------------------------------------------------------+ | Field | Value | +-----------------+---------------------------------------------------------------------------+ | created_at | 2024-08-06T05:49:57Z | | description | secgroup01 | | id | 9b8cb318-2949-4bdb-9978-89b547c4d461 | | name | secgroup01 | | project_id | 03a31c43ede44dcbaacb05c870920e81 | | revision_number | 1 | | rules | created_at='2024-08-06T05:49:57Z', direction='egress', ethertype='IPv6... | | | created_at='2024-08-06T05:49:57Z', direction='egress', ethertype='IPv4... | | shared | False | | stateful | True | | tags | [] | | updated_at | 2024-08-06T05:49:57Z | +-----------------+---------------------------------------------------------------------------+ # create an SSH keypair for connecting to instances [cent@dlp ~(keystone)]$ ssh-keygen -q -N "" Enter file in which to save the key (/home/cent/.ssh/id_rsa): # add public-key [cent@dlp ~(keystone)]$ openstack keypair create --public-key ~/.ssh/id_rsa.pub mykey +-------------+-------------------------------------------------+ | Field | Value | +-------------+-------------------------------------------------+ | created_at | None | | fingerprint | f6:ea:82:63:6b:66:c0:bf:58:44:94:a4:c5:9f:21:56 | | id | mykey | | is_deleted | None | | name | mykey | | type | ssh | | user_id | 8a9b46168b42480f889ad4fad3ca6ee4 | +-------------+-------------------------------------------------+[cent@dlp ~(keystone)]$ netID=$(openstack network list | grep private | awk '{ print $2 }')
[cent@dlp ~(keystone)]$
[cent@dlp ~(keystone)]$ openstack server create --flavor m1.small --image CentOS-Stream9 --security-group secgroup01 --nic net-id=$netID --key-name mykey CentOS-St9
openstack server list +--------------------------------------+------------+--------+-------------------------+----------------+----------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+------------+--------+-------------------------+----------------+----------+ | 339084d3-eeed-401e-84e8-4ffbc7d23441 | CentOS-St9 | ACTIVE | private=192.168.100.177 | CentOS-Stream9 | m1.small | +--------------------------------------+------------+--------+-------------------------+----------------+----------+ |
| [9] | Assign floating IP address to the Instance above. |
|
[cent@dlp ~(keystone)]$ openstack floating ip create public +---------------------+--------------------------------------+ | Field | Value | +---------------------+--------------------------------------+ | created_at | 2024-08-06T06:13:57Z | | description | | | dns_domain | None | | dns_name | None | | fixed_ip_address | None | | floating_ip_address | 10.0.0.218 | | floating_network_id | af151899-ed5a-486d-9b3a-e635f2f3c7af | | id | d4d6ad0d-16f6-4bda-92d3-5de4ce88a240 | | name | 10.0.0.218 | | port_details | None | | port_id | None | | project_id | 03a31c43ede44dcbaacb05c870920e81 | | qos_policy_id | None | | revision_number | 0 | | router_id | None | | status | DOWN | | subnet_id | None | | tags | [] | | updated_at | 2024-08-06T06:13:57Z | +---------------------+--------------------------------------+
[cent@dlp ~(keystone)]$
openstack server add floating ip CentOS-St9 10.0.0.218
# confirm settings [cent@dlp ~(keystone)]$ openstack floating ip show 10.0.0.218 +---------------------+--------------------------------------------------------------------------+ | Field | Value | +---------------------+--------------------------------------------------------------------------+ | created_at | 2024-08-06T06:13:57Z | | description | | | dns_domain | None | | dns_name | None | | fixed_ip_address | 192.168.100.177 | | floating_ip_address | 10.0.0.218 | | floating_network_id | af151899-ed5a-486d-9b3a-e635f2f3c7af | | id | d4d6ad0d-16f6-4bda-92d3-5de4ce88a240 | | name | 10.0.0.218 | | port_details | admin_state_up='True', device_id='339084d3-eeed-401e-84e8-4ffbc7d2344... | | port_id | 6f27cd7f-91c3-4b05-b7de-9683edf68092 | | project_id | 03a31c43ede44dcbaacb05c870920e81 | | qos_policy_id | None | | revision_number | 2 | | router_id | e048476f-038f-44a0-898b-0d7a946442a0 | | status | ACTIVE | | subnet_id | None | | tags | [] | | updated_at | 2024-08-06T06:14:33Z | +---------------------+--------------------------------------------------------------------------+[cent@dlp ~(keystone)]$ openstack server list +--------------------------------------+------------+--------+-------------------------------------+----------------+----------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+------------+--------+-------------------------------------+----------------+----------+ | 339084d3-eeed-401e-84e8-4ffbc7d23441 | CentOS-St9 | ACTIVE | private=10.0.0.218, 192.168.100.177 | CentOS-Stream9 | m1.small | +--------------------------------------+------------+--------+-------------------------------------+----------------+----------+ |
| [10] | Configure security settings for the security group you created above to access with SSH and ICMP. |
|
# permit ICMP [cent@dlp ~(keystone)]$ openstack security group rule create --protocol icmp --ingress secgroup01 +-------------------------+--------------------------------------+ | Field | Value | +-------------------------+--------------------------------------+ | belongs_to_default_sg | False | | created_at | 2024-08-06T05:53:05Z | | description | | | direction | ingress | | ether_type | IPv4 | | id | ba99ced1-47c0-4bb9-8beb-0e4caa8df2c7 | | name | None | | normalized_cidr | 0.0.0.0/0 | | port_range_max | None | | port_range_min | None | | project_id | 03a31c43ede44dcbaacb05c870920e81 | | protocol | icmp | | remote_address_group_id | None | | remote_group_id | None | | remote_ip_prefix | 0.0.0.0/0 | | revision_number | 0 | | security_group_id | 9b8cb318-2949-4bdb-9978-89b547c4d461 | | tags | [] | | updated_at | 2024-08-06T05:53:05Z | +-------------------------+--------------------------------------+ # permit SSH [cent@dlp ~(keystone)]$ openstack security group rule create --protocol tcp --dst-port 22:22 secgroup01 +-------------------------+--------------------------------------+ | Field | Value | +-------------------------+--------------------------------------+ | belongs_to_default_sg | False | | created_at | 2024-08-06T05:53:24Z | | description | | | direction | ingress | | ether_type | IPv4 | | id | 7efebe00-c472-4e55-bc02-62a23be2dfd9 | | name | None | | normalized_cidr | 0.0.0.0/0 | | port_range_max | 22 | | port_range_min | 22 | | project_id | 03a31c43ede44dcbaacb05c870920e81 | | protocol | tcp | | remote_address_group_id | None | | remote_group_id | None | | remote_ip_prefix | 0.0.0.0/0 | | revision_number | 0 | | security_group_id | 9b8cb318-2949-4bdb-9978-89b547c4d461 | | tags | [] | | updated_at | 2024-08-06T05:53:24Z | +-------------------------+--------------------------------------+[cent@dlp ~(keystone)]$ openstack security group rule list secgroup01 +--------------------------------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+ | ID | IP Protocol | Ethertype | IP Range | Port Range | Direction | Remote Security Group | Remote Address Group | +--------------------------------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+ | 0f65eebf-8d8d-45a5-94fa-a6e1b61b82e5 | None | IPv6 | ::/0 | | egress | None | None | | 7efebe00-c472-4e55-bc02-62a23be2dfd9 | tcp | IPv4 | 0.0.0.0/0 | 22:22 | ingress | None | None | | ba99ced1-47c0-4bb9-8beb-0e4caa8df2c7 | icmp | IPv4 | 0.0.0.0/0 | | ingress | None | None | | cf1e0c43-e08e-4179-a6fe-e48ab48425a1 | None | IPv4 | 0.0.0.0/0 | | egress | None | None | +--------------------------------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+ |
| [11] | It's possible to login to the Instance to connect to the floating IP address with SSH like follows. |
|
[cent@dlp ~(keystone)]$ openstack server list +--------------------------------------+------------+--------+-------------------------------------+----------------+----------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+------------+--------+-------------------------------------+----------------+----------+ | 339084d3-eeed-401e-84e8-4ffbc7d23441 | CentOS-St9 | ACTIVE | private=10.0.0.218, 192.168.100.177 | CentOS-Stream9 | m1.small | +--------------------------------------+------------+--------+-------------------------------------+----------------+----------+[cent@dlp ~(keystone)]$ ssh centos@10.0.0.241 The authenticity of host '10.0.0.218 (10.0.0.218)' can't be established. ED25519 key fingerprint is SHA256:NwRGo4xu2UxpA3uMSXzSEyumLMZLoiVLqH0y7lO6Kkg. This key is not known by any other names Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added '10.0.0.218' (ED25519) to the list of known hosts. [centos@centos-st9 ~]$ # logined |
| Sponsored Link |