Предварительная настройка
Openstack в этом сетапе использует Tungsten Fabric в качестве Core Network Plugin в Neutron.
Подробнее: Настройка Cisco ASR1001X как Edge Router для Tungsten Fabric
|
Данная страница находится в разработке.
Эта страница ещё не закончена. Информация, представленная здесь, может оказаться неполной или неверной.
Если вы считаете, что её стоило бы доработать как можно быстрее, пожалуйста, соообщите.
|
Создание ВМ по шагам
Дано: только что развернутый опенстек, в качестве внешнего роутреа используется ASR1001X
Требуется: Задеплоить 2 VM с Floating IP
openstack image create
Пример загрузки образа в OpenStack
openstack \
image create \
--container-format bare \
--disk-format qcow2 \
--file ~/Downloads/noble-server-cloudimg-amd64.img \
Ubuntu-24.04
+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| container_format | bare |
| created_at | 2025-08-19T12:10:19Z |
| disk_format | qcow2 |
| file | /v2/images/803782ba-c971-4b0a-9312-49e750601ccf/file |
| id | 803782ba-c971-4b0a-9312-49e750601ccf |
| min_disk | 0 |
| min_ram | 0 |
| name | Ubuntu-24.04 |
| owner | f39e087061ea48378c9c68348eebbb59 |
| properties | locations='[]', os_hidden='False', owner_specified.openstack.md5='', owner_specified.openstack.object='images/Ubuntu-24.04', owner_specified.openstack.sha256='' |
| protected | False |
| schema | /v2/schemas/image |
| status | queued |
| tags | |
| updated_at | 2025-08-19T12:10:19Z |
| visibility | shared |
+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
keypair create
Создать пару ключей, если нужно, приватную часть сохранить так как она больше нигде не сохраняется.
openstack keypair create mmazur
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEAp4Yv+iyTCrHSMwbPahlGRdSGuuMtG+JPMYdeIhi/QDA4Wvyh
Af/TlBUNkdiYJfOJp8R6xFCOv9wREs5VHlHHk3b3xcl/w8Vtz53G3jYSu/cRV0VY
<skipped>
4vyy0i8k2fkcZooAtU4I60g9GJEWhJLiLaytXcv0XXSralhV6hihICX4SxSL5HCP
DroCuM9W/AI4rK7gyfsMdqhF6yHri8lvVAYiQMHqmvrrS85WenuY
-----END RSA PRIVATE KEY-----
Публичная сеть
openstack network create
openstack network create --external public
--external public - сеть внешняя, использует для Floating IPs и будет маршрутизироваться наружу, за пределы OpenStack
+---------------------------+---------------------------------------+
| Field | Value |
+---------------------------+---------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | None |
| availability_zones | None |
| created_at | 2025-08-18T13:50:26.265216 |
| description | |
| dns_domain | None |
| fq_name | ['default-domain', 'admin', 'public'] |
| id | d8b0b12c-de94-4068-8b0c-0f1f39d85e26 |
| ipv4_address_scope | None |
| ipv6_address_scope | None |
| is_default | None |
| is_vlan_transparent | None |
| mtu | 0 |
| name | public |
| port_security_enabled | True |
| project_id | f39e087061ea48378c9c68348eebbb59 |
| provider:network_type | None |
| provider:physical_network | None |
| provider:segmentation_id | None |
| qos_policy_id | None |
| revision_number | None |
| router:external | External |
| segments | None |
| shared | False |
| status | ACTIVE |
| subnets | |
| tags | |
| tenant_id | f39e087061ea48378c9c68348eebbb59 |
| updated_at | 2025-08-18T13:50:26.265216 |
+---------------------------+---------------------------------------+
openstack subnet create
Сабнет определяет диапазон адресов
openstack subnet create \
--network public \
--subnet-range 10.170.6.0/24 \
--allocation-pool start=10.170.6.201,end=10.170.6.249 \
--dns-nameserver 8.8.8.8 \
--gateway none \
public-subnet
+----------------------+--------------------------------------+
| Field | Value |
+----------------------+--------------------------------------+
| allocation_pools | 10.170.6.201-10.170.6.249 |
| cidr | 10.170.6.0/24 |
| created_at | 2025-08-18T13:51:12.519366 |
| description | None |
| dns_nameservers | 8.8.8.8 |
| dns_publish_fixed_ip | None |
| enable_dhcp | True |
| gateway_ip | None |
| host_routes | |
| id | d55b6937-ff01-420a-94c5-d077a9e5049c |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| name | public-subnet |
| network_id | d8b0b12c-de94-4068-8b0c-0f1f39d85e26 |
| project_id | f39e087061ea48378c9c68348eebbb59 |
| revision_number | None |
| segment_id | None |
| service_types | None |
| subnetpool_id | None |
| tags | |
| updated_at | 2025-08-18T13:51:12.519366 |
+----------------------+--------------------------------------+
Приватная сеть
openstack network create internal
openstack network create internal
+---------------------------+-----------------------------------------+
| Field | Value |
+---------------------------+-----------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | None |
| availability_zones | None |
| created_at | 2025-08-18T13:58:20.948683 |
| description | |
| dns_domain | None |
| fq_name | ['default-domain', 'admin', 'internal'] |
| id | 8546fd5c-f9bc-4521-8f46-f54f42a5491d |
| ipv4_address_scope | None |
| ipv6_address_scope | None |
| is_default | None |
| is_vlan_transparent | None |
| mtu | 0 |
| name | internal |
| port_security_enabled | True |
| project_id | f39e087061ea48378c9c68348eebbb59 |
| provider:network_type | None |
| provider:physical_network | None |
| provider:segmentation_id | None |
| qos_policy_id | None |
| revision_number | None |
| router:external | Internal |
| segments | None |
| shared | False |
| status | ACTIVE |
| subnets | |
| tags | |
| tenant_id | f39e087061ea48378c9c68348eebbb59 |
| updated_at | 2025-08-18T13:58:20.948683 |
+---------------------------+-----------------------------------------+
openstack subnet create
openstack subnet create \
--subnet-range 192.168.77.0/24 \
--network internal \
--dns-nameserver 8.8.8.8 \
internal-subnet
+----------------------+--------------------------------------+
| Field | Value |
+----------------------+--------------------------------------+
| allocation_pools | 192.168.77.2-192.168.77.254 |
| cidr | 192.168.77.0/24 |
| created_at | 2025-08-18T14:00:35.578348 |
| description | None |
| dns_nameservers | 8.8.8.8 |
| dns_publish_fixed_ip | None |
| enable_dhcp | True |
| gateway_ip | 192.168.77.1 |
| host_routes | |
| id | 06488205-7fa6-416c-accb-d6cdc514ae13 |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| name | internal-subnet |
| network_id | 8546fd5c-f9bc-4521-8f46-f54f42a5491d |
| project_id | f39e087061ea48378c9c68348eebbb59 |
| revision_number | None |
| segment_id | None |
| service_types | None |
| subnetpool_id | None |
| tags | |
| updated_at | 2025-08-18T14:00:35.578348 |
+----------------------+--------------------------------------+
router
openstack router create
openstack router create rtr01
+-------------------------+--------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | None |
| availability_zones | None |
| created_at | 2025-08-18T13:59:16.759104 |
| description | |
| enable_ndp_proxy | None |
| external_gateway_info | null |
| flavor_id | None |
| fq_name | ['default-domain', 'admin', 'rtr01'] |
| id | 008de586-a2c6-4641-a54f-8218a21dacaf |
| name | rtr01 |
| project_id | f39e087061ea48378c9c68348eebbb59 |
| revision_number | None |
| routes | None |
| status | ACTIVE |
| tags | |
| tenant_id | f39e087061ea48378c9c68348eebbb59 |
| updated_at | 2025-08-18T13:59:16.759104 |
+-------------------------+--------------------------------------+
set external-gateway
openstack router set --external-gateway public rtr01
Вывод пустой
openstack router add subnet
openstack router add subnet rtr01 internal-subnet
openstack security group
openstack security group create
openstack security group create icmp_ssh
+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| created_at | 2025-08-18T14:01:30.519406 |
| description | icmp_ssh |
| fq_name | ['default-domain', 'admin', 'icmp_ssh'] |
| id | 990e0698-f9d0-4ee6-b567-676541f84344 |
| name | icmp_ssh |
| project_id | f39e087061ea48378c9c68348eebbb59 |
| revision_number | None |
| rules | created_at='2025-08-18T14:01:30.527379', direction='egress', ethertype='IPv4', id='a29fe0eb-01e5-41df-a012-88e1af4e4672', port_range_max='65535', protocol='any', remote_ip_prefix='0.0.0.0/0', updated_at='2025-08-18T14:01:30.527379' |
| | created_at='2025-08-18T14:01:30.776084', direction='egress', ethertype='IPv6', id='1b0bb642-8af6-4842-b41b-7f73ac5600e8', port_range_max='65535', protocol='any', remote_ip_prefix='::/0', updated_at='2025-08-18T14:01:30.776084' |
| stateful | None |
| tags | [] |
| updated_at | 2025-08-18T14:01:30.940176 |
+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
openstack security group rule create
openstack security group rule create \
--remote-ip 0.0.0.0/0 \
--protocol icmp \
icmp_ssh
+-------------------------+--------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------+
| created_at | 2025-08-18T14:01:58.366970 |
| description | |
| direction | ingress |
| ether_type | IPv4 |
| id | 8e754684-e017-4ddf-8ebe-91fd314fdf1c |
| name | None |
| port_range_max | None |
| port_range_min | None |
| project_id | f39e087061ea48378c9c68348eebbb59 |
| protocol | icmp |
| remote_address_group_id | None |
| remote_group_id | None |
| remote_ip_prefix | 0.0.0.0/0 |
| revision_number | None |
| security_group_id | 990e0698-f9d0-4ee6-b567-676541f84344 |
| tags | [] |
| updated_at | 2025-08-18T14:01:58.366970 |
+-------------------------+--------------------------------------+
openstack security group rule create \
--remote-ip 0.0.0.0/0 \
--protocol tcp \
--dst-port 22 \
icmp_ssh
+-------------------------+--------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------+
| created_at | 2025-08-18T14:15:58.444894 |
| description | |
| direction | ingress |
| ether_type | IPv4 |
| id | 1248e9a7-b1da-459d-bbe1-b98c566f68f4 |
| name | None |
| port_range_max | 22 |
| port_range_min | 22 |
| project_id | f39e087061ea48378c9c68348eebbb59 |
| protocol | tcp |
| remote_address_group_id | None |
| remote_group_id | None |
| remote_ip_prefix | 0.0.0.0/0 |
| revision_number | None |
| security_group_id | 990e0698-f9d0-4ee6-b567-676541f84344 |
| tags | [] |
| updated_at | 2025-08-18T14:15:58.444894 |
+-------------------------+--------------------------------------+
openstack server create (Cirros)
openstack server create \
--flavor m1.small \
--image Cirros-6.0.raw \
--network internal \
--security-group icmp_ssh \
test-01
+-------------------------------------+-------------------------------------------------------+
| Field | Value |
+-------------------------------------+-------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | 2d2PgcQjrkVa |
| config_drive | |
| created | 2025-08-18T14:21:32Z |
| flavor | m1.small (4eaad6dc-ce03-4f5b-868b-135e7719456d) |
| hostId | |
| id | 6d0d06b8-ebc3-4d00-9eb5-18ba705981e6 |
| image | Cirros-6.0.raw (2fff2f7b-dc7a-4fa6-b68b-49f8bc8caa8d) |
| key_name | None |
| name | test-01 |
| progress | 0 |
| project_id | f39e087061ea48378c9c68348eebbb59 |
| properties | |
| security_groups | name='990e0698-f9d0-4ee6-b567-676541f84344' |
| status | BUILD |
| updated | 2025-08-18T14:21:32Z |
| user_id | f81d6b6c4efa4f46af215dc9815d510a |
| volumes_attached | |
+-------------------------------------+-------------------------------------------------------+
openstack server create \
--flavor m1.small \
--image Cirros-6.0.raw \
--network internal \
--security-group icmp_ssh \
test-02
openstack floating
openstack floating ip create public
+---------------------+--------------------------------------+
| Field | Value |
+---------------------+--------------------------------------+
| created_at | 2025-08-18T16:40:33.022272 |
| description | |
| dns_domain | None |
| dns_name | None |
| fixed_ip_address | None |
| floating_ip_address | 10.170.6.202 |
| floating_network_id | d8b0b12c-de94-4068-8b0c-0f1f39d85e26 |
| id | 3856367c-c409-4840-9ff4-9528cd150873 |
| name | 10.170.6.202 |
| port_details | None |
| port_id | None |
| project_id | f39e087061ea48378c9c68348eebbb59 |
| qos_policy_id | None |
| revision_number | None |
| router_id | None |
| status | DOWN |
| subnet_id | None |
| tags | [] |
| updated_at | 2025-08-18T16:40:33.022272 |
+---------------------+--------------------------------------+
openstack server add floating ip
openstack server add floating ip test-01 10.170.6.202
SR-IOV
Простой случай - Access в сторонй VM
SR-IOV openstack network create
openstack \
network create \
--enable-port-security \
--provider-network-type vlan \
--provider-physical-network sriovnet0 \
--provider-segment 101 \
sriov-vlan101
--provider-network-type vlan ???--provider-physical-network sriovnet0 ???--provider-segment 100 ???
+---------------------------+----------------------------------------------+
| Field | Value |
+---------------------------+----------------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | None |
| availability_zones | None |
| created_at | 2025-08-23T09:48:54.265051 |
| description | |
| dns_domain | None |
| fq_name | ['default-domain', 'admin', 'sriov-vlan101'] |
| id | 3666ef64-9387-4c66-9e63-565124258268 |
| ipv4_address_scope | None |
| ipv6_address_scope | None |
| is_default | None |
| is_vlan_transparent | None |
| mtu | 0 |
| name | sriov-vlan101 |
| port_security_enabled | True |
| project_id | f39e087061ea48378c9c68348eebbb59 |
| provider:network_type | vlan |
| provider:physical_network | sriovnet0 |
| provider:segmentation_id | 101 |
| qos_policy_id | None |
| revision_number | None |
| router:external | Internal |
| segments | None |
| shared | False |
| status | ACTIVE |
| subnets | |
| tags | |
| tenant_id | f39e087061ea48378c9c68348eebbb59 |
| updated_at | 2025-08-23T09:48:54.265051 |
+---------------------------+----------------------------------------------+
SR-IOV openstack subnet create
openstack \
subnet create \
--network sriov-vlan101 \
--no-dhcp \
--ip-version 4 \
--gateway none \
--subnet-range 172.16.64.0/24 \
sriov-vlan101-subnet01
+----------------------+--------------------------------------+
| Field | Value |
+----------------------+--------------------------------------+
| allocation_pools | 172.16.64.1-172.16.64.254 |
| cidr | 172.16.64.0/24 |
| created_at | 2025-08-23T09:51:37.653049 |
| description | None |
| dns_nameservers | |
| dns_publish_fixed_ip | None |
| dns_server_address | 172.16.64.2 |
| enable_dhcp | False |
| gateway_ip | None |
| host_routes | |
| id | 399fa951-c115-4ab6-b50f-a2d94c0a69e0 |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| name | sriov-vlan101-subnet01 |
| network_id | 3666ef64-9387-4c66-9e63-565124258268 |
| project_id | f39e087061ea48378c9c68348eebbb59 |
| revision_number | None |
| segment_id | None |
| service_types | None |
| subnetpool_id | None |
| tags | |
| updated_at | 2025-08-23T09:51:37.653049 |
+----------------------+--------------------------------------+
SR-IOV openstack port create
openstack \
port create \
--network sriov-vlan101 \
--enable-port-security \
--fixed-ip subnet=sriov-vlan101-subnet01,ip-address=172.16.64.3 \
--vnic-type direct \
sriov-vlan101-subnet01-port01
|
|
Тут важно обратить внимаение что доступный адрес начинается с 3-го в сети, в случае с Tungsten Fabric,
так как второй "занят" под DNS, что видно в сабнете
dns_server_address | 172.16.64.2
Если попробовать его использовать то будет сложнодиагностируемая ошибка, с ничего не говорящим трейсом
|
+-------------------------+----------------------------------------------------------------------------+
| Field | Value |
+-------------------------+----------------------------------------------------------------------------+
| admin_state_up | UP |
| allowed_address_pairs | |
| binding_host_id | None |
| binding_profile | None |
| binding_vif_details | port_filter='True', vlan='101' |
| binding_vif_type | unbound |
| binding_vnic_type | direct |
| created_at | 2025-08-23T10:02:23.953447 |
| data_plane_status | None |
| description | |
| device_id | |
| device_owner | |
| device_profile | None |
| dns_assignment | None |
| dns_domain | None |
| dns_name | None |
| extra_dhcp_opts | None |
| fixed_ips | ip_address='172.16.64.3', subnet_id='399fa951-c115-4ab6-b50f-a2d94c0a69e0' |
| id | 709d705c-7cc4-47c2-9671-68399da51a7e |
| ip_allocation | None |
| mac_address | 02:70:9d:70:5c:7c |
| name | sriov-vlan101-subnet01-port01 |
| network_id | 3666ef64-9387-4c66-9e63-565124258268 |
| numa_affinity_policy | None |
| port_security_enabled | True |
| project_id | f39e087061ea48378c9c68348eebbb59 |
| propagate_uplink_status | None |
| qos_network_policy_id | None |
| qos_policy_id | None |
| resource_request | None |
| revision_number | None |
| security_group_ids | 762b2618-3a38-412e-b39c-ea6921183cbe |
| status | DOWN |
| tags | |
| trunk_details | None |
| updated_at | 2025-08-23T10:02:24.014059 |
+-------------------------+----------------------------------------------------------------------------+
openstack server add port
openstack server add port ubuntu-test-01 sriov-vlan101-subnet01-port01
После того как порт "прикреплен" к серверу, можно изучить его свойства
Подробности порта
openstack port show sriov-vlan101-subnet01-port01 -c binding_profile -c binding_vif_details -f json
Ниже видно следующее
{
"binding_profile": {
"vf_num": 62,
"capabilities": [
"rx",
"tx",
"sg",
"tso",
"gso",
"gro",
"rxvlan",
"txvlan",
"txudptnl"
],
"pf_mac_address": "00:e0:ed:da:5c:8e",
"physical_network": "sriovnet0",
"pci_slot": "0000:06:1f.5",
"pci_vendor_info": "8086:10ed"
},
"binding_vif_details": {
"port_filter": true,
"vlan": "101"
}
}
vf_num: 62, номер виртуальной функцииpf_mac_address: 00:e0:ed:da:5c:8e - это мак адрес корневого устройства, а не виртуальной функции
7: enp6s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 00:e0:ed:da:5c:8e brd ff:ff:ff:ff:ff:ff
physical_network: sriovnet0 - имя физической сети, описанной в конфигурации нейтрона, куда привязаны сетевые картыpci_slot: "0000:06:1f.5" - Адрес на шине PCI, что там "сидит" можно увидеть lspci -s 06:1f.5 -vv и извлечь имя устройства ls -l /sys/bus/pci/devices/0000:06:1f.5/netpci_vendor_info: 8086:10ed Ведор (то же самое покажет например lspci -s 06:1f.5 -mm -nnport_filter: truevlan: 101 Номер Vlan
lspci -s
Зная PCI ID можно получить информацию об устройстве
lspci -s 06:1f.5 -vv
06:1f.5 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
Subsystem: Intel Corporation 82599 Ethernet Controller Virtual Function
Control: I/O- Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
IOMMU group: 159
Region 0: Memory at d02f8000 (64-bit, prefetchable) [virtual] [size=16K]
Region 3: Memory at d03f8000 (64-bit, prefetchable) [virtual] [size=16K]
Capabilities: [70] MSI-X: Enable+ Count=3 Masked-
Vector table: BAR=3 offset=00000000
PBA: BAR=3 offset=00002000
Capabilities: [a0] Express (v0) Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed unknown, Width x0, ASPM not supported
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed unknown (ok), Width x0 (ok)
TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
AERCap: First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 0
ARICtl: MFVC- ACS-, Function Group: 0
Kernel driver in use: ixgbevf
Kernel modules: ixgbevf
ip link show
А так же он настройках виртуальной функции
enp6s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 00:e0:ed:da:5c:8e brd ff:ff:ff:ff:ff:ff
vf 0 link/ether 36:b8:ac:28:5a:83 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off, query_rss off
<skipped>
vf 62 link/ether 02:70:9d:70:5c:7c brd ff:ff:ff:ff:ff:ff, vlan 101, spoof checking on, link-state auto, trust off, query_rss off
Тут вижно что:
vf 62 - совпадает с "vf_num": 6202:70:9d:70:5c:7c - мак, совпадает с
openstack port show sriov-vlan101-subnet01-port01 -c mac_address -f shell
mac_address="02:70:9d:70:5c:7c"
vlan 101 - номер VLAN заданный при создании сетиspoof checking on - включен Port Securitytrust off - Об этой опции ниже
Вид "изунтри" виртуальной машины
dmesg -T
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: [10ec:8139] type 00 class 0x020000 conventional PCI endpoint
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: BAR 0 [io 0x0000-0x00ff]
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x000000ff]
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: ROM [mem 0x00000000-0x0007ffff pref]
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: ROM [mem 0x80000000-0x8007ffff pref]: assigned
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: BAR 0 [io 0x1000-0x10ff]: assigned
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: BAR 1 [mem 0x80080000-0x800800ff]: assigned
[Sat Aug 23 10:21:23 2025] 8139cp 0000:00:04.0: enabling device (0000 -> 0003)
[Sat Aug 23 10:21:23 2025] 8139cp 0000:00:04.0 eth0: RTL-8139C+ at 0x000000007f98c756, 02:70:9d:70:5c:7c, IRQ 11
[Sat Aug 23 10:21:23 2025] 8139cp 0000:00:04.0 ens4: renamed from eth0
Проверка работы Port Security
ip link show
5: ens4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether 02:70:9d:70:5c:7c brd ff:ff:ff:ff:ff:ff
altname enp0s4
Добавить IP и попробовать послать запросы, адрес взят "от фонаря", важно только что бы запрос ушел в сеть и свитч увидел МАК
ip addr add 10.90.0.2/24 dev ens4
ip link set up dev ens4
Далее запустить ping, естественно не ожидая ответов
ping 10.90.0.1
На хост-системе видно запросы (все кроме броадкастов может не попадать в дамп и это нормально!)
# tcpdump -n -i enp6s0f1 -ee
11:05:53.634812 02:70:9d:70:5c:7c > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP (0x0806), Request who-has 10.90.0.1 tell 10.90.0.2, length 46
Со стороны свитча мак виден
dell-lab#show mac-address-table vlan 101
Codes: *N - VLT Peer Synced MAC
*I - Internal MAC Address used for Inter Process Communication
VlanId Mac Address Type Interface State
101 02:70:9d:70:5c:7c Dynamic Te 0/57 Active
Если попробовать поменять МАК
ip link set dev ens4 address 02:a9:21:bc:e4:5b
Илм запустить утилиту arppoison ens4 из пакета arptools - новых мак-адресов на свитче не появляется, работает как ожидалось
Отключение Port Security
В качествет теста создам порт с отключенным Port Security
Можно отключить и у существующего порта - см
openstack port set --disable-port-security --binding-profile trusted=true test-sriov01-sriov_port_1-manual
openstack \
port create \
--network sriov-vlan101 \
--disable-port-security \
--fixed-ip subnet=sriov-vlan101-subnet01,ip-address=172.16.64.4 \
--vnic-type direct \
sriov-vlan101-subnet01-port02
В выводе единственное отличие
| port_security_enabled | False
+-------------------------+----------------------------------------------------------------------------+
| Field | Value |
+-------------------------+----------------------------------------------------------------------------+
| admin_state_up | UP |
| allowed_address_pairs | |
| binding_host_id | None |
| binding_profile | None |
| binding_vif_details | port_filter='True', vlan='101' |
| binding_vif_type | unbound |
| binding_vnic_type | direct |
| created_at | 2025-08-23T11:29:00.135520 |
| data_plane_status | None |
| description | |
| device_id | |
| device_owner | |
| device_profile | None |
| dns_assignment | None |
| dns_domain | None |
| dns_name | None |
| extra_dhcp_opts | None |
| fixed_ips | ip_address='172.16.64.4', subnet_id='399fa951-c115-4ab6-b50f-a2d94c0a69e0' |
| id | a8ba8629-eb70-46c1-b69f-3cfb9b04ab88 |
| ip_allocation | None |
| mac_address | 02:a8:ba:86:29:eb |
| name | sriov-vlan101-subnet01-port02 |
| network_id | 3666ef64-9387-4c66-9e63-565124258268 |
| numa_affinity_policy | None |
| port_security_enabled | False |
| project_id | f39e087061ea48378c9c68348eebbb59 |
| propagate_uplink_status | None |
| qos_network_policy_id | None |
| qos_policy_id | None |
| resource_request | None |
| revision_number | None |
| security_group_ids | |
| status | DOWN |
| tags | |
| trunk_details | None |
| updated_at | 2025-08-23T11:29:00.183637 |
+-------------------------+----------------------------------------------------------------------------+
openstack server add port ubuntu-test-01 sriov-vlan101-subnet01-port02
Со стороны сервера базовые настройки
ip link set up dev ens8
ip addr add 10.90.1.2/24 dev ens8
ip ro
default via 192.168.77.1 dev ens3 proto dhcp src 192.168.77.5 metric 100
8.8.8.8 via 192.168.77.1 dev ens3 proto dhcp src 192.168.77.5 metric 100
10.90.0.0/24 dev ens4 proto kernel scope link src 10.90.0.2
10.90.1.0/24 dev ens8 proto kernel scope link src 10.90.1.2
192.168.77.0/24 dev ens3 proto kernel scope link src 192.168.77.5 metric 100
192.168.77.1 dev ens3 proto dhcp scope link src 192.168.77.5 metric 100
На Хосте ожидаемо наблюдаем spoof checking off
vf 61 link/ether 02:a8:ba:86:29:eb brd ff:ff:ff:ff:ff:ff, vlan 101, spoof checking off, link-state auto, trust off, query_rss off
Смена мака
ip link set dev ens8 address 02:70:9d:70:5c:99
Видно что мак поменялся (99 на конце)
tcpdump -n -i enp6s0f1 -ee
11:53:56.233183 02:70:9d:70:5c:99 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP (0x0806), Request who-has 10.90.1.1 tell 10.90.1.2, length 46
Оба мака заехали на свитч (что ожидаемо при отключеном Port Security)
show mac-address-table vlan 101
Codes: *N - VLT Peer Synced MAC
*I - Internal MAC Address used for Inter Process Communication
VlanId Mac Address Type Interface State
101 02:70:9d:70:5c:7c Dynamic Te 0/57 Active
dell-lab#show mac-address-table vlan 101
Codes: *N - VLT Peer Synced MAC
*I - Internal MAC Address used for Inter Process Communication
VlanId Mac Address Type Interface State
101 02:70:9d:70:5c:7c Dynamic Te 0/57 Active
101 02:70:9d:70:5c:99 Dynamic Te 0/57 Active
101 02:a8:ba:86:29:eb Dynamic Te 0/57 Active
"протравить" свитч заполнив табличку коммутации:
arppoison ens8
Случайные пары мак-адресов/ip-адресов
11:56:51.593634 00:3d:9f:e3:5e:0b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP (0x0806), Reply 192.199.71.81 is-at 00:3d:9f:e3:5e:0b, length 46
11:56:51.594051 00:3d:52:f1:46:38 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP (0x0806), Reply 120.206.54.21 is-at 00:3d:52:f1:46:38, length 46
11:56:51.594419 00:26:5c:59:a4:fb > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP (0x0806), Reply 99.143.18.102 is-at 00:26:5c:59:a4:fb, length 46
11:56:51.594805 00:13:a2:48:f1:76 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP (0x0806), Reply 170.228.232.78 is-at 00:13:a2:48:f1:76, length 46
Свитчк такое нравится не очень, таблица забита полностью (а свитч довольно жирный)
dell-lab#Aug 23 11:57:27 %S4810:0 %MACAGT-5-HASH_COLLISION_LOG: Mac:00:17:71:90:4d:91/Vlan:101 could not be added to L2 CAM on portpipe 0 stack-unit 0 due to hash collision. Total number of hash collisions: 17888
Aug 23 11:57:40 %S4810:0 %MACAGT-5-HASH_COLLISION_LOG: Mac:00:2b:5a:e9:6c:d1/Vlan:101 could not be added to L2 CAM on portpipe 0 stack-unit 0 due to hash collision. Total number of hash collisions: 18365
Aug 23 11:57:52 %S4810:0 %MACAGT-5-HASH_COLLISION_LOG: Mac:00:10:04:a7:90:a9/Vlan:101 could not be added to L2 CAM on portpipe 0 stack-unit 0 due to hash collision. Total number of hash collisions: 18661
Aug 23 11:57:53 %STKUNIT0-M:CP %SYSADM-5-CPU_THRESHOLD_CLR: Overall cpu usage of management-unit drops below threshold. Cpu1minUsage (73%)
show mac-address-table count vlan 101
MAC Entries for vlan 101 :
Dynamic Address Count : 130988
Static Address (User-defined) Count : 0
Sticky Address Count : 0
Total MAC Addresses in Use: 130988
