Cisco ASR1001 Tungsten Fabric OpenStack VM: различия между версиями

Материал из noname.com.ua
Перейти к навигацииПерейти к поиску
 
(не показано 99 промежуточных версий этого же участника)
Строка 6: Строка 6:
 
Openstack в этом сетапе использует Tungsten Fabric в качестве Core Network Plugin в Neutron. <BR>
  
Openstack в этом сетапе использует Tungsten Fabric в качестве Core Network Plugin в Neutron. <BR>
 
Подробнее: [[Cisco_ASR1001_Tungsten_Fabric|Настройка Cisco ASR1001X как Edge Router для Tungsten Fabric]]
  
Подробнее: [[Cisco_ASR1001_Tungsten_Fabric|Настройка Cisco ASR1001X как Edge Router для Tungsten Fabric]]
  +
  +
{{stub}}
   
 
=Создание ВМ по шагам=
  
=Создание ВМ по шагам=
Строка 14: Строка 16:
   
   
−
==<code>image create</code>==
 +
==<code>openstack image create</code>==
 
Пример загрузки образа в OpenStack
  
Пример загрузки образа в OpenStack
 
<PRE>
  
<PRE>
Строка 24: Строка 26:
 
Ubuntu-24.04
  
Ubuntu-24.04
 
</PRE>
  
</PRE>
  +
{{#spoiler:show=Output: openstack image create|
 
<PRE>
  
<PRE>
 
+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  
+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Строка 46: Строка 49:
 
+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  
+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 
</PRE>
  
</PRE>
  +
}}
   
 
==<code>keypair create</code>==
  
==<code>keypair create</code>==
Строка 61: Строка 65:
 
-----END RSA PRIVATE KEY-----
  
-----END RSA PRIVATE KEY-----
 
</PRE>
  
</PRE>
  +
==Публичная сеть==
−
==<code><code> openstack network create</code></code>==
  
  +
===<code><code>openstack network create</code></code>===
   
 
<PRE>
  
<PRE>
Строка 67: Строка 72:
 
</PRE>
  
</PRE>
 
* <code>--external public</code> - сеть внешняя, использует для Floating IPs и будет маршрутизироваться наружу, за пределы OpenStack
  
* <code>--external public</code> - сеть внешняя, использует для Floating IPs и будет маршрутизироваться наружу, за пределы OpenStack
  +
{{#spoiler:show=Output: openstack network create|
 
<PRE>
  
<PRE>
 
+---------------------------+---------------------------------------+
  
+---------------------------+---------------------------------------+
Строка 102: Строка 108:
 
+---------------------------+---------------------------------------+
  
+---------------------------+---------------------------------------+
 
</PRE>
  
</PRE>
  +
}}
   
−
==<code><code>openstack subnet create</code></code>==
 +
===<code><code>openstack subnet create</code></code>===
 
Сабнет определяет диапазон адресов
  
Сабнет определяет диапазон адресов
 
<PRE>
  
<PRE>
Строка 114: Строка 121:
 
public-subnet
  
public-subnet
 
</PRE>
  
</PRE>
  +
{{#spoiler:show=Output: openstack subnet create|
−  
 
<PRE>
  
<PRE>
 
+----------------------+--------------------------------------+
  
+----------------------+--------------------------------------+
Строка 143: Строка 150:
 
+----------------------+--------------------------------------+
  
+----------------------+--------------------------------------+
 
</PRE>
  
</PRE>
  +
}}
   
  +
==Приватная сеть==
−
==<code><code> openstack network create internal</code></code>==
  
  +
===<code><code> openstack network create internal</code></code>===
 
<PRE>
  
<PRE>
 
openstack network create internal
  
openstack network create internal
  +
</PRE>
  +
{{#spoiler:show=Output: openstack network create internal|
  +
<PRE>
 
+---------------------------+-----------------------------------------+
  
+---------------------------+-----------------------------------------+
 
| Field | Value |
  
| Field | Value |
Строка 181: Строка 193:
 
+---------------------------+-----------------------------------------+
  
+---------------------------+-----------------------------------------+
 
</PRE>
  
</PRE>
  +
}}
   
−
==<code><code>openstack subnet create</code></code>==
 +
===<code><code>openstack subnet create</code></code>===
 
<PRE>
  
<PRE>
 
openstack subnet create \
  
openstack subnet create \
Строка 189: Строка 202:
 
--dns-nameserver 8.8.8.8 \
  
--dns-nameserver 8.8.8.8 \
 
internal-subnet
  
internal-subnet
  +
</PRE>
  +
  +
{{#spoiler:show=Output: openstack subnet create internal-subnet|
  +
  +
<PRE>
   
 
+----------------------+--------------------------------------+
  
+----------------------+--------------------------------------+
Строка 217: Строка 235:
 
+----------------------+--------------------------------------+
  
+----------------------+--------------------------------------+
 
</PRE>
  
</PRE>
  +
}}
   
−
=<code>2</code>=
 +
==<code>router</code>==
  +
===<code><code> openstack router create </code></code>===
 
<PRE>
  
<PRE>
 
openstack router create rtr01
  
openstack router create rtr01
  +
</PRE>
  +
{{#spoiler:show=Output: openstack router create rtr01|
  +
<PRE>
 
+-------------------------+--------------------------------------+
  
+-------------------------+--------------------------------------+
 
| Field | Value |
  
| Field | Value |
Строка 244: Строка 267:
 
+-------------------------+--------------------------------------+
  
+-------------------------+--------------------------------------+
 
</PRE>
  
</PRE>
  +
}}
   
−
=<code>3</code>=
 +
===<code>set external-gateway</code>===
 
<PRE>
  
<PRE>
 
openstack router set --external-gateway public rtr01
  
openstack router set --external-gateway public rtr01
Строка 251: Строка 275:
 
</PRE>
  
</PRE>
   
−
=<code>4</code>=
 +
===<code>openstack router add subnet</code>===
 
<PRE>
  
<PRE>
 
openstack router add subnet rtr01 internal-subnet
  
openstack router add subnet rtr01 internal-subnet
 
</PRE>
  
</PRE>
   
−
=<code>5</code>=
 +
==<code>openstack security group</code>==
  +
===<code>openstack security group create</code>===
 
<PRE>
  
<PRE>
 
openstack security group create icmp_ssh
  
openstack security group create icmp_ssh
  +
</PRE>
  +
{{#spoiler:show=Output: openstack security group create icmp_ssh|
  +
<PRE>
 
+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  
+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 
| Field | Value |
  
| Field | Value |
Строка 276: Строка 304:
 
+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  
+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 
</PRE>
  
</PRE>
  +
}}
−
=<code>6</code>=
  
  +
  +
===<code>openstack security group rule create</code>===
 
<PRE>
  
<PRE>
 
openstack security group rule create \
  
openstack security group rule create \
−
> --remote-ip 0.0.0.0/0 \
 +
--remote-ip 0.0.0.0/0 \
−
> --protocol icmp \
 +
--protocol icmp \
−
> icmp_ssh
 +
icmp_ssh
  +
</PRE>
  +
{{#spoiler:show=Output: openstack security group rule create|
  +
<PRE>
 
+-------------------------+--------------------------------------+
  
+-------------------------+--------------------------------------+
 
| Field | Value |
  
| Field | Value |
Строка 304: Строка 337:
 
+-------------------------+--------------------------------------+
  
+-------------------------+--------------------------------------+
 
</PRE>
  
</PRE>
  +
}}
−
=<code>7</code>=
  
  +
 
<PRE>
  
<PRE>
 
openstack security group rule create \
  
openstack security group rule create \
Строка 312: Строка 346:
 
icmp_ssh
  
icmp_ssh
 
</PRE>
  
</PRE>
  +
{{#spoiler:show=Output: openstack security group rule create|
−
=<code>8</code>=
  
 
<PRE>
  
<PRE>
−
openstack security group rule create \
  
−
--remote-ip 0.0.0.0/0 \
  
−
--protocol tcp \
  
−
--dst-port 22 \
  
−
icmp_ssh
  
−  
 
+-------------------------+--------------------------------------+
  
+-------------------------+--------------------------------------+
 
| Field | Value |
  
| Field | Value |
Строка 342: Строка 370:
 
+-------------------------+--------------------------------------+
  
+-------------------------+--------------------------------------+
 
</PRE>
  
</PRE>
  +
}}
   
−
=<code>9</code>=
 +
==<code>openstack server create</code> (Cirros)==
 
<PRE>
  
<PRE>
 
openstack server create \
  
openstack server create \
Строка 351: Строка 380:
 
--security-group icmp_ssh \
  
--security-group icmp_ssh \
 
test-01
  
test-01
  +
</PRE>
  +
{{#spoiler:show=Output: openstack server create|
  +
<PRE>
 
+-------------------------------------+-------------------------------------------------------+
  
+-------------------------------------+-------------------------------------------------------+
 
| Field | Value |
  
| Field | Value |
Строка 386: Строка 418:
 
+-------------------------------------+-------------------------------------------------------+
  
+-------------------------------------+-------------------------------------------------------+
 
</PRE>
  
</PRE>
  +
}}
−
=<code>10</code>=
  
  +
 
<PRE>
  
<PRE>
 
openstack server create \
  
openstack server create \
Строка 395: Строка 428:
 
test-02
  
test-02
 
</PRE>
  
</PRE>
  +
−
=<code>11</code>=
  
  +
==<code>openstack floating</code>==
 
<PRE>
  
<PRE>
 
openstack floating ip create public
  
openstack floating ip create public
  +
</PRE>
  +
{{#spoiler:show=Output: openstack floating ip create public|
  +
<PRE>
 
+---------------------+--------------------------------------+
  
+---------------------+--------------------------------------+
 
| Field | Value |
  
| Field | Value |
Строка 422: Строка 459:
 
+---------------------+--------------------------------------+
  
+---------------------+--------------------------------------+
 
</PRE>
  
</PRE>
  +
}}
−
=<code>12</code>=
  
  +
  +
==<code>openstack server add floating ip</code>==
 
<PRE>
  
<PRE>
 
openstack server add floating ip test-01 10.170.6.202
  
openstack server add floating ip test-01 10.170.6.202
 
</PRE>
  
</PRE>
  +
 
=<code>SR-IOV</code>=
  
=<code>SR-IOV</code>=
  +
==Простой случай - Access в сторонй VM==
  +
===<code>SR-IOV openstack network create</code>===
 
<PRE>
  
<PRE>
 
openstack \
  
openstack \
 
network create \
  
network create \
  +
--enable-port-security \
 
--provider-network-type vlan \
  
--provider-network-type vlan \
 
--provider-physical-network sriovnet0 \
  
--provider-physical-network sriovnet0 \
−
--provider-segment 100 \
 +
--provider-segment 101 \
  +
sriov-vlan101
−
physnet_sriovnet0_vlan_100
  
  +
</PRE>
  +
* <code>--provider-network-type vlan</code> ???
  +
* <code>--provider-physical-network sriovnet0</code> ???
  +
* <code>--provider-segment 100</code> ???
   
  +
{{#spoiler:show=Output: openstack network create|
−
+---------------------------+-----------------------------------------------------------+
  
  +
<PRE>
−
| Field | Value |
  
−
+---------------------------+-----------------------------------------------------------+
 +
+---------------------------+----------------------------------------------+
−
| admin_state_up | UP |
 +
| Field | Value |
  +
+---------------------------+----------------------------------------------+
−
| availability_zone_hints | None |
  
−
| availability_zones | None |
 +
| admin_state_up | UP |
−
| created_at | 2025-08-19T14:12:40.053861 |
 +
| availability_zone_hints | None |
−
| description | |
 +
| availability_zones | None |
−
| dns_domain | None |
 +
| created_at | 2025-08-23T09:48:54.265051 |
  +
| description | |
−
| fq_name | ['default-domain', 'admin', 'physnet_sriovnet0_vlan_100'] |
  
−
| id | 8c93c693-0e4c-4082-9fc8-fc8d9247c8b7 |
 +
| dns_domain | None |
  +
| fq_name | ['default-domain', 'admin', 'sriov-vlan101'] |
−
| ipv4_address_scope | None |
  
−
| ipv6_address_scope | None |
 +
| id | 3666ef64-9387-4c66-9e63-565124258268 |
−
| is_default | None |
 +
| ipv4_address_scope | None |
−
| is_vlan_transparent | None |
 +
| ipv6_address_scope | None |
−
| mtu | 0 |
 +
| is_default | None |
−
| name | physnet_sriovnet0_vlan_100 |
 +
| is_vlan_transparent | None |
−
| port_security_enabled | True |
 +
| mtu | 0 |
−
| project_id | f39e087061ea48378c9c68348eebbb59 |
 +
| name | sriov-vlan101 |
−
| provider:network_type | vlan |
 +
| port_security_enabled | True |
−
| provider:physical_network | sriovnet0 |
 +
| project_id | f39e087061ea48378c9c68348eebbb59 |
−
| provider:segmentation_id | 100 |
 +
| provider:network_type | vlan |
−
| qos_policy_id | None |
 +
| provider:physical_network | sriovnet0 |
−
| revision_number | None |
 +
| provider:segmentation_id | 101 |
−
| router:external | Internal |
 +
| qos_policy_id | None |
−
| segments | None |
 +
| revision_number | None |
−
| shared | False |
 +
| router:external | Internal |
−
| status | ACTIVE |
 +
| segments | None |
−
| subnets | |
 +
| shared | False |
−
| tags | |
 +
| status | ACTIVE |
−
| tenant_id | f39e087061ea48378c9c68348eebbb59 |
 +
| subnets | |
−
| updated_at | 2025-08-19T14:12:40.053861 |
 +
| tags | |
  +
| tenant_id | f39e087061ea48378c9c68348eebbb59 |
−
+---------------------------+-----------------------------------------------------------+
  
  +
| updated_at | 2025-08-23T09:48:54.265051 |
  +
+---------------------------+----------------------------------------------+
 
</PRE>
  
</PRE>
  +
}}
  +
  +
===<code>SR-IOV openstack subnet create</code>===
   
−
==1==
  
 
<PRE>
  
<PRE>
 
openstack \
  
openstack \
 
subnet create \
  
subnet create \
−
--network physnet_sriovnet0_vlan_100 \
 +
--network sriov-vlan101 \
 
--no-dhcp \
  
--no-dhcp \
 
--ip-version 4 \
  
--ip-version 4 \
 
--gateway none \
  
--gateway none \
−
--subnet-range 10.90.0.0/24 \
 +
--subnet-range 172.16.64.0/24 \
  +
sriov-vlan101-subnet01
−
sriov_subnet_vlan_100
  
 
</PRE>
  
</PRE>
  +
{{#spoiler:show=Output: openstack subnet create|
   
  +
<PRE>
  +
+----------------------+--------------------------------------+
  +
| Field | Value |
  +
+----------------------+--------------------------------------+
  +
| allocation_pools | 172.16.64.1-172.16.64.254 |
  +
| cidr | 172.16.64.0/24 |
  +
| created_at | 2025-08-23T09:51:37.653049 |
  +
| description | None |
  +
| dns_nameservers | |
  +
| dns_publish_fixed_ip | None |
  +
| dns_server_address | 172.16.64.2 |
  +
| enable_dhcp | False |
  +
| gateway_ip | None |
  +
| host_routes | |
  +
| id | 399fa951-c115-4ab6-b50f-a2d94c0a69e0 |
  +
| ip_version | 4 |
  +
| ipv6_address_mode | None |
  +
| ipv6_ra_mode | None |
  +
| name | sriov-vlan101-subnet01 |
  +
| network_id | 3666ef64-9387-4c66-9e63-565124258268 |
  +
| project_id | f39e087061ea48378c9c68348eebbb59 |
  +
| revision_number | None |
  +
| segment_id | None |
  +
| service_types | None |
  +
| subnetpool_id | None |
  +
| tags | |
  +
| updated_at | 2025-08-23T09:51:37.653049 |
  +
+----------------------+--------------------------------------+
  +
</PRE>
  +
}}
  +
*
  +
*
   
  +
===SR-IOV <code>openstack port create </code>===
 
<PRE>
  
<PRE>
  +
openstack \
  +
port create \
  +
--network sriov-vlan101 \
  +
--enable-port-security \
  +
--fixed-ip subnet=sriov-vlan101-subnet01,ip-address=172.16.64.3 \
  +
--vnic-type direct \
  +
sriov-vlan101-subnet01-port01
 
</PRE>
  
</PRE>
  +
{{caution|text=
  +
Тут важно обратить внимаение что доступный адрес начинается с 3-го в сети, в случае с Tungsten Fabric,<BR>
  +
так как второй "занят" под DNS, что видно в сабнете
  +
<PRE>
  +
dns_server_address | 172.16.64.2
  +
</PRE>
  +
Если попробовать его использовать то будет сложнодиагностируемая ошибка, с ничего не говорящим трейсом
  +
}}
  +
{{#spoiler:show=Output: openstack port create|
  +
<PRE>
  +
+-------------------------+----------------------------------------------------------------------------+
  +
| Field | Value |
  +
+-------------------------+----------------------------------------------------------------------------+
  +
| admin_state_up | UP |
  +
| allowed_address_pairs | |
  +
| binding_host_id | None |
  +
| binding_profile | None |
  +
| binding_vif_details | port_filter='True', vlan='101' |
  +
| binding_vif_type | unbound |
  +
| binding_vnic_type | direct |
  +
| created_at | 2025-08-23T10:02:23.953447 |
  +
| data_plane_status | None |
  +
| description | |
  +
| device_id | |
  +
| device_owner | |
  +
| device_profile | None |
  +
| dns_assignment | None |
  +
| dns_domain | None |
  +
| dns_name | None |
  +
| extra_dhcp_opts | None |
  +
| fixed_ips | ip_address='172.16.64.3', subnet_id='399fa951-c115-4ab6-b50f-a2d94c0a69e0' |
  +
| id | 709d705c-7cc4-47c2-9671-68399da51a7e |
  +
| ip_allocation | None |
  +
| mac_address | 02:70:9d:70:5c:7c |
  +
| name | sriov-vlan101-subnet01-port01 |
  +
| network_id | 3666ef64-9387-4c66-9e63-565124258268 |
  +
| numa_affinity_policy | None |
  +
| port_security_enabled | True |
  +
| project_id | f39e087061ea48378c9c68348eebbb59 |
  +
| propagate_uplink_status | None |
  +
| qos_network_policy_id | None |
  +
| qos_policy_id | None |
  +
| resource_request | None |
  +
| revision_number | None |
  +
| security_group_ids | 762b2618-3a38-412e-b39c-ea6921183cbe |
  +
| status | DOWN |
  +
| tags | |
  +
| trunk_details | None |
  +
| updated_at | 2025-08-23T10:02:24.014059 |
  +
+-------------------------+----------------------------------------------------------------------------+
  +
</PRE>
  +
}}
   
  +
===<code>openstack server add port</code>===
  +
<PRE>
  +
openstack server add port ubuntu-test-01 sriov-vlan101-subnet01-port01
  +
</PRE>
  +
После того как порт "прикреплен" к серверу, можно изучить его свойства
  +
====Подробности порта====
  +
<PRE>
  +
openstack port show sriov-vlan101-subnet01-port01 -c binding_profile -c binding_vif_details -f json
  +
</PRE>
  +
Ниже видно следующее
  +
<PRE>
  +
{
  +
"binding_profile": {
  +
"vf_num": 62,
  +
"capabilities": [
  +
"rx",
  +
"tx",
  +
"sg",
  +
"tso",
  +
"gso",
  +
"gro",
  +
"rxvlan",
  +
"txvlan",
  +
"txudptnl"
  +
],
  +
"pf_mac_address": "00:e0:ed:da:5c:8e",
  +
"physical_network": "sriovnet0",
  +
"pci_slot": "0000:06:1f.5",
  +
"pci_vendor_info": "8086:10ed"
  +
},
  +
"binding_vif_details": {
  +
"port_filter": true,
  +
"vlan": "101"
  +
}
  +
}
  +
</PRE>
  +
* <code>vf_num</code>: 62, номер виртуальной функции
  +
* <code>pf_mac_address</code>: <code>00:e0:ed:da:5c:8e</code> - это мак адрес корневого устройства, а не виртуальной функции
  +
<PRE>
  +
7: enp6s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT group default qlen 1000
  +
link/ether 00:e0:ed:da:5c:8e brd ff:ff:ff:ff:ff:ff
  +
</PRE>
  +
* <code>physical_network</code>: <code>sriovnet0</code> - имя физической сети, описанной в конфигурации нейтрона, куда привязаны сетевые карты
  +
* <code>pci_slot</code>: <code>"0000:06:1f.5"</code> - Адрес на шине PCI, что там "сидит" можно увидеть <code>lspci -s 06:1f.5 -vv</code> и извлечь имя устройства <code>ls -l /sys/bus/pci/devices/0000:06:1f.5/net</code>
  +
* <code>pci_vendor_info</code>: <code>8086:10ed</code> Ведор (то же самое покажет например <code>lspci -s 06:1f.5 -mm -nn</code>
  +
* <code>port_filter</code>: true
  +
* <code>vlan</code>: 101 Номер Vlan
   
  +
====<code>lspci -s</code>====
  +
Зная PCI ID можно получить информацию об устройстве
  +
<PRE>
  +
lspci -s 06:1f.5 -vv
  +
</PRE>
  +
<PRE>
  +
06:1f.5 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
  +
Subsystem: Intel Corporation 82599 Ethernet Controller Virtual Function
  +
Control: I/O- Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
  +
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
  +
Latency: 0
  +
IOMMU group: 159
  +
Region 0: Memory at d02f8000 (64-bit, prefetchable) [virtual] [size=16K]
  +
Region 3: Memory at d03f8000 (64-bit, prefetchable) [virtual] [size=16K]
  +
Capabilities: [70] MSI-X: Enable+ Count=3 Masked-
  +
Vector table: BAR=3 offset=00000000
  +
PBA: BAR=3 offset=00002000
  +
Capabilities: [a0] Express (v0) Endpoint, MSI 00
  +
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
  +
ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- SlotPowerLimit 0.000W
  +
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
  +
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
  +
MaxPayload 128 bytes, MaxReadReq 128 bytes
  +
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
  +
LnkCap: Port #0, Speed unknown, Width x0, ASPM not supported
  +
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
  +
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
  +
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
  +
LnkSta: Speed unknown (ok), Width x0 (ok)
  +
TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
  +
Capabilities: [100 v1] Advanced Error Reporting
  +
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
  +
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
  +
UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
  +
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
  +
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
  +
AERCap: First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn-
  +
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
  +
HeaderLog: 00000000 00000000 00000000 00000000
  +
Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
  +
ARICap: MFVC- ACS-, Next Function: 0
  +
ARICtl: MFVC- ACS-, Function Group: 0
  +
Kernel driver in use: ixgbevf
  +
Kernel modules: ixgbevf
  +
</PRE>
   
  +
====<code> ip link show</code>====
  +
А так же он настройках виртуальной функции
 
<PRE>
  
<PRE>
  +
enp6s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT group default qlen 1000
  +
link/ether 00:e0:ed:da:5c:8e brd ff:ff:ff:ff:ff:ff
  +
vf 0 link/ether 36:b8:ac:28:5a:83 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off, query_rss off
  +
<skipped>
  +
vf 62 link/ether 02:70:9d:70:5c:7c brd ff:ff:ff:ff:ff:ff, vlan 101, spoof checking on, link-state auto, trust off, query_rss off
 
</PRE>
  
</PRE>
  +
Тут вижно что:
  +
* <code>vf 62</code> - совпадает с <code>"vf_num": 62</code>
  +
* <code>02:70:9d:70:5c:7c</code> - мак, совпадает с
  +
<PRE>
  +
openstack port show sriov-vlan101-subnet01-port01 -c mac_address -f shell
  +
mac_address="02:70:9d:70:5c:7c"
  +
</PRE>
  +
* <code>vlan 101</code> - номер VLAN заданный при создании сети
  +
* <code>spoof checking on</code> - включен Port Security
  +
* <code>trust off</code> - Об этой опции ниже
   
  +
====Вид "изунтри" виртуальной машины====
  +
<PRE>
  +
dmesg -T
  +
</PRE>
   
  +
<PRE>
  +
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: [10ec:8139] type 00 class 0x020000 conventional PCI endpoint
  +
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: BAR 0 [io 0x0000-0x00ff]
  +
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x000000ff]
  +
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: ROM [mem 0x00000000-0x0007ffff pref]
  +
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: ROM [mem 0x80000000-0x8007ffff pref]: assigned
  +
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: BAR 0 [io 0x1000-0x10ff]: assigned
  +
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: BAR 1 [mem 0x80080000-0x800800ff]: assigned
  +
[Sat Aug 23 10:21:23 2025] 8139cp 0000:00:04.0: enabling device (0000 -> 0003)
  +
[Sat Aug 23 10:21:23 2025] 8139cp 0000:00:04.0 eth0: RTL-8139C+ at 0x000000007f98c756, 02:70:9d:70:5c:7c, IRQ 11
  +
[Sat Aug 23 10:21:23 2025] 8139cp 0000:00:04.0 ens4: renamed from eth0
  +
</PRE>
  +
  +
  +
====Проверка работы <code>Port Security</code>====
   
 
<PRE>
  
<PRE>
  +
ip link show
  +
</PRE>
  +
<PRE>
  +
5: ens4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
  +
link/ether 02:70:9d:70:5c:7c brd ff:ff:ff:ff:ff:ff
  +
altname enp0s4
 
</PRE>
  
</PRE>
   
  +
Добавить IP и попробовать послать запросы, адрес взят "от фонаря", важно только что бы запрос ушел в сеть и свитч увидел МАК
  +
<PRE>
  +
ip addr add 10.90.0.2/24 dev ens4
  +
</PRE>
   
  +
<PRE>
  +
ip link set up dev ens4
  +
</PRE>
  +
Далее запустить ping, естественно не ожидая ответов
  +
<PRE>
  +
ping 10.90.0.1
  +
</PRE>
  +
На хост-системе видно запросы (все кроме броадкастов может не попадать в дамп и это нормально!)
  +
<PRE>
  +
# tcpdump -n -i enp6s0f1 -ee
  +
</PRE>
  +
<PRE>
  +
11:05:53.634812 02:70:9d:70:5c:7c > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP (0x0806), Request who-has 10.90.0.1 tell 10.90.0.2, length 46
  +
</PRE>
  +
Со стороны свитча мак виден
  +
<PRE>
  +
dell-lab#show mac-address-table vlan 101
   
  +
Codes: *N - VLT Peer Synced MAC
  +
*I - Internal MAC Address used for Inter Process Communication
  +
VlanId Mac Address Type Interface State
  +
101 02:70:9d:70:5c:7c Dynamic Te 0/57 Active
  +
</PRE>
  +
Если попробовать поменять МАК
 
<PRE>
  
<PRE>
  +
ip link set dev ens4 address 02:a9:21:bc:e4:5b
 
</PRE>
  
</PRE>
  +
Илм запустить утилиту <code>arppoison ens4</code> из пакета [https://github.com/burghardt/arptools arptools] - новых мак-адресов на свитче не появляется, работает как ожидалось
   
  +
==Отключение Port Security==
  +
В качествет теста создам порт с отключенным Port Security
  +
<BR>
  +
Можно отключить и у существующего порта - см {{#spoiler:show=пример disable-port-security|
  +
<PRE>
  +
openstack port set --disable-port-security --binding-profile trusted=true test-sriov01-sriov_port_1-manual
  +
</PRE>
  +
}}
  +
===<code>openstack port create --disable-port-security</code>===
  +
<PRE>
  +
openstack \
  +
port create \
  +
--network sriov-vlan101 \
  +
--disable-port-security \
  +
--fixed-ip subnet=sriov-vlan101-subnet01,ip-address=172.16.64.4 \
  +
--vnic-type direct \
  +
sriov-vlan101-subnet01-port02
  +
</PRE>
  +
В выводе единственное отличие
  +
<PRE>
  +
| port_security_enabled | False
  +
</PRE>
  +
{{#spoiler:show=пример openstack port create|
  +
<PRE>
  +
+-------------------------+----------------------------------------------------------------------------+
  +
| Field | Value |
  +
+-------------------------+----------------------------------------------------------------------------+
  +
| admin_state_up | UP |
  +
| allowed_address_pairs | |
  +
| binding_host_id | None |
  +
| binding_profile | None |
  +
| binding_vif_details | port_filter='True', vlan='101' |
  +
| binding_vif_type | unbound |
  +
| binding_vnic_type | direct |
  +
| created_at | 2025-08-23T11:29:00.135520 |
  +
| data_plane_status | None |
  +
| description | |
  +
| device_id | |
  +
| device_owner | |
  +
| device_profile | None |
  +
| dns_assignment | None |
  +
| dns_domain | None |
  +
| dns_name | None |
  +
| extra_dhcp_opts | None |
  +
| fixed_ips | ip_address='172.16.64.4', subnet_id='399fa951-c115-4ab6-b50f-a2d94c0a69e0' |
  +
| id | a8ba8629-eb70-46c1-b69f-3cfb9b04ab88 |
  +
| ip_allocation | None |
  +
| mac_address | 02:a8:ba:86:29:eb |
  +
| name | sriov-vlan101-subnet01-port02 |
  +
| network_id | 3666ef64-9387-4c66-9e63-565124258268 |
  +
| numa_affinity_policy | None |
  +
| port_security_enabled | False |
  +
| project_id | f39e087061ea48378c9c68348eebbb59 |
  +
| propagate_uplink_status | None |
  +
| qos_network_policy_id | None |
  +
| qos_policy_id | None |
  +
| resource_request | None |
  +
| revision_number | None |
  +
| security_group_ids | |
  +
| status | DOWN |
  +
| tags | |
  +
| trunk_details | None |
  +
| updated_at | 2025-08-23T11:29:00.183637 |
  +
+-------------------------+----------------------------------------------------------------------------+
  +
</PRE>
  +
}}
  +
===<code>openstack server add port ubuntu-test-01 sriov-vlan101-subnet01-port02</code>===
  +
Прикрепить второй порт к виртуальной машине
  +
<PRE>
  +
openstack server add port ubuntu-test-01 sriov-vlan101-subnet01-port02
  +
</PRE>
  +
===Настройки виртуальной машины===
  +
Со стороны сервера базовые настройки
  +
<PRE>
  +
ip link set up dev ens8
  +
</PRE>
   
  +
<PRE>
  +
ip addr add 10.90.1.2/24 dev ens8
  +
</PRE>
   
 
<PRE>
  
<PRE>
  +
ip ro
  +
default via 192.168.77.1 dev ens3 proto dhcp src 192.168.77.5 metric 100
  +
8.8.8.8 via 192.168.77.1 dev ens3 proto dhcp src 192.168.77.5 metric 100
  +
10.90.0.0/24 dev ens4 proto kernel scope link src 10.90.0.2
  +
10.90.1.0/24 dev ens8 proto kernel scope link src 10.90.1.2
  +
192.168.77.0/24 dev ens3 proto kernel scope link src 192.168.77.5 metric 100
  +
192.168.77.1 dev ens3 proto dhcp scope link src 192.168.77.5 metric 100
 
</PRE>
  
</PRE>
   
  +
На Хосте ожидаемо наблюдаем <code>spoof checking off</code>
  +
<PRE>
  +
vf 61 link/ether 02:a8:ba:86:29:eb brd ff:ff:ff:ff:ff:ff, vlan 101, spoof checking off, link-state auto, trust off, query_rss off
  +
</PRE>
   
  +
===Проверка что ограничений на src-mac нет===
  +
Смена мака
  +
<PRE>
  +
ip link set dev ens8 address 02:70:9d:70:5c:99
  +
</PRE>
   
  +
  +
Видно что мак поменялся (99 на конце)
 
<PRE>
  
<PRE>
  +
tcpdump -n -i enp6s0f1 -ee
  +
11:53:56.233183 02:70:9d:70:5c:99 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP (0x0806), Request who-has 10.90.1.1 tell 10.90.1.2, length 46
 
</PRE>
  
</PRE>
   
  +
Оба мака заехали на свитч (что ожидаемо при отключеном Port Security)
  +
<PRE>
  +
show mac-address-table vlan 101
   
  +
Codes: *N - VLT Peer Synced MAC
  +
*I - Internal MAC Address used for Inter Process Communication
  +
VlanId Mac Address Type Interface State
  +
101 02:70:9d:70:5c:7c Dynamic Te 0/57 Active
  +
dell-lab#show mac-address-table vlan 101
  +
  +
Codes: *N - VLT Peer Synced MAC
  +
*I - Internal MAC Address used for Inter Process Communication
  +
VlanId Mac Address Type Interface State
  +
101 02:70:9d:70:5c:7c Dynamic Te 0/57 Active
  +
101 02:70:9d:70:5c:99 Dynamic Te 0/57 Active
  +
101 02:a8:ba:86:29:eb Dynamic Te 0/57 Active
  +
</PRE>
   
  +
"протравить" свитч заполнив табличку коммутации:
 
<PRE>
  
<PRE>
  +
arppoison ens8
 
</PRE>
  
</PRE>
  +
  +
Случайные пары мак-адресов/ip-адресов
  +
<PRE>
  +
11:56:51.593634 00:3d:9f:e3:5e:0b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP (0x0806), Reply 192.199.71.81 is-at 00:3d:9f:e3:5e:0b, length 46
  +
11:56:51.594051 00:3d:52:f1:46:38 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP (0x0806), Reply 120.206.54.21 is-at 00:3d:52:f1:46:38, length 46
  +
11:56:51.594419 00:26:5c:59:a4:fb > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP (0x0806), Reply 99.143.18.102 is-at 00:26:5c:59:a4:fb, length 46
  +
11:56:51.594805 00:13:a2:48:f1:76 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP (0x0806), Reply 170.228.232.78 is-at 00:13:a2:48:f1:76, length 46
  +
</PRE>
  +
  +
Свитчк такое нравится не очень, таблица забита полностью (а свитч довольно жирный)
  +
<PRE>
  +
dell-lab#Aug 23 11:57:27 %S4810:0 %MACAGT-5-HASH_COLLISION_LOG: Mac:00:17:71:90:4d:91/Vlan:101 could not be added to L2 CAM on portpipe 0 stack-unit 0 due to hash collision. Total number of hash collisions: 17888
  +
Aug 23 11:57:40 %S4810:0 %MACAGT-5-HASH_COLLISION_LOG: Mac:00:2b:5a:e9:6c:d1/Vlan:101 could not be added to L2 CAM on portpipe 0 stack-unit 0 due to hash collision. Total number of hash collisions: 18365
  +
Aug 23 11:57:52 %S4810:0 %MACAGT-5-HASH_COLLISION_LOG: Mac:00:10:04:a7:90:a9/Vlan:101 could not be added to L2 CAM on portpipe 0 stack-unit 0 due to hash collision. Total number of hash collisions: 18661
  +
Aug 23 11:57:53 %STKUNIT0-M:CP %SYSADM-5-CPU_THRESHOLD_CLR: Overall cpu usage of management-unit drops below threshold. Cpu1minUsage (73%)
  +
show mac-address-table count vlan 101
  +
MAC Entries for vlan 101 :
  +
Dynamic Address Count : 130988
  +
Static Address (User-defined) Count : 0
  +
Sticky Address Count : 0
  +
Total MAC Addresses in Use: 130988
  +
</PRE>
  +
===Краткий вывод===
  +
Отключение Port Security может быть опасно
  +
<BR>
  +
Конечно можно городить ограничения и со стороны порта свитча, но все же.
  +
   
   
  +
==Транковый порт (с ограниченным набором VLAN)==
   
  +
===1===
  +
Проверка, что транки доступны
 
<PRE>
  
<PRE>
  +
openstack extension list --network | grep -i trunk
  +
| Tag support for resources with standard attribute: port, subnet, subnetpool, network, security_group, router, floatingip, policy, trunk | standard-attr-tag | Enables to set tag on resources with standard attribute. |
  +
| Trunk Extension | trunk | Provides support for trunk ports |
  +
| Trunk port details | trunk-details | Expose trunk port details |
 
</PRE>
  
</PRE>
   
   
  +
===Подготовка сетей===
  +
Обычно: одна «родительская» (untagged / access), и несколько «дочерних» сетей под VLAN’ы.
   
 
<PRE>
  
<PRE>
  +
openstack network create sriov-vlan102 --provider-network-type vlan --provider-physical-network sriovnet0 --provider-segment 102
  +
+---------------------------+----------------------------------------------+
  +
| Field | Value |
  +
+---------------------------+----------------------------------------------+
  +
| admin_state_up | UP |
  +
| availability_zone_hints | None |
  +
| availability_zones | None |
  +
| created_at | 2025-08-23T12:49:55.679015 |
  +
| description | |
  +
| dns_domain | None |
  +
| fq_name | ['default-domain', 'admin', 'sriov-vlan102'] |
  +
| id | fafb4688-64e9-454b-840d-d486246c30f8 |
  +
| ipv4_address_scope | None |
  +
| ipv6_address_scope | None |
  +
| is_default | None |
  +
| is_vlan_transparent | None |
  +
| mtu | 0 |
  +
| name | sriov-vlan102 |
  +
| port_security_enabled | True |
  +
| project_id | f39e087061ea48378c9c68348eebbb59 |
  +
| provider:network_type | vlan |
  +
| provider:physical_network | sriovnet0 |
  +
| provider:segmentation_id | 102 |
  +
| qos_policy_id | None |
  +
| revision_number | None |
  +
| router:external | Internal |
  +
| segments | None |
  +
| shared | False |
  +
| status | ACTIVE |
  +
| subnets | |
  +
| tags | |
  +
| tenant_id | f39e087061ea48378c9c68348eebbb59 |
  +
| updated_at | 2025-08-23T12:49:55.679015 |
  +
+---------------------------+----------------------------------------------+
 
</PRE>
  
</PRE>
   
  +
<PRE>
  +
openstack network create sriov-vlan103 --provider-network-type vlan --provider-physical-network sriovnet0 --provider-segment 103
  +
+---------------------------+----------------------------------------------+
  +
| Field | Value |
  +
+---------------------------+----------------------------------------------+
  +
| admin_state_up | UP |
  +
| availability_zone_hints | None |
  +
| availability_zones | None |
  +
| created_at | 2025-08-23T12:50:36.487602 |
  +
| description | |
  +
| dns_domain | None |
  +
| fq_name | ['default-domain', 'admin', 'sriov-vlan103'] |
  +
| id | bd3ed51e-1cdd-4c9a-9c4f-b92dafd31daa |
  +
| ipv4_address_scope | None |
  +
| ipv6_address_scope | None |
  +
| is_default | None |
  +
| is_vlan_transparent | None |
  +
| mtu | 0 |
  +
| name | sriov-vlan103 |
  +
| port_security_enabled | True |
  +
| project_id | f39e087061ea48378c9c68348eebbb59 |
  +
| provider:network_type | vlan |
  +
| provider:physical_network | sriovnet0 |
  +
| provider:segmentation_id | 103 |
  +
| qos_policy_id | None |
  +
| revision_number | None |
  +
| router:external | Internal |
  +
| segments | None |
  +
| shared | False |
  +
| status | ACTIVE |
  +
| subnets | |
  +
| tags | |
  +
| tenant_id | f39e087061ea48378c9c68348eebbb59 |
  +
| updated_at | 2025-08-23T12:50:36.487602 |
  +
+---------------------------+----------------------------------------------+
  +
</PRE>
   
   
 
<PRE>
  
<PRE>
  +
openstack network create sriov-vlan104 --provider-network-type vlan --provider-physical-network sriovnet0 --provider-segment 104
  +
+---------------------------+----------------------------------------------+
  +
| Field | Value |
  +
+---------------------------+----------------------------------------------+
  +
| admin_state_up | UP |
  +
| availability_zone_hints | None |
  +
| availability_zones | None |
  +
| created_at | 2025-08-23T12:51:04.201571 |
  +
| description | |
  +
| dns_domain | None |
  +
| fq_name | ['default-domain', 'admin', 'sriov-vlan104'] |
  +
| id | fd609f3d-2561-4c39-9c71-0c2199ba2748 |
  +
| ipv4_address_scope | None |
  +
| ipv6_address_scope | None |
  +
| is_default | None |
  +
| is_vlan_transparent | None |
  +
| mtu | 0 |
  +
| name | sriov-vlan104 |
  +
| port_security_enabled | True |
  +
| project_id | f39e087061ea48378c9c68348eebbb59 |
  +
| provider:network_type | vlan |
  +
| provider:physical_network | sriovnet0 |
  +
| provider:segmentation_id | 104 |
  +
| qos_policy_id | None |
  +
| revision_number | None |
  +
| router:external | Internal |
  +
| segments | None |
  +
| shared | False |
  +
| status | ACTIVE |
  +
| subnets | |
  +
| tags | |
  +
| tenant_id | f39e087061ea48378c9c68348eebbb59 |
  +
| updated_at | 2025-08-23T12:51:04.201571 |
  +
+---------------------------+----------------------------------------------+
 
</PRE>
  
</PRE>
   
  +
<PRE>
  +
</PRE>
   
  +
===Сабнеты===
  +
<PRE>
  +
openstack subnet create sriov-vlan102-subnet01 --network sriov-vlan102 --subnet-range 10.102.0.0/24
  +
+----------------------+--------------------------------------+
  +
| Field | Value |
  +
+----------------------+--------------------------------------+
  +
| allocation_pools | 10.102.0.2-10.102.0.254 |
  +
| cidr | 10.102.0.0/24 |
  +
| created_at | 2025-08-23T12:52:35.449241 |
  +
| description | None |
  +
| dns_nameservers | |
  +
| dns_publish_fixed_ip | None |
  +
| dns_server_address | 10.102.0.2 |
  +
| enable_dhcp | True |
  +
| gateway_ip | 10.102.0.1 |
  +
| host_routes | |
  +
| id | fed5208c-6aed-42b8-9aa9-18a7444e3fa5 |
  +
| ip_version | 4 |
  +
| ipv6_address_mode | None |
  +
| ipv6_ra_mode | None |
  +
| name | sriov-vlan102-subnet01 |
  +
| network_id | fafb4688-64e9-454b-840d-d486246c30f8 |
  +
| project_id | f39e087061ea48378c9c68348eebbb59 |
  +
| revision_number | None |
  +
| segment_id | None |
  +
| service_types | None |
  +
| subnetpool_id | None |
  +
| tags | |
  +
| updated_at | 2025-08-23T12:52:35.449241 |
  +
+----------------------+--------------------------------------+
  +
</PRE>
   
 
<PRE>
  
<PRE>
  +
openstack subnet create sriov-vlan103-subnet01 --network sriov-vlan103 --subnet-range 10.103.0.0/24
  +
+----------------------+--------------------------------------+
  +
| Field | Value |
  +
+----------------------+--------------------------------------+
  +
| allocation_pools | 10.103.0.2-10.103.0.254 |
  +
| cidr | 10.103.0.0/24 |
  +
| created_at | 2025-08-23T12:52:55.647034 |
  +
| description | None |
  +
| dns_nameservers | |
  +
| dns_publish_fixed_ip | None |
  +
| dns_server_address | 10.103.0.2 |
  +
| enable_dhcp | True |
  +
| gateway_ip | 10.103.0.1 |
  +
| host_routes | |
  +
| id | ec2f665b-b510-4fe2-85ec-20ed46a8f7ab |
  +
| ip_version | 4 |
  +
| ipv6_address_mode | None |
  +
| ipv6_ra_mode | None |
  +
| name | sriov-vlan103-subnet01 |
  +
| network_id | bd3ed51e-1cdd-4c9a-9c4f-b92dafd31daa |
  +
| project_id | f39e087061ea48378c9c68348eebbb59 |
  +
| revision_number | None |
  +
| segment_id | None |
  +
| service_types | None |
  +
| subnetpool_id | None |
  +
| tags | |
  +
| updated_at | 2025-08-23T12:52:55.647034 |
  +
+----------------------+--------------------------------------+
 
</PRE>
  
</PRE>
   
  +
<PRE>
  +
openstack subnet create sriov-vlan104-subnet01 --network sriov-vlan104 --subnet-range 10.104.0.0/24
  +
+----------------------+--------------------------------------+
  +
| Field | Value |
  +
+----------------------+--------------------------------------+
  +
| allocation_pools | 10.104.0.2-10.104.0.254 |
  +
| cidr | 10.104.0.0/24 |
  +
| created_at | 2025-08-23T12:53:20.156279 |
  +
| description | None |
  +
| dns_nameservers | |
  +
| dns_publish_fixed_ip | None |
  +
| dns_server_address | 10.104.0.2 |
  +
| enable_dhcp | True |
  +
| gateway_ip | 10.104.0.1 |
  +
| host_routes | |
  +
| id | a1c11284-d06a-4605-85ac-526cf4675a1f |
  +
| ip_version | 4 |
  +
| ipv6_address_mode | None |
  +
| ipv6_ra_mode | None |
  +
| name | sriov-vlan104-subnet01 |
  +
| network_id | fd609f3d-2561-4c39-9c71-0c2199ba2748 |
  +
| project_id | f39e087061ea48378c9c68348eebbb59 |
  +
| revision_number | None |
  +
| segment_id | None |
  +
| service_types | None |
  +
| subnetpool_id | None |
  +
| tags | |
  +
| updated_at | 2025-08-23T12:53:20.156279 |
  +
+----------------------+--------------------------------------+
  +
</PRE>
   
   
Строка 542: Строка 1197:
   
   
  +
<PRE>
  +
</PRE>
   
  +
<PRE>
  +
</PRE>
  +
<PRE>
  +
</PRE>
  +
<PRE>
  +
</PRE>
 
<PRE>
  
<PRE>
 
</PRE>
  
</PRE>
   
  +
===Порты===
  +
Родительский порт (вставится в ВМ)
  +
Субпорты (по одному на каждую VLAN/сеть)
  +
Важное: subport сам по себе не присоединяется к ВМ — он «подвешивается» к транку с указанием segmentation-id.
  +
<PRE>
  +
openstack port create sriov-vlan102-subnet01-port01 --network sriov-vlan102 --vnic-type direct --disable-port-security
  +
+-------------------------+---------------------------------------------------------------------------+
  +
| Field | Value |
  +
+-------------------------+---------------------------------------------------------------------------+
  +
| admin_state_up | UP |
  +
| allowed_address_pairs | |
  +
| binding_host_id | None |
  +
| binding_profile | None |
  +
| binding_vif_details | port_filter='True', vlan='102' |
  +
| binding_vif_type | unbound |
  +
| binding_vnic_type | direct |
  +
| created_at | 2025-08-23T13:13:50.569474 |
  +
| data_plane_status | None |
  +
| description | |
  +
| device_id | |
  +
| device_owner | |
  +
| device_profile | None |
  +
| dns_assignment | None |
  +
| dns_domain | None |
  +
| dns_name | None |
  +
| extra_dhcp_opts | None |
  +
| fixed_ips | ip_address='10.102.0.3', subnet_id='fed5208c-6aed-42b8-9aa9-18a7444e3fa5' |
  +
| id | b6428934-8ef3-4a2d-bc81-23622644a6b6 |
  +
| ip_allocation | None |
  +
| mac_address | 02:b6:42:89:34:8e |
  +
| name | sriov-vlan102-subnet01-port01 |
  +
| network_id | fafb4688-64e9-454b-840d-d486246c30f8 |
  +
| numa_affinity_policy | None |
  +
| port_security_enabled | False |
  +
| project_id | f39e087061ea48378c9c68348eebbb59 |
  +
| propagate_uplink_status | None |
  +
| qos_network_policy_id | None |
  +
| qos_policy_id | None |
  +
| resource_request | None |
  +
| revision_number | None |
  +
| security_group_ids | |
  +
| status | DOWN |
  +
| tags | |
  +
| trunk_details | None |
  +
| updated_at | 2025-08-23T13:13:50.612751 |
  +
+-------------------------+---------------------------------------------------------------------------+
  +
</PRE>
  +
  +
<PRE>
  +
openstack port create sriov-vlan103-subnet01-port01-subport --network sriov-vlan103 --vnic-type direct --disable-port-security
  +
+-------------------------+---------------------------------------------------------------------------+
  +
| Field | Value |
  +
+-------------------------+---------------------------------------------------------------------------+
  +
| admin_state_up | UP |
  +
| allowed_address_pairs | |
  +
| binding_host_id | None |
  +
| binding_profile | None |
  +
| binding_vif_details | port_filter='True', vlan='103' |
  +
| binding_vif_type | unbound |
  +
| binding_vnic_type | direct |
  +
| created_at | 2025-08-23T13:14:24.688277 |
  +
| data_plane_status | None |
  +
| description | |
  +
| device_id | |
  +
| device_owner | |
  +
| device_profile | None |
  +
| dns_assignment | None |
  +
| dns_domain | None |
  +
| dns_name | None |
  +
| extra_dhcp_opts | None |
  +
| fixed_ips | ip_address='10.103.0.3', subnet_id='ec2f665b-b510-4fe2-85ec-20ed46a8f7ab' |
  +
| id | eda0d1a6-efb7-4c92-bdf9-36be29308aef |
  +
| ip_allocation | None |
  +
| mac_address | 02:ed:a0:d1:a6:ef |
  +
| name | sriov-vlan103-subnet01-port01-subport |
  +
| network_id | bd3ed51e-1cdd-4c9a-9c4f-b92dafd31daa |
  +
| numa_affinity_policy | None |
  +
| port_security_enabled | False |
  +
| project_id | f39e087061ea48378c9c68348eebbb59 |
  +
| propagate_uplink_status | None |
  +
| qos_network_policy_id | None |
  +
| qos_policy_id | None |
  +
| resource_request | None |
  +
| revision_number | None |
  +
| security_group_ids | |
  +
| status | DOWN |
  +
| tags | |
  +
| trunk_details | None |
  +
| updated_at | 2025-08-23T13:14:24.727028 |
  +
+-------------------------+---------------------------------------------------------------------------+
  +
</PRE>
   
   
 
<PRE>
  
<PRE>
  +
openstack port create sriov-vlan104-subnet01-port01-subport --network sriov-vlan104 --vnic-type direct --disable-port-security
  +
+-------------------------+---------------------------------------------------------------------------+
  +
| Field | Value |
  +
+-------------------------+---------------------------------------------------------------------------+
  +
| admin_state_up | UP |
  +
| allowed_address_pairs | |
  +
| binding_host_id | None |
  +
| binding_profile | None |
  +
| binding_vif_details | port_filter='True', vlan='104' |
  +
| binding_vif_type | unbound |
  +
| binding_vnic_type | direct |
  +
| created_at | 2025-08-23T13:14:51.060012 |
  +
| data_plane_status | None |
  +
| description | |
  +
| device_id | |
  +
| device_owner | |
  +
| device_profile | None |
  +
| dns_assignment | None |
  +
| dns_domain | None |
  +
| dns_name | None |
  +
| extra_dhcp_opts | None |
  +
| fixed_ips | ip_address='10.104.0.3', subnet_id='a1c11284-d06a-4605-85ac-526cf4675a1f' |
  +
| id | 243ab831-9c29-40a4-af84-4277d81ecbe4 |
  +
| ip_allocation | None |
  +
| mac_address | 02:24:3a:b8:31:9c |
  +
| name | sriov-vlan104-subnet01-port01-subport |
  +
| network_id | fd609f3d-2561-4c39-9c71-0c2199ba2748 |
  +
| numa_affinity_policy | None |
  +
| port_security_enabled | False |
  +
| project_id | f39e087061ea48378c9c68348eebbb59 |
  +
| propagate_uplink_status | None |
  +
| qos_network_policy_id | None |
  +
| qos_policy_id | None |
  +
| resource_request | None |
  +
| revision_number | None |
  +
| security_group_ids | |
  +
| status | DOWN |
  +
| tags | |
  +
| trunk_details | None |
  +
| updated_at | 2025-08-23T13:14:51.099710 |
  +
+-------------------------+---------------------------------------------------------------------------+
 
</PRE>
  
</PRE>
   
  +
===trunk===
  +
<PRE>
  +
openstack network trunk create sriov-trunk-vlan102-103-104 --parent-port sriov-vlan102-subnet01-port01 --subport port=sriov-vlan103-subnet01-port01-subport,segmentation-type=vlan,segmentation-id=103 --subport port=sriov-vlan104-subnet01-port01-subport,segmentation-type=vlan,segmentation-id=104
  +
+-------------------+-------------------------------------------------------------------------------------------------+
  +
| Field | Value |
  +
+-------------------+-------------------------------------------------------------------------------------------------+
  +
| created_at | 2025-08-23T13:00:01.594180 |
  +
| description | |
  +
| id | 536c8ce4-6c72-45c8-bf92-ee042b96ba19 |
  +
| is_admin_state_up | True |
  +
| name | sriov-trunk-vlan102-103-104 |
  +
| port_id | 58f758a8-e30f-48ab-a5f6-7c7ee7a550bb |
  +
| project_id | f39e087061ea48378c9c68348eebbb59 |
  +
| status | DOWN |
  +
| sub_ports | port_id='1d508ca2-4cf4-4327-8b24-3a50ebfa21b9', segmentation_id='103', segmentation_type='vlan' |
  +
| | port_id='899e65f0-2a59-4241-bdc2-9e1f1e6a4606', segmentation_id='104', segmentation_type='vlan' |
  +
| tags | [] |
  +
| updated_at | 2025-08-23T13:00:01.594180 |
  +
+-------------------+-------------------------------------------------------------------------------------------------+
  +
</PRE>
   
  +
Или добавлять позже:
  +
  +
bash
  +
Copy
  +
Edit
  +
openstack network trunk set trunk1 \
  +
--subport port=subport-100,segmentation-type=vlan,segmentation-id=100
  +
openstack network trunk set trunk1 \
  +
--subport port=subport-200,segmentation-type=vlan,segmentation-id=200
  +
  +
===add port===
  +
<PRE>
  +
openstack server add port ubuntu-test-01 sriov-vlan102-subnet01-port01
  +
</PRE>
  +
  +
<PRE>
  +
vf 60 link/ether 02:58:f7:58:a8:e3 brd ff:ff:ff:ff:ff:ff, vlan 102, spoof checking on, link-state auto, trust off, query_rss off
  +
</PRE>
  +
  +
===2===
  +
<PRE>
  +
</PRE>
  +
  +
===2===
  +
<PRE>
  +
</PRE>
  +
  +
===2===
  +
<PRE>
  +
</PRE>
   
  +
===2===
 
<PRE>
  
<PRE>
 
</PRE>
  
</PRE>

Текущая версия на 15:15, 23 августа 2025


Предварительная настройка

Openstack в этом сетапе использует Tungsten Fabric в качестве Core Network Plugin в Neutron.
Подробнее: Настройка Cisco ASR1001X как Edge Router для Tungsten Fabric

Stub.png
Данная страница находится в разработке.
Эта страница ещё не закончена. Информация, представленная здесь, может оказаться неполной или неверной.

Если вы считаете, что её стоило бы доработать как можно быстрее, пожалуйста, соообщите.


Создание ВМ по шагам

Дано: только что развернутый опенстек, в качестве внешнего роутреа используется ASR1001X
Требуется: Задеплоить 2 VM с Floating IP


openstack image create

Пример загрузки образа в OpenStack 

openstack \
    image create \
    --container-format bare  \
    --disk-format qcow2 \
    --file ~/Downloads/noble-server-cloudimg-amd64.img \
    Ubuntu-24.04

keypair create

Создать пару ключей, если нужно, приватную часть сохранить так как она больше нигде не сохраняется. 

openstack keypair create mmazur

-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEAp4Yv+iyTCrHSMwbPahlGRdSGuuMtG+JPMYdeIhi/QDA4Wvyh
Af/TlBUNkdiYJfOJp8R6xFCOv9wREs5VHlHHk3b3xcl/w8Vtz53G3jYSu/cRV0VY
<skipped>
4vyy0i8k2fkcZooAtU4I60g9GJEWhJLiLaytXcv0XXSralhV6hihICX4SxSL5HCP
DroCuM9W/AI4rK7gyfsMdqhF6yHri8lvVAYiQMHqmvrrS85WenuY
-----END RSA PRIVATE KEY-----

Публичная сеть

openstack network create

 openstack network create --external public
  • --external public - сеть внешняя, использует для Floating IPs и будет маршрутизироваться наружу, за пределы OpenStack

openstack subnet create

Сабнет определяет диапазон адресов 

openstack subnet create \
     --network public \
     --subnet-range 10.170.6.0/24 \
     --allocation-pool start=10.170.6.201,end=10.170.6.249 \
     --dns-nameserver 8.8.8.8 \
     --gateway none \
     public-subnet

Приватная сеть

openstack network create internal

 openstack network create internal

openstack subnet create

 openstack subnet create \
    --subnet-range 192.168.77.0/24 \
    --network internal \
    --dns-nameserver 8.8.8.8 \
    internal-subnet

router

openstack router create 

 openstack router create rtr01

set external-gateway

openstack router set --external-gateway public rtr01
Вывод пустой

openstack router add subnet

openstack router add subnet rtr01 internal-subnet

openstack security group

openstack security group create

openstack security group create icmp_ssh

openstack security group rule create

openstack security group rule create \
     --remote-ip 0.0.0.0/0 \
     --protocol icmp \
     icmp_ssh

openstack security group rule create \
    --remote-ip 0.0.0.0/0 \
    --protocol tcp \
    --dst-port 22 \
    icmp_ssh

openstack server create (Cirros)

openstack server create \
    --flavor m1.small \
    --image Cirros-6.0.raw \
    --network internal \
    --security-group icmp_ssh \
    test-01

openstack server create \
    --flavor m1.small \
    --image Cirros-6.0.raw \
    --network internal \
    --security-group icmp_ssh \
    test-02

openstack floating

openstack floating ip create public

openstack server add floating ip

openstack server add floating ip test-01 10.170.6.202

SR-IOV

Простой случай - Access в сторонй VM

SR-IOV openstack network create

openstack \
  network create \
  --enable-port-security \
  --provider-network-type vlan \
  --provider-physical-network sriovnet0  \
  --provider-segment  101 \
  sriov-vlan101
  • --provider-network-type vlan ???
  • --provider-physical-network sriovnet0 ???
  • --provider-segment 100 ???

SR-IOV openstack subnet create

openstack \
  subnet create \
  --network  sriov-vlan101 \
  --no-dhcp \
  --ip-version 4 \
  --gateway none \
  --subnet-range 172.16.64.0/24 \
  sriov-vlan101-subnet01

SR-IOV openstack port create 

openstack \
  port create \
    --network sriov-vlan101 \
    --enable-port-security \
    --fixed-ip subnet=sriov-vlan101-subnet01,ip-address=172.16.64.3 \
    --vnic-type direct \
    sriov-vlan101-subnet01-port01

Icon-caution.gif

Тут важно обратить внимаение что доступный адрес начинается с 3-го в сети, в случае с Tungsten Fabric,
так как второй "занят" под DNS, что видно в сабнете 

dns_server_address   | 172.16.64.2

Если попробовать его использовать то будет сложнодиагностируемая ошибка, с ничего не говорящим трейсом

openstack server add port

openstack server add port ubuntu-test-01 sriov-vlan101-subnet01-port01

После того как порт "прикреплен" к серверу, можно изучить его свойства

Подробности порта

openstack port show sriov-vlan101-subnet01-port01 -c binding_profile -c binding_vif_details  -f json

Ниже видно следующее 

{
  "binding_profile": {
    "vf_num": 62,
    "capabilities": [
      "rx",
      "tx",
      "sg",
      "tso",
      "gso",
      "gro",
      "rxvlan",
      "txvlan",
      "txudptnl"
    ],
    "pf_mac_address": "00:e0:ed:da:5c:8e",
    "physical_network": "sriovnet0",
    "pci_slot": "0000:06:1f.5",
    "pci_vendor_info": "8086:10ed"
  },
  "binding_vif_details": {
    "port_filter": true,
    "vlan": "101"
  }
}
  • vf_num: 62, номер виртуальной функции
  • pf_mac_address: 00:e0:ed:da:5c:8e - это мак адрес корневого устройства, а не виртуальной функции
7: enp6s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 00:e0:ed:da:5c:8e brd ff:ff:ff:ff:ff:ff
  • physical_network: sriovnet0 - имя физической сети, описанной в конфигурации нейтрона, куда привязаны сетевые карты
  • pci_slot: "0000:06:1f.5" - Адрес на шине PCI, что там "сидит" можно увидеть lspci -s 06:1f.5 -vv и извлечь имя устройства ls -l /sys/bus/pci/devices/0000:06:1f.5/net
  • pci_vendor_info: 8086:10ed Ведор (то же самое покажет например lspci -s 06:1f.5 -mm -nn
  • port_filter: true
  • vlan: 101 Номер Vlan

lspci -s

Зная PCI ID можно получить информацию об устройстве 

lspci -s 06:1f.5 -vv

06:1f.5 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
	Subsystem: Intel Corporation 82599 Ethernet Controller Virtual Function
	Control: I/O- Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	IOMMU group: 159
	Region 0: Memory at d02f8000 (64-bit, prefetchable) [virtual] [size=16K]
	Region 3: Memory at d03f8000 (64-bit, prefetchable) [virtual] [size=16K]
	Capabilities: [70] MSI-X: Enable+ Count=3 Masked-
		Vector table: BAR=3 offset=00000000
		PBA: BAR=3 offset=00002000
	Capabilities: [a0] Express (v0) Endpoint, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- SlotPowerLimit 0.000W
		DevCtl:	CorrErr- NonFatalErr- FatalErr- UnsupReq-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed unknown, Width x0, ASPM not supported
			ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
		LnkCtl:	ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed unknown (ok), Width x0 (ok)
			TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
		AERCap:	First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn-
			MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
		HeaderLog: 00000000 00000000 00000000 00000000
	Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 0
		ARICtl:	MFVC- ACS-, Function Group: 0
	Kernel driver in use: ixgbevf
	Kernel modules: ixgbevf

ip link show

А так же он настройках виртуальной функции 

enp6s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 00:e0:ed:da:5c:8e brd ff:ff:ff:ff:ff:ff
    vf 0     link/ether 36:b8:ac:28:5a:83 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off, query_rss off
<skipped>
    vf 62     link/ether 02:70:9d:70:5c:7c brd ff:ff:ff:ff:ff:ff, vlan 101, spoof checking on, link-state auto, trust off, query_rss off

Тут вижно что:

  • vf 62 - совпадает с "vf_num": 62
  • 02:70:9d:70:5c:7c - мак, совпадает с
openstack port show sriov-vlan101-subnet01-port01 -c mac_address  -f shell
mac_address="02:70:9d:70:5c:7c"
  • vlan 101 - номер VLAN заданный при создании сети
  • spoof checking on - включен Port Security
  • trust off - Об этой опции ниже

Вид "изунтри" виртуальной машины

dmesg -T

[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: [10ec:8139] type 00 class 0x020000 conventional PCI endpoint
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: BAR 0 [io  0x0000-0x00ff]
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x000000ff]
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: ROM [mem 0x00000000-0x0007ffff pref]
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: ROM [mem 0x80000000-0x8007ffff pref]: assigned
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: BAR 0 [io  0x1000-0x10ff]: assigned
[Sat Aug 23 10:21:23 2025] pci 0000:00:04.0: BAR 1 [mem 0x80080000-0x800800ff]: assigned
[Sat Aug 23 10:21:23 2025] 8139cp 0000:00:04.0: enabling device (0000 -> 0003)
[Sat Aug 23 10:21:23 2025] 8139cp 0000:00:04.0 eth0: RTL-8139C+ at 0x000000007f98c756, 02:70:9d:70:5c:7c, IRQ 11
[Sat Aug 23 10:21:23 2025] 8139cp 0000:00:04.0 ens4: renamed from eth0


Проверка работы Port Security

ip link show

5: ens4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 02:70:9d:70:5c:7c brd ff:ff:ff:ff:ff:ff
    altname enp0s4

Добавить IP и попробовать послать запросы, адрес взят "от фонаря", важно только что бы запрос ушел в сеть и свитч увидел МАК 

ip addr add 10.90.0.2/24 dev ens4

ip link set up dev ens4

Далее запустить ping, естественно не ожидая ответов 

ping 10.90.0.1

На хост-системе видно запросы (все кроме броадкастов может не попадать в дамп и это нормально!) 

# tcpdump  -n -i enp6s0f1 -ee

11:05:53.634812 02:70:9d:70:5c:7c > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP (0x0806), Request who-has 10.90.0.1 tell 10.90.0.2, length 46

Со стороны свитча мак виден 

dell-lab#show mac-address-table vlan 101

Codes: *N - VLT Peer Synced MAC
*I - Internal MAC Address used for Inter Process Communication
VlanId     Mac Address           Type          Interface        State
 101	02:70:9d:70:5c:7c	Dynamic    	Te 0/57   	Active

Если попробовать поменять МАК 

ip link set dev ens4 address 02:a9:21:bc:e4:5b

Илм запустить утилиту arppoison ens4 из пакета arptools - новых мак-адресов на свитче не появляется, работает как ожидалось

Отключение Port Security

В качествет теста создам порт с отключенным Port Security

Можно отключить и у существующего порта - см

openstack port create --disable-port-security

openstack \
  port create \
    --network sriov-vlan101 \
    --disable-port-security \
    --fixed-ip subnet=sriov-vlan101-subnet01,ip-address=172.16.64.4 \
    --vnic-type direct \
    sriov-vlan101-subnet01-port02

В выводе единственное отличие 

| port_security_enabled   | False

openstack server add port ubuntu-test-01 sriov-vlan101-subnet01-port02

Прикрепить второй порт к виртуальной машине 

openstack server add port ubuntu-test-01 sriov-vlan101-subnet01-port02

Настройки виртуальной машины

Со стороны сервера базовые настройки 

ip link set up dev ens8

ip addr add 10.90.1.2/24 dev ens8

ip ro
default via 192.168.77.1 dev ens3 proto dhcp src 192.168.77.5 metric 100
8.8.8.8 via 192.168.77.1 dev ens3 proto dhcp src 192.168.77.5 metric 100
10.90.0.0/24 dev ens4 proto kernel scope link src 10.90.0.2
10.90.1.0/24 dev ens8 proto kernel scope link src 10.90.1.2
192.168.77.0/24 dev ens3 proto kernel scope link src 192.168.77.5 metric 100
192.168.77.1 dev ens3 proto dhcp scope link src 192.168.77.5 metric 100

На Хосте ожидаемо наблюдаем spoof checking off 

    vf 61     link/ether 02:a8:ba:86:29:eb brd ff:ff:ff:ff:ff:ff, vlan 101, spoof checking off, link-state auto, trust off, query_rss off

Проверка что ограничений на src-mac нет

Смена мака 

ip link set dev ens8 address 02:70:9d:70:5c:99


Видно что мак поменялся (99 на конце) 

tcpdump  -n -i enp6s0f1 -ee
11:53:56.233183 02:70:9d:70:5c:99 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP (0x0806), Request who-has 10.90.1.1 tell 10.90.1.2, length 46

Оба мака заехали на свитч (что ожидаемо при отключеном Port Security) 

show mac-address-table vlan 101

Codes: *N - VLT Peer Synced MAC
*I - Internal MAC Address used for Inter Process Communication
VlanId     Mac Address           Type          Interface        State
 101	02:70:9d:70:5c:7c	Dynamic    	Te 0/57   	Active
dell-lab#show mac-address-table vlan 101

Codes: *N - VLT Peer Synced MAC
*I - Internal MAC Address used for Inter Process Communication
VlanId     Mac Address           Type          Interface        State
 101	02:70:9d:70:5c:7c	Dynamic    	Te 0/57   	Active
 101	02:70:9d:70:5c:99	Dynamic    	Te 0/57   	Active
 101	02:a8:ba:86:29:eb	Dynamic    	Te 0/57   	Active

"протравить" свитч заполнив табличку коммутации: 

arppoison ens8

Случайные пары мак-адресов/ip-адресов 

11:56:51.593634 00:3d:9f:e3:5e:0b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP (0x0806), Reply 192.199.71.81 is-at 00:3d:9f:e3:5e:0b, length 46
11:56:51.594051 00:3d:52:f1:46:38 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP (0x0806), Reply 120.206.54.21 is-at 00:3d:52:f1:46:38, length 46
11:56:51.594419 00:26:5c:59:a4:fb > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP (0x0806), Reply 99.143.18.102 is-at 00:26:5c:59:a4:fb, length 46
11:56:51.594805 00:13:a2:48:f1:76 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP (0x0806), Reply 170.228.232.78 is-at 00:13:a2:48:f1:76, length 46

Свитчк такое нравится не очень, таблица забита полностью (а свитч довольно жирный) 

dell-lab#Aug 23 11:57:27 %S4810:0 %MACAGT-5-HASH_COLLISION_LOG: Mac:00:17:71:90:4d:91/Vlan:101 could not be added to L2 CAM on portpipe 0 stack-unit 0 due to hash collision. Total number of hash collisions: 17888
Aug 23 11:57:40 %S4810:0 %MACAGT-5-HASH_COLLISION_LOG: Mac:00:2b:5a:e9:6c:d1/Vlan:101 could not be added to L2 CAM on portpipe 0 stack-unit 0 due to hash collision. Total number of hash collisions: 18365
Aug 23 11:57:52 %S4810:0 %MACAGT-5-HASH_COLLISION_LOG: Mac:00:10:04:a7:90:a9/Vlan:101 could not be added to L2 CAM on portpipe 0 stack-unit 0 due to hash collision. Total number of hash collisions: 18661
Aug 23 11:57:53 %STKUNIT0-M:CP %SYSADM-5-CPU_THRESHOLD_CLR: Overall cpu usage of management-unit drops below threshold. Cpu1minUsage (73%)
show mac-address-table count vlan 101
MAC Entries for vlan 101 :
Dynamic Address Count : 		 130988
Static Address (User-defined) Count : 	 0
Sticky Address Count  : 		 0
Total MAC Addresses in Use: 		 130988

Краткий вывод

Отключение Port Security может быть опасно
Конечно можно городить ограничения и со стороны порта свитча, но все же.


Транковый порт (с ограниченным набором VLAN)

1

Проверка, что транки доступны 

openstack extension list --network | grep -i trunk
| Tag support for resources with standard attribute: port, subnet, subnetpool, network, security_group, router, floatingip, policy, trunk | standard-attr-tag           | Enables to set tag on resources with standard attribute.                                                                                                 |
| Trunk Extension                                                                                                                         | trunk                       | Provides support for trunk ports                                                                                                                         |
| Trunk port details                                                                                                                      | trunk-details               | Expose trunk port details                                                                                                                                |


Подготовка сетей

Обычно: одна «родительская» (untagged / access), и несколько «дочерних» сетей под VLAN’ы. 

openstack network create sriov-vlan102  --provider-network-type vlan --provider-physical-network sriovnet0 --provider-segment 102
+---------------------------+----------------------------------------------+
| Field                     | Value                                        |
+---------------------------+----------------------------------------------+
| admin_state_up            | UP                                           |
| availability_zone_hints   | None                                         |
| availability_zones        | None                                         |
| created_at                | 2025-08-23T12:49:55.679015                   |
| description               |                                              |
| dns_domain                | None                                         |
| fq_name                   | ['default-domain', 'admin', 'sriov-vlan102'] |
| id                        | fafb4688-64e9-454b-840d-d486246c30f8         |
| ipv4_address_scope        | None                                         |
| ipv6_address_scope        | None                                         |
| is_default                | None                                         |
| is_vlan_transparent       | None                                         |
| mtu                       | 0                                            |
| name                      | sriov-vlan102                                |
| port_security_enabled     | True                                         |
| project_id                | f39e087061ea48378c9c68348eebbb59             |
| provider:network_type     | vlan                                         |
| provider:physical_network | sriovnet0                                    |
| provider:segmentation_id  | 102                                          |
| qos_policy_id             | None                                         |
| revision_number           | None                                         |
| router:external           | Internal                                     |
| segments                  | None                                         |
| shared                    | False                                        |
| status                    | ACTIVE                                       |
| subnets                   |                                              |
| tags                      |                                              |
| tenant_id                 | f39e087061ea48378c9c68348eebbb59             |
| updated_at                | 2025-08-23T12:49:55.679015                   |
+---------------------------+----------------------------------------------+

openstack network create sriov-vlan103  --provider-network-type vlan --provider-physical-network sriovnet0 --provider-segment 103
+---------------------------+----------------------------------------------+
| Field                     | Value                                        |
+---------------------------+----------------------------------------------+
| admin_state_up            | UP                                           |
| availability_zone_hints   | None                                         |
| availability_zones        | None                                         |
| created_at                | 2025-08-23T12:50:36.487602                   |
| description               |                                              |
| dns_domain                | None                                         |
| fq_name                   | ['default-domain', 'admin', 'sriov-vlan103'] |
| id                        | bd3ed51e-1cdd-4c9a-9c4f-b92dafd31daa         |
| ipv4_address_scope        | None                                         |
| ipv6_address_scope        | None                                         |
| is_default                | None                                         |
| is_vlan_transparent       | None                                         |
| mtu                       | 0                                            |
| name                      | sriov-vlan103                                |
| port_security_enabled     | True                                         |
| project_id                | f39e087061ea48378c9c68348eebbb59             |
| provider:network_type     | vlan                                         |
| provider:physical_network | sriovnet0                                    |
| provider:segmentation_id  | 103                                          |
| qos_policy_id             | None                                         |
| revision_number           | None                                         |
| router:external           | Internal                                     |
| segments                  | None                                         |
| shared                    | False                                        |
| status                    | ACTIVE                                       |
| subnets                   |                                              |
| tags                      |                                              |
| tenant_id                 | f39e087061ea48378c9c68348eebbb59             |
| updated_at                | 2025-08-23T12:50:36.487602                   |
+---------------------------+----------------------------------------------+



 openstack network create sriov-vlan104  --provider-network-type vlan --provider-physical-network sriovnet0 --provider-segment 104
+---------------------------+----------------------------------------------+
| Field                     | Value                                        |
+---------------------------+----------------------------------------------+
| admin_state_up            | UP                                           |
| availability_zone_hints   | None                                         |
| availability_zones        | None                                         |
| created_at                | 2025-08-23T12:51:04.201571                   |
| description               |                                              |
| dns_domain                | None                                         |
| fq_name                   | ['default-domain', 'admin', 'sriov-vlan104'] |
| id                        | fd609f3d-2561-4c39-9c71-0c2199ba2748         |
| ipv4_address_scope        | None                                         |
| ipv6_address_scope        | None                                         |
| is_default                | None                                         |
| is_vlan_transparent       | None                                         |
| mtu                       | 0                                            |
| name                      | sriov-vlan104                                |
| port_security_enabled     | True                                         |
| project_id                | f39e087061ea48378c9c68348eebbb59             |
| provider:network_type     | vlan                                         |
| provider:physical_network | sriovnet0                                    |
| provider:segmentation_id  | 104                                          |
| qos_policy_id             | None                                         |
| revision_number           | None                                         |
| router:external           | Internal                                     |
| segments                  | None                                         |
| shared                    | False                                        |
| status                    | ACTIVE                                       |
| subnets                   |                                              |
| tags                      |                                              |
| tenant_id                 | f39e087061ea48378c9c68348eebbb59             |
| updated_at                | 2025-08-23T12:51:04.201571                   |
+---------------------------+----------------------------------------------+



Сабнеты


openstack subnet  create sriov-vlan102-subnet01 --network sriov-vlan102 --subnet-range 10.102.0.0/24
+----------------------+--------------------------------------+
| Field                | Value                                |
+----------------------+--------------------------------------+
| allocation_pools     | 10.102.0.2-10.102.0.254              |
| cidr                 | 10.102.0.0/24                        |
| created_at           | 2025-08-23T12:52:35.449241           |
| description          | None                                 |
| dns_nameservers      |                                      |
| dns_publish_fixed_ip | None                                 |
| dns_server_address   | 10.102.0.2                           |
| enable_dhcp          | True                                 |
| gateway_ip           | 10.102.0.1                           |
| host_routes          |                                      |
| id                   | fed5208c-6aed-42b8-9aa9-18a7444e3fa5 |
| ip_version           | 4                                    |
| ipv6_address_mode    | None                                 |
| ipv6_ra_mode         | None                                 |
| name                 | sriov-vlan102-subnet01               |
| network_id           | fafb4688-64e9-454b-840d-d486246c30f8 |
| project_id           | f39e087061ea48378c9c68348eebbb59     |
| revision_number      | None                                 |
| segment_id           | None                                 |
| service_types        | None                                 |
| subnetpool_id        | None                                 |
| tags                 |                                      |
| updated_at           | 2025-08-23T12:52:35.449241           |
+----------------------+--------------------------------------+



openstack subnet  create sriov-vlan103-subnet01 --network sriov-vlan103 --subnet-range 10.103.0.0/24
+----------------------+--------------------------------------+
| Field                | Value                                |
+----------------------+--------------------------------------+
| allocation_pools     | 10.103.0.2-10.103.0.254              |
| cidr                 | 10.103.0.0/24                        |
| created_at           | 2025-08-23T12:52:55.647034           |
| description          | None                                 |
| dns_nameservers      |                                      |
| dns_publish_fixed_ip | None                                 |
| dns_server_address   | 10.103.0.2                           |
| enable_dhcp          | True                                 |
| gateway_ip           | 10.103.0.1                           |
| host_routes          |                                      |
| id                   | ec2f665b-b510-4fe2-85ec-20ed46a8f7ab |
| ip_version           | 4                                    |
| ipv6_address_mode    | None                                 |
| ipv6_ra_mode         | None                                 |
| name                 | sriov-vlan103-subnet01               |
| network_id           | bd3ed51e-1cdd-4c9a-9c4f-b92dafd31daa |
| project_id           | f39e087061ea48378c9c68348eebbb59     |
| revision_number      | None                                 |
| segment_id           | None                                 |
| service_types        | None                                 |
| subnetpool_id        | None                                 |
| tags                 |                                      |
| updated_at           | 2025-08-23T12:52:55.647034           |
+----------------------+--------------------------------------+



openstack subnet  create sriov-vlan104-subnet01 --network sriov-vlan104 --subnet-range 10.104.0.0/24
+----------------------+--------------------------------------+
| Field                | Value                                |
+----------------------+--------------------------------------+
| allocation_pools     | 10.104.0.2-10.104.0.254              |
| cidr                 | 10.104.0.0/24                        |
| created_at           | 2025-08-23T12:53:20.156279           |
| description          | None                                 |
| dns_nameservers      |                                      |
| dns_publish_fixed_ip | None                                 |
| dns_server_address   | 10.104.0.2                           |
| enable_dhcp          | True                                 |
| gateway_ip           | 10.104.0.1                           |
| host_routes          |                                      |
| id                   | a1c11284-d06a-4605-85ac-526cf4675a1f |
| ip_version           | 4                                    |
| ipv6_address_mode    | None                                 |
| ipv6_ra_mode         | None                                 |
| name                 | sriov-vlan104-subnet01               |
| network_id           | fd609f3d-2561-4c39-9c71-0c2199ba2748 |
| project_id           | f39e087061ea48378c9c68348eebbb59     |
| revision_number      | None                                 |
| segment_id           | None                                 |
| service_types        | None                                 |
| subnetpool_id        | None                                 |
| tags                 |                                      |
| updated_at           | 2025-08-23T12:53:20.156279           |
+----------------------+--------------------------------------+

























Порты


Родительский порт (вставится в ВМ)
Субпорты (по одному на каждую VLAN/сеть)
Важное: subport сам по себе не присоединяется к ВМ — он «подвешивается» к транку с указанием segmentation-id.



openstack port create sriov-vlan102-subnet01-port01         --network sriov-vlan102 --vnic-type direct  --disable-port-security
+-------------------------+---------------------------------------------------------------------------+
| Field                   | Value                                                                     |
+-------------------------+---------------------------------------------------------------------------+
| admin_state_up          | UP                                                                        |
| allowed_address_pairs   |                                                                           |
| binding_host_id         | None                                                                      |
| binding_profile         | None                                                                      |
| binding_vif_details     | port_filter='True', vlan='102'                                            |
| binding_vif_type        | unbound                                                                   |
| binding_vnic_type       | direct                                                                    |
| created_at              | 2025-08-23T13:13:50.569474                                                |
| data_plane_status       | None                                                                      |
| description             |                                                                           |
| device_id               |                                                                           |
| device_owner            |                                                                           |
| device_profile          | None                                                                      |
| dns_assignment          | None                                                                      |
| dns_domain              | None                                                                      |
| dns_name                | None                                                                      |
| extra_dhcp_opts         | None                                                                      |
| fixed_ips               | ip_address='10.102.0.3', subnet_id='fed5208c-6aed-42b8-9aa9-18a7444e3fa5' |
| id                      | b6428934-8ef3-4a2d-bc81-23622644a6b6                                      |
| ip_allocation           | None                                                                      |
| mac_address             | 02:b6:42:89:34:8e                                                         |
| name                    | sriov-vlan102-subnet01-port01                                             |
| network_id              | fafb4688-64e9-454b-840d-d486246c30f8                                      |
| numa_affinity_policy    | None                                                                      |
| port_security_enabled   | False                                                                     |
| project_id              | f39e087061ea48378c9c68348eebbb59                                          |
| propagate_uplink_status | None                                                                      |
| qos_network_policy_id   | None                                                                      |
| qos_policy_id           | None                                                                      |
| resource_request        | None                                                                      |
| revision_number         | None                                                                      |
| security_group_ids      |                                                                           |
| status                  | DOWN                                                                      |
| tags                    |                                                                           |
| trunk_details           | None                                                                      |
| updated_at              | 2025-08-23T13:13:50.612751                                                |
+-------------------------+---------------------------------------------------------------------------+



 openstack port create sriov-vlan103-subnet01-port01-subport --network sriov-vlan103 --vnic-type direct  --disable-port-security
+-------------------------+---------------------------------------------------------------------------+
| Field                   | Value                                                                     |
+-------------------------+---------------------------------------------------------------------------+
| admin_state_up          | UP                                                                        |
| allowed_address_pairs   |                                                                           |
| binding_host_id         | None                                                                      |
| binding_profile         | None                                                                      |
| binding_vif_details     | port_filter='True', vlan='103'                                            |
| binding_vif_type        | unbound                                                                   |
| binding_vnic_type       | direct                                                                    |
| created_at              | 2025-08-23T13:14:24.688277                                                |
| data_plane_status       | None                                                                      |
| description             |                                                                           |
| device_id               |                                                                           |
| device_owner            |                                                                           |
| device_profile          | None                                                                      |
| dns_assignment          | None                                                                      |
| dns_domain              | None                                                                      |
| dns_name                | None                                                                      |
| extra_dhcp_opts         | None                                                                      |
| fixed_ips               | ip_address='10.103.0.3', subnet_id='ec2f665b-b510-4fe2-85ec-20ed46a8f7ab' |
| id                      | eda0d1a6-efb7-4c92-bdf9-36be29308aef                                      |
| ip_allocation           | None                                                                      |
| mac_address             | 02:ed:a0:d1:a6:ef                                                         |
| name                    | sriov-vlan103-subnet01-port01-subport                                     |
| network_id              | bd3ed51e-1cdd-4c9a-9c4f-b92dafd31daa                                      |
| numa_affinity_policy    | None                                                                      |
| port_security_enabled   | False                                                                     |
| project_id              | f39e087061ea48378c9c68348eebbb59                                          |
| propagate_uplink_status | None                                                                      |
| qos_network_policy_id   | None                                                                      |
| qos_policy_id           | None                                                                      |
| resource_request        | None                                                                      |
| revision_number         | None                                                                      |
| security_group_ids      |                                                                           |
| status                  | DOWN                                                                      |
| tags                    |                                                                           |
| trunk_details           | None                                                                      |
| updated_at              | 2025-08-23T13:14:24.727028                                                |
+-------------------------+---------------------------------------------------------------------------+








openstack port create sriov-vlan104-subnet01-port01-subport --network sriov-vlan104 --vnic-type direct  --disable-port-security
+-------------------------+---------------------------------------------------------------------------+
| Field                   | Value                                                                     |
+-------------------------+---------------------------------------------------------------------------+
| admin_state_up          | UP                                                                        |
| allowed_address_pairs   |                                                                           |
| binding_host_id         | None                                                                      |
| binding_profile         | None                                                                      |
| binding_vif_details     | port_filter='True', vlan='104'                                            |
| binding_vif_type        | unbound                                                                   |
| binding_vnic_type       | direct                                                                    |
| created_at              | 2025-08-23T13:14:51.060012                                                |
| data_plane_status       | None                                                                      |
| description             |                                                                           |
| device_id               |                                                                           |
| device_owner            |                                                                           |
| device_profile          | None                                                                      |
| dns_assignment          | None                                                                      |
| dns_domain              | None                                                                      |
| dns_name                | None                                                                      |
| extra_dhcp_opts         | None                                                                      |
| fixed_ips               | ip_address='10.104.0.3', subnet_id='a1c11284-d06a-4605-85ac-526cf4675a1f' |
| id                      | 243ab831-9c29-40a4-af84-4277d81ecbe4                                      |
| ip_allocation           | None                                                                      |
| mac_address             | 02:24:3a:b8:31:9c                                                         |
| name                    | sriov-vlan104-subnet01-port01-subport                                     |
| network_id              | fd609f3d-2561-4c39-9c71-0c2199ba2748                                      |
| numa_affinity_policy    | None                                                                      |
| port_security_enabled   | False                                                                     |
| project_id              | f39e087061ea48378c9c68348eebbb59                                          |
| propagate_uplink_status | None                                                                      |
| qos_network_policy_id   | None                                                                      |
| qos_policy_id           | None                                                                      |
| resource_request        | None                                                                      |
| revision_number         | None                                                                      |
| security_group_ids      |                                                                           |
| status                  | DOWN                                                                      |
| tags                    |                                                                           |
| trunk_details           | None                                                                      |
| updated_at              | 2025-08-23T13:14:51.099710                                                |
+-------------------------+---------------------------------------------------------------------------+



trunk


openstack network trunk create sriov-trunk-vlan102-103-104   --parent-port sriov-vlan102-subnet01-port01   --subport port=sriov-vlan103-subnet01-port01-subport,segmentation-type=vlan,segmentation-id=103 --subport port=sriov-vlan104-subnet01-port01-subport,segmentation-type=vlan,segmentation-id=104
+-------------------+-------------------------------------------------------------------------------------------------+
| Field             | Value                                                                                           |
+-------------------+-------------------------------------------------------------------------------------------------+
| created_at        | 2025-08-23T13:00:01.594180                                                                      |
| description       |                                                                                                 |
| id                | 536c8ce4-6c72-45c8-bf92-ee042b96ba19                                                            |
| is_admin_state_up | True                                                                                            |
| name              | sriov-trunk-vlan102-103-104                                                                     |
| port_id           | 58f758a8-e30f-48ab-a5f6-7c7ee7a550bb                                                            |
| project_id        | f39e087061ea48378c9c68348eebbb59                                                                |
| status            | DOWN                                                                                            |
| sub_ports         | port_id='1d508ca2-4cf4-4327-8b24-3a50ebfa21b9', segmentation_id='103', segmentation_type='vlan' |
|                   | port_id='899e65f0-2a59-4241-bdc2-9e1f1e6a4606', segmentation_id='104', segmentation_type='vlan' |
| tags              | []                                                                                              |
| updated_at        | 2025-08-23T13:00:01.594180                                                                      |
+-------------------+-------------------------------------------------------------------------------------------------+



Или добавлять позже:

bash
Copy
Edit
openstack network trunk set trunk1 \



 --subport port=subport-100,segmentation-type=vlan,segmentation-id=100



openstack network trunk set trunk1 \



 --subport port=subport-200,segmentation-type=vlan,segmentation-id=200



add port


 openstack server add port ubuntu-test-01 sriov-vlan102-subnet01-port01



    vf 60     link/ether 02:58:f7:58:a8:e3 brd ff:ff:ff:ff:ff:ff, vlan 102, spoof checking on, link-state auto, trust off, query_rss off



2




2




2




2









Источник — https://noname.com.ua/mediawiki/index.php?title=Cisco_ASR1001_Tungsten_Fabric_OpenStack_VM&oldid=14895