Check out my first novel, midnight's simulacra!

Azure: Difference between revisions

From dankwiki
Line 11: Line 11:
Azure hosts use ConnectX devices from Mellanox (now NVIDIA). ConnectX-4 needs the [https://docs.kernel.org/networking/device_drivers/ethernet/mellanox/mlx5.html mlx5_core] driver, while ConnectX-3 needs mlx4_en. It is not possible as of February 2023 to force allocation of one or the other for a given VM, and the NIC type can change if the VM is moved. Azure documentation typically refers to both classes as "SmartNICs".
Azure hosts use ConnectX devices from Mellanox (now NVIDIA). ConnectX-4 needs the [https://docs.kernel.org/networking/device_drivers/ethernet/mellanox/mlx5.html mlx5_core] driver, while ConnectX-3 needs mlx4_en. It is not possible as of February 2023 to force allocation of one or the other for a given VM, and the NIC type can change if the VM is moved. Azure documentation typically refers to both classes as "SmartNICs".


The fundamental network object of Azure is the VNet. Hosts connected to the same VNet can communicate with one another without intermediary gateways. VNets ought use RFC1918 or RFC6598 addresses, and best practices include using distinct sections of these address spaces in your distinct VNets (since VNets can be peered with one another). Broadcast and multicast are not supported with an VNet. ARP requests are answered by the (virtual) gateway (actually SDN), not broadcast to the various machines of the VNet. You will not typically see incoming ARP requests (though you will see your own requests, and any replies you receive).
The fundamental network object of Azure is the VNet. Hosts connected to the same VNet can communicate with one another without intermediary gateways. VNets ought use RFC1918 or RFC6598 addresses, and best practices include using distinct sections of these address spaces in your distinct VNets (since VNets can be peered with one another). Broadcast and multicast are not supported with an VNet. ARP requests are answered by the (virtual) gateway (actually SDN), not broadcast to the various machines of the VNet. You will not typically see incoming ARP requests (though you will see your own requests, and any replies you receive). ARP replies will generally contain the bogus hardware address 12:34:56:78:9a:bc, and this will typically be the only MAC address in your neighbor tables. You do see a "real" MAC locally when examining your interfaces. Your private IP addresses will be acquired using DHCP.


You'll see traffic with the IP 168.63.129.16; it is used by numerous internal Microsoft services. No host can transmit with this address.
You'll see traffic with the IP 168.63.129.16; it is used by numerous internal Microsoft services, including DHCP. No host can transmit with this address.
 
VMs can be assigned public IPs, but they will not be visible on the VM proper, and should not be added using <tt>ip-address</tt>. The incoming traffic will be directed to your VM by the SDN, and DNATted such that you see your internal IP as the destination address.


===Accelerated Networking===
===Accelerated Networking===

Revision as of 11:42, 17 February 2023

Microsoft's cloud platform, similar to Amazon's AWS or Google's GCP.

To get KeyVault stuff working with Debian's 2.18.0, I had to run: pip3 install azure-keyvault==1.1.0

CLI

Networking

Azure hosts use ConnectX devices from Mellanox (now NVIDIA). ConnectX-4 needs the mlx5_core driver, while ConnectX-3 needs mlx4_en. It is not possible as of February 2023 to force allocation of one or the other for a given VM, and the NIC type can change if the VM is moved. Azure documentation typically refers to both classes as "SmartNICs".

The fundamental network object of Azure is the VNet. Hosts connected to the same VNet can communicate with one another without intermediary gateways. VNets ought use RFC1918 or RFC6598 addresses, and best practices include using distinct sections of these address spaces in your distinct VNets (since VNets can be peered with one another). Broadcast and multicast are not supported with an VNet. ARP requests are answered by the (virtual) gateway (actually SDN), not broadcast to the various machines of the VNet. You will not typically see incoming ARP requests (though you will see your own requests, and any replies you receive). ARP replies will generally contain the bogus hardware address 12:34:56:78:9a:bc, and this will typically be the only MAC address in your neighbor tables. You do see a "real" MAC locally when examining your interfaces. Your private IP addresses will be acquired using DHCP.

You'll see traffic with the IP 168.63.129.16; it is used by numerous internal Microsoft services, including DHCP. No host can transmit with this address.

VMs can be assigned public IPs, but they will not be visible on the VM proper, and should not be added using ip-address. The incoming traffic will be directed to your VM by the SDN, and DNATted such that you see your internal IP as the destination address.

Accelerated Networking

"Accelerated Networking" (AN) is available on most VM classes, and provides the VM with an SR-IOV Virtual Function; this will show up as a distinct interface within the VM. It can be selected at VM creation time, or whenever the VM is deallocated (stopped). Some VM classes *require* AN. The mapping of VF devices to synthetic devices is undefined, and ought be managed with systemd-networkd or udev rules in the presence of multiple virtual interfaces. The synthetic interface and its associated VF interface will have the same MAC address. From the perspective of other network entities, these are a single interface. They can be distinguished by checking ethtool for the driver (the synthetic interface is "hv_netsvc") or checking ip-link output for the slave designator (the VF is the slave). From the Azure docs: 

Applications should interact only with the synthetic interface...Outgoing network packets are passed from the netvsc driver to the VF driver and then transmitted through the VF interface. Incoming packets are received and processed on the VF interface before being passed to the synthetic interface. Exceptions are incoming TCP SYN packets and broadcast/multicast packets that are processed by the synthetic interface only.

VMs

  • Fsv2: 8370C (Ice Lake), 8272CL (Cascade Lake), 8168 (Skylake), 3.4--3.7
  • FX: 8246R (Cascade Lake), 4.0
  • Dv2: 8370C (Ice Lake), 8272CL (Cascade Lake), 8171M (Skylake), 2673v3 (Haswell), 2673v4 (Broadwell)
  • DSv2: 8370C (Ice Lake), 8272CL (Cascade Lake), 8171M (Skylake), 2673v3 (Haswell), 2673v4 (Broadwell)
  • Eav4, Easv4: EPYC 7452
  • H: 2667v3 without hyperthreading or SR-IOV but w/ MPI
Class vCPUS GiB SSD Disks NICs Mbps
F2sv2 2 4 16 4 2 5000
F4sv2 4 8 32 8 2 10000
F8sv2 8 16 64 16 4 12500
F16sv2 16 32 128 32 4 12500
F32sv2 32 64 256 32 8 16000
F48sv2 48 96 384 32 8 21000
F64sv2 64 128 512 32 8 28000
F72sv2 72 144 576 32 8 30000
FX4mds 4 84 168 8 2 4000
FX12mds 12 252 504 24 4 8000
FX24mds 24 504 1008 32 4 16000
FX36mds 36 756 1512 32 8 24000
FX48mds 48 1008 2016 32 8 32000
D1v2 1 3.5 504 4 2 750
D2v2 2 7 100 8 2 1500
D3v2 4 14 200 16 4 3000
D4v2 8 28 400 32 8 6000
D5v2 16 56 800 64 8 12000
D11v2 2 14 100 ? 2 1500
D12v2 4 28 200 ? 4 3000
D13v2 8 56 400 ? 8 6000
D14v2 16 112 800 ? 8 12000
D15v2 20 140 1000 ? 8 25000
DS1v2 1 3.5 7 4 2 750
DS2v2 2 7 14 8 2 1500
DS3v2 4 14 28 16 4 3000
DS4v2 8 28 56 32 8 6000
DS5v2 16 56 112 64 8 12000
DS11v2 2 14 28 8 2 1500
DS12v2 4 28 56 16 4 3000
DS13v2 8 56 112 32 8 6000
DS14v2 16 112 224 64 8 12000
DS15v2 20 140 280 64 8 25000
H8 8 56 1000 32 2 ?
H16 16 112 2000 64 4 ?
H8m 8 112 1000 32 2 ?
H16m 16 224 2000 64 4 ?
H16r 16 112 2000 64 4 ?
H16mr 16 224 2000 64 4 ?

NUMA (multipackage) machines are only available from the HB class.

External Links

Retrieved from "https://nick-black.com/dankwiki/index.php?title=Azure&oldid=9616"