VXLAN: Difference between revisions
| (10 intermediate revisions by the same user not shown) | |||
| Line 5: | Line 5: | ||
VTEPs are not allowed to fragment packets. It is thus important to choose a network MTU which allows the VXLAN header to be inserted without exceeding physical MTUs. | VTEPs are not allowed to fragment packets. It is thus important to choose a network MTU which allows the VXLAN header to be inserted without exceeding physical MTUs. | ||
VXLAN runs over udp/4789 by default. VXLANs can be stacked. | VXLAN runs over udp/4789 by default (some vendors used 8472 prior to standardization). VXLANs can be stacked. | ||
==VXLAN encapsulation== | ==VXLAN encapsulation== | ||
| Line 21: | Line 21: | ||
==Use on Linux== | ==Use on Linux== | ||
Note that Openvswitch has its own, distinct VXLAN implementation. This describes the Linux kernel [[tunneling]] device. | VXLAN was added to the Linux kernel in 3.7, with the DOVE extensions added in 3.8. Note that Openvswitch has its own, distinct VXLAN implementation. This describes the Linux kernel [[tunneling]] device. | ||
* Create the interface using <tt>ip</tt>: <tt>ip l a VXLANDEV type vxlan id VNI dstport DSTPORT</tt> | * Create the interface using <tt>ip</tt>: <tt>ip l a VXLANDEV type vxlan id VNI dstport DSTPORT local VTEPIP</tt> | ||
** Add <tt>group MCASTIP dev MCASTDEV</tt> for a multicast-based VXLAN (see below) | ** Add <tt>group MCASTIP dev MCASTDEV</tt> for a multicast-based VXLAN (see below) | ||
*** The default multicast TTL is 1; change it by adding <tt>ttl MCASTTTL</tt> if necessary | |||
** Add the <tt>nolearning</tt> flag to disable source address-based FDB learning | |||
* Destroy the interface with <tt>ip l d VXLANDEV</tt> | * Destroy the interface with <tt>ip l d VXLANDEV</tt> | ||
* Get information with <tt>ip -d l s | * Get information with <tt>ip -d -s l sh VXLANDEV</tt> | ||
** Note: <tt>ip l s</tt> will ''not'' work, as <tt>s</tt> will be interpreted as <tt>set</tt> | |||
===[[systemd]]=== | |||
Minimal <tt>systemd-networkd</tt> netdev file: | |||
<pre> | |||
[NetDev] | |||
Name=vx0 | |||
Kind=vxlan | |||
[VXLAN] | |||
VNI=whatever | |||
Independent=yes | |||
</pre> | |||
Without <tt>Independent=yes</tt>, the device must be bound to some other device via a <tt>network</tt> file, or it will be more or less silently ignored by <tt>systemd-networkd</tt>. | |||
===Forwarding tables=== | ===Forwarding tables=== | ||
* On Linux, the forwarding tables can be built up | * On Linux, the forwarding tables can be built up four different ways: | ||
** Automatically using multicast: supply the multicast IP as the <tt>group</tt> argument when creating the VXLAN interface | ** Automatically using multicast: supply the multicast IP as the <tt>group</tt> argument when creating the VXLAN interface | ||
** Manually using <tt>bridge</tt>: add the entries using <tt>bridge fdb add to L2ADDR dst L3ADDR dev VXLANDEV</tt> | ** Manually using <tt>bridge</tt>: add the entries using <tt>bridge fdb add to L2ADDR dst L3ADDR dev VXLANDEV</tt> | ||
** Automatically using flooding: <tt>bridge fdb append 00:00:00:00:00:00 dev VXLANDEV dst RVTEPIP</tt> | |||
*** Unknown l2 destinations will see [[ARP]] flooded to all 00:00:00:00:00:00 peers | |||
** BGP-EVPN (see [https://tools.ietf.org/html/rfc7432 RFC 7432]) | |||
* Delete an entry: <tt>bridge fdb delete L2ADDR dev VXLANDEV</tt> | * Delete an entry: <tt>bridge fdb delete L2ADDR dev VXLANDEV</tt> | ||
* Dump the forwarding table: <tt>bridge fdb show dev VXLANDEV</tt> | * Dump the forwarding table: <tt>bridge fdb show dev VXLANDEV</tt> | ||
===Offloading=== | ===Offloading=== | ||
Several NICs provide hardware offloading functionality, typically configured with <tt>[[ethtool]]</tt>. | Several NICs provide hardware offloading functionality, typically configured with <tt>[[ethtool]]</tt>. VXLAN offloading mainly means that other NIC offloading (UDP checksums, GRO, etc.) can be performed in the presence of VXLAN. | ||
* [https://community.mellanox.com/s/article/vxlan-considerations-for-connectx-3-pro Mellanox Connect-X 3] | |||
* Telefonica's "[https://engineering.telefonica.com/maximizing-performance-in-vxlan-overlay-networks-ec35ebe29440 Maximizing Performance in VXLAN Networks]", 2017-01-25 | |||
==External links== | ==External links== | ||
* [[Linux]] kernel [https://www.kernel.org/doc/Documentation/networking/vxlan.txt vxlan] documentation | * [[Linux]] kernel [https://www.kernel.org/doc/Documentation/networking/vxlan.txt vxlan] documentation | ||
* Vincent Bernat's "[https://vincent.bernat.ch/en/blog/2017-vxlan-linux VXLAN & Linux]", 2017-05-03 | * Vincent Bernat's "[https://vincent.bernat.ch/en/blog/2017-vxlan-linux VXLAN & Linux]", 2017-05-03 | ||