VXLAN: Difference between revisions

 
(9 intermediate revisions by the same user not shown)
Line 5: Line 5:
VTEPs are not allowed to fragment packets. It is thus important to choose a network MTU which allows the VXLAN header to be inserted without exceeding physical MTUs.
VTEPs are not allowed to fragment packets. It is thus important to choose a network MTU which allows the VXLAN header to be inserted without exceeding physical MTUs.


VXLAN runs over udp/4789 by default. VXLANs can be stacked.
VXLAN runs over udp/4789 by default (some vendors used 8472 prior to standardization). VXLANs can be stacked.


==VXLAN encapsulation==
==VXLAN encapsulation==
Line 21: Line 21:


==Use on Linux==
==Use on Linux==
Note that Openvswitch has its own, distinct VXLAN implementation. This describes the Linux kernel [[tunneling]] device.
VXLAN was added to the Linux kernel in 3.7, with the DOVE extensions added in 3.8. Note that Openvswitch has its own, distinct VXLAN implementation. This describes the Linux kernel [[tunneling]] device.
* Create the interface using <tt>ip</tt>: <tt>ip l a VXLANDEV type vxlan id VNI dstport DSTPORT</tt>
* Create the interface using <tt>ip</tt>: <tt>ip l a VXLANDEV type vxlan id VNI dstport DSTPORT local VTEPIP</tt>
** Add <tt>group MCASTIP dev MCASTDEV</tt> for a multicast-based VXLAN (see below)
** Add <tt>group MCASTIP dev MCASTDEV</tt> for a multicast-based VXLAN (see below)
*** The default multicast TTL is 1; change it by adding <tt>ttl MCASTTTL</tt> if necessary
** Add the <tt>nolearning</tt> flag to disable source address-based FDB learning
* Destroy the interface with <tt>ip l d VXLANDEV</tt>
* Destroy the interface with <tt>ip l d VXLANDEV</tt>
* Get information with <tt>ip -d l s VXLANDEV</tt>
* Get information with <tt>ip -d -s l sh VXLANDEV</tt>
** Note: <tt>ip l s</tt> will ''not'' work, as <tt>s</tt> will be interpreted as <tt>set</tt>
 
===[[systemd]]===
Minimal <tt>systemd-networkd</tt> netdev file:
<pre>
[NetDev]
Name=vx0
Kind=vxlan
 
[VXLAN]
VNI=whatever
Independent=yes
</pre>
 
Without <tt>Independent=yes</tt>, the device must be bound to some other device via a <tt>network</tt> file, or it will be more or less silently ignored by <tt>systemd-networkd</tt>.


===Forwarding tables===
===Forwarding tables===
* On Linux, the forwarding tables can be built up three different ways:
* On Linux, the forwarding tables can be built up four different ways:
** Automatically using multicast: supply the multicast IP as the <tt>group</tt> argument when creating the VXLAN interface
** Automatically using multicast: supply the multicast IP as the <tt>group</tt> argument when creating the VXLAN interface
** Manually using <tt>bridge</tt>: add the entries using <tt>bridge fdb add to L2ADDR dst L3ADDR dev VXLANDEV</tt>
** Manually using <tt>bridge</tt>: add the entries using <tt>bridge fdb add to L2ADDR dst L3ADDR dev VXLANDEV</tt>
** Automatically using flooding: <tt>bridge fdb append 00:00:00:00:00:00 dev VXLANDEV dst VTEPIP</tt>
** Automatically using flooding: <tt>bridge fdb append 00:00:00:00:00:00 dev VXLANDEV dst RVTEPIP</tt>
*** Unknown l2 destinations will see [[ARP]] flooded to all 00:00:00:00:00:00 peers
** BGP-EVPN (see [https://tools.ietf.org/html/rfc7432 RFC 7432])
* Delete an entry: <tt>bridge fdb delete L2ADDR dev VXLANDEV</tt>
* Delete an entry: <tt>bridge fdb delete L2ADDR dev VXLANDEV</tt>
* Dump the forwarding table: <tt>bridge fdb show dev VXLANDEV</tt>
* Dump the forwarding table: <tt>bridge fdb show dev VXLANDEV</tt>


===Offloading===
===Offloading===
Several NICs provide hardware offloading functionality, typically configured with <tt>[[ethtool]]</tt>.
Several NICs provide hardware offloading functionality, typically configured with <tt>[[ethtool]]</tt>. VXLAN offloading mainly means that other NIC offloading (UDP checksums, GRO, etc.) can be performed in the presence of VXLAN.
 
* [https://community.mellanox.com/s/article/vxlan-considerations-for-connectx-3-pro Mellanox Connect-X 3]
* Telefonica's "[https://engineering.telefonica.com/maximizing-performance-in-vxlan-overlay-networks-ec35ebe29440 Maximizing Performance in VXLAN Networks]", 2017-01-25


==External links==
==External links==
* [[Linux]] kernel [https://www.kernel.org/doc/Documentation/networking/vxlan.txt vxlan] documentation
* [[Linux]] kernel [https://www.kernel.org/doc/Documentation/networking/vxlan.txt vxlan] documentation
* Vincent Bernat's "[https://vincent.bernat.ch/en/blog/2017-vxlan-linux VXLAN & Linux]", 2017-05-03
* Vincent Bernat's "[https://vincent.bernat.ch/en/blog/2017-vxlan-linux VXLAN & Linux]", 2017-05-03