VXLAN: The savior of modern Data Centers

Jose Chapa • July 21 2023

Introduction

In modern data centers, the rapid growth of virtualization and cloud computing has created an immediate need for scalable, flexible, and efficient network architectures and technologies. As we continue to incorporate containers and multiple layers of virtualization, the antiquated network protocols are not sufficient. VLANs, MPLS, and STP struggled with handling the sheer number of end hosts, whether they were bare-metal, virtualized, or even in the public cloud. VXLAN is a technology that has significantly mitigated these barriers and allowed our system engineers the freedom and flexibility they needed.

What is VXLAN?

Virtual eXtensible Local Area Network (VXLAN) is an encapsulation protocol that addresses the need for overlay networks within virtualized data centers. Tunnels are created at the edge of the underlay, adjacent to hosts, every time a data unit needs to be transported from host to host. The result is that, although the frames between the hosts may have been routed over multiple gateways, the hosts are able to communicate as if they are on the same logical network. They believe they're in the same VLAN, they believe they're in the same broadcast domain, and their traffic is directly addressed to each other's MAC addresses.

VXLAN vs VLAN

The 24-bit VXLAN Network Identifier (VNID) allows for over 16 million logical networks, in contrast to VLAN's 12-bit VLAN ID, which only allows over 4000 logical networks. Aside from the VNID, the VXLAN header also contains flags and reserved fields for future feature expansion.

VTEP

The VXLAN Tunnel EndPoint (VTEP) is the device that encapsulates and de-encapsulates VXLAN traffic. The logical interface on the VTEP that performs this function is called the Network Virtual Interface (NVE). The VTEP serves as the connection between the overlay and the underlay, and it can be either a layer 3 network device or the hypervisor that is in control of the virtual host. In the former case, it will usually be the leaf switch in the data center.

Let's assume a data center spine-leaf architecture. There are two hosts attempting communications. The first host will send the frame to the second host, and it will contain the correct destination MAC. But the hosts have different gateways and, therefore, different broadcast domains. They are connected to different leaf switches. So the leaf switch connected to host 1 must create a tunnel to host 2's leaf switch. Leaf switch 1 will encapsulate the traffic with a VXLAN header, a UDP header, and with leaf switch 2's IP as the destination IP. In the overlay, we see the traffic arrive directly at leaf switch 2, but we know that in the underlay, it got routed through the spine switch. Once leaf switch 2 receives the encapsulated traffic, it de-encapsulates it and delivers the original frame to host 2.

Because of the nature of these tunnels, they work great in environments with super spines connecting multiple fabrics. The hosts will not know that they might be in separate data centers that are continents apart. Or perhaps one of the hosts is located in a public cloud. Aside from the clear latency differences, the hosts will not know anything about the underlay network that is allowing the communications to happen, nor will they notice a difference in communication.

Conclusion

VXLAN has seen widespread adoption in modern data centers and cloud environments. It has replaced older platforms such as Cisco's fabric path. Some modern implementations include Cisco's ACI, which uses a modified and proprietary version of VXLAN called iVXLAN; It provides flexible automation solutions at the expense of being vendor-locked. VMware's NSX is another product that has a strong reliance on VXLAN. Other environments will build an EVPN fabric using open standards and create their own automation/orchestration platform. There are many implementations of software-defined networking that allow a scalable and flexible configuration for our servers.

VXLAN has emerged as a crucial technology in modern data centers and cloud environments, providing a scalable, flexible, and efficient network solution for the challenges posed by virtualization and cloud computing. Its ability to create virtual Layer 2 networks over existing Layer 3 infrastructure facilitates seamless VM mobility, multi-tenancy, and improved network utilization. As data centers and cloud environments continue to evolve, VXLAN will remain a vital component for achieving the agility and performance required to meet the demands of the digital age.