Key takeaways

  • MPLS (RFC 3031) forwards packets by short, locally significant labels instead of looking at the IP header at every hop. BGP/MPLS IP VPNs (RFC 4364) use it to give each customer a private routed network across a shared provider backbone.
  • In an RFC 4364 VPN, each provider-edge router keeps a separate VRF per customer attachment. Route distinguishers keep overlapping prefixes distinct and route targets decide which sites share routes. The RFC states the model aims for isolation comparable to a layer 2 backbone such as Frame Relay, but it does not encrypt data.
  • MEF 70.1 describes SD-WAN as an application-aware, over-the-top connectivity service whose policies steer application flows over multiple underlay networks, kept distinct from the underlay services themselves.
  • EVPN (RFC 7432) is a BGP control plane for Ethernet and IP reachability. RFC 8365 describes using it as the control plane of a network virtualization overlay with VXLAN or NVGRE, which is why it dominates inside data centers and between them.
  • The technologies combine. A common design uses a provider MPLS or internet underlay, an SD-WAN overlay for branches and an EVPN fabric inside the data centers, with the company owning the routing policy at each boundary.
  • The decision is mostly about who needs to control what: ownership of routing policy, number of sites, application sensitivity to loss and delay, and whether encryption is required in transit.

Three answers to one question

When a company has an office in New York, a warehouse in Atlanta, a data center and two cloud accounts, somebody has to decide how those places reach each other. The sales conversation usually presents MPLS, SD-WAN and EVPN as competitors. They are better understood as answers at different layers. MPLS is a forwarding and service technique that a carrier uses to build private networks. SD-WAN is a management and path-selection layer that runs on top of whatever transport you have. EVPN is a control plane, built on BGP, that spreads Ethernet and IP reachability information between the devices that make up a fabric or overlay. Treating them as layers rather than rivals makes the choice clearer.

This article goes through each in turn, quotes what the standards actually say, and finishes with a way to decide. The routing and segmentation mechanics are covered in more depth in our articles on VRFs, VLANs and VXLAN-EVPN across datacenters, on BGP at the edge, and on WireGuard versus IPsec for site-to-site meshes. Here we stay at the level of the backbone decision.

MPLS and the provider VPN

RFC 3031 describes the MPLS architecture. A router that supports it is a Label Switching Router. A packet entering the MPLS domain is assigned to a Forwarding Equivalence Class, defined as a group of packets forwarded in the same manner, and is given a short fixed-length label. The RFC stresses that a label is locally significant and is never an encoding of the network-layer address. Each following router looks only at the label, swaps it for the next one and forwards the packet along a Label Switched Path. Labels can be stacked, last in first out, and processing always uses the top label.

The RFC lists reasons for doing this. Forwarding can be done by switches that can look up and replace a label but cannot analyze network-layer headers at adequate speed. The ingress router can classify packets using more than the header, such as the arrival port. And explicit routes can be represented by labels, which supports traffic engineering and policy. Today the first reason matters less, because hardware routes IP quickly, while the last one remains the reason carriers keep using MPLS: they can place traffic on chosen paths and carry many customers on one core.

The service most companies buy is the one in RFC 4364, BGP/MPLS IP VPNs. Its abstract describes a service provider using an IP backbone to provide VPNs for its customers, with customer-edge routers handing routes to provider-edge routers in a peer model, so there is no overlay of tunnels that the customer has to build, and the core routers do not need to know the VPN routes. Each provider-edge router keeps a separate VPN Routing and Forwarding table, a VRF, for each attached site group, and only routes from sites that share a VPN enter that table.

Two inventions make overlapping customers workable. The route distinguisher is 8 bytes prepended to the 4‑byte IPv4 prefix to form a 12‑byte VPN-IPv4 address, so two customers who both use 10.1.0.0/16 stay distinct inside BGP. The route target is a BGP extended community: routes leaving a site carry export targets, and a VRF accepts a route only when one of its import targets matches. Provider-edge routers exchange these routes over multiprotocol BGP, using a VPN label chosen per route, per circuit or per VRF, and the packet then travels through a second, tunnel label to the egress router.

What does a customer get? Predictable private routing, a carrier that carries the traffic edge to edge, and usually a service-level agreement on latency, loss and availability for a class of traffic. What does it not get? The RFC is careful here: the model aims for isolation comparable to a layer 2 backbone like Frame Relay, but it does not encrypt data or detect tampering, so cryptography must be added separately. A private circuit is private by contract and engineering, not by cryptography. Companies that handle regulated data usually still encrypt across it, which is a design choice we revisit below.

SD-WAN: a layer over whatever transport you have

SD-WAN grew out of a cost problem. Dedicated circuits are expensive and slow to order, while broadband and business internet are cheap and fast to install but vary in quality. An SD-WAN edge device at each site builds encrypted tunnels across all available links, measures loss, delay and jitter on each, and steers each application over the path that suits it. The standards body for carrier services, MEF, has defined the vocabulary. MEF 70 described an application-aware, over-the-top WAN connectivity service that uses policies to decide how application flows are directed over multiple underlay networks, irrespective of the underlay technologies or providers. MEF 70.1, which is now the current reference, adds service attributes for underlay connectivity services, performance metrics that show application performance across providers, and support for application-based security.

The important idea in that definition is the separation between the SD-WAN service and the underlay connectivity services it runs over. That separation is what lets a company use a carrier MPLS circuit at headquarters, fiber at the data center, cable at a small branch and a cellular link as backup, and manage them with one policy. It is also why SD-WAN is not a replacement for the underlay: a branch on a congested cable line is still on a congested cable line. SD-WAN hides failures well and steers around degradation. It cannot create bandwidth.

Where SD-WAN fits best is a company with many small sites, a growing share of traffic going to cloud and software-as-a-service applications rather than to a central data center, and little appetite for managing routers at every location. Where it fits worse is a small number of large sites with predictable traffic, strict latency requirements between them, or a need to peer with partners using their own routing policy. Vendor products also differ a great deal in how open their control planes are, which matters if you want to leave one day. Our article on microsegmentation and the technologies pages on SD-WAN and SASE cover the security side.

EVPN: the control plane you own

EVPN started life as a replacement for older layer 2 VPN services. RFC 7432 describes procedures for BGP MPLS-based Ethernet VPNs. It defines route types that carry the information a layer 2 network normally learns by flooding. A MAC/IP Advertisement route (type 2) announces a learned MAC address and optionally its IP address. An Inclusive Multicast route (type 3) sets up flooding of broadcast, unknown and multicast traffic. Ethernet Auto-Discovery (type 1) and Ethernet Segment (type 4) routes support multihoming, where a customer device connects to two or more provider devices at once. The RFC also defines all-active and single-active redundancy, a MAC mobility extended community with a sequence number so that a moved host is recognised, and a way to carry IP addresses in the MAC route to reduce ARP and neighbor-discovery flooding.

RFC 8365 moved EVPN into the data center. It describes EVPN as the control plane of a network virtualization overlay: per-tenant isolation, a very large number of tenants, layer 2 extension across pods or data centers and virtual machine mobility, using VXLAN or NVGRE as the encapsulation, or MPLS over GRE. BGP distributes MAC and IP reachability and membership information, replacing flooding and learning in the data plane. The VXLAN network identifier is carried in the label field of the routes, and a BGP Encapsulation extended community signals which tunnel type applies. The RFC also covers data-center interconnect, comparing gateway-based and ASBR-based designs.

The consequence for a mid-size company is that EVPN is rarely a product you buy from a carrier for branches. It is a technique you use to build and connect your own fabrics: inside a data center, between two data centers, or between a data center and a colo facility. Carriers also sell EVPN-based layer 2 and layer 3 services, but what you are buying then is a provider service, described by the same RFCs, and the decision belongs back in the MPLS conversation. If you run your own fabric, the benefits are vendor-neutral standards, active-active redundancy, fast host mobility and strong multi-tenancy by VRF. The cost is operational: you need people who can run BGP and understand the underlay.

Comparing them honestly

  • Who owns the routing policy: with a provider MPLS VPN the carrier does, and you hand it routes at the edge. With SD-WAN the vendor's controller does, within the policies you set. With EVPN you do, end to end.
  • Number and size of sites: dozens of small sites favor SD-WAN. A handful of large sites with strict requirements favor private circuits or your own EVPN fabric.
  • Application sensitivity: voice, trading and real-time control are sensitive to jitter and loss. A carrier class-of-service guarantee on MPLS, or dedicated fiber under your own overlay, gives the most predictable path. SD-WAN can steer around trouble but cannot guarantee a path across the public internet.
  • Encryption: MPLS VPNs do not encrypt, per RFC 4364. SD-WAN edges normally encrypt their tunnels. EVPN and VXLAN do not encrypt either, so a fabric that crosses untrusted infrastructure needs IPsec or MACsec under it.
  • Cloud access: SD-WAN products integrate with cloud on-ramps most easily. Providers also offer cloud interconnect over MPLS or cross-connects, and EVPN designs reach clouds through routers at the edge.
  • Exit cost: standards-based MPLS and EVPN can be moved between providers or vendors with effort. A proprietary SD-WAN controller is the hardest to leave.

How the pieces combine

Most real networks use all three ideas. A typical design for a company with a data center, a colo site and twenty branches runs EVPN inside and between the two data centers, so that tenants and applications are separated by VRF and move freely. It connects branches with an SD-WAN overlay over a mix of business internet and one or two carrier circuits. At the head end the SD-WAN hub hands each overlay to a VRF in the data center, and BGP at the edge connects to carriers and the internet. The boundaries are where the policy lives: which prefixes are accepted from a carrier, which tenants may reach which branches, and what is encrypted.

This is why the question to ask a vendor is not which technology is best, but which of those boundaries it takes over and which it leaves to you. A managed MPLS service takes over the whole backbone and gives you a handoff. A managed SD-WAN takes over the branch edge and path selection. If you want to keep the ability to change carriers or vendors, keep the data-center side and the routing policy in open protocols, and treat the branch overlay as replaceable.

A decision path for a mid-size firm

Start with the sites and the applications, not the technology. List each site, the circuits available there, who uses it and which applications cross the WAN. Mark the traffic that cannot tolerate variation, such as voice and market-data or control traffic, and the traffic that is mostly cloud-bound. If most traffic goes to the internet and cloud applications, the case for an expensive private backbone weakens quickly, and a well-chosen SD-WAN with two diverse links per site is usually enough. If a few large sites exchange large volumes with strict requirements between them, price private circuits or a carrier MPLS service for those, and use SD-WAN for the rest.

Then decide what must be encrypted. Regulations such as HIPAA and the Gramm-Leach-Bliley Act do not name MPLS or SD-WAN, but a documented risk analysis will usually conclude that data leaving a building should be encrypted in transit even over a private circuit. That favors an SD-WAN or IPsec layer on top of any underlay. Finally, decide who runs it. A mid-size firm with one network engineer should be wary of designs whose failure modes are understood only by the vendor, and equally wary of a bespoke EVPN fabric that only one person understands. Whatever you choose, insist on a monitoring and flow-data plan, as in our NetFlow, sFlow and IPFIX article, and on a tested failover plan before the first outage.

Clayton Cybernetics designs and operates backbones of all three kinds, and the usual starting point is a short assessment that maps sites, circuits and applications before any product is named.

Sources

This article is general technical information, not a design for your network. We read the RFCs through a summarizing fetch tool; each page was longer than one fetch, so the later sections of RFC 4364 (including its security section), RFC 3031 and RFC 7432 were not read in full. For MEF 70 and 70.1 we relied on a search summary and the MEF text excerpts it returned; the MEF page itself moved from mef.net to mplify.net and its full text was not read. The comparison table, the advice on encryption, the combination design and the decision path are general engineering knowledge, not quoted from the sources listed. Product capabilities and carrier offers differ; confirm against current documentation and contracts. Last reviewed .

Keep reading

Need help with this?

Cloud cost review and management, without compromising on quality or security.

Cloud Management