In-Depth Comparison of MetalLB Modes in Kubernetes: L2 vs BGP (with FRR Example)

1. Introduction

In public cloud environments such as EKS, GKE, and AKS, a LoadBalancer Service is automatically mapped to the provider’s native load balancer.
In an on-premises Kubernetes cluster, however, there is no built-in way to expose a Service with a dedicated external IP.

This is where MetalLB comes in. MetalLB assigns a Service VIP (virtual IP) within your cluster and advertises it to the outside network, enabling external access to Kubernetes Services.

MetalLB provides two major operating modes:

• L2 Mode (Layer 2)
• BGP Mode (Border Gateway Protocol)

These two modes differ significantly in architecture, scalability, and operational complexity. In this article, we’ll take a deep dive into both, and walk through real YAML configuration and router integration examples.

2. L2 Mode (Layer 2)

How It Works

• MetalLB assigns a VIP to one of the nodes.
• That node responds to ARP/NDP requests on behalf of the VIP.
• Devices in the same subnet associate the VIP with that node’s MAC address.
• Incoming traffic to the VIP lands on the selected node and is forwarded to Pods via kube-proxy (iptables/IPVS).

Pros

• Simple to configure (no router integration needed).
• Works out of the box inside a single subnet.
• Perfect for PoC, dev, and small clusters.

Cons

• Relies on ARP/NDP broadcast → limited scalability.
• Failover can be slow if ARP cache entries are stale.
• Cannot span multiple subnets or racks.

3. BGP Mode (Border Gateway Protocol)

How It Works

• MetalLB nodes establish BGP sessions with external routers.
• Service VIPs are advertised via BGP routes.
• The wider network learns the VIPs through routing, and traffic can be distributed across nodes.

Pros

• Fast failover (routing updates are quick).
• Scales beyond a single subnet; works across racks and even data centers.
• Supports ECMP (Equal-Cost Multi-Path) for efficient load sharing.

Cons

• Requires router configuration and cooperation with the network team.
• Misconfiguration can lead to routing loops or blackholes.
• Higher learning curve compared to L2.

4. Performance and Reliability Comparison

5. Configuration Guide

(1) L2 Mode Example

metallb-config-l2.yaml

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: default-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.10.240-192.168.10.250   # Range of VIPs to allocate

---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: l2adv
  namespace: metallb-system
spec:
  ipAddressPools:
  - default-pool

→ VIPs will be allocated from the 192.168.10.240–250 range.

(2) BGP Mode Example

metallb-config-bgp.yaml

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: bgp-pool
  namespace: metallb-system
spec:
  addresses:
  - 10.10.20.100-10.10.20.110

---
apiVersion: metallb.io/v1beta1
kind: BGPPeer
metadata:
  name: router-peer1
  namespace: metallb-system
spec:
  myASN: 64512                # Cluster ASN
  peerASN: 64513              # Router ASN
  peerAddress: 10.10.20.1     # Router address
  holdTime: 30s

---
apiVersion: metallb.io/v1beta1
kind: BGPAdvertisement
metadata:
  name: bgp-adv
  namespace: metallb-system
spec:
  ipAddressPools:
  - bgp-pool

(3) Cisco IOS Router Example

router bgp 64513
 bgp log-neighbor-changes
 neighbor 10.10.20.101 remote-as 64512
 neighbor 10.10.20.102 remote-as 64512
 !
 address-family ipv4
  neighbor 10.10.20.101 activate
  neighbor 10.10.20.102 activate
 exit-address-family

6. FRR (Free Range Routing)

What is FRR?

• Free Range Routing is an open-source routing stack for Linux/Unix.
• Originally forked from Quagga, now far more actively developed.
• Turns a Linux server into a software router.

Supported Protocols

• BGP, OSPF, OSPFv3
• RIP, RIPng
• IS-IS
• PIM-SM (multicast)
• EVPN, MPLS, and more

Why Use FRR?

• Great for lab setups — no need for Cisco/Juniper hardware.
• Works perfectly with MetalLB BGP mode for testing.
• Can act as an internal or external BGP peer in Kubernetes clusters.

(1) Installing FRR (Ubuntu)

apt-get update
apt-get install frr frr-pythontools -y

(2) FRR BGP Configuration Example

/etc/frr/bgpd.conf

router bgp 64513
 bgp router-id 10.10.20.1
 neighbor 10.10.20.101 remote-as 64512
 neighbor 10.10.20.102 remote-as 64512

 address-family ipv4 unicast
  network 10.10.20.0/24
  neighbor 10.10.20.101 activate
  neighbor 10.10.20.102 activate
 exit-address-family

• 64513: ASN for the FRR router
• 10.10.20.101, 10.10.20.102: Kubernetes node IPs running MetalLB BGP peers
• 10.10.20.0/24: network to advertise

(3) Verify BGP Session

vtysh -c "show bgp summary"

If the session is established, MetalLB and FRR are successfully peering over BGP.

7. Operational Considerations

• PoC / Small-scale → Start with L2 mode for simplicity.
• Production / Large-scale → Use BGP mode for reliability and scalability.
• Limited network team access → Start with L2, migrate to BGP later.
• Lab / Training → Use FRR as a BGP router replacement.

8. Conclusion

• L2 mode is the easiest entry point into MetalLB.
• BGP mode is essential for production-grade, scalable deployments.
• FRR offers a powerful way to practice BGP or even serve as a software router in smaller environments.

In short:

• Quick start → L2
• Reliable production → BGP
• Lab and testing → FRR

This combination gives you flexibility across different environments.