Cisco SD-WAN Lab Part 2 - TLOC Extensions and VRRP

Following up on the last article, I´d like to talk about TLOC Extensions. Here´s a reminder of the topology:

We will focus on Site 6:

What are TLOC Extensions? Site 6 in my topology has two routers installed for redundancy, in a scenario where you could ignore circuit costs, you would have two or more circuits used for transport per Router. But what if you could only afford two circuits? You would have to connect one circuit to each of your routers, and to use transport one, the traffic would flow to Router 1, and to use transport 2 the traffic would flow to Router 2, right? Well, not if you use TLOC extensions.

TLOC extensions allow you to extend a TLOC across an interface. In my example there are two links interconnecting both vEdges, "Site_6_NET" is extending It´s Internet circuit via interface ge0/1 to "Site_6_MPLS" vEdge, and "Site_6_MPLS" is also extending it´s MPLS circuit via interface ge0/2 to "Site_6_NET".

How does it work? For the vEdge connecting to the TLOC Extension the configuration is completely transparent, It's just like connecting to a circuit provided by the ISP. The Router that is extending the TLOC is the one with the extra configuration, stating that the interface is an extension of another interface, that is connected to a circuit. Let's look at each of the Extensions.

"Site_6_NET" Extension, stating the interface that is going to be extended:

 interface ge0/1

  ip address 172.30.0.1/30

  tloc-extension ge0/0

  no shutdown
        

Configuration on "Site_6_MPLS", as you can see it´s the normal config to connect to a circuit:

 interface ge0/1

  description TLOC_EXT_to_public-internet

  ip address 172.30.0.2/30

  tunnel-interface

   encapsulation ipsec

   color public-internet

   no allow-service bgp

   allow-service dhcp

   allow-service dns

   allow-service icmp

   no allow-service sshd

   no allow-service netconf

   no allow-service ntp

   no allow-service ospf

   no allow-service stun

   allow-service https

        

"Site_6_MPLS" Extension:

 interface ge0/2

  ip address 172.30.0.6/30

  tloc-extension ge0/0

  no shutdown

        

"Site_6_NET" usual config to a transport interface:

interface ge0/2

  ip address 172.30.0.5/30

  tunnel-interface

   encapsulation ipsec

   color mpls

   no allow-service bgp

   allow-service dhcp

   allow-service dns

   allow-service icmp

   no allow-service sshd

   no allow-service netconf

   no allow-service ntp

   no allow-service ospf

   no allow-service stun

   allow-service https

        

Easy and self-explanatory once you look at the config right?

Another thing to take into consideration is the addressing used for these interconnections. I´m using private IP´s with a /30 subnet mask, but we need to think about the underlay for a bit.

For instance, as Site_6_MPLS is trying to build a tunnel from interface ge0/1 which is now his public-internet color, It´s source is going to be 172.30.0.2 and that´s not routable on the public internet. The solution is to enable NAT on vEdge Site_6_NET on the interface connecting to the ISP:

vpn 0

 interface ge0/0

  description ***Circuito_NET***

  ip address 25.12.10.2/30

  ipv6 dhcp-client

  nat   ←-----

  !

        

(...)
        

Now when Site_6_MPLS is trying to build a tunnel from the public-internet color, It´s source address is going to be 25.12.10.2, the connectivity issue is solved.

We also need to do the same for the MPLS extension to Site_6_NET, right now the MPLS PE does not know about the 172.30.0.4/30 subnet. We could again use NAT, but since we are talking about private networks and MPLS circuits we could use a static route on the PE, or use BGP to exchange routes between the PE and vEdge Site_6_MPLS. Let's use BGP for fun. In this case we´re running BGP on VPN 0 (transport), so we´re using it on the underlay like on a tradicional MPLS deployment:

Site_6_MPLS# sh run vpn 0               

vpn 0

 router

  bgp 65006

   address-family ipv4-unicast

    network 172.30.0.4/30

   !

   neighbor 192.168.100.17

    no shutdown

    remote-as 65000

        

Don´t forget to allow-service bgp, It's disabled by default on interfaces that belong to VPN 0:

 interface ge0/0

  description Circuito_WAN_MPLS

  ip address 192.168.100.18/30

  ipv6 dhcp-client

  tunnel-interface

   encapsulation ipsec

   color mpls

   allow-service bgp  ←--------

   allow-service dhcp

   allow-service dns

   allow-service icmp

   no allow-service sshd

   no allow-service netconf

   no allow-service ntp

   no allow-service ospf

   no allow-service stun

   allow-service https

        

The ISP´s PE now knows about the subnet:

MPLS#sh ip bgp

BGP table version is 2, local router ID is 192.168.100.17

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 

              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 

              x best-external, a additional-path, c RIB-compressed, 

              t secondary path, 

Origin codes: i - IGP, e - EGP, ? - incomplete

RPKI validation codes: V valid, I invalid, N Not found




     Network          Next Hop            Metric LocPrf Weight Path

 *>   172.30.0.4/30    192.168.100.18           0             0 65006 i

        

At this point we built our TLOC Extensions and resolved connectivity issues, so we should have a full mesh to Site 6 at this point (as long as we are allowing a full mesh from a control plane point of view).

Let's check our BFD sessions to Site 6 from another site´s perspective:

As expected we have a full mesh!

Note that from "vEdge_Site_3" perspective for the remote TLOC color MPLS we have two destination IP´s 192.168.100.18 wich it the MPLS circuit at site 6 used by "Site_6_MPLS", and 172.30.0.5 wich is the TLOC extension for "Site_6_NET".

As for the remote TLOC color public-internet the destination IP is always 25.12.10.2 because we are using NAT.

So far we covered the transport VPN 0 for site 6, but how do we apply redundancy to the service side VPN´s? If we go for a Layer 3 design we can use OSPF and route to the CORE Switch at that side, but if we want to stick to a Layer 3 design we need to use a First Hop Redundancy Protocol. The vEdge´s support VRRP, the configuration is very straightforward, if you have implemented HSRP or VRRP in the past there's not much to it, but in order to the failover to happen quickly we need to track a route or route´s that we receive via OMP, this is the config:

Site_6_MPLS# show running-config policy 

policy

 lists

  prefix-list VRRP

   ip-prefix 172.16.2.0/24

   ip-prefix 172.16.7.0/24





Site_6_MPLS# sh run vpn 10

vpn 10

 interface ge0/3

  ip address 172.16.6.252/24

  no shutdown

  vrrp 1

   priority          120

   timer             2

   track-prefix-list VRRP

   ipv4 172.16.6.254

        

Show VRRP VPN 10:

Site_6_MPLS# sh vrrp vpn 10

vrrp vpn 10

 interfaces ge0/3

  groups 1

   virtual-ip             172.16.6.254

   virtual-mac            00:00:5e:00:01:01

   priority               120

   vrrp-state             master

   omp-state              up

   advertisement-timer    2

   master-down-timer      6

   last-state-change-time 2021-08-14T18:26:21+00:00

   track-prefix-list      VRRP

   prefix-list-state      resolved

        

With the last command we can see that "Site_6_MPLS" vEdge is the master Router, and that the prefix list state is resolved, meaning the prefixes we are tracking are on the omp routing table:

Finally we can also see on vManage that if we simulate a flow for "Site_6_MPLS", that traffic is going to be load balanced across the available transports, although this is the master Router It's still going to be able to route traffic to the TLOC extension which is being provided by the Backup Router: