Type-2 interconnects are fun, but mtu issues are not. When faced with migrating subnets  from one MPLS cloud to another -different AS numbers you see- the three common inter-AS types were taken off the shelf and dusted off…

A quick refresh for all:

1. PE physical link(s), ties one VRF to another using one subnet for each VRF. Commonly deployed using vlans across a DOT1q trunk (anyone still using ISL? ok, ok a tagged trunk but don’t tell me I’m not allowed to say “trunk”). Possible to use routing protocols between each individual VRF, most commonly staticly routed as it’s the safest model, the other side isn’t trusted…
2. PE MP-eBGP, ties the two MPLS clouds together exchanging VPN labels. BGP can be used to exchange the vpn labels, no need for tag-switching of IP traffic. Requires the inter- link to be known in both IGPs as the next-hop changes, only the vpn label is preserved end-to-end.
3. P tag-switching, joins the two domains together at the lowest level. The inter-link can be between P devices all labels are preserved end-to-end. Even the IGPs must have ‘full’ connectivity, of course I mean PE-to-PE…

Type-2 seemed to make most sense due to the amount of vrfs involved, no need to go for type-3 and type-1 would add way too much complexity to be able to support a multi month long migration.

Now for the MTU issue. The old MPLS cloud uses 3600’s as PE’s in the relevant sites and the new network uses 7600’s. Admittedly a bit slow on the ball -mind you I was not the original designer- we implemented an mtu of 1536 on the sup720 GE based infrastructure links on the new network. However the vpn label on the 4 FE inter-AS/MP-eBGP links was forgotten untill a service using TCP keepalives was partially migrated. Sessions flapped due to the keepalives being 1500 bytes in size and having their df-bit set. I wonder why they used TCP but as of yet I’m left guessing.

The moment we noticed the error of our ways we found that IOS threw us a few curve balls:

1. A 3600 running 12.3 doesn’t accept anything over 1500 as mtu on a FastEthernet interface
2. A 7600 with 6748-GE (LAN interfaces) doesn’t allow a larger than 1500 mtu size either (routed interface)

The short answers are the following:

1. On the 3600: tag switching mtu 1508
2. On the 7600: mpls mtu 1508

fyi, 1504 would have done fine as an mpls label is just 4 bytes we just threw in another 4 bytes for luck 🙂

Some links to back things up:

• Cisco MTU config for 7600 12.2SR & layer3 ports
• Cisco IOS hints and tricks: The tale of three MTUs
• You’ll have to take my word for this quote from Cisco: “I can confirm that using tag-switching mtu at 1508 on the 3600 should resolve the issue from that side” (despite not being able to set the interface mtu higher than 1500)

Command to verity mpls mtu:

sh mpls [intf] det

Tags: 3600,7600,cisco,IP,linkedin,MP-eBGP,mpls,mtu
Categories: Networking
Comments Off on Inter-AS MPLS and MTU