Problem statement

Continuing our Inter-AS MPLS topic, today we will focus on troubleshooting LSP's towards non-/32 routes. Typical use case for this kind of a problem is OSPF announcing /32 prefix for a loopback that have /24 mask configured. This one is probably more of a CCIE lab scenario, however there are certain valid use cases for this. And one of them is Inter-AS MPLS Option B. In this kind of scenario, there is a transit link with probably /30 or even /31 mask (ARIN is exhausted, right?). Let's say VPNv4 session between providers runs directly over physical interface IP's (not loopbacks). When labels are announced for routes, next hops are /30 IP's and even though control plane looks "fine", data plane will broken.

Hint: In Classic IOS, this problems is solved automatically and as soon as BGP VPNv4 session comes up over transit link two things will be done in background:
  • "bgp mpls forwarding" command will be configured on interface to install BGP advertised labels in LFIB
  • Connected /32 route pointing to transit interface will be installed into RIB

Initial data

* Transit link is 150.0.0.0/31

* CE advertised ranges:

198.51.100.0/24 - advertised by neighbouring AS

14.14.14.14/32 - advertised by local to ASBR AS

Verification

PE

RP/0/0/CPU0:PE#show route vrf CUSTOMER_A
Mon Oct 5 14:59:06.338 UTC
---omitted for brevity---
B 14.14.14.14/32 [20/0] via 10.13.14.14, 4d23h
B 198.51.100.0/24 [200/0] via 11.11.11.11 (nexthop in vrf default), 2d21h

RP/0/0/CPU0:PE#show bgp vpnv4 uni rd 5.5.5.5:1 198.51.100.0/24
Mon Oct 5 15:02:11.965 UTC
BGP routing table entry for 198.51.100.0/24, Route Distinguisher: 5.5.5.5:1
Versions:
Process bRIB/RIB SendTblVer
Speaker 34 34
Last Modified: Oct 2 17:55:33.842 for 2d21h
Paths: (1 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer
65099 65003 65101
11.11.11.11 (metric 20) from 12.12.12.12 (11.11.11.11)
Received Label 24006
Origin IGP, localpref 100, valid, internal, best, group-best, import-candidate, not-in-vrf
Received Path ID 0, Local Path ID 1, version 34
Extended community: RT:65101:101
Originator: 11.11.11.11, Cluster list: 12.12.12.12

RP/0/0/CPU0:R13#show cef 11.11.11.11 detail
Mon Oct 5 15:04:09.267 UTC
11.11.11.11/32, version 14, internal 0x1000001 0x0 (ptr 0xa140c674) [1], 0x0 (0xa13d7830), 0xa28 (0xa156d190)
Updated Sep 30 18:09:09.131
local adjacency 10.12.13.12
Prefix Len 32, traffic index 0, precedence n/a, priority 3
gateway array (0xa12a0980) reference count 6, flags 0x68, source lsd (5), 1 backups
[3 type 5 flags 0x8081 (0xa1587320) ext 0x0 (0x0)]
LW-LDI[type=5, refc=3, ptr=0xa13d7830, sh-ldi=0xa1587320]
gateway array update type-time 1 Sep 30 13:44:19.599
LDI Update time Sep 30 13:44:19.599
LW-LDI-TS Sep 30 13:44:19.599
via 10.12.13.12, GigabitEthernet0/0/0/0.1213, 5 dependencies, weight 0, class 0 [flags 0x0]
path-idx 0 NHID 0x0 [0xa107d3a0 0x0]
next hop 10.12.13.12
local adjacency
local label 24001 labels imposed {24001}


Load distribution: 0 (refcount 3)

Hash OK Interface Address
0 Y GigabitEthernet0/0/0/0.1213 10.12.13.12

ASBR

RP/0/0/CPU0:R11#show mpls forwarding 
Mon Oct 5 15:04:25.530 UTC
Local Outgoing Prefix Outgoing Next Hop Bytes
Label Label or ID Interface Switched
------ ----------- ------------------ ------------ --------------- ------------
24004 24002 13.13.13.13:65002:14.14.14.14/32 \
13.13.13.13 371616
24005 Aggregate 150.0.0.0/31 default 0
24006 299856 5.5.5.5:1:198.51.100.0/24 \
150.0.0.0 303216

Someone experienced in IOS-XR would notice the problem immediately. Let's see detailed output:

RP/0/0/CPU0:R11#show mpls forwarding detail 
Mon Oct 5 15:08:22.634 UTC
Local Outgoing Prefix Outgoing Next Hop Bytes
Label Label or ID Interface Switched
------ ----------- ------------------ ------------ --------------- ------------
24006 299856 5.5.5.5:1:198.51.100.0/24 \
150.0.0.0 303216
Updated Oct 2 17:51:25.110
Path Flags: 0x6000 [ ]
MAC/Encaps: 0/0, MTU: 0
Label Stack (Top -> Bottom): { }
Packets Switched: 2916

Let's verify next hop:

RP/0/0/CPU0:R11#show route ipv4 150.0.0.0       
Mon Oct 5 15:10:38.315 UTC

Routing entry for 150.0.0.0/31
Known via "connected", distance 0, metric 0 (connected)
Installed Oct 2 17:33:12.435 for 2d21h
Routing Descriptor Blocks
directly connected, via GigabitEthernet0/0/0/1
Route metric is 0
No advertising protos.
RP/0/0/CPU0:R11#show mpls forwarding prefix ipv4 unicast 150.0.0.0/31
Mon Oct 5 15:12:10.369 UTC
Local Outgoing Prefix Outgoing Next Hop Bytes
Label Label or ID Interface Switched
------ ----------- ------------------ ------------ --------------- ------------
24005 Aggregate 150.0.0.0/31 default 0

Outgoing label "Aggregate" means that router will pop the whole label stack and do IP lookup.

And if we do debug, we see the following message:

RP/0/0/CPU0:Oct  5 16:50:19.095 : netio[309]: [mpls_switch:2818] Pkt Drop: mpls_switch: GigabitEthernet0_0_0_0.1112, No LFIB entry found for in_label 24006

It becomes even more confusing if you look back at LFIB table in one of the previous outputs, where you can see that outgoing label for 24006 actually exists and it is 299856.

The root cause of this problem is missing /32 route for BGP next hop.

Let's add this connected route manually and see what happens.

router static
address-family ipv4 unicast
150.0.0.0/32 GigabitEthernet0/0/0/1
!
!
RP/0/0/CPU0:ASBR#show mpls forwarding detail
24006  299856      5.5.5.5:1:198.51.100.0/24   \
Gi0/0/0/1 150.0.0.0 303840
Updated Oct 2 17:51:25.110
Path Flags: 0x6000 [ ]
Version: 30, Priority: 4
MAC/Encaps: 14/18, MTU: 1500
Label Stack (Top -> Bottom): { 299856 }
NHID: 0
Packets Switched: 2922

Now we see that outgoing interface was determined and label stack is no longer empty. As soon as /32 route was added, forwarding was restored.