Thursday, May 5, 2011

How Multi is MP-BGP in IOS-XR?

This caught me on surprise. I had an impression that IOS-XR as an advanced operating system would support all kinds of multi-protocol transferability over BGP.

As it seems, there is an issue when transferring IPv6 prefixes over an IPv4 peering or IPv4 prefixes over an IPv6 peering. This happens for sure on ASR9k running latest 4.1.0, but i haven't verified it on the CRS yet.

IPv4 prefixes over IPv6 peering
This doesn't seem to be supported based on the available configuration options.
What is even more worrying, is that no other address family is supported too.

RP/0/RSP0/CPU0:ASR#conf t
RP/0/RSP0/CPU0:ASR(config)#router bgp 100
RP/0/RSP0/CPU0:ASR(config-bgp)#neighbor 2001::1:2:3
RP/0/RSP0/CPU0:ASR(config-bgp-nbr)#address-family ?
  ipv6  IPv6 Address Family

IPv6 prefixes over IPv4 peering
This is supported according to the configuration options, but it doesn't work.
Cisco also insists that this is definitely supported.

RP/0/RSP0/CPU0:ASR#conf t
RP/0/RSP0/CPU0:ASR(config)#router bgp 100
RP/0/RSP0/CPU0:ASR(config-bgp)#neighbor 10.11.254.37
RP/0/RSP0/CPU0:ASR(config-bgp-nbr)#address-family ?
  ipv4   IPv4 Address Family
  ipv6   IPv6 Address Family
  l2vpn  L2VPN Address Family
  vpnv4  VPNv4 Address Family
  vpnv6  VPNv6 Address Family

As soon as you enable the IPv6 address family under the IPv4 neighbor, the BGP session is dropped and it never comes up.

RP/0/RSP0/CPU0:ASR#sh bgp sum
BGP router identifier 10.11.254.38, local AS number 100
BGP generic scan interval 60 secs
BGP table state: Active
Table ID: 0xe0000000   RD version: 1
BGP main routing table version 1
BGP scan interval 60 secs

BGP is operating in STANDALONE mode.


Process       RcvTblVer   bRIB/RIB   LabelVer  ImportVer  SendTblVer  StandbyVer
Speaker               1          1          1          1           1           1

Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd
10.11.254.37      0  100        0       0        0    0    0 00:00:00 Idle

Also, debug shows that there are no tries of BGP to establish a session. It's like BGP gets disabled.

The only doc that refers such a limitation (in IOS-XR 3.3 for CRS) is the one in http://www.cisco.com/en/US/docs/ios_xr_sw/iosxr_r3.3/conversion/reference/guide/cn33main.html#wp1028960

A given address family is only supported with a neighbor whose address is from that address family. For instance, IPv4 neighbors support IPv4 unicast and multicast address families, and IPv6 neighbors support IPv6 unicast and multicast address families. However, you cannot exchange IPv6 routing information with an IPv4 neighbor and vice versa.

I searched all CCO for more information, but i didn't manage to find something useful. Does anyone have extra information to share? TAC is struggling (as usual) to find an answer...

Update #1
Cisco verified once more that this is a supported configuration. Arie Vayner (and later tac) proposed to add an IPv6 address to the interface being used as an IPv4 next-hop. Indeed, this solved the problem and the BGP session came up. But then it became even more interesting...

Two IPv6 prefixes are learned from the IPv4 neighbor. Next-hop is an IPv6 address.

RP/0/RSP0/CPU0:ASR#sh bgp ipv6 uni 
BGP router identifier 10.11.254.38, local AS number 100
BGP generic scan interval 60 secs
BGP table state: Active
Table ID: 0xe0800000   RD version: 5
BGP main routing table version 5
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale
Origin codes: i - IGP, e - EGP, ? - incomplete
Network            Next Hop            Metric LocPrf Weight Path
* i2001::1:2:3/128    2003::1:2:3              0    100      0 ?
* i2003::/64          2003::1:2:3              0    100      0 ?

Processed 2 prefixes, 2 paths

If i remove the IPv6 address from the interface that is being used as next-hop (the one i added before), then i automatically get an IPv6 prefix with an IPv4 next-hop!!!

RP/0/RSP0/CPU0:core-distr-kln-02#sh bgp ipv6 uni 
BGP router identifier 10.11.254.38, local AS number 100
BGP generic scan interval 60 secs
BGP table state: Active
Table ID: 0xe0800000   RD version: 6
BGP main routing table version 6
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale
Origin codes: i - IGP, e - EGP, ? - incomplete
Network            Next Hop            Metric LocPrf Weight Path
*>i2001::1:2:3/128    10.11.254.41             0    100      0 ?

Processed 1 prefixes, 1 paths

The BGP session stays up, until something happens that will reset it. Then it will stay down forever, as it was happening in the beginning.

I must say that i cannot endorse such an implementation. Using exactly the same configuration, you get different results, depending on the order of (un)configuring things. Also, i cannot understand why the establishment of an IPv4 BGP session that is going to negotiate IPv4/IPv6 address-family capabilities should depend on whether an IPv6 next-hop exists or not. That should be left for the NLRI exchange routine.

After all, RFC 4271 defines among others two error conditions for the NEXT_HOP attribute:

If the NEXT_HOP attribute field is syntactically incorrect, then the Error Subcode MUST be set to Invalid NEXT_HOP Attribute. The Data field MUST contain the incorrect attribute (type, length, and value). Syntactic correctness means that the NEXT_HOP attribute represents a valid IP host address.

If the NEXT_HOP attribute is semantically incorrect, the error SHOULD be logged, and the route SHOULD be ignored. In this case, a NOTIFICATION message SHOULD NOT be sent, and the connection SHOULD NOT be closed.



Update #2
After the developers got involved, we ended up with the following:

  1. In IOS-XR you need an IPv6 NH in order to activate the IPv6 AF for an IPv4 BGP session.
  2. If you don't have an IPv6 NH, then the IPv4 BGP session won't even come up.
  3. The above was done to protect against misconfiguration, because otherwise you would get a misleading v4 mapped v6 address as NH.
  4. If you have an IPv6 NH, then the IPv4 BGP session with the IPv6 AF will come up.
  5. If afterwards you remove the IPv6 NH, then the session deliberately remains up and you get a misleading v4 mapped v6 address as NH.
Cisco agreed (thx Xander) that the behavior in 3 and 5 contradict each other, so a short-term solution (update the documentation and print a warning message) got recorded in CSCtq26829.

10 comments:

  1. Michael Bonnett, Jr.05 May, 2011 22:01

    This may be a dumb question, but have you added the "address-family ipv4 unicast" and "address-family ipv6 unicast" commands under the bgp global config? Example below:

    router bgp 100
    address-family ipv4 unicast
    !
    address-family ipv6 unicast
    !
    neighbor 10.11.254.37
    remote-as 100
    address-family ipv4 unicast
    !
    address-family ipv6 unicast
    !

    ReplyDelete
  2. Yes, these were configured (otherwise you're not allowed to configure them under a neighbor).

    ReplyDelete
  3. Hi Man,
    On CRS-1, Could you please help to check this issue, because when i remove interface bundle-pos back to use interface pos i can't ping to neighbor with source loopback, such # ping 10.10.10.10 source 10.10.10.20 Type escape sequence to abort.
    Sending 5, 100-byte ICMP Echos to 10.10.10.20, timeout is 2 seconds:
    .....
    Success rate is 0 percent (0/5)

    Thank so much
    KversionMan

    ReplyDelete
  4. Hi I'm KversionMan again,
    Why i can't run L2VPN on interface-pos (i used C3-POS-16, E3-OC3-POS-8, E3-OC3-POS-4)
    now i run ios-xr version 3.8 and run MPLS&OSPS&BGP&IPv4&IPv6.

    ReplyDelete
  5. Hi,
    In your output, after configuring IPv6 addreses, you have
    * i2001::1:2:3/128 2003::1:2:3
    but still not valid next-hop.

    Is it displayed like 2003::1:2:3 (FE80:::: - local IPv6 address) in show bgp ipv6 unicast 2001::1:2:3/128?
    It seems it's a matter of order of operations in XR or IOS. If I de-configure and configure back IPv6 addresses on the link and put the route-policy to manually set the IPv6 address then it works. My test is done with IPv6 peering in VRF on XR.

    So, to summarize:
    XR, compared to IOS, does advertise an IPv6 next-hop for IPv6 prefixes to an IPv4 peer; it should work without the manually setting the next-hop in a route-policy.
    IOS however complains about the next-hop not beeing connected so you have to configure ebgp-multihop
    After it accepts the prefix IOS still complains about the next-hop beeing inaccessible (when the global IPv6 address is seen with an local IPv6 address).

    So, did you got it fully working besides the prefix exchange?

    Thanks,
    Edi

    ReplyDelete
  6. Edi,

    From my tests i have concluded that in order to have IPv6 prefixes exchanged over an IPv4 BGP peering in IOS-XR, you need to do:

    1) enable IPv6 on the interfaces that are used as NH
    or
    2) set manually the IPv6 NH through RPL

    It's obvious that packets won't pass through unless there is IPv6 connectivity end-to-end, but that's another story.

    ReplyDelete
  7. What are some of the benefits to using IPv6?

    ReplyDelete
    Replies
    1. If talking in general, then this is going to be a long discussion.
      If talking specifically, then this was just an experiment in order to verify transfer of IPv6 BGP table over an IPv4 session.

      Delete
  8. I came to this blog post after search a while on the internet, It helped my to TS an issue with a CRS-1 of one of hour customers.

    Thanks for the info :)

    ReplyDelete
  9. Read more on BGP

    http://www.kgpramod.com/go/bgp

    or

    http://www.kgpramod.com

    ReplyDelete

 
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Greece License.