eBGP L2VPN EVPN route are not propagate by spine when NextHop is the same as the eBGP peer
shujaaa opened this issue · comments
Description
This takes place in a bgp/evpn vxlan fabric with spine-leaf architecture and multiple-as e-bgp (each leaf and the spine have different AS).
When a leaf uses the same loopback address for bgp evpn peering AND vxlan next hop, then the spine does not propagate the reiceived T2/T3/T5 routes to the other leafs.
When the leaf uses a separate loopback for eBGP peering and for VxLAN interface source (and nextHop) then the routes are correctly propagated by the spine.
Version
FRRouting 8.4.3 (leaf-2) on Linux(5.10.0-cl-1-amd64).
How to reproduce
Leaf-2 is cumulus/FRR swp1 goes to spine-1
Current configuration:
!
frr version 8.4.3
frr defaults datacenter
hostname leaf-2
log syslog informational
service integrated-vtysh-config
!
router bgp 65106
bgp router-id 172.30.0.6
neighbor underlay peer-group
neighbor underlay remote-as external
neighbor underlay advertisement-interval 0
neighbor underlay timers 3 9
neighbor underlay timers connect 10
neighbor swp1 interface peer-group underlay
neighbor swp1 advertisement-interval 0
neighbor swp1 timers 3 9
neighbor swp1 timers connect 10
!
address-family ipv4 unicast
redistribute connected
maximum-paths 64
maximum-paths ibgp 64
exit-address-family
!
address-family l2vpn evpn
neighbor underlay activate
advertise-all-vni
vni 165
rd 172.30.0.6:165
route-target import 11:165
route-target export 11:165
exit-vni
advertise-svi-ip
exit-address-family
exit
!
Spine is cumulus/FRR, swp6 goes to leaf-2
frr version 8.4.3
frr defaults datacenter
hostname spine-1
log syslog informational
service integrated-vtysh-config
!
router bgp 65100
bgp router-id 172.30.0.3
neighbor underlay peer-group
neighbor underlay remote-as external
neighbor underlay advertisement-interval 0
neighbor underlay timers 3 9
neighbor underlay timers connect 10
neighbor swp6 interface peer-group underlay
neighbor swp6 advertisement-interval 0
neighbor swp6 timers 3 9
neighbor swp6 timers connect 10
neighbor 172.30.0.7 peer-group underlay
neighbor 172.30.0.7 update-source 172.30.0.3
neighbor 172.30.0.7 advertisement-interval 0
neighbor 172.30.0.7 timers 3 9
neighbor 172.30.0.7 timers connect 10
neighbor 172.32.0.3 peer-group underlay
neighbor 172.32.0.3 advertisement-interval 0
neighbor 172.32.0.3 timers 3 9
neighbor 172.32.0.3 timers connect 10
!
address-family ipv4 unicast
redistribute connected
maximum-paths 64
maximum-paths ibgp 64
exit-address-family
!
address-family l2vpn evpn
neighbor underlay activate
advertise-all-vni
exit-address-family
exit
!
the next leaf is runnning on an HPE Aruba switch in our example but can be reproduced with another cumulus/FRR machine, the key aspect is whether or not the vxlan source address is the same as the bgp-evpn neighbor address.
evpn
vlan 165
rd 172.30.0.7:165
route-target export 11:165
route-target import 11:165
interface loopback 0
ip address 172.30.0.7/32
interface vlan 165
vrf attach LR1
ip address 10.162.0.53/20
interface vxlan 1
source ip 172.30.0.7
no shutdown
vni 2
vlan 2
vni 165
vlan 165
router bgp 65107
bgp router-id 172.32.0.3
bgp deterministic-med
neighbor 172.30.0.3 remote-as 65100
neighbor 172.30.0.3 ebgp-multihop 5
neighbor 172.30.0.3 update-source 172.30.0.7
neighbor 172.32.0.1 remote-as 65100
address-family ipv4 unicast
neighbor 172.32.0.1 activate
redistribute connected
redistribute local loopback
exit-address-family
address-family l2vpn evpn
neighbor 172.30.0.3 send-community extended
neighbor 172.30.0.3 activate
exit-address-family
!
"show bgp l2vpn evpn" on the spine will display the routes sent by 172.30.0.7. The same command on leaf-2 does not.
when looking at routes received/advertised, spine-1 will show that it does not advertise 172.30.0.7 routes to leaf-2.
Expected behavior
All routes from neighbor 172.30.0.7 should be advertised to leaf-2
When changing the 172.30.0.7 config to the following
interface loopback 0
ip address 172.30.0.7/32
interface loopback 1
ip address 172.31.0.4/32
interface vxlan 1
source ip 172.31.0.4
It works as expected.
No perceivable difference in the routes when changing the next hop IP address.
Expected when it works (with a different next hop)
leaf-2# show bgp l2vpn evpn rd 172.30.0.7:165
BGP table version is 37, local router ID is 172.30.0.6
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-1 prefix: [1]:[EthTag]:[ESI]:[IPlen]:[VTEP-IP]:[Frag-id]
EVPN type-2 prefix: 2:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: 3:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 172.30.0.7:165
- 172.31.0.4 (leaf-1)
0 65105 65100 65107 ?
RT:11:165 ET:8
*> 172.31.0.4 (spine-1)
0 65100 65107 ?
RT:11:165 ET:8 - 172.31.0.4 (leaf-1)
0 65105 65100 65107 ?
RT:11:165 ET:8
*> 172.31.0.4 (spine-1)
0 65100 65107 ?
RT:11:165 ET:8
Displayed 2 out of 4 total prefixes
Actual behavior
leaf-2# show bgp l2vpn evpn rd 172.30.0.7:165
No prefixes displayed, 0 exist
Additional context
No response
Checklist
- I have searched the open issues for this bug.
- I have not included sensitive information in this report.
8.4.3 is an unsupported version anymore, can you test recent versions (9.1, 10.0)?