eBGP multihop over loopbacks without session roulette

#How update‑source, multihop TTL, next‑hop handling, and GTSM interact when eBGP is moved to loopbacks

Introduction

External BGP (eBGP) exchanges NLRI between routers in different autonomous systems using a TCP session on port 179. The protocol relies on several key attributes—Router ID, Hold Time, Optional Parameters, and the NEXT_HOP attribute—to select the best path and install routes in the FIB.

A loopback interface provides a stable, always‑up IP address that is not tied to any physical port. Using a loopback for eBGP transport offers:

Session stability – The BGP session survives physical link flaps because the transport endpoint (source IP) does not change.
Load‑sharing and redundancy – Multiple parallel links can carry the traffic while the peer address remains constant, simplifying policy and monitoring.

Moving the BGP endpoint to a loopback, however, introduces dependencies on update‑source, multihop TTL, next‑hop handling, and the Generalized TTL Security Mechanism (GTSM). If these are not configured correctly, the session may flap or never reach ESTABLISHED even though basic IP reachability (e.g., ping) appears fine.

Update‑Source and Next‑Hop Handling

Update‑Source

The update‑source knob tells the BGP process which interface (or IP address) to use as the source IP for outgoing TCP packets. Without it, the router selects the primary IP of the outgoing physical interface that matches the route to the peer, based on the local RIB entry for the peer’s address.

Control‑plane trace (no update‑source)

BGP initiates a TCP OPEN. 2. The TCP stack queries the FIB for the destination peer address.
The FIB returns an outgoing interface (e.g., GigabitEthernet0/0/0).
The TCP source address is chosen from the addresses configured on that interface (usually the lowest‑numbered IP).

With update‑source configured – step 4 is overridden and the specified address (often a loopback) is used instead.

Next‑Hop Attribute

The NEXT_HOP attribute in a BGP UPDATE indicates the IP address that should be used as the next hop toward the advertised prefix. For eBGP, the default NEXT_HOP is the IP address of the BGP speaker that sent the UPDATE (i.e., the neighbor’s source IP). The receiving router must resolve this NEXT_HOP via its IGP (or static routes) to a reachable outgoing interface before installing the route in the FIB.

Control‑plane trace

Router A receives an UPDATE from Router B with NEXT_HOP = Router B’s BGP source IP.
Router A performs a recursive lookup: it searches its RIB for a route to that NEXT_HOP.
If a matching route exists, the corresponding outgoing interface and next‑hop MAC are programmed into the FIB.
If the lookup fails, the route is marked unreachable and is not installed, even though the BGP session may be ESTABLISHED.

Interaction Between update‑source and Next‑Hop

When the BGP transport endpoint is moved to a loopback, the source IP of the TCP packets (controlled by update‑source) often differs from the interface IP used for ordinary data traffic. This can cause two distinct problems:

Session establishment failure – If the neighbor is configured with a static peer address that does not match the source IP seen in the incoming TCP SYN, the neighbor rejects the connection (TCP reset or silent drop).
NEXT_HOP mismatch – The NEXT_HOP advertised by the peer will be the loopback address (if update‑source points to the loopback). Remote routers must have a route to that loopback via the IGP; otherwise NEXT_HOP resolution fails and the route is not installed. Thus, for a successful session we need:

update‑source (local)  ==  neighbor address (remote)
IGP route to peer’s loopback exists  ==>  NEXT_HOP reachable

Multihop TTL and GTSM

Multihop TTL

By default, eBGP packets are sent with an IP TTL of 1 (RFC 4271, §8.2), limiting eBGP to directly connected peers. To allow eBGP over multiple hops, the multihop feature (often configured via neighbor ebgp-multihop or ttl) increases the TTL value in the outgoing IP header so packets can traverse the required number of routers before expiring.

Control‑plane trace

BGP process constructs a TCP segment.
IP layer sets the TTL field to the value specified by the multihop setting (or to 255 if GTSM is enabled).
Each hop decrements TTL; if it reaches 0 before arriving at the peer, the packet is discarded and an ICMP Time‑Exceeded may be generated.
The peer never sees the TCP SYN, so the session stays in Idle/Active state.

Generalized TTL Security Mechanism (GTSM)

Defined in RFC 5082, GTSM protects BGP sessions from spoofed or forged packets by expecting that legitimate BGP packets arrive with an IP TTL of 255 (or 255 − expected hop count when multihop is used).

When GTSM is enabled on a BGP peer, the router installs an incoming‑packet filter that accepts only packets whose TTL matches the expected value. * For a directly connected eBGP peer, the expected TTL is 255 (sender sets TTL = 255, receiver sees no decrement).
For a multihop peer, the expected TTL is 255 − configured hop‑count. The sender must set TTL = 255 − actual hop count; the receiver will see TTL = 255 − hop‑count after decrements along the path. If the incoming TTL does not match the expected value, the packet is silently discarded and the BGP state machine does not advance.

GTSM Assumptions and Potential Issues GTSM assumes:

Symmetric path length – the number of hops from sender to receiver equals the number from receiver to sender.
Static TTL value – the sender always transmits with TTL = 255 (direct) or 255 − hop‑count (multihop).
No TTL‑altering devices in the path – devices that rewrite TTL (e.g., certain firewalls, load balancers, or MPLS nodes with TTL propagation disabled) break the assumption.

When these assumptions are violated, legitimate BGP packets are dropped, causing the session to flap or never reach ESTABLISHED. Common misconfigurations include:

Forgetting to disable GTSM when using a non‑standard TTL (e.g., setting ebgp-multihop 2 but leaving ttl-security enabled).
Using a loopback as update‑source while the underlying physical path has a different hop count than anticipated, leading to a TTL mismatch.
Enabling GTSM on only one side of the session; the side without GTSM accepts any TTL, but the side with GTSM drops the return traffic, causing asymmetric session state.

Moving eBGP to Loopbacks

Motivations

Resilience to link flaps – the BGP transport endpoint stays constant as long as an alternate IGP path to the loopback exists.
Simplified policy – ACLs, route‑maps, and QoS can reference a single peer address rather than multiple physical IPs.
Load‑balancing over multiple links – ECMP or unequal‑cost load‑balancing in the IGP distributes traffic while the BGP peer address remains unchanged. * Facilitates route‑reflector and confederation designs – loopbacks provide stable identifiers for intra‑cluster communication.

Configuration Examples

The examples below assume AS 65001 (local) peering with AS 65002 (remote) using loopback 0 addresses 10.0.0.1/32 and 10.0.0.2/32 respectively. The physical path consists of two hops (local → transit → remote). #### Cisco IOS / IOS‑XE

! Local router (AS 65001)
interface Loopback0
 ip address 10.0.0.1 255.255.255.255!
router bgp 65001
 neighbor 10.0.0.2 remote-as 65002 neighbor 10.0.0.2 update-source Loopback0   ! source TCP from loopback
 neighbor 10.0.0.2 ebgp-multihop 2          ! allow 2 hops neighbor 10.0.0.2 ttl-security hops 2      ! enable GTSM for 2‑hop path
!
! Ensure IGP (OSPF/IS‑IS) advertises the loopbackrouter ospf 1
 network 10.0.0.1 0.0.0.0 area 0

Juniper Junos

# Local router (AS 65001)
set interfaces lo0 unit 0 family inet address 10.0.0.1/32
set protocols bgp group ext-peers type external
set protocols bgp group ext-peers peer-as 65002
set protocols bgp group ext-peers neighbor 10.0.0.2 local-address 10.0.0.1
set protocols bgp group ext-peers neighbor 10.0.0.2 multihop ttl 2
set protocols bgp group ext-peers neighbor 10.0.0.2 multihop
set protocols bgp group ext-peers neighbor 10.0.0.2 ttl-security
# OSPF to advertise loopback
set protocols ospf area 0.0.0.0 interface lo0.0

FRR (Free Range Routing) ```bash

! Local router (AS 65001) interface lo ip address 10.0.0.1/32 ! router bgp 65001 neighbor 10.0.0.2 remote-as 65002 neighbor 10.0.0.2 update-source lo neighbor 10.0.0.2 ebgp-multihop 2 neighbor 10.0.0.2 ttl-security hops 2 ! ! OSPF example router ospf ospf router-id 10.0.0.1 network 10.0.0.1/32 area 0


**Key points**  

* `update-source` (or `local-address`) forces the TCP source to be the loopback.  
* `ebgp-multihop` (or `multihop ttl`) raises the outgoing TTL to allow the required number of hops.  * `ttl-security` (or `ttl-security hops`) enables GTSM and tells the router the expected hop count; the router will then only accept incoming packets with TTL = 255 − hop‑count.  

### Verification (Cisco example)  

```bash
show bgp neighbors 10.0.0.2

Expected output snippet:

BGP neighbor is 10.0.0.2, remote AS 65002, external link
  BGP version 4, remote router ID 10.0.0.2
  BGP state = Established, up for 00:15:23
  Last read 00:00:02, last write 00:00:02
  Hold time is 180, keepalive interval is 60
  Received 1200 packets, sent 1180 packets
  ...
  Local host: 10.0.0.1, Local port: 179
  Foreign host: 10.0.0.2, Foreign port: 54321```

---

## Conclusion  

Moving eBGP to loopback interfaces improves stability and simplifies policy, but it hinges on the correct interplay of four mechanisms:

1. **update‑source** – must match the neighbor’s configured peer address.  
2. **next‑hop resolution** – the advertised loopback must be reachable via the IGP.  
3. **multihop TTL** – must be set high enough for the actual path length.  
4. **GTSM** – must be enabled consistently on both sides with the correct hop‑count expectation.  

Misconfiguring any of these can leave the session stuck in Idle/Active or cause it to flap, even when basic IP reachability (ping) appears successful. Proper verification—checking BGP state, confirming IGP routes to loopbacks, and validating TTL/GTSM settings—is essential for a reliable eBGP-over‑loopback deployment.