Network Engineering, November 2020

BGP, Quick Reference Notes

BGP does not compute a shortest path. It applies policy to received paths, selects one per prefix, and advertises what policy permits, which makes the interesting behaviour configuration rather than algorithm.

Loop prevention differs by session type and this is the root of most of the design. External sessions use the path attribute, rejecting anything already containing the local autonomous system number. Internal sessions have no equivalent, so a route learned internally is never re-advertised internally.

The selection order

StepCriterionPrefers
1WeightHigher, local to one device
2Local preferenceHigher, carried across the domain
3Locally originatedLocal origination
4Path lengthShorter
5OriginInternal over external over incomplete
6Multi exit discriminatorLower, same neighbour only
7Session typeExternal over internal
8Metric to next hopLower interior cost
9AgeOldest external path
10Router identifierLower

Steps one and two are where deliberate policy belongs, because they are evaluated before anything topological and therefore override it. Steps nine and ten are tie breaks whose purpose is determinism rather than quality.

Step six is the one that misleads. The discriminator is only compared between paths from the same neighbouring autonomous system unless that restriction is explicitly disabled, so a lower value from a different provider is frequently never considered at all.

Attribute classes decide propagation

ClassExamplesBehaviour at a router that does not understand it
Well known mandatoryOrigin, path, next hopMust be present, must be understood
Well known discretionaryLocal preferenceUnderstood, may be absent
Optional transitiveCommunity, aggregatorPassed on unchanged
Optional non transitiveDiscriminator, originatorDiscarded

The last two rows explain propagation scope without any further rules. Communities survive across autonomous systems, which is what makes them usable as a signalling mechanism between operators, while the discriminator does not, which is why it only ever influences an immediate neighbour.

Next hop is not rewritten internally

A route learned externally and passed to an internal peer retains the external next hop address. If that address is not reachable through the interior protocol, the path is valid, unusable, and not installed.

The two remedies are to carry external links in the interior protocol, which enlarges its topology for no routing benefit, or to rewrite the next hop to the advertising router at the internal session, which is the standard approach and the reason the option exists.

Scaling the internal mesh

Because internal routes are not re-advertised internally, every speaker must peer with every other, which grows with the square of the count and becomes unmanageable in the tens.

Route reflection relaxes the rule for a designated device, which may re-advertise between clients while adding originator and cluster attributes to prevent loops. Confederations divide the domain into sub autonomous systems that behave externally as one, and are less common because the operational complexity is higher.

Reflection has a consequence worth designing around. A reflector advertises only its own best path, so clients see fewer paths than a full mesh would provide, which reduces diversity for fast reroute and can cause suboptimal exit selection. Additional path advertisement addresses it where the platform supports it.

Where control actually exists

Outbound traffic is straightforward to control, because local preference is applied on receipt and decides which learned path is used, and it propagates across the domain.

Inbound is the difficult direction, because the decision belongs to somebody else. Path prepending is blunt and often ignored by policy several hops away. The discriminator only influences a single adjacent autonomous system. Communities work where the provider publishes and honours them, which makes them the most reliable of a weak set.

The honest position is that inbound engineering is a request rather than a control, and designs that depend on it holding precisely are fragile.

Failure modes that are not obvious

Discriminator comparison is order dependent under some conditions, so the selected path can vary with the sequence in which paths arrived, producing an inconsistency between routers that is entirely legitimate and extremely confusing. Always comparing, or ignoring the attribute, resolves it.

Flap damping penalises unstable prefixes and, with historical default values, suppresses prefixes that flapped only briefly, which withdraws reachability for far longer than the original fault. It is either tuned to modern recommendations or left disabled.

Reflector placement matters because the interior metric to the next hop is a selection step, so a reflector positioned away from the traffic path can steer clients toward an exit that is optimal for the reflector and not for them.

Controls that belong on every session

A maximum prefix limit bounds the damage from a neighbour that leaks a full table, which is the most common cause of a router exhausting memory. Explicit inbound and outbound filtering means a policy error produces no announcement rather than an accidental one.

Origin validation checks an announcement against a signed record of which autonomous system is authorised to originate the prefix. It addresses accidental origination directly and does not validate the remainder of the path, which is the distinction worth stating rather than assuming.

Note: a session that establishes and exchanges nothing is usually policy rather than protocol. Confirm what was advertised and what was received before the sent and received tables are compared, since a filter denying everything looks identical to a neighbour with nothing to send.