user-guide: add gateway-failover documentation #259
+111
−2
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,104 @@ | ||||||||||||||||||||||||||
| # Gateway fail-over and redundancy | ||||||||||||||||||||||||||
| ## Overview | ||||||||||||||||||||||||||
| When VPC *peerings* are configured to use a gateway, the latter is responsible for the delivery of the traffic exchanged between the VPCs on each side of the peering, fabric-wide. Failures in a gateway, its interconnects, or neighboring nodes can cause connectivity interruptions. Much as link protection is accomplished with interconnect redundancy, gateway failures are mitigated by deploying additional gateways. When more than one gateway is deployed in a Hedgehog Fabric, flexible fail-over strategies are possible to minimize service interruptions, as explained next. | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| !!! note | ||||||||||||||||||||||||||
| Gateway "failures" do not necessarily refer to physical issues with the gateway device, its cabling or its software. Any condition that prevents a gateway from being reachable by fabric edges (such as multiple neighbor failures or their cabling) falls in this category. The fail-over strategy of the Hedgehog Fabric is designed to protect against those as well. | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| ### Gateway groups | ||||||||||||||||||||||||||
| Gateway fail-over strategies build on the concept of *gateway groups*. A gateway group is a configurable, named set of gateways ranked by priority, such that the member gateway with highest priority is preferred over the rest, provided, of course, that it is operational. There is no limit on the number of groups that can be defined, and gateways can be members of as many groups as desired. | ||||||||||||||||||||||||||
| However: | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| !!! warning | ||||||||||||||||||||||||||
| Currently, group sizes are limited to 10 members at the most. Such a limit may only affect in case you have more than 10 gateways deployed on the same fabric. | ||||||||||||||||||||||||||
|
There was a problem hiding this comment. Minor grammar: “at the most” should be “at most”.
Suggested change
|
||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| Declaring gateway groups is done by means of the `GatewayGroup` object. The following sample snippet shows the declaration of a group called **group-1**. | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| ```yaml | ||||||||||||||||||||||||||
| apiVersion: gateway.githedgehog.com/v1alpha1 | ||||||||||||||||||||||||||
| items: | ||||||||||||||||||||||||||
| - apiVersion: gateway.githedgehog.com/v1alpha1 | ||||||||||||||||||||||||||
| kind: GatewayGroup | ||||||||||||||||||||||||||
| metadata: | ||||||||||||||||||||||||||
| name: group-1 | ||||||||||||||||||||||||||
| namespace: default | ||||||||||||||||||||||||||
| spec: {} | ||||||||||||||||||||||||||
|
Comment on lines
+19
to
+25
There was a problem hiding this comment. The GatewayGroup YAML example is not a valid single-object manifest: it uses a top-level
Suggested change
|
||||||||||||||||||||||||||
| ``` | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| The set of gateways that belong to a *GatewayGroup* is not explicitly enumerated in the GatewayGroup. Instead, group membership is declared in the definition of the Gateway objects by referring to groups by their name and indicating a certain priority. The following snippet shows how **gateway-1** is a member of group **group-1** with priority 10. | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| !!! note inline end | ||||||||||||||||||||||||||
| A `default` gateway group is implicitly created by the fabric and all gateways are members of it. Therefore, a gateway always belongs to at least one group even if no groups are explicitly declared. | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| ```yaml | ||||||||||||||||||||||||||
| apiVersion: gateway.githedgehog.com/v1alpha1 | ||||||||||||||||||||||||||
| kind: Gateway | ||||||||||||||||||||||||||
| metadata: | ||||||||||||||||||||||||||
| name: gateway-1 | ||||||||||||||||||||||||||
| [..] | ||||||||||||||||||||||||||
| spec: | ||||||||||||||||||||||||||
| asn: 65534 | ||||||||||||||||||||||||||
| groups: | ||||||||||||||||||||||||||
| - name: group-1 | ||||||||||||||||||||||||||
| priority: 10 | ||||||||||||||||||||||||||
| [..] | ||||||||||||||||||||||||||
| ``` | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| !!! note | ||||||||||||||||||||||||||
| The priority assigned to a gateway in a group has no significance in absolute terms. Configuring three gateways in the same group with priorities 300, 200 and 100 has the same effect as configuring them with priorities 51, 29 and 3. | ||||||||||||||||||||||||||
|
There was a problem hiding this comment. Several admonition bodies are indented with a tab character. MkDocs/Material admonitions require consistent space indentation; tabs can cause the admonition content to render as a code block or not be associated with the admonition at all. Replace the leading tab with 4 spaces in these blocks.
Suggested change
|
||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| Gateways implement services that are, in many cases, stateful. To correctly handle flows, the packets in the forward and reverse direction should be processed by the same gateway. The Hedgehog Fabric fail-over strategy is such that only one gateway handles a particular flow at any point in time. Gateway group priorities help to ensure that edge devices participating in a VPC peering select the same gateway. In future releases, it may be possible to balance the traffic of a single VPC peering over multiple gateways. | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| !!! note | ||||||||||||||||||||||||||
| Since group membership priorities are specified in the gateways themselves (instead of the `GatewayGroup`s), with many groups and gateways, two or more gateways may end up being assigned the same priority in a given group. The fabric will not reject such a configuration: despite having the same priorities, only one of the gateways will be the preferred; the first when ordering the gateways within the group alphabetically by name. This tie-breaking criteria is implemented by all gateways so that only one gateway per group is selected consistently across the fabric. | ||||||||||||||||||||||||||
|
Member
There was a problem hiding this comment. I think we should clarify here that it applies to when prio are the same, but ok for me
Contributor
Author
There was a problem hiding this comment.
Sorry @Frostman . I don't understand your point. Isn't it clear that we're talking about the case when you have two or more gateways with the same priority (and it is higher than the rest)? ... two or more gateways may end up being assigned the same priority in a given group... ... despite having the same priorities, only one of the gateways will be the preferred; the first when ordering the gateways within the group alphabetically by name There was a problem hiding this comment. Minor grammar: “This tie-breaking criteria is …” is ungrammatical (either use singular “criterion” or plural “criteria are”).
Suggested change
|
||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| ## Using gateway groups: sample setups | ||||||||||||||||||||||||||
| ### Minimal, default fail-over setup | ||||||||||||||||||||||||||
| The simplest fail-over setup consists of two gateways and a single gateway group. In this setup, one of the gateways is the preferred one and all of the traffic exchanged over all VPC peerings is serviced by that gateway. In the event that it fails, the other gateway takes over. This strategy is commonly called Active-Backup in many redundant systems. | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| !!! note | ||||||||||||||||||||||||||
| Since there always exists a `default` gateway group (containing all of the gateways), this Active-Backup behavior is (for any number of gateways) the default when no additional configuration is provided. | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| ### Customizing fail-over setups: traffic mapping to gateway groups | ||||||||||||||||||||||||||
| With the previous setup, one of the two gateways remains idle, which can be sub-optimal and under-utilize resources. In order to overcome this, VPC peerings can be specified with a `GatewayGroup` to indicate the name of the gateway group that should serve the traffic for that peering, as shown in the following snippet. | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| !!! info inline end | ||||||||||||||||||||||||||
|
Member
There was a problem hiding this comment. Why |
||||||||||||||||||||||||||
| A VPC peering always has a single `GatewayGroup`: if not explicitly set, the system automatically assigns it the `default` group. This is the reason why the behavior without any configuration is Active-Backup. | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| ```yaml | ||||||||||||||||||||||||||
| apiVersion: gateway.githedgehog.com/v1alpha1 | ||||||||||||||||||||||||||
| kind: Peering | ||||||||||||||||||||||||||
| metadata: | ||||||||||||||||||||||||||
| name: vpc-1--vpc-2 | ||||||||||||||||||||||||||
| namespace: default | ||||||||||||||||||||||||||
| spec: | ||||||||||||||||||||||||||
| peering: | ||||||||||||||||||||||||||
| gatewayGroup: group-1 | ||||||||||||||||||||||||||
| vpc-1: | ||||||||||||||||||||||||||
| expose: [..] | ||||||||||||||||||||||||||
| vpc-2: | ||||||||||||||||||||||||||
| expose: [..] | ||||||||||||||||||||||||||
| ``` | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| We refer to this as "_mapping_ the peering" to the gateway group. | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| The possibility of creating multiple gateway groups (with distinct gateways and priorities), and of mapping VPC peerings individually to them provides greater flexibility and advantages. For instance, consider a fabric with 3 gateways deployed G1, G2 and G3, and take the case of a VPC _vpc-1_ that peers with N other VPCs, _vpc-n_ n=2..N. Three groups could be defined, each containing the 3 gateways but with distinct priorities such that a distinct gateway would be the preferred in each group. By mapping the N-1 peerings of _vpc-1_ to the 3 groups, the traffic exchanged by _vpc-1_ would use distinct gateways to reach each of the VPCs it peered with, while guaranteeing that each flow exchanged be handled by a single gateway. By mapping the peerings the other VPCs may have, this type of configuration would allow balancing all of the load among the three gateways and, in turn, the links and fabric nodes these were connected to. Plus, in the event of a failure, only the peerings mapped to the failed gateway may be momentarily affected and be re-routed. | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| !!! note | ||||||||||||||||||||||||||
| One consequence of mapping a peering to a non-default `GatewayGroup` is that any gateway that is not a member of that group will not be used to serve the traffic for that peering, even if all gateways in that group become unavailable. | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| !!! tip | ||||||||||||||||||||||||||
| Gateway groups and the peering mappings can be handy for other purposes. For instance, removing a gateway from a group allows pulling the traffic of all peerings mapped to that group out of that gateway. Or, by adjusting member priorities, traffic can be re-mapped without changing the peering mappings to groups. | ||||||||||||||||||||||||||
|
Comment on lines
+88
to
+91
There was a problem hiding this comment. These admonition bodies are also tab-indented; use spaces to ensure the content is rendered as part of the admonition (and not as a code block).
Suggested change
|
||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| ## Fail-over under the hood and recommendations | ||||||||||||||||||||||||||
| The gateway fail-over strategy in the Hedgehog Fabric is implemented in a distributed manner. Gateways announce VPC peering prefixes with the specified priorities, while edge nodes (such as leaf switches) select the gateway that handles each packet based on the destination and the priorities. If a VPC peering refers to a group that has K members, the edge devices participating in the VPC will have K BGP routes (one per gateway) for each of the peering destinations. However, only one of those routes will be active at any point in time; the one advertised by (and pointing to) the preferred active gateway. | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| Because edge devices perform the fail-over when a gateway ceases to be reachable, downtime in case of failure depends on how quickly those devices reckon the anomaly. To minimize the detection time (and the volume of traffic blackholed), it is recommended to enable BFD on gateway links in order to expedite the propagation of failures. | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| !!! note "Takeaways and configuration summary" | ||||||||||||||||||||||||||
Fredi-raspall marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||||||||||||||
| Redundancy works out of the box in an Active-Backup fashion. To customize the behavior, you can: | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| 1. Declare gateway groups (`GatewayGroup` objects) depending on the number of gateways available. | ||||||||||||||||||||||||||
| 2. Assign gateways to those groups, with suitable priorities. The recommendation is to assign a high priority to each gateway in at least one of the groups so that load is shared evenly among all gateways. | ||||||||||||||||||||||||||
| 3. Map VPC peerings to the groups defined. | ||||||||||||||||||||||||||
| 4. Enable BFD on gateway links. | ||||||||||||||||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The new title is plural (“Adding Gateways…”), but the opening sentence still describes adding a single gateway node. Consider aligning the wording (either keep the title singular or update the intro to cover adding one or more gateways) to avoid confusing readers.