Congestion-Aware Adaptive Flow Table Switching for ECMP

Introduction Equal-Cost Multi-Path (ECMP) routing is a widely deployed load balancing technology in data center networks . Traditional ECMP distributes traffic across multiple equal-cost paths by hashing packet header fields, typically the five-tuple. To ensure packet ordering within a flow, the mapping between a flow and its egress port typically remains unchanged throughout the flow's lifetime. However, this static mapping approach exhibits significant limitations in the following scenarios: Traffic Surge Scenario: Network traffic is highly dynamic and may cause sudden increases on certain ports. The flow table mapping cannot be adjusted in time to alleviate congestion. Multicast Traffic Scenario: The replication characteristics of multicast traffic may cause it to concentrate on a small number of ports, exacerbating load imbalance. Existing congestion response strategies typically adopt two extreme approaches: either no switching (maintaining the original mapping until flow aging) or full switching (simultaneously migrating all flows on a congested port). The former cannot respond to congestion in a timely manner, while the latter may cause congestion transfer and resource fluctuations. This document defines a congestion-aware adaptive flow table switching mechanism that quantifies port congestion levels and progressively adjusts flow table mappings to achieve dynamic optimization of load balancing while preserving packet ordering.

Terminology and Conventions

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Definitions

ECMP (Equal-Cost Multi-Path):: A routing strategy that distributes traffic across multiple paths of equal cost.
Flow Table:: A data structure that stores the mapping between flow identifiers and egress ports, ensuring that packets of the same flow are forwarded from the same port.
Congestion Quantification Index (CQI):: A quantified value representing the degree of port congestion, ranging from 0 to a configured maximum. A CQI of 0 indicates no congestion.
Assessment Interval:: The time interval for port congestion state assessment.
Flow Table Migration:: The operation of remapping a flow table entry from one egress port to another.

Problem Statement

Limitations of Traditional ECMP Traditional ECMP load balancing uses static hash mapping. Once a flow is assigned to a port, the mapping remains unchanged throughout the flow's lifetime. This design has the following deficiencies: Delayed Response: When a port becomes congested, flows already mapped to that port cannot be migrated in time, causing congestion to persist. Load Imbalance: The randomness of traffic and the presence of elephant flows may cause severe load imbalance between ports.

Inadequacy of Existing Solutions Flowlet Switching: This mechanism switches based on inter-packet gaps within a flow and relies on manually configured time thresholds. If the threshold is too large, it degrades to traditional ECMP; if too small, it may cause packet reordering. Full-Switch Strategy: Migrating all relevant flows simultaneously when congestion is detected may cause the target port to be instantly overloaded, resulting in congestion transfer.

Requirements Summary A mechanism is needed that can:

Perceive port congestion state in real-time
Progressively adjust flow table mappings based on congestion level
Avoid congestion transfer and resource fluctuation
Preserve packet ordering

Solution Overview This mechanism defines two core functional components: Port Congestion Assessment: Periodically assesses the congestion state of each egress port and generates a Congestion Quantification Index (CQI). Adaptive Flow Table Migration: Progressively migrates flow table entries from congested ports to less loaded ports based on the CQI value. The fundamental design principle is that the higher the CQI value, the more flow table entries are allowed to migrate in the current assessment interval. For each entry migrated, the CQI is decremented by 1 until the CQI reaches zero or no more entries need migration.

Protocol Specification

Port Congestion Assessment

Assessment Interval Implementations MUST support a configurable assessment interval. The RECOMMENDED default value is between 10ms and 100ms. Implementations MAY adaptively adjust the assessment interval based on overall traffic levels: shortening the interval during high traffic to improve responsiveness, and lengthening it during low traffic to reduce overhead.

Congestion Quantification Index Calculation CQI calculation SHOULD be based on one or more of the following metrics: port egress queue depth, port buffer utilization, and port packet drop counter increment. The CQI value range is 0 to CQI_MAX. The RECOMMENDED value for CQI_MAX is 16. The recommended CQI calculation method is: where congestion_threshold is the congestion determination threshold, RECOMMENDED to be 10% of queue capacity.

State Advertisement At the end of each assessment interval, the Port Congestion Assessment component MUST synchronize each port's CQI value to the Flow Table Migration component.

Adaptive Flow Table Migration

Migration Decision When a packet arrives, implementations MUST process it according to the following rules: Rule 1 (Flow Table Does Not Exist): Perform normal flow table learning and select the port with the lightest current load. Rule 2 (Port Failure): If the flow table exists but the corresponding port is unavailable, a new port MUST be selected. Rule 3 (No Congestion): If the flow table exists and the corresponding port's CQI is 0, the implementation MUST continue using the current port and MUST NOT perform migration. Rule 4 (Congestion Exists): If the flow table exists and the corresponding port's CQI is greater than 0, the implementation SHOULD perform flow table migration.

Migration Operation When migration is triggered, implementations MUST perform the following steps: Step 1: Select the port with the smallest CQI from all available ports as the target. If multiple candidate ports have the same CQI, implementations MAY use random selection or round-robin. Step 2: Update the flow table entry's egress port to the target port. Step 3: Decrement the original port's CQI by 1.

Migration Quantity Control A key property of this mechanism is that the migration quantity is proportional to the congestion level. When the CQI value is high, more flow table entries may be migrated within a single assessment interval. When the CQI value is low, the migration quantity decreases accordingly. Implementations MUST ensure that within a single assessment interval, the number of flow table entries migrated from a port does not exceed that port's initial CQI value.

Continuous Migration If the CQI does not drop to 0 within an assessment interval, subsequent assessment intervals will recalculate the CQI. If congestion persists, migration will continue; if congestion is alleviated, migration will decrease or stop.

Data Structures

Flow Table Entry A flow table entry MUST contain the following fields: flow identifier (obtained through hash calculation), egress port identifier, valid bit, and timestamp (for aging).

Port Status Table The port status table MUST contain the following fields: port identifier, port status (UP/DOWN), current CQI value, and queue depth.

Operational Procedures

Initialization Implementations MUST perform the following at startup:

Clear the flow table
Initialize all ports' CQI to 0
Start the periodic assessment task

Packet Processing The packet processing flow is as follows:

Packet arrives
Calculate flow identifier
Query flow table
If flow table does not exist: learn new entry, select lightest loaded port
If flow table exists: check port status and CQI, perform migration if needed
Forward packet

Relationship with Existing Mechanisms

Relationship with ECMP This mechanism is an enhancement extension to traditional ECMP, adding congestion awareness and adaptive migration capabilities on top of ECMP. Implementations MAY overlay this mechanism on existing ECMP implementations.

Relationship with Flowlet This mechanism MAY be used in conjunction with flowlet switching. Flowlet uses inter-packet gaps within a flow for switching, while this mechanism uses port congestion state to trigger switching. The two can be complementary.

Relationship with Congestion Control This mechanism operates at the forwarding layer and is orthogonal to end-to-end congestion control mechanisms such as ECN and DCQCN. Implementations SHOULD consider coordination with congestion control mechanisms.

Security Considerations

Denial of Service Risk Attackers may induce frequent migration by forging traffic, consuming device resources. Mitigation Measures: Implementations SHOULD set a maximum number of migrations per unit time. Implementations SHOULD use smoothing algorithms for CQI calculation to avoid overreaction to instantaneous fluctuations.

Information Disclosure Risk CQI values and migration decisions may reveal network topology or traffic pattern information. Mitigation Measures: Implementations MUST implement access control for related data. Inter-module communication SHOULD use security mechanisms.

Configuration Integrity Mitigation Measures: Implementations MUST ensure configuration parameter integrity. Implementations SHOULD log configuration changes.

IANA Considerations This document does not require IANA to allocate any resources.