Abstract
As artificial intelligence (AI) advances rapidly, traditional data center networks face major challenges when carrying AI traffic. High bandwidth utilization and a small number of elephant flows can make conventional per-flow load balancing ineffective, resulting in congestion and packet loss that significantly increase AI model training and job completion times. This white paper examines a dynamic intelligent routing technology that combines real-time awareness with intelligent decision-making. It uses path-quality measurement, BGP extended-attribute synchronization, dynamic weighted-cost multipath (WCMP), abnormal-path removal, and
Flowlet-level auto load balancing (ALB) to improve traffic scheduling in AI environments. The design also uses virtualization to provide multi-tenant isolation and offers application scenarios and planning guidance for efficient, stable, and intelligent AI data center networks.
Introduction: Network Evolution in the AI Era
Why Traditional Networks Struggle with AI Traffic
In traditional data center networks, a large number of small flows enables per-flow load balancing to achieve acceptable distribution and congestion avoidance even without real-time awareness of network conditions. AI traffic is very different: bandwidth utilization is extremely high and traffic is often dominated by only a few elephant flows. Hash collisions can therefore place multiple large flows on the same path while other paths remain underused. Once packet loss occurs, the completion time of the entire AI workload may increase significantly. For this reason, the industry increasingly focuses on optimized load-balancing algorithms for AI fabrics so that traffic is distributed more evenly across multiple paths .
Figure 1. Comparison of AI elephant-flow traffic and traditional small-flow traffic.
Combining Dynamic Awareness with Intelligent Decision-Making
Dynamic intelligent routing is an awareness-based load-balancing technology. It uses path quality sensed by switches in the fabric to adjust local path selection and supports dynamically weighted load balancing. Built on BGP, the technology defines a new extended-community attribute.
Multi-dimensional, high-precision measurements are used to evaluate path quality, and the result is propagated through BGP to guide subsequent traffic forwarding. This improves network-wide load distribution and reduces application response time [1].
Figure 2. Dynamic awareness, decision, and execution workflow.
Core Technologies: Intelligent Traffic Scheduling
Mainstream network load-balancing approaches include:
- Per-flow ECMP: The most common approach, based on a five-tuple hash. It works well when there are many flows and preserves packet order, but hash collisions can produce poor balancing when flow counts are low, as in AI training [1].
- Flowlet-based balancing: Depends on correct configuration of the inter-flowlet gap. Accurate configuration is difficult when global path-level latency is unknown [1].
- Per-packet ECMP: Provides excellent theoretical balance but may create extensive packet reordering at the receiver [1].
The dynamic intelligent routing approach used by XingRongyuan CX-N Series RoCE switches (SONiC-based) combines per-flow ECMP and Flowlet-based balancing. It introduces dynamic WCMP (Weighted Cost Multipath) and Flowlet-based ALB (Auto Load Balancing) [1].
Path Quality Measurement
The system evaluates network path quality using factors that strongly affect AI cluster fabrics, including bandwidth utilization, queue utilization, and forwarding latency .
Statistical Counters
Bandwidth and queue utilization are measured through ASIC hardware counters with accuracy at the hundred-millisecond level. The ASIC records port-forwarding and queue-forwarding counters in real time. The SONiC control plane reads the counters through the SAI interface at sub-second precision and stores them in Redis. The routing-control process evaluates interface quality using these counters and advertises the result through BGP. To limit control-plane load, advertisements are currently sent at second-level intervals, with a weighted average applied to multiple samples; more recent samples receive higher weight .
In-Band Telemetry
Forwarding-delay measurement is based on INT (In-band Network Telemetry) and can reach nanosecond-level precision. HDC (High Delay Capture) captures packets that experience high latency inside the ASIC. When a packet exceeds a user-defined delay threshold, the switch sends the first 150 bytes of the original packet together with metadata, including ingress port, egress port, and latency, to a collector. In this design, the CPU acts as the HDC collector and analyzer, enabling high-precision forwarding-delay measurements and more accurate path-quality evaluation.
Figure 3. Path-quality propagation and WCMP installation.
The technology uses a new BGP extended-community attribute, the Path Bandwidth Extended Community, to indicate the aggregate quality of a path to a destination. The high-order byte of the extended-type field is 0x00 and the low-order byte is 0x05. In the value field, the Global Administrator subfield represents the AS number. Path quality is encoded in four bytes using IEEE floating-point format and expressed in GB/s .
When NIC1 communicates with NIC2, NIC2 first advertises its IP address to Leaf2. Leaf2 advertises the address to the Spine together with the link-quality value toward NIC2 multiplied by the Leaf2 downlink weight. The Spine adds its own weighted link quality and passes the accumulated value to Leaf1. Leaf1 summarizes the path quality and installs routes that guide forwarding. In a two-tier Leaf-Spine fabric, ports are classified as Leaf uplinks, Leaf downlinks, and Spine ports, with configurable calculation coefficients for each class.
Dynamic WCMP
Load balancing distributes traffic across multiple links. In AI environments, this is essential for building a lossless Ethernet fabric with minimal packet loss, latency, and throughput degradation. ECMP is the conventional mechanism in data centers . WCMP extends ECMP by distributing
traffic proportionally across links. In dynamic intelligent routing, WCMP weights are adjusted in real time according to path quality. For example, if two paths exist between NIC1 and NIC2, the system may calculate a 3:7 ratio. As traffic changes, updated path-quality information is propagated through BGP to every Leaf switch, where dynamic WCMP routes are generated to guide forwarding .
Abnormal Path Removal
When a path's aggregate quality falls below an agreed threshold, the path is considered unusable for the AI workload and is removed from forwarding. The remaining paths continue to carry traffic through dynamic WCMP. Once the path returns to normal, it is restored. Although this may temporarily leave some capacity unused, it prevents more serious congestion and packet loss [1].
Intelligent Load Balancing (ALB)
ALB provides Flowlet-based load distribution. The ASIC measures load and latency on different ports in real time and directs each Flowlet to the link with lower load or lower delay. This adds finer-grained scheduling on top of traditional ECMP. ALB also supports port failover and automatically redistributes traffic when an egress link fails [1].
Virtualization
Front-end networks often need multi-tenant support so that different GPU resources can be allocated to different users. The solution uses VRF (Virtual Routing and Forwarding) to isolate tenants, with one VRF assigned to each user. GPU-related subnets are placed into the user's VRF, and PRE ACLs in the ASIC classify ingress traffic so that each user's traffic is forwarded only within the corresponding VRF [1].
Application Scenarios
1.How Dynamic WCMP Absorbs Traffic Surges
For a fabric with 256 x 400G GPU ports, a two-tier Clos architecture can be designed with a 1:1 downlink-to-uplink convergence ratio to maintain high throughput, high bandwidth, and symmetrical capacity. Suitable product models include the XingRongyuan CX864E-N or CX732Q-N [1]. Under conventional ECMP, a small number of AI elephant flows may hash to the same Spine, causing uplink congestion or packet loss. Dynamic WCMP continuously adjusts traffic distribution according to real-time path quality, reducing traffic surges and avoiding congestion [1].
2.Flowlet-Level Load Balancing
In the same 256 x 400G GPU fabric, not every device or workload is best served by dynamic WCMP. The switch can dynamically select Flowlet-based ALB. ALB measures the load and latency of links in the ECMP group and sends Flowlets to links with lower utilization or delay. If an interface becomes overloaded or its forwarding delay rises, the ASIC automatically avoids that egress until conditions return to normal [1].
AI Network Planning and Design
AI services generally include three stages: data collection and preprocessing, model training, and AI inference. Each stage imposes different requirements on the network .
1.Data Collection and Preprocessing
A global training-data collection and preprocessing network must provide large-scale, elastic public IP and bandwidth resources so that massive raw datasets can be collected efficiently from the global internet. The design must ensure secure and stable transmission while improving data aggregation and preprocessing performance [2].
1.1 VPC Segmentation
VPC design for cloud-based data collection should follow least-privilege, layered isolation, security control, and elastic scalability principles. Either multiple VPCs or multiple subnets inside a single VPC may be used [2].
- Segment by service stage: Deploy data collection, preprocessing, storage, training, and inference in separate subnets or VPCs. Place collection entry points in a DMZ VPC or public subnet; place preprocessing and temporary data in a dedicated processing VPC; place sensitive raw data and AI training in a high-security VPC with no direct public access.
- Isolate by security level: Create high-, medium-, and low-security VPCs or subnets and use security groups, network ACLs, and private connectivity to control cross-VPC and cross-subnet traffic.
- Lsolate by tenant or project: Assign separate VPCs to different tenants or projects to isolate resources, networks, and data.
1.2 Data Collection Egress Network
NAT gateway: Use a pool of EIPs and SNAT through a NAT gateway, allowing collection programs to access the internet from randomly selected public IP addresses.
Self-built proxy: Deploy proxy servers that maintain mappings between collection processes and public IP addresses, and distribute a single collection request across multiple proxies.
IP service provider: Connect to an IP service provider over the cloud private network for lower cost, controlled security, and stable quality [2].
1.3 Cross-Region Data Aggregation and Preprocessing
Select the OSS region according to the preprocessing-cluster plan and create a data-aggregation VPC in the same region. Create a Transit Router (TR) in each region, connect local VPCs, and interconnect the TRs so that regional collection VPCs can directly access the central OSS. Services such as PAI-iTAG, MaxCompute, and Flink can then perform data labeling and preprocessing in the OSS region [2].
2. Model Training Network
The model-training network is tightly integrated with the collection and preprocessing network to form a global fabric that connects cloud data centers, edge nodes, and on-premises IDCs.
High-speed interconnection provides low-latency, high-throughput data movement, while bandwidth, security policies, access controls, and disaster recovery must be designed end to end [2].
- In-region cross-cluster compute pool: A Transit Router connects multiple VPCs within the same region. Server ENIs connect to VPC switches and can be configured for the required throughput and RDMA.
- Cross-cloud compute pool: Dedicated circuits connect an IDC or another cloud to a same-region VPC. Alibaba Cloud VBRs connect at Layer 3 to the peer device, with BGP, BFD, and fast failover for rapid convergence.
Global compute pool: Cross-region links between TRs are provisioned according to peak inter-region traffic, with optional inter-region QoS to create a unified compute pool across regions [2].
3. Global Inference-Service Network
A global inference network selects deployment models according to the resources and cloud services available in each region and provides efficient service distribution and access [2].
- Model or Agent deployment: Use Model Studio MaaS deployment, PAI-EAS or Function Compute PaaS deployment, or IaaS deployment on EGS, ACK, ACS, or Lingjun bare-metal infrastructure inside the customer VPC.
- Service distribution: Create an inference-distribution VPC and connect private networks using VPC Peering, associated cloud-service VPCs, or PrivateLink. Deploy ALB or enhanced ALB as the access point and proxy requests to different models or Agents.
- Service access: Local internet users can access enhanced ALB directly; remote users can use Global Accelerator (GA); internal or external Alibaba Cloud enterprise users can connect privately through PrivateLink .
Conclusion and Outlook
This white paper explains the challenges faced by IDC networks in the AI era and how dynamic intelligent routing can address them. Dynamic awareness, intelligent decision-making, WCMP, ALB, and VRF virtualization improve load-balancing efficiency, reduce congestion and packet loss, and accelerate AI training and inference. The planning section emphasizes VPC segmentation, collection-egress design, cross-region aggregation and preprocessing, model training, and global inference delivery. As AI continues to evolve, data center networks will face stricter requirements. Dynamic intelligent routing will continue to improve for edge AI, federated learning, and other complex scenarios. Future IDC networks will place greater emphasis on compute-network convergence, energy efficiency, and automated operations, creating a more intelligent, efficient, and reliable foundation for the digital economy.
References
User-provided technical source document.
Alibaba Cloud AI Network White Paper. Accessed May 27, 2026.
https://help.aliyun.com/zh/cloud-network-well-architected-design/alibaba-cloud-ai-network-white-paper