The main objective of the Application Centric Infrastructure is to decouple endpoints from the underlying physical network. It provides a distributed Layer 3 gateway that ensures optimal Layer 3 and Layer 2 forwarding. The ACI fabric supports standard bridging and routing without location restrictions (any IP address anywhere) and also removes flooding requirements for Address Resolution Protocol (ARP)/Gratuitous Address Resolution Protocol (GARP). All traffic within the fabric is encapsulated within VXLAN.

For application high availability and performance, the new application is designed to be distributed. Distributed applications drive east-west traffic from server to server through the data center access layer (Layer 2). Applications driving this shift include big data distribution, live virtual machine, or workload migration, as with VMware vMotion, server clustering, and multitier applications.

North-south traffic drives traditional data center design with core, aggregation, and access layers, or collapsed core and access layers. Client data comes in from the WAN or Internet, a server processes it, and then it exits the data center, which permits data center hardware oversubscription due to WAN or Internet bandwidth constraints. However, Spanning Tree Protocol (STP) is required to block loops. This limits bandwidth availability due to blocked links and potentially forces traffic to take a suboptimal path.

In traditional data center designs, IEEE 802.1Q VLANs provide logical segmentation of Layer 2 boundaries or broadcast domains. However, VLAN use of network links is inefficient, requirements for device placements in the data center network can be rigid, and the VLAN maximum of 4094 VLANs can be a limitation. As IT departments and cloud providers build large multitenant data centers, VLAN limitations become problematic.

A spine-leaf architecture addresses these limitations. The ACI fabric appears as a single switch to the outside world, capable of bridging and routing, as shown in Figure 4-18. Moving Layer 3 routing to the access layer would limit the Layer 2 reachability that modern applications require. Applications like virtual machine workload mobility and some clustering software require Layer 2 adjacency between source and destination servers. By routing at the access layer, only servers connected to the same access switch with the same VLANs trunked down would be Layer 2-adjacent. In ACI, VXLAN solves this problem by decoupling Layer 2 domains from the underlying Layer 3 network infrastructure.

Figure 4-18 ACI Fabric

As traffic enters the fabric, the ACI encapsulates and applies policy to it, forwards it as needed across the fabric through a spine switch (maximum two-hops), and decapsulates it upon exiting the fabric.

Within the fabric, ACI uses IS-IS and Council of Oracles Protocol (COOP) for all forwarding of endpoint-to-endpoint communications. This enables all ACI links to be active, equal-cost multipath (ECMP) forwarding in the fabric, and fast-reconverging. ACI uses MP-BGP to propagate routing information between software-defined networks within the fabric and routers external to the fabric.