Cisco Nexus 9000 Series switches offer modular and fixed 1-, 10-, 40-, and 100-Gigabit Ethernet switches. Nexus 9000 configurations can operate either in Cisco NX-OS standalone mode for compatibility and consistency with the current Cisco Nexus switches, or run in ACI mode to take full advantage of the APIC’s application policy-driven services and infrastructure automation features.

Cisco Nexus 9000 Series switches are designed for leaf-and-spine APIC deployment. Leaf-and-spine is a two-tier CLOS “fat-tree” architecture, as shown in Figure 4-4. Every lower-tier switch (leaf layer) is connected to each of the top-tier switches (spine layer) in a full-mesh topology. The leaf layer consists of access switches that connect to devices such as servers. The spine layer is the backbone of the network and is responsible for interconnecting all leaf switches. Every leaf switch connects to every spine switch in the fabric. The path is randomly chosen so that the traffic load is evenly distributed among the top-tier switches. If one of the top-tier switches were to fail, it would only slightly degrade performance throughout the data center.

Figure 4-4 ACI Fabric Overview

Starting with the Cisco APIC Release 4.1(1), you can now create a multitier fabric topology that corresponds to the core-aggregation-access architecture. The new design for the Cisco ACI incorporates the addition of a tier-2 leaf layer for connectivity to hosts or servers on the downlink ports and connectivity to the leaf layer (aggregation) on the uplink ports, as shown in Figure 4-5.

Figure 4-5 Cisco ACI Multitier Architecture (Spine, Tier-1 Leaf, and Tier-2 Leaf) Topology

In a multitier topology, all switch-to-switch links must be configured as fabric ports. You can also use ports that, by default, are downlink ports by configuring them to be fabric ports. Special care must be taken when choosing which port from tier-2 leaf switches is connected to tier-1 leaf switches. The reason is that the downlink to fabric link port conversion can be done only after the APIC discovers a leaf. If the APIC is connected to a tier-2 leaf, at least one tier-2 leaf default fabric port must be connected to a default fabric port on a tier-1 leaf. By ensuring this, the APIC can discover both the tier-2 leaf that is attached to the APIC and the tier-1 leaf that the tier-2 leaf is attached to. After this initial discovery, you can convert additional ports on the tier-1 and tier-2 leaf switches to be fabric ports. Also, each tier-1 leaf should have at least one default fabric port connected to a tier-2 leaf port.

In summary, these are the connectivity requirements for tier-2 leaf switches:

If the APIC is connected to a tier-2 leaf, one default fabric port of the tier-2 leaf must be connected to a default fabric port of a tier-1 leaf.

If no APIC is connected to a tier-2 leaf, one default fabric port of the tier-2 leaf must be connected to any port of a tier-1 leaf.