Cisco Catalyst Blade Switch Series for HP BladeSystem c-Class
Easy, Smart, and Flexible Blade Server I/O Solution Challenges
Organizations of all types continue to invest in IT to develop new revenue opportunities, reduce costs, and improve service levels. However, this continuing investment is exacerbating facilities concerns for organizations, especially in the areas of power, cooling, rack space, and cable management. These challenges are especially salient in the area of server infrastructure, where application proliferation along with the widespread adoption of x86 based scale-out architectures has lead to significant server sprawl in the data center. While blade servers offer a high-density, cost-effective solution for delivering scale-out x86 architectures, they significantly increase the number of network access switches and operational challenges. Cisco Catalyst Blades Switches address these challenges with innovations such as virtual blade switch (VBS) technology. Cisco blade switches provide an easy, smart, and flexible blade servers I/O solution that enables customers to gain the full benefits offered by the blade server architecture. The HP c-Class enclosure can hold up to 16 half-height servers and up to 8 blade switches. The HP c-Class BladeSystem backplane provides power and network connectivity to the blades. The base I/O module slots house a pair of Cisco Catalyst Blade Switches for HP, which provide a highly available and multihomed environment wherein each server blade is attached through a Gigabit Ethernet port to each Cisco Catalyst Blade Switch for HP. Two Cisco Catalyst Blade Switches within the blade enclosure connect the blade server modules to external network devices such as aggregation-layer switches.
Figure 1: Front and Back Views of HP c-Class BladeSystem Enclosure

At-A-Glance

Cisco Catalyst Blade Switch 3120 for HP
Cisco has coordinated development with HP to bring the Cisco Catalyst Blade Switches to the HP BladeSystem c-Class blade server enclosure with Cisco Catalyst Blade Switch 3120G and 3120X for HP (Figure 2). The Cisco Catalyst Blade Switch 3120G supports up to eight Gigabit Ethernet uplinks, and the Cisco Catalyst Blade Switch 3120X supports four Gigabit Ethernet uplinks and two 10 Gigabit Ethernet ports. The Cisco Catalyst Blade Switch 3120G configuration allows customers to connect the switch to the existing network fabric using Gigabit Ethernet uplinks, and the Cisco Catalyst Blade Switch 3120X configuration allows customers to connect the switch to the existing network fabric using Gigabit and 10 Gigabit Ethernet uplinks. The Cisco Catalyst Blade Switch 3120 for HP is the next-generation Ethernet blade switch for the HP c-Class BladeSystem, and it provides a high-performance, cost-effective, and scalable solution that can help simplify the network and meet growing bandwidth needs.
Figure 2: Cisco Catalyst Blade Switch 3120 for HP
Cisco Catalyst Blade Switches Overview
Cisco Catalyst Blade Switches are a comprehensive I/O solution for blade server network services that transparently extend network fabric from the blade server edge to clients at the network edge. The Cisco Catalyst Blade Switches product family includes Cisco Catalyst Ethernet blade switches and Cisco MDS 9000 family Fibre Channel blade switches, helping ensure blade server and virtual machine availability, simplify operations, and provide virtualized data center infrastructure while lowering total cost of ownership (TCO) and improving end-user and IT productivity.
Cisco Catalyst Blade Switches for HP
Cisco Catalyst Blade Switches represent the next-generation networking solution for blade server environments. Cisco Catalyst Blade Switches implement a number of technologies and features that provide new options when designing blade server architectures. Specifically, Cisco Catalyst Blade Switches provide options for bandwidth, availability, and flexibility. Cisco Blade Switches are designed to deliver scaleable, high-performance, highly resilient connectivity while supporting ongoing IT initiatives to reduce server infrastructure complexity, and TCO.
Cisco Catalyst Blade Switch 3020 for HP
The Cisco Catalyst Blade Switch 3020 for HP (Figure 3) is an integrated switch for the HP c-Class BladeSystem that extends resilient and secure Cisco infrastructure services to the server edge and takes advantage of existing network investments to help reduce operating expenses. The Cisco Catalyst Blade Switch 3020 provides an integrated switching solution that dramatically reduces cable complexity. This solution offers consistent network services such as high availability, quality of service (QoS), and security. It also uses a comprehensive Cisco management framework to simplify ongoing operations.

HP c-Class BladeSystem Enclosure Overview
The HP c-Class BladeSystem Enclosure represents the next generation of blade server and blade switch integration. Figure 1 shows front and back views of the cabinet.
2009 Cisco Systems, Inc. All rights reserved. Cisco, the Cisco logo, and Cisco Systems are registered trademarks or trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries. All other trademarks mentioned in this document or Website are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (0812R)
Easy, Smart, and Flexible Blade Server I/O Solution
Figure 3: Cisco Catalyst Blade Switch 3020 for HP
Benefits of Cisco Catalyst Blade Switches
Built on the market-leading Cisco hardware and Cisco IOS Software, the Cisco Catalyst Blade Switches are engineered with innovative technologies specifically designed to meet the rigors of blade serverbased application infrastructure. Specifically, the switch is designed to support blade servers in their new role by delivering scalable, high-performance, highly resilient connectivity while supporting ongoing initiatives to reduce server infrastructure complexity and TCO.

Virtual Blade Switch

With the Cisco Catalyst Blade Switch 3120, Cisco has a unique technology called VBS. Much like server virtualization technology, this switch virtualization technology treats the individual physical switches within a rack as one logical switch (Figure 4). As with server virtualization technology, this innovation allows the switches to deliver better utilization, increased performance, and greater resilience while simplifying operations and management.
Figure 4: VBS Allows Up to Eight Switches to Be Treated as a Single Virtual Switch
Easy: Easy to operate and deploy Fewer switches to manage with VBS technology, which solves problem of switch sprawl Consistent management interface and tools throughout the Cisco data center and blade switch portfolio, which accelerates service provisioning and simplifies troubleshooting VBS technology provides operational transparency and efficiency during replacement and addition of switches Smart: Feature-rich server-smart networking solution Provides highly resilient LAN uplinks to increase blade server and virtual machine availability using innovations such as trunk failover and EtherChannel Helps secure application servers and virtual machines by using private VLANs and accesscontrol lists (ACLs) Provides intelligent congestion management mechanisms to optimize network bandwidth using QoS Provides advanced Layer 2 and 3, IPv4, IPv6, and multicast capabilities to facilitate smart end-toend server networking in the data center

Flexible: Flexible solution to scale resources and facilitate data center virtualization Provides flexibility with VBS to configure network topology based on application needs such as performance, scalability, and resiliency Provides investment protection and a flexible transition path with VBS and the capability to mix and match Gigabit Ethernet and 10 Gigabit Ethernet switches Provides flexible options for configuration and management such as a command-line interface (CLI) and a GUI

Conclusion

The Cisco Catalyst Blade Switches product family designed for Data Center 3.0 provides the following benefits to server, storage, and network organizations:
Lowers costs with VBS by reducing the number of cables required to connect blade servers Increases IT productivity with simplified and consistent management and troubleshooting tools Helps ensure blade server and virtual machine availability after live migrations with innovative LAN features Provides faster server deployments by taking advantage of an end-to-end virtualized Data Center 3.0 architecture

For More Information

See http://www.cisco.com/go/bladeswitch.
2009 Cisco Systems, Inc. All rights reserved. Cisco, the Cisco logo, and Cisco Systems are registered trademarks or trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries. All other trademarks mentioned in this document or Website are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (0812R) C45-450758-03 03/09

Integrating the Cisco Catalyst Blade Switch 3020 for the HP c-Class BladeSystem into the Cisco Data Center Network Architecture

Design Guide

2008 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.

Contents

Introduction..... 3 HP c-Class BladeSystem Enclosure Overview.... 3 Cisco Catalyst Blade Switch 3020 for HP... 5 Cisco Catalyst Blade Switch 3020 Features... 6 Spanning Tree.... 6 Traffic Monitoring.... 8 Link Aggregation Protocols.... 9 Data Center Network Architecture.... 10 Data Center Network Components... 10 Aggregation Layer..... 11 Access Layer.... 11 High Availability.... 12 Design Goals.... 12 High Availability.... 12 High Availability for the BladeSystem Switching Infrastructure... 13 High Availability for the Blade Servers... 13 Scalability..... 14 Physical Port Count.... 14 Slot Count.... 15 Management.... 16 Out-of-Band Management.... 16 In-Band Management... 17 Serial Console Port.... 17 Management Options.... 18 HP c-Class BladeSystem iLO Connectivity... 18 Design and Implementation Details... 18 Network Management Recommendations.... 18 Network Topologies Using the Cisco Catalyst Blade Switch 3020.. 19 Recommended Topology... 19 Alternative Topology.... 22 Configuration Details.... 23 VLAN Configuration.... 24 RPVST+ Configuration.... 24 Inter-Switch Link Configuration.... 24 Server-Port Configuration.... 26 Server Default Gateway Configuration... 27 RSPAN Configuration... 28

Page 2 of 28

Introduction
This guide provides best design practices for deploying the Cisco Catalyst Blade Switch 3020 for the HP c-Class BladeSystem enclosure within the Cisco Data Center Networking Architecture. It describes the internal components of the blade-server enclosure and Cisco Catalyst Blade Switch 3020 and explores different methods of deployment.
HP c-Class BladeSystem Enclosure Overview
The HP c-Class BladeSystem enclosure represents the next generation of blade-server and bladeswitch integration. Figure 1 shows both a front and back side view of the cabinet. The c-Class enclosure can hold up to 16 half-height servers and up to 8 switch modules. The servers are available with either Intel or AMD processors. HP also offers full-height servers with two Intel processors. Both support dual-core processors. The first two switch bays must contain Ethernet switches because the onboard LAN adapters are routed to those bays. The additional six bays are available for additional Ethernet switches, Fibre Channel switches, InfiniBand switches, or copper or fiber pass-through modules. Each full-height server contains four Gigabit Ethernet interfaces, two running each module in module slots 1 and 2. Full-height servers also have three mezzanine slots for additional I/O connections such as Fibre Channel, InfiniBand, or even more Ethernet switches.
Figure 1. Front and Back Views of HP c-Class BladeSystem Enclosure
The HP c-Class BladeSystem backplane provides power and network connectivity to the blades. The base I/O module slots house a pair of Cisco Catalyst Blade Switch 3020s, which provide a highly available and multihomed environment wherein each server blade is attached through a Gigabit Ethernet port to each Cisco Catalyst Blade Switch 3020. Two Cisco Catalyst Blade Switch 3020s within the blade enclosure connect the blade-server modules to external network devices such as aggregation layer switches. Figures 2 and 3 show the logical connections between the servers, the two internal blade switches, and the outside network.

Page 3 of 28

Figure 2.
Enclosure Interconnections Using Full-Height Servers

Figure 3.

Enclosure Interconnections Using Half-Height Servers

Page 4 of 28

Cisco Catalyst Blade Switch 3020 for HP
This section briefly describes the Cisco Catalyst Blade Switch 3020 for HP and explains how the blade servers within the HP c-Class BladeSystem are physically connected to the switching modules. The Cisco Catalyst Blade Switch 3020 provides enhanced Layer 2 services (known as Layer 2+ or Intelligent Ethernet switching) to the HP c-Class BladeSystem. The Cisco Catalyst Blade Switch 3020 enhances basic Layer 2 switching by including Cisco proprietary protocols, access control lists (ACLs), and quality of service (QoS) based on Layer 3 information. With Simple Network Management Protocol (SNMP), command-line interface (CLI), or HTTP management options available and a robust set of Cisco IOS Software switching features, the Cisco Catalyst Blade Switch 3020 naturally integrates into the data center environment. The following features highlight this capacity:
Loop protection and rapid convergence with support for Per VLAN Spanning Tree Plus (PVST+), IEEE 802.1w, IEEE 802.1s, Bridge Protocol Data Unit (BDPU) Guard, Loop Guard, PortFast, UplinkFast, and Unidirectional Link Detection (UDLD)
Advanced management protocols, including Cisco Discovery Protocol, VLAN Trunking Protocol (VTP), and Dynamic Trunking Protocol (DTP) Port Aggregation Protocol (PAgP) and Link Aggregation Control Protocol (LACP) for link load balancing and high availability Support for authentication services, including RADIUS and TACACS+ client support Support for protection mechanisms, such as limiting the number of MAC addresses allowed or shutting down the port in response to security violations
Each Ethernet switch provides eight external Ethernet ports for connecting the blade enclosure to the external network. Four Small Form-Factor Pluggable (SFP) ports provide 1000BASE-SX interfaces and are shared with four of the copper Gigabit Ethernet links. Two additional copper Gigabit Ethernet ports are shared with two internal crossover interfaces connecting the pair of switches (labeled X-Crossovers in Figures 2 and 3). All of these ports can be grouped to support the IEEE 802.3ad LACP. Each blade server is connected to the backplane using the available Gigabit Ethernet network interface cards (NICs). The number of NICs on each blade server varies. Each server, whether it is full- or half-height, supports an additional Ethernet interface providing Integrated Lights Out (iLO) support. Note: The iLO interface supports a management interface that resides on each server blade. For

more information about the iLO system, refer to the Management section of this guide.

Page 5 of 28

Cisco Catalyst Blade Switch 3020 Features
This section highlights information about the protocols and features provided by the Cisco Catalyst Blade Switch 3020 that help integrate the HP c-Class BladeSystem enclosure into the Cisco Data Center Network Architecture.

Spanning Tree

The Cisco Catalyst Blade Switch 3020 supports different versions of the Spanning Tree Protocol and associated features, including the following:
Rapid Spanning Tree Protocol (RSTP), based on IEEE 802.1w Multiple Spanning Tree (MST), based on IEEE 802.1s (and includes IEEE 802.1w support) PVST+ Rapid PVST+ (RPVST+) Loop Guard UDLD BPDU Guard PortFast UplinkFast (Cisco proprietary enhancement for IEEE 802.1d deployments) BackboneFast (Cisco proprietary enhancement for IEEE 802.1d deployments)
The IEEE 802.1w protocol is the standard for rapid spanning tree convergence, whereas IEEE 802.1s is the standard for multiple spanning-tree instances. Support for these protocols is essential in a server-farm environment for allowing rapid Layer 2 convergence after a failure occurs in the primary path. The primary benefits of IEEE 802.1w include the following:
The spanning-tree topology converges quickly after a switch or link failure. Convergence is accelerated by a handshake, known as the proposal agreement mechanism.
The user need not enable PortFast, BackboneFast, or UplinkFast if running RSTP.
In terms of convergence, Spanning Tree Protocol algorithms based on IEEE 802.1w are much faster than the traditional Spanning Tree Protocol IEEE 802.1d algorithms. The proposal agreement mechanism allows the Cisco Catalyst Blade Switch 3020 to decide new port roles by exchanging proposals with its neighbors. With IEEE 802.1w, as with other versions of the Spanning Tree Protocol, BPDUs are sent by default every 2 seconds (called the hello time). If three BPDUs are missed, Spanning Tree Protocol recalculates the topology, a process that takes less than 1 second for IEEE 802.1w. Because the data center is made of point-to-point links, the only failures are physical failures of the networking devices or links. The IEEE 802.1w protocol can actively confirm that a port can safely transition to forwarding without relying on any timer configuration, meaning that the actual convergence time is less than 1 second.

Page 6 of 28

A scenario wherein BPDUs are lost may be caused by unidirectional links, which can cause Layer 2 loops. To prevent this problem, use Loop Guard and UDLD. Loop Guard prevents a port from forwarding as a result of missed BPDUs, which might cause a Layer 2 loop that could bring down the network. UDLD allows devices to monitor the physical configuration of fiberoptic or copper Ethernet cables and detect when a unidirectional link exists. When a unidirectional link is detected, UDLD shuts down the affected port and generates an alert. BPDU Guard prevents a port from being active in a spanning-tree topology as a result of an attack or a misconfigured device connected to the switch port. The port that sees unexpected BPDUs is automatically disabled and must then be manually enabled, giving the network administrator full control over port and switch behavior. The Cisco Catalyst Blade Switch 3020 supports Per VLAN Spanning Tree (PVST) and a maximum of 128 spanning- tree instances. RPVST+ is a combination of Cisco PVST Plus (PVST+) and RSTP, provides the flexibility of one spanning-tree instance per VLAN and the fast convergence benefits of IEEE 802.1w. MST allows the switch to map several VLANs to one spanning-tree instance, reducing the total number of spanning-tree topologies the switch processor must manage. A maximum of 16 MST instances is supported. In addition, MST uses IEEE 802.1w for rapid convergence. MST and RPVST+ create a more predictable and resilient spanning-tree topology, while providing downward compatibility for integration with devices that use IEEE 802.1d and PVST+ protocols. Figure 4 illustrates an example of Spanning Tree Protocol when using two switches in the crossover configuration. Each blade switch is dual homed to each aggregation switch through a 2port Cisco EtherChannel interface. In this figure the blocked links are indicated in red. In this example, only four of the eight uplinks from each blade switch are used. The network designer can make those EtherChannel uplinks more robust (up to four 4 ports each), or use them to connect other devices such as intrusion detection systems (IDSs) or standalone servers.

Figure 4. Spanning-Tree Example with the HP c-Class Enclosure and Cisco Catalyst Blade Switch 3020s

Page 7 of 28

The IEEE 802.1w protocol is enabled by default when running spanning tree in RPVST+
or MST mode on the Cisco Catalyst Blade Switch 3020. The Cisco Catalyst Blade Switch 3020 enables PVST+ for VLAN 1 by default. The Spanning Tree Protocol uses the path cost value to determine the shortest distance to the root bridge. The port path cost value represents the media speed of the link and is configurable on a per-interface basis, including Cisco EtherChannel interfaces. To allow for more granular Spanning Tree Protocol calculations, enable the use of a 32-bit value instead of the default 16-bit value. The longer path cost better reflects changes in the speed of channels and allows the Spanning Tree Protocol to optimize the network in the presence of loops. Note: The Cisco Catalyst Blade Switch 3020 supports IEEE 802.1t, which allows for spanning-
tree calculations based on a 32-bit path cost value instead of the default 16 bits. For more information about the standards supported by the Cisco Catalyst Blade Switch 3020, refer to the Cisco Catalyst Blade Switch 3020 Overview document: http://www.cisco.com/go/bladeswitch. For more information regarding spanning tree and Layer 2 design in the data center, visit: http://www.cisco.com/en/US/solutions/ns340/ns517/ns224/ns304/net_design_guidance0900aecd80 0e4d2e.pdf.

Traffic Monitoring

The Cisco Catalyst Blade Switch 3020 supports the following traffic-monitoring features, which are useful for monitoring blade-enclosure traffic in data center environments:
Switched Port Analyzer (SPAN) Remote SPAN (RSPAN)
SPAN mirrors traffic transmitted or received on source ports or source VLANs to another local switch port. This traffic can be analyzed by connecting a switch or Remote Monitoring (RMON) probe to the destination port of the mirrored traffic. Only traffic that enters or leaves source ports or source VLANs can be monitored using SPAN. RSPAN facilitates remote monitoring of multiple switches across your network. The traffic for each RSPAN session is carried over a user-specified VLAN that is dedicated to that RSPAN session for all participating switches. The SPAN traffic from the source ports or source VLANs is copied to the RSPAN VLAN. This mirrored traffic is then forwarded over trunk ports to any destination session that is monitoring the RSPAN VLAN. Figure 5 illustrates the use of RSPAN in a dual-blade switch environment. Here the internal crossconnects can allow the RSPAN traffic to traverse the backplane from one switch to the other. The second switch can either send the SPAN traffic out an uplink port to a local IDS device or pass it up the EtherChannel uplink to the aggregation switch above. Because RSPAN uses its own unique VLAN, it can use ports that may be blocked by other data VLANs.

Page 8 of 28

Figure 5.

RSPAN Example

Link Aggregation Protocols
Cisco Fast EtherChannel interfaces and Gigabit EtherChannel interfaces are logically bundled, and they provide link redundancy and scalable bandwidth between network devices. PAgP and LACP help automatically create these channels by exchanging packets between Ethernet interfaces and negotiating a logical connection. PAgP is a Cisco proprietary protocol that can be run only on Cisco switches or on switches manufactured by vendors that are licensed to support PAgP. LACP is a standard protocol that allows Cisco switches to manage Ethernet channels between any switches that conform to the IEEE 802.3ad protocol. Because the Cisco Catalyst Blade Switch 3020 supports both protocols, you can use either IEEE 802.3ad or PAgP to form port channels between Cisco switches. For both of these protocols, a switch learns the identity of partners capable of supporting either PAgP or LACP and identifies the capabilities of each interface. The switch dynamically groups similarly configured interfaces into a single, logical link, called a channel or aggregate port. The interface grouping is based on hardware, administrative, and port parameter attributes. For example, PAgP groups interface with the same speed, duplex mode, native VLAN, VLAN range, trunking status, and trunking type. After grouping the links into a port channel, PAgP adds the group to the spanning tree as a single switch port. In Figure 6, each blade switch uses an alternative configuration. The switch is no longer dual homed; instead all the ports are put into a single Cisco EtherChannel uplink to the aggregation switch above. This single EtherChannel uplink can use up to the full 8 ports, providing a 2-to-1 cable reduction from the servers. In this configuration, the Spanning Tree Protocol may not be needed because there is no loop in the network if the interconnect ports between the two blade switches are disabled.

Page 9 of 28

Figure 6.
Alternative Network Configuration
Data Center Network Architecture
The architecture of the data center infrastructure must address the requirements necessary to create a highly available, scalable, and secure network. This section describes the basic architecture necessary to meet these goals. It is a synopsis of the Cisco Data Center Network Architecture; for details about this architecture, visit: http://www.cisco.com/en/US/solutions/ns340/ns517/ns224/ns304/net_design_guidance0900aecd80 0e4d2e.pdf.

High Availability

High availability in the data center is a goal that must be achieved systematically. A highly available environment is attainable by addressing each layer of the data center and each of the devices that comprise that particular data center layer. Network and software features help achieve high availability, as well as physical redundancy of links and devices. The aggregation and access layers use redundant devices and links to help ensure no single point of failure occurs. The Layer 2 and Layer 3 features supported by these switches also create a highly available infrastructure. Spanning Tree Protocol support on both the aggregation and access switches creates a deterministic topology that converges quickly. Logical redundancy or fault tolerance may be achieved with Layer 3 technologies such as Hot Standby Router Protocol (HSRP) or Virtual Router Redundancy Protocol (VRRP). These protocols allow for virtualization of the gateways for servers or clients across the physical routing devices in the network. This virtualization mitigates the effect of a routing-device failure on the availability of data center services. Load-balancing services deployed in the aggregation layer allow the network to monitor server health and application availability. These devices and features combined produce a more resilient application environment. Dual homing a server in relation to separate access layer switches is another method to achieve a higher level of availability in the data center. NIC teaming removes the possibility of a single NIC failure isolating the server. It requires the server to have two separate NICs that support teaming software. Typically, teaming software detects failures over an external network probe between members of the team by monitoring the local status of each NIC in the team. The combination of dual-homed servers and a network load balancer provides an even greater level of availability for the server and the applications it supports.

Design Goals

This section describes the design goals for deploying blade servers and the functions that the Cisco Catalyst Blade Switch 3020 supports in data centers. It discusses high availability, scalability, and management.
Data centers are the repository of critical business applications that support the continual operation of an enterprise. These applications must be accessible throughout the working day during peak times, and some on a 24-hour basis. The infrastructure of the data center, network devices, and servers must address these diverse requirements. The network infrastructure provides device and

2008 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information. Page 12 of 28
link redundancy combined with a deterministic topology design to achieve application-availability requirements. Servers are typically configured with multiple NICs and dual homed to the access layer switches to provide backup connectivity to the business application. High availability is an important design consideration in the data center. The Cisco Catalyst Blade Switch 3020 has numerous features and characteristics that contribute to a reliable, highly available network. High Availability for the BladeSystem Switching Infrastructure High availability between the Cisco Catalyst Blade Switch 3020s in the HP c-Class BladeSystem and the aggregation layer switches requires link redundancy. Each Cisco Catalyst Blade Switch 3020 in the HP c-Class BladeSystem uses four SFP uplinks for connectivity to the external network, allowing for redundant paths using two links each for more redundancy. Redundant paths implemented between the HP c-Class BladeSystem and each aggregation layer switch when each path uses two links provide a highly resilient design. However, this setup introduces the possibility of Layer 2 loops; therefore, a mechanism is required to manage the physical topology. The implementation of RSTP helps ensure a fast-converging, predictable Layer 2 domain between the aggregation layer and access switches (the Cisco Catalyst Blade Switch 3020s) when redundant paths are present. The recommended design is a triangle topology (as shown in Figure 4 earlier), which delivers a highly available environment through redundant links and a spanning tree. It allows for multiple switch or link failures without compromising the availability of the data center applications. These channels support the publicly available subnets in the data center and traffic between servers. The server-to-server traffic that uses these uplinks is logically segmented through VLANs and can use network services available in the aggregation layer. There is also a port channel defined between the two blade-enclosure switches. This path provides intraenclosure connectivity between the servers for VLANs defined locally on the blade-enclosure switches. Clustering applications that require Layer 2 communication can use this traffic path, as well as mirrored traffic. Each of these port channels is composed of two Gigabit Ethernet ports. RPVST+ is recommended as the method for controlling the Layer 2 domain because of its predictable behavior and fast convergence. A meshed topology combined with RPVST+ allows only one active link from each blade switch to the root of the spanning-tree domain. This design creates a highly available server farm through controlled traffic paths and the rapid convergence of the spanning tree. The details of the recommended design are discussed in a later section. High Availability for the Blade Servers The HP c-Class BladeSystem provides high availability to blade servers by multihoming each server to the Cisco Catalyst Blade Switch 3020s. The two Cisco Catalyst Blade Switch 3020s housed in the interconnect bays are connected to the blade server over the backplane. Four backplane Gigabit Ethernet connections are available to every blade-server slot. Multihoming the server blades allows the use of a NIC teaming driver, which provides another highavailability mechanism to fail over and load balance at the server level. Three modes of teaming are supported:

These traffic paths provide three different management options for network administration and support different user and application interfaces to the Cisco Catalyst Blade Switch 3020. The remote management of the blade servers within the HP c-Class BladeSystem is critical to an efficient and scalable data center. This section discusses these topics, as well as the iLO connectivity options provided using the enclosure to the blade servers. Out-of-Band Management OOB management is the practice of dedicating an interface on the managed device for carrying management traffic. It is also the recommended management method for blade systems. OOB management isolates the management and data traffic and provides a more secure environment.

Page 16 of 28

The Cisco Catalyst Blade Switch 3020 contains an additional Fast Ethernet port, which connects to the HP c-Class BladeSystem Onboard Administrator, providing OOB management using the insight manager interface. The user may also use this path to access the CLI functions of the switch, transfer SNMP information, and upload software images and configuration files. This path is independent of the switch fabric. This Fast Ethernet port defaults to a Dynamic Host Configuration Protocol (DHCP) client from a DHCP server either as part of the Onboard Administrator or external on the network attached to the enclosure. The user can also set a static IP address for the Fast Ethernet port. The Cisco Catalyst Blade Switch 3020 supports multiple switched virtual interfaces (SVIs) to be active at the same time; however, it does not perform any routing functions between SVIs. By default, the SVI is created as VLAN 1 and enabled during the setup phase of the installation. The VLAN is often referred to as the management VLAN. Cisco recommends that the user change the management VLAN to something other than VLAN 1. Therefore, it is important to create an SVI with another VLAN and allow this VLAN on the external front-panel ports. In addition, you can manage the switch using the Fa0 port using the Onboard Administrator on the back of the enclosure. By default, the Cisco Catalyst Blade Switch 3020 provides no routing functions and can have only one default gateway defined. Even though the Fa0 interface is called routed, it cannot route user traffic. Therefore, if you enable multiple SVIs or enable the Fast Ethernet port, you will not be able to access all these interfaces from other subnets. The recent migration (12.2(22)SE) of the Cisco Catalyst Blade Switch 3020 from the LANBase image to IP Base provides basic Layer 3 routing (RIP and Static Routing and EIRGP Stub). For best practices in selecting the management VLAN, please visit: http://www.cisco.com/en/US/products/hw/switches/ps700/products_white_paper09186a00801b49a 4.shtml. In-Band Management In-band management uses logical isolation to separate management traffic from data traffic. VLANs segregate the two traffic types that are sharing the bandwidth of the uplink ports. This practice is common in situations in which multiple applications running on the servers must be managed along with the network infrastructure devices. In-band management traffic uses the uplink trunk ports located on the back of the Cisco Catalyst Blade Switch 3020s for management. Cisco recommends that the Data VLANs not be the same VLAN as the management VLAN. Serial Console Port The front panel of the Cisco Catalyst Blade Switch 3020 has an RJ-45 serial port that can be used to manage the switch through the CLI. The CLI can be accessed by connecting directly to the console port with the serial port of a workstation or remotely by using terminal servers and IP connectivity protocols such as Telnet.

Page 17 of 28

Management Options The Cisco Catalyst Blade Switch 3020 switch is manageable with the following methods:
HTTP-based device-manager GUI SNMP-based management applications Cisco IOS Software CLI
The embedded device manager on the Cisco Catalyst Blade Switch 3020 provides a GUI to configure and monitor the switch through a Web browser. This scenario requires using either inband or out-of-band management and enabling the HTTP or HTTPS server on the switch. The HTTP server and SSL are enabled by default. SNMP-compatible management utilities are supported through a comprehensive set of MIB extensions and through four Remote Monitoring (RMON) groups. CiscoWorks 2000 and HP OpenView are two such management applications. SNMP Versions 1, 2, and 3 are available on the switch (Cisco IOS Software Crypto image). The CLI delivers the standard Cisco IOS Software interface over Telnet or the console port. Cisco recommends that you use the Secure Shell (SSH) Protocol for CLI access. Note: For more information about the embedded device manager, refer to the online help on the
switch CLI. For more information about the management options for the HP c-Class BladeSystem, please visit: http://h18004.www1.hp.com/products/blades/components/management.html. HP c-Class BladeSystem iLO Connectivity The iLO provides remote-management capabilities and is standard with all c-Class server blades. Remote power, console, and diagnostics are just a few of the advanced functions iLO provides. The HP c-Class BladeSystem provides two methods to access this management interface through its Onboard Administrator. The iLO connection is independent of the Cisco Catalyst Blade Switch 3020. The blade-servers Onboard Administrator located on the back of the enclosure provides access to each of the iLO interfaces through a single Ethernet cable. A redundant Onboard Administrator is also available.
Design and Implementation Details
Network Management Recommendations
An OOB network is recommended for managing the Cisco Catalyst Blade Switch 3020. OOB management provides an isolated environment for monitoring and configuring the switch. Isolation is achieved by deploying a physically separate management network or by logically separating the traffic with management VLANs. The Cisco Catalyst Blade Switch 3020 has 8 external Gigabit Ethernet ports; any of them may be used to support network-monitoring devices and network-management traffic. Using secure protocols, such as SSH or HTTPS, maintains the integrity of communications between the switch and the management station. The console port positioned at the front of the Cisco Catalyst Blade Switch 3020 is another option for connectivity to the OOB network.

Page 20 of 28

RSPAN requires a VLAN to carry the mirrored traffic to the remote destination switch. In the recommended topology, the secondary aggregation switch is the RSPAN destination, where an analysis device, such as the integrated Cisco Network Analysis Module (NAM), resides. The RSPAN VLAN uses the uplink between the blade switch and the secondary aggregation switch. This uplink is blocking under normal conditions for regular VLANs. As a result, bandwidth usage is a concern only when the uplink is forwarding and sharing the path with production traffic.
Configuring the Aggregate Switches
Complete the following steps on the aggregate switches: Step 1. VLAN configuration Step 2. RPVST+ configuration Step 3. Primary and secondary root configuration Step 4. Configuration of port channels between aggregate switches Step 5. Configuration of port channels between aggregate switches and Cisco Catalyst Blade Switch 3020s Step 6. Trunking of port channels between aggregate switches Step 7. Configuration of default gateway for each VLAN Note: The Configuration Details section describes each of these steps.
Configuring the Cisco Catalyst Blade Switch 3020s
Complete the following steps on the Cisco Catalyst Blade Switch 3020s: Step 1. VLAN configuration Step 2. RPVST+ configuration Step 3. Configuration of port channels between the Cisco Catalyst Blade Switch 3020 and aggregate switches Step 4. Trunking of port channels between the Cisco Catalyst Blade Switch 3020 and aggregate switches Step 5. Configuration of server ports on the Cisco Catalyst Blade Switch 3020
Additional Aggregation-Switch Configuration
The following recommendations help integrate the Cisco Catalyst Blade Switch 3020s into the data center: Step 1. Enable Root Guard on the aggregate-switch links connected to the switches in the blade enclosure. The spanning-tree topology is calculated, and one of the primary parameters involved in this equation is the location of the root switch. Determining the position of the root switch in the network allows the network administrator to create an optimized forwarding path for traffic. The Root Guard feature is designed to control the location of the root switch. The aggregation switches should employ the spanning-tree guard root command on the port-channel interfaces connected to the blade switches.

Page 21 of 28

Step 2. Allow only those VLANs that are necessary on the port channel between the aggregate and the blade switches. Use the switchport trunk allowed vlan vlanID command to configure the port-channel interfaces of the aggregate switch to allow only those VLANs indicated with the vlanID option.

Additional Cisco Catalyst Blade Switch 3020 Configuration
Step 1. Enable BPDU Guard on the internal server ports of the switch. Use the spanning-tree bpduguard enable command to shut down a port that receives a BPDU when it should not be participating in the spanning tree. Step 2. Allow only those VLANs that are necessary on the port channels between the aggregate and the blade switches. Use the switchport trunk allowed vlan vlanID command to configure the port-channel interfaces of the switch to allow only those VLANs indicated with the vlanID option. Alternative Topology An alternative topology that does not rely on the Spanning Tree Protocol to account for redundant paths in the network (because there are none) is to have the two Cisco Catalyst Blade Switch 3020s connect directly to two aggregate switches using a port channel supporting the server-farm VLANs. Four to 8 of the external uplinks of each Cisco Catalyst Blade Switch 3020 are channeled and connected to one of the two aggregate switches. (The internal connections between the two Cisco Catalyst Blade Switch 3020s complete the loop and thus would require Spanning Tree Protocol.) Alternatively, if you enable the internal interconnects, you can user Layer 3 interconnects between the aggregation layer switches and still maintain a loop-free environment. This design uses the links between the two Cisco Catalyst Blade Switch 3020s as a redundant path for blade-server traffic. The use of a longer path cost value provides for a more granular calculation of the topology based on the available link bandwidth (refer to the Cisco Catalyst Blade Switch 3020 Features section). This feature is enabled with the spanning-tree pathcost method long CLI command. RPVST+ should be used in this network design for its fast convergence and predictable behavior. The following convergence tests were conducted against this alternative topology:
Uplink failure and recovery between switch-A and the primary root Uplink failure and recovery between switch B and the secondary root Failure and recovery of switch A and switch B Failure and recovery of the primary and secondary root switches
These tests yielded results similar to those of the recommended topology. Layer 2 convergence occurs in approximately 1 second. As stated previously, recovery at Layer 3 depends on the HSRP settings of the aggregate switches (refer to the Recommended Topology section). In our testbed, the failure of the active HSRP device typically increased the convergence time to 5 seconds.

Aggregate 1 to aggregate 2 Aggregate 1 or aggregate 2 to HP c-Class BladeSystem switch A or switch B HP BladeSystem switch A to switch B
Each of these connections is a Layer 2 Cisco EtherChannel connection consisting of multiple physical interfaces bound together as a channel group or port channel. These point-to-point links between the switches should carry more than one VLAN; therefore, each is a trunk.
Port-Channel Configuration
Link Aggregate Control Protocol (LACP) is the IEEE standard for creating and managing Cisco EtherChannel connections between switches. Each aggregate switch uses this feature to create a port channel across the line cards. The use of multiple line cards within a single switch reduces the possibility of the point-to-point port channel becoming a single point of failure in the network. Configure the active LACP members on aggregate 1 to Cisco Catalyst Blade Switch 3020 switch A as follows: (config)# interface GigabitEthernet12/1 (config-if)# description <<** Connected to Switch-A **>> (config-if)# channel-protocol lacp (config-if)# channel-group 1 mode active (config)# interface GigabitEthernet11/1 (config-if)# description <<** Connected to Switch-A **>> (config-if)# channel-protocol lacp

Page 24 of 28

(config-if)# channel-group 1 mode active Configure the passive LACP members on Cisco Catalyst Blade Switch 3020 switch A as follows: (config) # interface GigabitEthernet0/19 (config-if)# description <<** Connected to Aggregation-1 **>> (config-if)# channel-group 1 mode on (config) # interface GigabitEthernet0/20 (config-if)# description <<** Connected to Aggregation-1 **>> (config-if)# channel-group 1 mode on

Trunking Configuration

Use the following guidelines when configuring trunks:
Allow only those that are necessary on the trunk. Use IEEE 802.1q trunking. Tag all VLANs over a trunk from the aggregation switches.
Configure trunks using the standard encapsulation method IEEE 802.1q as follows: (config-if)# switchport trunk encapsulation dot1q Define the VLANs permitted on a trunk as follows: (config-if)# switchport trunk allowed vlan <VLAN IDs> Modify the VLANs allowed on a trunk using one of the following commands: (config-if)# switchport trunk allowed vlan add <VLAN IDs> (config-if)# switchport trunk allowed vlan remove <VLAN IDs> Define a port as a trunk port as follows: (config-if)# switchport mode trunk Note: The autonegotiation of a trunk requires that the ports be in the same VTP domain and be

Printed in USA

C07-468192-00 04/08

Page 28 of 28