Quantcast
Channel: Colocation to Virtualization » iom
Viewing all articles
Browse latest Browse all 2

Cisco UCS and Nexus 5000 DataCenter – Our Implementation

0
0

Our Cisco 10gbE implementation consists of 2 Chassis fully populated UCS with a mix of full and half width blades. The servers are all boot from SAN with no local disks. “PALO” cards are used in all servers which allow us to do FCOE. 7 of the blades are running VMware ESX 4 (vSphere) and the rest are a mix of RedHat Linux and Windows 2008.

Storage:

  • All servers boot from SAN over FCOE
  • Primary storage is via Fibre Channel to an EMC CX4-240
  • A few servers are also connected via Fibre Channel to RamSan SSD SANs
  • The CX4-240 is also connected to the UCS environment via quad 10gbE links over fiberNetwork:
  • Core 10gbE switching is provided via Cisco Nexus 5010s
  • 1 Gigabit Ethernet to legacy servers are provided by dual 48 port C2148 Fabric Extenders
  • Each C2148 is connected to a Nexus 5010 by dual 10gbE uplinks
  • Each 5010 is connected via quad 4gig fiber to the CX4-240 to provide FCoE to physical servers outside of the UCS environment
  • Each 5010 is also connected to the CX4-240 by dual 10gbE fiber ports to provide ISCSI to all of the network
  • Each UCS switch (6120s) are uplinked to the 5010s by dual 10gbE interfaces
  • Each UCS chassis is uplinked to the 6120s by 8 qty 10gbE ports (4 connections per IO module)
  • The fiber switches are a redundant pair of Cisco MDS 9124s10gb ISCSI was added after the initial build. We are running Oracle with ASM in our VMware environment. This allowed for easier management of storage inside of oracle. Using ASM allowed dynamic growth of the database without having to do a lot of resizing of ext3 filesystem when expanding LUNs. We wound up with a ton of VMware RDMs (Raw Device Mappings). To remedy this issue, we have gone with 10gbE ISCSI to the EMC.

    Below is a diagram of how we are currently setup.

    10gbE DataCenter

    Notes: Sounds like Cisco will be able to do multi-hop FCoE soon.  This should remove the need of having the CX4 connected via fiber to both the 5010 and the 6120.  I definitely would love feedback on this.  How is everyone else implementing 10gb? Anyone considering 10gb ISCSI?


    Filed under: Colocation, EMC, Hardware, Networking, RamSan, SAN (Storage Area Network)

    Viewing all articles
    Browse latest Browse all 2

    Latest Images

    Trending Articles





    Latest Images