Recently we have had issues with our chassis 2 iom (IO Module) 1 port 1. In the process of troubleshooting, Cisco sent out a replacement IO module hoping it would solve the issue. Turns out one thing that I did not know is that once the “fabric ports” on the IOM have been wired, it must keep the same cabling.
I had the great (not really) idea to swap cable 1 and 2 on the IOM while I was replacing the module. I figured that if it was a port issue on the 6120 blade switch that the problem would move to the new port. Turns out the IOM freaked out when this happened and downed the ports that were not where they had been.
The part which made it more difficult to track down was the the “fabric ports” (links from chassis to switch) all showed green. It was 3 of the 8 backend ports on the IOM that were errored.
Here is a server interface that was pinned to a removed port from the UCS Manager CLI. Below shows the downed interfaces as “Error disabled”.
ucsm01-A(nxos)# sh interface brief | inc "Error disabled" Eth2/1/3 1 eth access down Error disabled 10G(D) --
Once the 10g cables (SFPs) were swapped back to their origional locations, this ports came back. Here the status now shows “up”
ucsm01-A(nxos)# sh interface brief | inc Eth2/1/3 Eth2/1/3 1 eth access up none 10G(D) --
Again, the resolution here was to put the cabling back in their origional locations. Even with this being a new IO module, UCS loaded it with the configuration from the origional card. While talking with Cisco TAC, they said that the development group is working on making the IO module switch uplinks swappable (dynamic) and not statically pinned.
This really wasn’t a major issue due to UCS having redundancy built in, we never lost connectivity to a server. It’s just one of those “gotchas” that I ran into while troubleshooting a port.
Filed under: Uncategorized Image may be NSFW.
Clik here to view.
Image may be NSFW.Clik here to view.
