Cisco MDS9124 fools around: IP management interface unreachable
Gosh … today seems to be the day of a thousand troubles.
First I had to fiddle arround getting my RS232-to-usb dongle to work on Snow Leopard, second the Cisco MDS9124 switches fools me around with an unreachable IP management interface.
My intent was to add two cisco MDS9124 SAN switches to the management network.
For the sake of simplicity I leave away unneeded information to give a simple overview of the topoligy:
The two MDS switches need to be connected physically to either of the Catalyst 4948 L2/L3 switches.
A fully redundant management interconnection is not possible, as each MDS switch has only one management port.
The two Catalyst 4948 are interconnected by 4 1Gig wires, which form a logical Port Channel group.
Additionally we span an access VLAN 11 accross the two switches, to allow connecting the MDS switches to the management network.
For the internal routing, VLAN11 is bound to a VRRP interface, which ensures ongoing available of the default gateway.
So, here’s the chart:
And some config stuff as well for the Cat 4948 switches. For better readability I omit some statements like storm control and port security and alike.
!
vlan 11
name MGMT_VLAN
!
vlan 99
name NATIVE_VLAN
!
!
interface Port-channel10
description "Trunk Port: Uplink between SWI01 and SWI02
switchport
switchport trunk encapsulation dot1q
switchport trunk native vlan 99
switchport trunk allowed vlan 11
switchport mode trunk
switchport nonegotiate
!
interface range GigabitEthernet1/17 - 20
switchport access vlan 11
logging event link-status
!
interface range GigabitEthernet1/45 - 48
switchport trunk encapsulation dot1q
switchport trunk native vlan 99
switchport trunk allowed vlan 11
switchport mode trunk
switchport nonegotiate
udld port aggressive
channel-protocol lacp
channel-group 10 mode active
!
!
interface Vlan11
ip address 192.0.2.11 255.255.255.0
no ip redirects
no ip unreachables
no ip proxy-arp
no ip mroute-cache
standby 11 ip 192.0.2.1
standby 11 timers 1 3
standby 11 priority 110
standby 11 preempt delay minimum 60
standby 11 track 1 decrement 30
arp timeout 300
!
The second catalyst switch would be identical in setup except for interface vlan 11, which looks like this:
interface Vlan11
ip address 192.0.2.12 255.255.255.0
no ip redirects
no ip unreachables
no ip proxy-arp
no ip mroute-cache
standby 11 ip 192.0.2.1
standby 11 timers 1 3
standby 11 priority 90
standby 11 preempt
arp timeout 300
!
Now, for the first MDS switch the configuration would like this:
ip default-gateway 192.0.2.1
interface mgmt0
ip address 192.0.2.21 255.255.255.0
And for the second switch:
ip default-gateway 192.0.2.1
interface mgmt0
ip address 192.0.2.22 255.255.255.0
You now should have the basic idea of what I was trying to achieve.
As I was on my configuration spree, the first MDS switch worked right away and was immediately reachable.
But for the second one, it simply didn’t work out.
Although the physical interface came up right away, it was unreachable by any means.
Checking on the switch gave me this:
mgmt0 is up
Hardware is FastEthernet
Address is 000d.ec9b.6dcc
Internet address is 192.0.2.22/24
MTU 1500 bytes, BW 100 Mbps full Duplex
0 packets input, 0 bytes
0 multicast frames, 0 compressed
0 input errors, 0 frame, 0 overrun 0 fifo
0 packets output, 0 bytes, 0 underruns
0 output errors, 0 collisions, 0 fifo
0 carrier errors
Pinging the switch itself worked:
switch# ping 192.0.2.22
PING 192.0.2.22 (192.0.2.22) 56(84) bytes of data.
64 bytes from 192.0.2.22: icmp_seq=1 ttl=64 time=0.079 ms
Besides that, nothing else could be pinged from the switch.
Checking from the Catalyst switches (both of them) turned to be no better, the switch was simply unreachable.
Also checking for L2 traffic, which would cause the mac address table to be populated, there was absolutely nothing to be seen on the physical interface.
4924-B#sh mac address-table int gi1/17
Mac Address Table
-------------------------------------------
Vlan Mac Address Type Ports
---- ----------- -------- -----
Additionally, only the MAC address entries for the port channel and the VRRP interface as well as the other MDS switch, seen through the port channel, were know to the second 4924 switch.
4924-B##sh mac address-table vlan 11
Unicast Entries
vlan mac address type protocols port
-------+---------------+--------+---------------------+--------------------
11 0000.0c07.ac0b dynamic ip Port-channel10
11 000d.ec9b.6e0c dynamic ip Port-channel10
11 d0d0.fdc2.9bbf static ip,ipx,assigned,other Switch
11 f866.f2d7.533f dynamic ip Port-channel10
Multicast Entries
vlan mac address type ports
-------+---------------+-------+--------------------------------------------
11 ffff.ffff.ffff system Switch,Po10
After some fiddling and twisting, even disabling the mgmt0 interface, deleting and readding whilst double-checking the configuration, I assumed that this might be well one of this "bogus phenomens" that never should happen. But as we all know, stuff like this DOES happen.
And guess what: After rebooting the MDS, all worked perfectly as it was supposed to.
What was the reason to this? I honestly don't know. It was surely NOT a configuration issue.
To me it seems very odd, that a MDS should need to be rebootet after applying a first-time configuration. The first one proved that it actually CAN work. So to conclude: bogus that took me valuable time for no apparent reason.
