MAP7030 Private network isolation procedure 30

The Network Topology tool has been used to determine that one or both management consoles lost connectivity to all nodes on a single storage facility (both CEC enclosures) through the Black network (eth0).

MAP7030 Section-1

Procedure

  1. Does the failing management console have working links through interface eth0 (the Black network) to other storage facilities or management consoles?
    Figure 1. Ethernet connectors on the rear of the X335 management console server
    Management console rear view with Ethernet locations
    Figure 2. Ethernet connectors on the rear of the X336 management console server
    X336 management console Ethernet ports
    Figure 3. Ethernet connectors X3550 management console server (rear view)
    X3550 management console server (front view)
    Figure 4. eth0 (black network) and eth3 (gray network) USB Ethernet adapters for Thinkpad W500 (similar for Thinkpad T510, T520, T530, T540)
    eth0 (black network) and eth3 (gray network) USB Ethernet adapters for Thinkpad W500 (similar for Thinkpad T510)
    Figure 5. Ethx and Ethx LEDs on the rear of the model 98x management console server

    Ethernet ports: eth3  2 ; eth2  3 ; eth0  4 

    Model 98x management console cable connections (rear)
  2. Locate the Ethernet switch SW1 (Black network) at the rear of the storage facility with which the management console cannot communicate. See Figure 6 for 16-port switches, and Figure 7, Figure 8, and Figure 9 for 8-port switches.
    Is the Ethernet switch SW1 powered on?
    Figure 6. 16-port Ethernet switch port designations (SW1, SW2 - Tx) - Models 921/931, 922/932, 9A2/9B2
    Ethernet switch port designations (SW1, SW2 - Tx)
    Figure 7. 8-port Ethernet switch port designations (SW1, SW2-Tx) Models 921/931, 922/932, 9A2/9B2
    8-port Ethernet switch port designations (SW1, SW2-Tx)
    Figure 8. 8-port Ethernet switch port designations (SW1, SW2-Tx) Model 941, Model 951, and Model 961
    8-port Ethernet switch port designations (SW1, SW2-Tx)
    Figure 9. 8-port Ethernet switch port locations, Model 98x
    Note: Upper switch SW1 is "black" network; lower switch SW2 is "gray" network.
    8-port Ethernet switch port locations, Model 98x
  3. Select the type of Ethernet switches in the affected storage facility:
    • 16-port Ethernet switches, go to step 4.
    • 8-port Ethernet switches, go to step 8.
  4. Observe the 16-port Ethernet switch SW1-T16 port LEDs in the storage facility with which the management console cannot communicate. Reference Table 1 for a description and interpretation of the Ethernet switch port LEDs.
    Table 1. 16-port Ethernet switch port LED descriptions
    LED State Meaning
    Upper On 1000 Mb/s connection
    Off 100 Mb/s connection (Normal for FSP connection only)
    Lower Off No link
    On Link, No Activity
    Flashing Link, Activity

    There are two LEDs for each port. Is the lower Link LED lit solidly or flashing?

  5. Temporarily move the Ethernet cable to an unused port on the same switch. Observe the Ethernet switch port LEDs corresponding to the new location.
    Is the lower Link LED lit solidly or flashing?
    • Yes, it appears that the Ethernet switch in this rack is failing. Move the Ethernet cable back to its original port and then go to MAP7030 Section-3.
    • No, move the Ethernet cable back to port T16 and continue with the next step.
  6. Return to the storage facility that houses the failing management console and locate Ethernet switch SW1. Use Table 2 to determine the SW1 Ethernet port where the cable from the failing storage facility should be plugged. Observe the Ethernet switch port LEDs.
    Is the lower Link LED lit solidly or flashing?
    • Yes, verify that you have the correct port location. If necessary, trace the cable back from the failing storage facility.
    • No, continue with the next step.
    Table 2. Storage facility Ethernet switch ports for 16-port Ethernet switches
    Storage Facility # SW1 port # Black Network SW2 port # Gray Network
    SF #2 (all except model 98x) SW1-T15 SW2-T15
  7. Temporarily move the Ethernet cable to an unused port on the same switch. Observe the Ethernet switch port LEDs corresponding to the new location.
    Is the lower Link LED lit solidly or flashing?
    • Yes, the Ethernet switch in this rack (housing the failing MC) is failing. Go to MAP7030 Section-3.
    • No, the Ethernet cable between the storage facilities is failing. Go to MAP7030 Section-4.
  8. Observe the 8-port Ethernet switch SW1-T2 (model 98x, SW1-T7) port LEDs in the storage facility with which the management console cannot communicate. See Figure 7 and Figure 8 .
    • When viewed from the rear of the rack, the Ethernet ports are numbered 1 - 8 from right to left (top to bottom for model 98x). The LED indicator for each port is on the top front edge of the switch and are also numbered T1 to T8 from right to left (top to bottom for model 98x).
    • See Table 3 for a description and interpretation of the Ethernet switch port LED.
    Table 3. 8-port Ethernet switch port LED description
    LED State Meaning
    Blue port LED Off No link
    On Link, No Activity
    Flashing Link, Activity
    Is the blue port LED lit solidly or flashing?
  9. Is there an unused port on the same switch?
    • Yes, go to the next step.
    • No, this can only occur on an 8-port switch. Temporarily unplug the cable from port T8 (T1 for model 98x), and then continue at the next step.
      Note: For Models 9A2/9B2, this can cause a new serviceable event to be opened for Ethernet communications from the second LPAR on this CEC enclosure. After this cable is reconnected in the following step, you can close that serviceable event.
  10. Temporarily move the Ethernet cable to an unused port on the same switch. Observe the Ethernet switch port LEDs corresponding to the new location.
    Is the port LED lit solidly or flashing?
    • Yes, the Ethernet switch in this rack is failing. Move the Ethernet cable back to its original port. If you unplugged the cable from T8 (T1 for model 98x) to create an unused port, then reconnect that cable now. Then, go to MAP7030 Section-3.
    • No, move the Ethernet cable back to its original port and continue with the next step.
  11. Return to the storage facility that houses the failing management console and locate Ethernet switch SW1. Use Table 4 to determine the SW1 Ethernet port where the cable from the failing storage facility should be plugged. Observe the Ethernet switch port LEDs.
    Is the lower Link LED lit solidly or flashing?
    • Yes, verify that you have the correct port location. If necessary, trace the cable back from the failing storage facility.
    • No, continue with the next step.
    Table 4. Storage facility Ethernet switch ports for 8-port Ethernet switches
    Storage Facility # SW1 port # Black Network SW2 port # Gray Network
    SF #2 (all except model 98x) SW1-T2 SW2-T2
  12. Is there an unused port on the same switch?
    • Yes, go to the next step.
    • No, temporarily unplug the cable from port T8 (T1 for model 98x), and then continue at the next step.
      Note: For Models 9A2/9B2, this can cause a new serviceable event to be opened for Ethernet communications from the second LPAR on this CEC enclosure. After this cable is reconnected in the following step, you can close that serviceable event.
  13. Temporarily move the Ethernet cable to an unused port on the same switch. Observe the Ethernet switch port LEDs corresponding to the new location.
    Is the port LED lit solidly or flashing?
    • Yes, the Ethernet switch in this rack (housing the failing MC) is failing. Move the Ethernet cable back to its original port. If you unplugged the cable from T8 (T1 for model 98x) to create an unused port, then reconnect that cable now. Then, go to MAP7030 Section-3.
    • No, the Ethernet cable between the storage facilities is failing. Move the Ethernet cable back to its original port. If you unplugged the cable from T8 (T1 for model 98x) to create an unused port, then reconnect that cable now. Then, go to MAP7030 Section-4.

MAP7030 Section-2

Procedure

  1. Verify that the link failure is still present by using the Network topology test tool as described in MAP7001 Using the network topology tool.
    Is the link still failing?
  2. Return to the storage facility with which the management console cannot communicate. Select the type of Ethernet switches it has:
    • 16-port Ethernet switches, go to step 3.
    • 8-port Ethernet switches, go to step 4.
  3. At the storage facility with which the management console cannot communicate, temporarily move the Ethernet cable plugged in SW1-T16 to an unused port on the same switch. Verify if the link failure is still present by using the Network topology test tool as described in MAP7001 Using the network topology tool.
    Is the link still failing?
    • Yes, move the Ethernet cable back to port T16 and go to step 6.
    • No, the Ethernet switch is failing. Go to MAP7030 Section-3.
  4. Is there an unused port on the 8-port SW1 switch?
    • Yes, go to the next step.
    • No, temporarily unplug the cable from port T8 (T1 for model 98x), and then continue at the next step.
      Note: For Models 9A2 and 9B2, this might cause a new serviceable event to be opened for Ethernet communications from the second LPAR on this CEC enclosure. After this cable is reconnected in the following step, you can close that serviceable event.
  5. At the storage facility with which the management console cannot communicate, temporarily move the Ethernet cable plugged in SW1-T2 (SW1-T7 for model 98x) to an unused port on the same switch. Verify if the link failure is still present using the Network topology test tool.
    Is the link still failing?
    • Yes, move the Ethernet cables back to their original ports and continue with the next step.
    • No, the Ethernet switch is failing. Go to MAP7030 Section-3
  6. Return to the storage facility that houses the failing management console and locate Ethernet switch SW1. Select the type of Ethernet switches it has:
    • 16-port Ethernet switches, go to step 7.
    • 8-port Ethernet switches, go to step 8.
  7. Use Table 2 to determine the SW1 Ethernet port where the cable from the failing storage facility should be plugged. Temporarily move the Ethernet cable to an unused port on the same switch. Observe the Ethernet switch port LEDs corresponding to the new location. Verify if the link failure is still present by using the Network topology test tool as described in MAP7001 Using the network topology tool.
    • Yes, the Ethernet cable between the storage facilities is failing. Move the Ethernet cable back to its original port and then go to MAP7030 Section-4.
    • No, the Ethernet switch is failing. Move the Ethernet cable back to its original port and then go to MAP7030 Section-3.
  8. Is there an unused port on the 8-port SW1 switch?
    • Yes, go to the next step.
    • No, temporarily unplug the cable from port T2 (T7 for model 98x), and then continue at step 7.
      Note: For Models 9A2 and 9B2, this can cause a new serviceable event to be opened for Ethernet communications from the second LPAR on this CEC enclosure. After this cable is reconnected in the following step, you can close that serviceable event.
  9. Temporarily move the Ethernet cable plugged in SW1-T2 (SW1-T7 for model 98x) to an unused port on the same switch. Verify if the link failure is still present by using the Network topology test tool.
    Is the link still failing?
    • Yes, the Ethernet cable between the storage facilities is failing. Move the Ethernet cables back to their original ports and then go to MAP7030 Section-4.
    • No, the Ethernet switch is failing. Move the Ethernet cable back to its original port and then go to MAP7030 Section-3.

MAP7030 Section-3

About this task

Note: In this section, the SW1 Ethernet switch will be replaced.

Procedure

  1. Go to Exchanging parts to replace the SW1 Ethernet switch. Return here when the Ethernet switch has been replaced.
  2. Verify the fix by using the Network topology test tool as described in MAP7001 Using the network topology tool.
    Is the network problem resolved?

MAP7030 Section-4

About this task

Note: In this section, the Ethernet cable between the storage facilities will be replaced.

Procedure

  1. Replace the cable between the storage facilities. Continue with the next step when the cable has been replaced.
  2. Verify the fix by using the Network topology test tool as described in MAP7000 Section-1, step 3.
    Is the network problem resolved?