Sun Microsystems, Inc.  Sun System Handbook - ISO 3.4 June 2011 Internal/Partner Edition
   Home | Current Systems | Former STK Products | EOL Systems | Components | General Info | Search | Feedback

Asset ID: 1-77-1000451.1
Update Date:2011-03-01
Keywords:

Solution Type  Sun Alert Sure

Solution  1000451.1 :   Availability and/or Performance Issues With Dual-Controller Configuration or Replacement on 3510 Array  


Related Items
  • Sun Storage 3510 FC Array
  •  
Related Categories
  • GCS>Sun Microsystems>Sun Alert>Criteria Category>Availability
  •  
  • GCS>Sun Microsystems>Sun Alert>Release Phase>Resolved
  •  

PreviouslyPublishedAs
200589


Product
Sun StorageTek 3510 FC Array

Bug Id
<SUNBUG: 6217572>, <SUNBUG: 6248596>

Date of Workaround Release
28-JAN-2005

Date of Resolved Release
26-OCT-2007

Impact

On a Sun StorEdge 3510 FC array, both controllers in a dual-controller configuration may appear as "Primary", causing a race condition. Also, a controller replacement in a dual controller configuration may cause a performance impact or result in the replaced controller to be unrecognized.

One of the following four conditions can occur with this issue:

A) Neither controller will function as the primary, causing a "race condition" (also known as a "deadlock")

B) Both controllers in a dual-controller configuration appear as primary, causing the host to lose access to the LUNs.

C) A controller that has been replaced might not be recognized and can go into a failed status with an amber LED.

D) Performance slows down after controller replacement.

Note: Cases C) and D) occur when there is a controller replacement. 


Contributing Factors

This issue can occur on the following platforms:

  • Sun StorEdge 3510 FC Array without firmware version 4.21

Note: This issue has been seen on SE3510 but may also occur on the SE3511.

To determine the 3510 firmware revision, run the following command:

    sccli> show inquiry
    Vendor: SUN
    Product: StorEdge 3510
    Revision: 327P
    Peripheral Device Type: 0x0

Symptoms

There are two possible conditions to indicate a race condition has occurred:

Normally, in a dual-controller configuration, one controller is the primary controller and the other is the secondary. The primary controller is identified by a blinking green LED, the secondary is identified by a solid green LED.

A) When replacing a failed controller, LEDs on both controllers display as solid green, indicating neither of them are the primary (also known as a "race condition" or "deadlock"). This condition may also cause issues with the serial port and not allow a telnet(1) session to the array, and the host will be unable to access the array.

Or,

B) LEDs on both controllers are blinking green (indicating that both are primary) and the host loses access to LUNs.

In the case of the unrecognized controller:

C) the affected controller has an amber LED and viewing peripheral devices indicated "Scanning"(I/O is continued, ctl+w didn't work):

    redundancy-mode
    Primary controller serial number: 8013914
    Primary controller location: Lower
    Redundancy mode: Active-Active
    Redundancy status: Scanning
    Secondary controller serial number: 0 <-- couldn't identify inserted
    controller.

In the slow performance case:

D) the array front disk LED blinks rarely every couple of seconds and host/applications may complain.


Workaround

Follow the procedure given below when inserting a controller, in order to avoid the controller replacement issues:

  1. Insert the controller FRU into the slot, and push forward until you begin to feel resistance as the connecting pins engage.
  2. Slowly push the controller FRU the rest of the way until the connecting pins are fully seated and the RAID controller fits flush against the back plate of the RAID array.

Refer to the Sun StorEdge 3000 Family FRU Installation Guide (816-7326-19.pdf) for more details on how to install an I/O controller module.

Note: For cases A, B, and D:

A reset or power cycle may only temporarily avoid the "race condition". Resetting controller or powering-off the array also helps recover from case D.

For case C, stopping all I/O and reseating the replaced controller may help.


Resolution

This issue is addressed in the following release:

  • Sun StorEdge 3510 FC Array with firmware version 4.21 (as delivered in patch 113723-18 or later)


Modification History
Date: 16-JUN-2005

Change History

16-Jun-2005:

  • Updated Impact, Contributing Factors, and Workaround sections

Date: 26-OCT-2007
  • Updated Contributing Factors and Resolution sections
  • State: Resolved


Previously Published As
101644
Internal Comments


This issue has been seen on SE3510 but may also occur on the SE3511.



This (below) applies to cases A & B only:



This issue has been filed as bug 6217572 and still under investigation by the PDE. It may be very difficult to reproduce this issue and hence it would take a lot of time to find the root cause and the same is true with the resolution.



We could reproduce a variant of the problem "a" mentioned above seen at customer place. The details are in the bug report. What we found out was that if there is a controller failure and when we do the controller replacement, if the replacement controller came from a standalone unit "and" the replacment controller is configured as be active controller only (with redundant controller option disabled) then upon replacement both controllers would want to be active and freeze the box. This problem was reproduced in the PTS and the engineering lab too. However, with the customer system, the problem was resolved after the reset whereas in our test cases, the problem still remained after the array reset and only after the controller was pulled out, the problem went away.



For additional information, please see SRDB 79042 at http://sunsolve.central.sun.com/search/document.do?assetkey=1-25-79042-1



Cases C & D can occur when inserting a new controller with excessive insertion force and speed.


Internal Contributor/submitter
chaya.adatrao@sun.com

Internal Eng Business Unit Group
NWS (Network Storage)

Internal Eng Responsible Engineer
chaya.adatrao@sun.com

Internal Services Knowledge Engineer
david.mariotto@sun.com

Internal Escalation ID
1-6252550, 1-6318677

Internal Sun Alert Kasp Legacy ID
101644, 57731 (Sun Alert)

Internal Sun Alert & FAB Admin Info
Critical Category: Availability ==> Pervasive
Significant Change Date: 2005-01-28, 2007-10-26
Avoidance: Patch
Responsible Manager: bagher.vahdatinia@sun.com
Original Admin Info: [WF 15-Jul-2005 Dave M. engineering requested incorrect info be removed, remove a Note entry]
This document has been imported from KMS Creator and may need adjustment before re-publishing.

This imported document has been reviewed/adjusted by:
Review Name:
Review Date:

Original KMS Creator attributes below:

--- PLEASE DO NOT MAKE ANY CHANGES BELOW THIS LINE! ---

Sun Alert ID: 57731
Synopsis: Both Controllers in a Dual-Controller Configuration May Appear as "Primary", Causing a Race Condition
Category: Availability
Product: Sun StorEdge 3510, 3511 FC Arrays
BugIDs: 6217572
Avoidance: None
State: Committed
Date Released: 28-Jan-2005
Date Closed:
Date Modified:
Escalation IDs: 1-6252550, 1-6318677
Pending Patches:
Resolution Patches:
FIN:
FCO:
Date Submitted: 24-Jan-2005
Submitter: sailesh.thanki@sun.com
Responsible Engineer: sailesh.thanki@sun.com
Responsible Manager: bagher.vahdatinia@sun.com
CTE group: NWS
Responsible Writer: david.mariotto@sun.com
Distribution: Preliminary-Contract SunSolve

Workflow History:

WF State: Issued, 02-Feb-2005, David Mariotto
WF Note: added SRDB 79042 information to "Internal" section

WF State: Issued, 28-Jan-2005, David Mariotto
WF Note: sent for release

WF State: Draft, 28-Jan-2005, David Mariotto
WF Note: all comments recorded. send for release

WF State: Draft, 26-Jan-2005, David Mariotto
WF Note: waiting on comments from BU per Anne

WF State: Draft, 25-Jan-2005, David Mariotto
WF Note: sent for review

WF State: Draft, 24-Jan-2005, David Mariotto
WF Note: waiting for BUPO approval

WF State: Draft, 24-Jan-2005, David Mariotto
WF Note: Article created.

Exported from KMS Creator Sat May 21 09:16:29 2005 GMT, olaf.reineke@sun.com
Product_uuid
58553d0e-11f4-11d7-9b05-ad24fcfd42fa|Sun StorageTek 3510 FC Array

Attachments
This solution has no attachment
  Copyright © 2011 Sun Microsystems, Inc.  All rights reserved.
 Feedback