Logical Devices in ST9990 may block when the DKU Array Frame is running on only one redundant power supply and a sufficient quantity of DKS2D-K146FC HDDs are installed.

Asset ID:	1-73-1022191.1
Update Date:	2010-08-05
Keywords:

Solution Type FAB (standard) Sure

Solution 1022191.1 : Logical Devices in ST9990 may block when the DKU Array Frame is running on only one redundant power supply and a sufficient quantity of DKS2D-K146FC HDDs are installed.

Impact

A problem may occur where a power supply has failed in a ST9990 array frame and the array frame has powered off, blocking logical devices (LDEV). This results in a loss of access to data.

Contributing Factors

This issue may occur when the following conditions below are met for the same DKU Array frame:

1) If the ST9990 is being used with an attached Array Frame (R1, R2, L1, L2). Note that
   the Control Unit (DKC) Array Frame R0 is not affected. The ST9985 is not affected.

2) A DKU frame in item 1 above contains DKS2D-K146FC model disk drives installed in
   146GB 15Krpm parity groups located in its Lower B4 HDU-BOX (HDU = Hard Disk Unit)
   where the DKS2D-K146 model drives quantity installed are equal to or greater than
   one of the entries in row 1 of Table 1.0, listed in Conditions of Occurrence
   section of the alert at below (internal only) link;

     http://se9990.eng/HECN/510V09/usp_01_062012.pdf

   SUPPORT: The lower ST9990 Lower B4 is comprised of all HDU = XY0, XY1, XY2, XY3
            where X=any and Y<>0 (Not DKC R0 Frame).

3) The DKU frame identified in item 2 above, in its Lower B4 HDU-BOX also contains a
   total number of all disk drives (regardless of size, speed or model) that can be
   found in row 2 of Table 1.0, listed in Conditions of Occurrence section of the
   alert at below (internal only) link;

     http://se9990.eng/HECN/510V09/usp_01_062012.pdf

Example (using Table 1.0 inabove link): This DKU Frame can be affected because there are 70 DKS2D-K1456FC HDDs out of a total of 126 HDDs (any size, speed or model) installed in the Lower B4 HDU-BOX. For this example only these two values are highlighted in the table below, the 126 total drives and 67, which is the lowest quantity of DKS2D-K1456FC HDDs that can cause the DKU frame to be affected.

4) One of the following occurs for the same frame (effectively forcing the lower
   portion of the frame to operate on only one of its 56v power supplies):

. Power outage or site power maintenance where the same DKU frame is left running
    on one leg of power.
. Maintenance is performed on one of the following power supplies in the same DKU
    frame:

    i. DKU-ACBOX
    ii. DKUPS (DKUPS-xx1, DKUPS-xx2) for the lower half of the DKU frame.

5) Simultaneously, a battery charging cycle occurs for any batteries installed in the
   same DKU frame. Battery charging occurs for 1.5 hours once every 40 hours starting
   after a power up or after battery maintenance or installation. When the charging
   cycle occurs in combination with the other conditions, then the failure will most
   definitely occur.

6) Formatting drives located in the Lower B4 HDU-BOX, when the other conditions are
   present, will increase the chance that the problem will occur but is not necessary
   for the failure to occur.

7) Field Change "Addition of New FAN cable for DKU505" mentioned in the Resolution
   section is not installed to this same DKU frame. This Field Change redistributes
   the power source for the DKU frame fans so that one-half of the fans will draw
   power from the upper B4 DKU-Box.

Refer to the ECN, FCB and alert (interal only) links below for more details.

ECN: http://se9990.eng/HECN/510V09/5h061.pdf
FCB: http://se9990.eng/HECN/510V09/f5h011.pdf
HDS Alert: http://se9990.eng/HECN/510V09/usp_01_062012.pdf

Symptoms

Logical Devices may block when the DKU Array Frame is running on only one redundant power supply and a sufficient quantity of DKS2D-K146FC HDDs are installed. This results in a loss of access to data.

Root Cause

The power consumption level of the DKS2D-K146FC model HDD (Hard Disk Drive) is greater than other model HDDs that can be used in the ST9990 array. If a sufficient quantity of DKS2D-K146FC model HDDs are installed in the lower half of a Disk Array (DKU) frame (per Table 1.0), and specific conditions are met, it is possible for an over-current condition to be detected by the DKU AC/DC +56V PS, resulting in the power supply shutting down. If the lower DKU is operating on one of the two redundant DKUPS then all installed HDDs and FSWs in the lower half of the DKU will power off, resulting in blocked LDEVs and the customer losing access to their data.

Note that this is not a problem with the DKS2D-K146FC HDD itself, only that its normal operation draws slightly more power than the other HDD models.

Corrective Action

Workaround:

No workaround available - see Resolution section.

Resolution:

Audit contracted systems and refer to the below (internal only) link to affected ST9990 Subsystems Serial Number Listing (Machines reporting on Hi-Track/rm-portal) with frames affected or almost affected as of October 10, 2009;

http://se9990.eng/tech_docs/fab/9990_dku_issue/FAB_DKC510I-H011_CustomerList.ods

Site Ranking is based on the number of DKS2D-K146FC HDDs installed into lower disk chassis. Higher number of 146GB drives rank the highest in priority requiring immediate actions.

Units identified with a lower priority may not currently be at risk but may be at risk if more 146GB drives are added to the lower disk chassis. Additional ranking details can be found on page 2 of the customer list.

Note: Please audit your machines not reporting on Hi-Track.

Implement "Addition of New FAN cable for DKU505" for affected systems by following the procedure at the below (internal only) link;

http://se9990.eng/HECN/510V09/f5h011.pdf

Instructions for requesting the New Fan cable parts:

A request e-mail must be sent to ST9900_FCO_Parts@Sun.COM with "FCB DKC510I-H011 Hardware Request and system S/N" in the subject line.

Be sure to include all serial numbers for systems which you are requesting parts. Template information can be provided as an attachment or in body text of email.

The Part Order Template can be found at the below (internal only) URL;

http://se9990.eng/tech_docs/fab/9990_dku_issue/Sun_FCB_H011Form.doc

Parts are identified by the FCB # Parts name: FCB DKC510I-H011 Cable Kits

NOTE: C Country units will follow a separate order process, and will be coordinated through Sun Logistics. C Countries are China, Nigeria, Korea, Thailand, and Pakistan and are highlighted in yellow in the customer list.

Comments

Please contact backline support for any additional help or questions about this FAB.

There will be no Customer List downloaded into SunFIT and there is no implementation tracking requirement associated with this knowledge asset.

Subscribe to ST9900 up-to-date Alerts by refering the below link:

http://sejsc.ebay/alerts_via_alias.html

For ST9900 Maintenance Manuals go to;

http://pts-storage.west/products/T99x0/documentation.html

References:

Related URL(s):

ECN: http://se9990.eng/HECN/510V09/5h061.pdf
FCB procedure to implment FAN cable parts: http://se9990.eng/HECN/510V09/f5h011.pdf
HDS Alert: http://se9990.eng/HECN/510V09/usp_01_062012.pdf

For information about FAB documents, its release processes, implementation strategies and billing information, go to the following URL:

http://tns.central/fab

For Sun Authorized Service Providers go to:

http://tns.central/SPE/fab/

In addition to the above you may email:

FAB-Manager@sun.com

Internal Contributor/submitter
Suresh.Gummanur@Sun.COM, Brian.Sutcliffe@Sun.COM

Internal Eng Responsible Engineer
Suresh.Gummanur@Sun.COM Responsible Manager: Tejinder.Singh@Sun.COM

Internal Services Knowledge Engineer
Joe.Davis@Sun.COM

Internal Eng Business Unit Group
NWS (Storage)

Internal Sun Alert & FAB Admin Info
09-Feb-2010: Completed draft and sent to Extended Review.
12-Feb-2010: Incorporated feedback from Ext Rvw - sending to Publish.

Attachments

This solution has no attachment