Sun StorageTek 9990V and 9985V Arrays may experience Performance Delays or Microprocessor Hangs on Fibre Front End Directors (CHAs) when adding or deleting LU Paths.

Asset ID:	1-73-1161864.1
Update Date:	2010-07-30
Keywords:

Solution Type FAB (standard) Sure

Solution 1161864.1 : Sun StorageTek 9990V and 9985V Arrays may experience Performance Delays or Microprocessor Hangs on Fibre Front End Directors (CHAs) when adding or deleting LU Paths.

Applies to:

Sun Storage 9990V System - Version: Not Applicable to Not Applicable - Release: NA to NA
Sun Storage 9985V System - Version: Not Applicable to Not Applicable [Release: NA to NA]
Information in this document applies to any platform.

Symptoms

Host jobs may become delayed by temporary performance degradation, or Host access may be stopped temporarily by a link down caused by a hang-up/reset (SSB EC=1609) in the storage system.

Impact

Delays in response time may occur during a host I/O operation on a port after performing a LU path configuration change. The change may also cause multiple microprocessors (MPs) on the fibre front end director (FED) to hang, which eventually cause Host unable to process IO due to a Link Down caused by hang-up/reset.

The following will occur depending upon the Contributing Factors listed below:

. Spike in response time for ST9990V and ST9985V
. Host unable to process IO due to a Link Down caused by hang-up/reset (ST9990V only)

Changes

Contributing Factors

All of the following conditions below must be met for the problem to occur:

1. Use of any storage model listed in the Products Affected section.
2. Use of any microcode ranges listed below;

    60-06-05-00/00-M132 through 60-07-01-00/00-M172

3. Any combination of the following 16FS, 8FS or 8US open systems
    fibre FED feature (CHAs) product codes are installed:

   - DKC-F610I-8FS Fibre 8-port Adapter (1-4Gbps)
   - DKC-F610I-16FS Fibre 16-port Adapter (1-4Gbps)
   - DKC-F610I-8US Fibre 8-port Adapter (2-8Gbps)

Refer to the Maintenance Manuals for details on CHAs at the below (internal only URL;

   http://pts-storage.west/products/T99x0/documentation.html

4. LUN Path Definition is added or deleted from any of the following:

   a. Service Processor (SVP)
   b. Hitachi Storage Navigator
   c. Hitachi Device Manager software (HDvM, formerly known as HiCommand Device Manager)

5. The number of fibre FED PCBs installed in item(3) above will determine
    the phenomena that can occur:

   - If less than five pairs (8 or less PCB) installed a spike in respone time
     may occur. This can occur on either the ST9990V or ST9985V.

   - If five pairs or more (10 PCBs or more) installed a Host unable to process IO
    due to a Link Down caused by hang-up/reset. This can only occur on the ST9990V
    as the ST9985V cannot have this many pairs.

Cause

Root Cause

When a change in LUN path definition occurs in the processing that references or updates the path definition information for Logical Devices (LDEV), an MP busy condition continues to occur for a long time due to a microcode bug. As a result, dependending upon the number of fibre FED PCBs, a spike in response time may result, or the link may go down if a hang-up/reset occurs.

Solution

Workaround

No workaround available - see Resolution section below.

Resolution

Upgrade to fixed Microcode 60-07-32-00/00-M171 (or above).

Note 1: Do not perform LUN changes during periods of high I/O activity.
         Stop I/O prior to making changes if an application that is
         running is sensitive to response time spikes.

Note 2: Avoid Microcode upgrades to versions 60-06-05-00/00-M132 through
         60-07-01-00/00-M172 if you have open systems fibre FED PCBs.

Note 3: Do not add/upgrade fibre FEDs to create a configuration with
         five pairs (10 PCBs or more) of fibre FED PCBs if microcode
         60-06-05-00/00-M132 through 60-07-01-00/00-M172 is installed.
         Upgrade to a modified fixed microcode version first.

For your reference the below (internal only) link lists the exposed systems (with a microcode range 60-06-05-00/00-M132 through 60-07-01-00/00-M172) requiring microcode upgrade to the fixed version;

http://se9990.eng/tech_docs/fab/perf_spike/Alert_USPV_065283_CustomerList.ods

Note: The list doesn't include systems which are NOT connected to Hi-Track.

Comments

The above Customer List will not be loaded into SunFIT and the implementation of this FAB will not be monitored or tracked.

Subscribe to ST9900 up-to-date Alerts by referring the below link;

   http://sejsc.ebay/alerts_via_alias.html

For ST9900 Maintenance Manuals go to;

   http://pts-storage.west/products/T99x0/documentation.html

References

Related URL(s):

   HDS Alert: http://se9990.eng/tech_docs/fab/perf_spike/USPV_065283R2.pdf

For information about FAB documents, its release processes, implementation strategies and billing information, go to the following URL:

* http://tns.central/fab

In addition to the above you may email:

* FAB-Manager@sun.com

Contacts

Contributor: Suresh.Gummanur@Oracle.COM, Brian.Sutcliffe@Oracle.com
Responsible Engineer: Suresh.Gummanur@Oracle.COM
Responsible Manager: Tejinder.Singh@Oracle.com
Business Unit Group: NWS (Storage)

Attachments

This solution has no attachment