Document Audience:INTERNAL
Document ID:I1144-1
Title:Clustered Netra D130 array causes various SCSI errors under load if LVD SCSI Disk Drives are installed to replace failed single ended SCSI disk drives in a cross cabled configuration.
Copyright Notice:Copyright © 2005 Sun Microsystems, Inc. All Rights Reserved
Update Date:2005-03-29

------------------------------------------------------------
            - Sun Proprietary/Confidential: Internal Use Only -
------------------------------------------------------------------------

  ***  Sun Confidential:  Internal Use and Authorized VARs Only  ***
________________________________________________________________________

  This message including any attachments is confidential information
  of Sun Microsystems, Inc.  Disclosure, copying or distribution is
  prohibited without permission of Sun.  If you are not the intended
  recipient, please reply to the sender and then delete this message.
________________________________________________________________________
  
                        FIELD INFORMATION NOTICE
               (For Authorized Distribution by Sun Service)
FIN #: I1144-1
Synopsis: Clustered Netra D130 array causes various SCSI errors under load if LVD SCSI Disk Drives are installed to replace failed single ended SCSI disk drives in a cross cabled configuration.
Create Date: Jan/12/05
SunAlert: No
Top FIN/FCO Report: No
Products Reference: Sun StorEdge st D130 Array
Product Category: Storage / Diag-Doc-Service
Product Affected: 
Systems Affected:
-----------------  
Mkt_ID         Platform      Model       Description           Serial Number
------         --------      -----       -----------           -------------
  -              N14          ALL        Netra t1405                 -
  -              N15          ALL        Netra t1400                 -


X-Options Affected:
-------------------
Mkt_ID      Platform     Model     Description                 Serial Number
------      --------     -----     -----------                 ------------- 
  -         st D130       ALL      Netra st D130 Array               -
X1032A         -           -       OPT INT PCI 10/100BASET NIC       -
Parts Affected: 
----------------------
Part Number        Description                        Model
-----------        -----------                        -----
390-0069-03        DRV SEA 36GB 10K 1-in SCSI3          -
390-0109-05        DRV SEA 36GB 10K 1-in SCSI4          -
390-0156-03        DRV FJ 36GB 10K1 SCSI4-T485-61       -
References: 
ESC: 1-3909668
Issue Description: 
Installation of LVD SCSI disk drives as replacements for failed single
ended SCSI disk drives in a cross cabled configuration with clustered 
Netra D130 array causes various SCSI errors under load.

The following is a SunCluster 3.0 configuration in which the two host
nodes are Netra t 1400 servers connected to a pair of Netra D130
arrays.  A Single-Ended Ultra/Wide SCSI/FastEthernet (SunSwift PCI) has
been installed in each host.  These hosts are cabled to the D130 arrays
using 4 0.8 Meter SCSI cables in a cross connected fashion.

The onboard SCSI connecter of the Node 0 Netra 1400 is connected to
the SCSI in port of the Array 1 Netra D130, the SunSwift HBA of this
host is connected to the SCSI out port of the Array 2 netra D130.

The onboard SCSI connecter of the Node 1 Netra 1400 is connected to
the SCSI in port of the Array 2 Netra D130, the SunSwift HBA of this
host is connected to the SCSI out port of the Array 1 netra D130.

Diagrams are shown below:

        	+-------------+
  Netra t 1400	|           |-|-----+ 
    Node 0	| =           |     |
		+-|-----------+     |
		  +---------+	    |	
		+-----------|-+     |
  Netra st D130	|           = |     |
    Array 1	|           =-|--+  |
		+-------------+  |  |
		                 |  |
		                 |  |
		+-------------+  |  |
  Netra st D130	|        +--= |  |  |
    Array 2	|        |  =-|--)--+
		+--------|----+  |
		         |       |
		  +------+       |
		+-|-----------+  |
  Netra t 1400	| |         |-|--+
    Node 1	| =           |
		+-------------+
		
There are several factors contributing to the observed issue.

   . In this configuration given the 1 meter internal bus length of
     the D130 array, the 0.8 meter SCSI cables used to connect the
     hosts, and the 5 total targets on the SCSI bus, the entire overall
     SCSI bus length exceeds the 1.5 meter maximum bus length for this
     configuration.  This introduces a potential SCSI signal
     degradation issue and thus SCSI errors.

   . The onboard SCSI adapter on one host is cross connected to the
     SunSwift HBA on the second host which is typically not done.

   . It appears that the LVD SCSI disk drives are more sensitive to the
     extended bus length / cross cabled configuraton than single ended
     SCSI disks and thus genenerate the SCSI errors.

All of these factors combined to produce the SCSI errors observed onsite.

The following are examples of the SCSI errors seen on the D130 array.

SCSI Errors reported against the SunSwift HBA:
 
  Sep  20 03:16:47 DS2cable0 SCSI: [ID 107833 kern.warning] WARNING: 
             /pci@1f,4000/pci@5/SUNW,isptwo@4 (isp0):
  Sep  20 03:16:47 DS2cable0  Interrupt bit still set after 10 seconds.
             Card or firmware failure.
  Sep  20 03:16:51 DS2cable0 SCSI: [ID 107833 kern.warning] WARNING: 
             /pci@1f,4000/pci@5/SUNW,isptwo@4/sd@a,0 (sd9):
  Sep  20 03:16:51 DS2cable0 SCSI transport failed: reason 'reset':
             retrying command.

SCSI Errors reported against the onboard SCSI adapter:

  Sep 20 03:27:38 DS2cable0 SCSI: [ID 107833 kern.warning] WARNING: 
             /pci@1f,4000/scsi@3,1 (glm1):
  Sep 20 03:27:38 DS2cable0  Resetting SCSI bus, Message-In was expected  
             from (11,0)
  Sep 20 03:27:38 DS2cable0 genunix: [ID 408822 kern.info] NOTICE: glm1:  
             fault detected in device; service still available
  Sep 20 03:27:38 DS2cable0 genunix: [ID 611667 kern.info] NOTICE: glm1:  
             Resetting SCSI bus, Message-In was expected from (11,0)
  Sep 20 03:27:38 DS2cable0 SCSI: [ID 107833 kern.warning] WARNING: 
             /pci@1f,4000/scsi@3,1 (glm1):
  Sep 20 03:27:38 DS2cable0 	Target 11 reducing sync. transfer rate
  Sep 20 03:27:38 DS2cable0 glm: [ID 923092 kern.warning] WARNING: 
             ID[SUNWpd.glm.sync_wide_backoff.6014]
  Sep 20 03:27:38 DS2cable0 SCSI: [ID 107833 kern.warning] WARNING:
             /pci@1f,4000/scsi@3,1 (glm1):
  Sep 20 03:27:38 DS2cable0 	got SCSI bus reset

Issue an 'iostat -En' command from an operating system prompt, review the
output and look for one of the three Vendor/Product combinations listed
below:

   Vendor: FUJITSU  Product: MAP3367NC
   Vendor: SEAGATE  Product: ST336605LC
   Vendor: SEAGATE  Product: ST336607LC

See the sample iostat -En output below:

   #iostat -En 
   sd10     Soft Errors: 0 Hard Errors: 2 Transport Errors: 0 
   Vendor: FUJITSU  Product: MAP3367NC SUN18G  Revision: 0804 Serial No: 05P32232     
   Size: 18.11GB <18110967808 bytes>
   Media Error: 0 Device Not Ready: 0 No Device: 2 Recoverable: 0  
   Illegal Request: 0 Predictive Failure Analysis: 0

Under normal conditions, LVD drives can be used as a one for one
replacement for Single Ended SCSI disks.  But in the configuration
detailed above, the LVD disks combined with an Extended SCSI bus length
for cluster configurations and a cluster configuration using differing
HBAs in cross connected fashion are contributing to the SCSI signal
degradaton which leads to the SCSI errors.

Removal and replacement of existing LVD disks in affected Netra D130
arrays resolves the SCSI error issue.
Implementation: 
---
        |   |   MANDATORY (Fully Proactive)
         ---    
         
  
         ---
        |   |   CONTROLLED PROACTIVE (per Sun Geo Plan) 
         --- 
         
                                
         ---
        | X |   REACTIVE (As Required)
         ---
Corrective Action: 
The following recommendation is provided as a guideline for authorized
Sun Services Field Representatives who may encounter the above
mentioned issue.

When replacing failed drives in the D130 arrays, only use a single ended
SCSI drive for a replacement disk. The following details the procedure that 
must be followed to guarantee that an appropriate drive will be shipped 
as a replacement.

   1. NOTE that the FE/SSE needs to contact the RSL directly
      (the Parts Call Center will not be able to do this for them).
      Upon doing so, DO NOT advise the RSL to open the box.  Having
      RSLs open boxes will introduce risk to the part, as they do not
      have any ESD or parts handling training.  Instead, the RSL
      should check the drive base part number that is noted on the
      external box label:

           TOP LEVEL FRU:  540-4689

             Different versions of bare drives that go into this
             same top level FRU part number are noted on the label:
               
                 * 390-0050 Single Ended SCSI (OK)
                 * 390-0051 Single Ended SCSI (OK)
                 * 390-0052 Single Ended SCSI (OK)
                 * 390-0069 LVD SCSI (not acceptable)
                 * 390-0109 LVD SCSI (not acceptable)
                 * 390-0156 LVD SCSI (not acceptable)

    2. Make sure FE/SSEs are aware that having the RSLs manually check 
       the boxes may slightly increase the order processing time.
Comments: 
Acronyms used for this FIN:

   LVD - Low Voltage Differential

============================================================================

NOTE: FIN Tracking Instructions for Radiance/SPWeb:
--------------------------------------------------

If a Radiance case involves the application of a FIN to solve a customer
issue, please complete the following steps in Radiance/SPWeb prior to
closing the case:
 
    o Select "Field Information Notice" in the REFERENCE TYPE field.

    o Enter FIN ID number in the REFERENCE ID field.
      For example; I1111-1.

If possible, include additional details in the REFERENCE SUMMARY field
(ie. implementation complete, customer declined, etc.)
--------------------------------------------------------------------------


Implementation Notes:
--------------------

In case of "Mandatory" FINs, Sun Services will attempt to contact
all known customers to recommend proactive implementation.

For "Controlled Proactive" FINs, Sun Services mission critical
support teams will initiate proactive implementation efforts for
their respective accounts as required.

For "Reactive" FINs, Sun Services and partners will implement
the necessary corrective actions as the need arises.


Billing Information:
-------------------

Warranty: On-Site Labor Rates are based on specified Warranty deliverables
          for the affected product.

Contract: On-Site Labor Rates are based on the type of service contract.

Non Contract: On-Site implementation by Sun is available based on On-Site
              Labor Rates defined in the Price List.

--------------------------------------------------------------------------

All FIN documents are accessible via Internal SunSolve.  Type "sunsolve"
in a browser and follow the prompts to Search Collections.

For questions on this document, please email:

        finfco-manager@Sun.com

The FIN and FCO homepage is available at:

        http://sdpsweb.central/FIN_FCO/index.html

For more information on how to submit a FIN, go to:

        http://pronto.central/fin.html

To access the Service Partner Exchange, use:

        https://spe.sun.com
--------------------------------------------------------------------------