Sun Microsystems, Inc.  Sun System Handbook - ISO 3.4 June 2011 Internal/Partner Edition
   Home | Current Systems | Former STK Products | EOL Systems | Components | General Info | Search | Feedback

Asset ID: 1-77-1000427.1
Update Date:2010-08-04
Keywords:

Solution Type  Sun Alert Sure

Solution  1000427.1 :   Certain Sun StorageTek CSM200 X-Option Trays Are Shipped With Existing DACstore  


Related Items
  • Sun Storage Flexline 380 Array
  •  
  • Sun Storage Flexline 240 Array
  •  
  • Sun Storage Flexline 280 Array
  •  
  • Sun Storage 6540 Array
  •  
  • Sun External I/O Expansion Unit
  •  
Related Categories
  • GCS>Sun Microsystems>Sun Alert>Criteria Category>Availability
  •  
  • GCS>Sun Microsystems>Sun Alert>Release Phase>Resolved
  •  

PreviouslyPublishedAs
200560


Product
Sun StorageTek CSM200 Expansion Tray

Bug Id
<SUNBUG: 6600793>, <SUNBUG: 6455157>, <SUNBUG: 6596898>

Date of Resolved Release
17-SEP-2007

Impact

Certain Sun StorageTek CSM200 X-Option trays are shipped with an existing DACstore (array metadata), resulting in one or more of the following conditions:

- Loss of management and data access

- Inability to apply feature licenses

- Inability to upgrade firmware

- Component details showing up incorrectly or receiving alarms

- Host operating systems reporting the wrong product identifier, array registration or discovery (may hang or fail to complete)

- Issues with multipathing failover 


Contributing Factors

This issue can occur on the following platforms:

  • Sun StorageTek CSM200 Expansion Tray for 6130, 6140 and 6540 Storage Arrays
  • Sun StorageTek CSM200 Expansion Tray for FLX240, FLX280 and FLX380 Storage Arrays

when either of the following conditions are met:

- CSM200 Expansion Tray X-Options are added to the array with the RAID module powered off

- CSM200 trays and associated drives that may have been pre-configured prior to shipping


Symptoms

The most prominent symptom is that an affected storage array appears to the management interface or data host inquiry as a 6140 or CSM200_R instead of 6130, 6540, 6140, or FLX240, FLX280, FLX380, CSM100_R, or OPENstorage.

- CSM100_R_FC is the 6130 Array

- FLEXLINE 380 is the Flexline 380 or Sun StorageTek 6540 Array

- OPENstorage D280 is the Flexline 280 Array

- OPENstorage D240 is the Flexline 240 Array

To confirm this symptom, a comparison must be made between the Controller Board ID, Product ID, and NVSRAM version.

For Common Array Manager (CAM):

1. Use the Troubleshooting-> FRUs-> Controllers-> Controller Name link to get the entry for the Board ID field.

2. Compare that to the output of option 1 of "csmservice -s" for the NVSRAM loaded. Example "csmservice -s" option 1 output (see Section 4 for command path):

    Enter your selection 0 - 6: 1
    Analyzing array ... Please wait ...
    Controller: CSM200_R - Version 06.19.25.10
    NVSRAM: N399X-619843-004

For SANtricity Storage Manager(SANtricity):

Use the Controller Menu-> Properties, and compare the Board ID to the NVSRAM loaded. NVSRAM is identified by a standard format of Nxxxx-xxxxxx-xxx. The first and second field of the NVSRAM version, "Nxxxx-xxxxxx", should match the Board ID and Product ID as follows:

N288X-619855 should have a board ID of "288x" and a Product ID of OPENstorage D240

N2882--619843 should have a board ID of "2882" and a Product ID of CSM100_R

N399X-619843 should have a board ID of "3992" or "3994" and a Product ID of CSM200_R

N6091-619843 should have a board ID of "6091" and a Product ID of FLEXLINE 380

N588X-619855 should have a board ID of "5884" and a Product ID of OPENstorage D280

Other symptoms may include:

  • Alarms or alerts are generated for drive channels degraded on FLX380 or 6540 arrays, where the array channels shown are 1, 1, 1, 2, and should be 1 through 4 for each tray. This causes a false display and false alarm in the management interface.
  • Loss of access to the array, and the controller 7-Segment LEDs show a constantly changing letter and number combination instead of a stable number. This indicates a controller boot loop.
  • The IP address on the controller is wrong or has been changed.
  • CAM cannot register the array.
  • Firmware updates fail when done via CAM.
  • Feature Enabler Identifiers are duplicated in SANtricity/CAM, preventing the addition of Feature Licenses.
  • Array WWN changes (can cause loss of access within SAN switch zoning).

Workaround

The corrective action requires a service interruption.

Note: This issue is documented in the STK6540 Release Notes at http://docs.sun.com/source/819-6521-14/6540_Release_Notes.html  on pgs 16-17. However, the workaround mentioned in the Release Notes does not work (no -p option with csmservice).

For any addition of new CSM200 expansion trays to an existing array in a production or active environment, cable and add the trays while the RAID Controller Module is powered on. The issue will be avoided altogether.

For new installations of RAID Controllers plus expansions, or for systems that have no data on them but match the symptoms above:

1. Register the RAID head and check for the correct NVSRAM revision. If incorrect, continue to step 2.

2. Reload the correct NVSRAM. For CAM 5.1 and above, use "csmservice -s".

Solaris Location:

/opt/SUNWstksm/bin/csmservice

/var/sadm/swimages/<array_model>/CRM-F-NVSRAM/image.fw

Windows Location:

%Program Files%\Sun\Common Array Manager\Component\SunStorageTekSoftwareManager\bin\csmservice

%Program Files%\Sun\Common Array Manager\Component\SunStorageTekSoftwareManager\<array_model>\CRM-F-NVSRAM\image.fw

Linux Location:

/opt/sun/cam/private/csmservice

/opt/sun/cam/share/fw/<array_model>/CRM-F-NVSRAM

For SANtricity:

Use the Advanced Menu-> Maintenance-> Sub-menu-> Download Sub-menu-> Download NVSRAM

The NVSRAM files are located on the SANtricity CD-ROM Shipped with the unit.

3. Reset the configuration. For CAM, use the "Reset Configuration" button under the "Administration" menu tree for the array name.

4. Verify that the NVSRAM version is correct and the product ID strings have been corrected on the array. For CAM, you will need to remove and re-register the array to pick up the product ID correctly.

Notes:

  1. All corrective actions for this issue require loss of host access to the array during the NVSRAM update, due to the boot cycle of the array controllers. An outage will be required.
  2. Some corrective actions require a configuration reset and restore of the affected array. It is recommended that backups of the data on the affected array be up to date.

Resolution

CSM200 Expansion trays packaged in at the factory after August 29, 2007 no longer have this issue due to a change in the manufacturing process, ensuring that the drives enclosed no longer contain any system data.

Customers choosing to migrate drives or trays from a Sun StorageTek 6140 array to another array that supports the CSM200 trays are still exposed to this issue, if performed while the RAID Controller Module is powered off. A "Best Practice" recommendation is to involve Sun Support Services for any type of drive or data migration between arrays, due to the symptoms and resolutions described in this alert.



Modification History
Date: 12-OCT-2007
  • Update Synopsis, Impact, and Contributing Factors sections

Date: 13-DEC-2007
  • Updated Relief/Workaround section

Previously Published As 103067
Internal Comments
When a CSM200 drive tray is received with more than three drives with
DACStore already present on the drives (dirty dacstore), the problem
of pushing up the incorrect DACStore and NVSRAM to the controller
exists. This can happen when the CSM200 is attached to an array while
it is powered off. Upon power up, the array controller scans all
attached drives for sundry drives, and loads dacstore from the first
set discovered. This can result in a mixed (merged) configuration when
dirty CSM200 trays are added to existing
systems, or just a wrong
(mismatched) configuration on "new" systems. In either event, if the
6140 NVSRAM is copied from the sundry drives on the
dirty CSM200 tray to the 6540, FLX380, or FLX280 controller, several
things happen: the NVSRAM, PID, the array WWN, the Feature ID, drive
channel information in the profile, and the Network information will be incorrect.
A storageArrayProfile.txt will show the following fields for a 6540:
Tray.85.Controller.A
Status: Optimal
Current configuration
Firmware version:
Appware version: 06.19.25.10
Bootware version: 06.19.25.10
-->> NVSRAM version: N399X-619843-004 <<--
-->> Board ID: 6091 <<--
Replacment part number: 371-1810-01
-->> Product ID: CSM200_R<<--
The first field of the NVSRAM version(Nxxx), should match the Board ID field.
N2882 should have a board ID of "2882"
N399x should have a board ID of "3992" or "3994"
N6091 should have a board ID of "6091"
N5884 should have a board ID of "5884"
A storageArrayProfile.txt will show the following on a FLX380 or 6540
array with this issue:
CHANNEL STATUS CTRL A LINK CTRL B LINK
1 Optimal Up Up
1 Optimal Failed Failed
1 Optimal Failed Failed
2 Optimal Up Up
Corrective Action for New Trays being added to an array:
For any addition of new CSM200 expansion trays to an existing array in
a production or active environment, cable and add the trays while the
RAID Controller Module is powered on. The issue will be avoided
altogether.Alternatively, the processes documented in
InfoDoc 91030: "Clearing DACstore Before Adding Trays for Sun
StorageTek[TM], Sun StorEdge[TM], and StorageTek[TM] Flexline Arrays"
http://sunsolve.sun.com/search/document.do?assetkey=1-9-91030-1
Corrective Action For Arrays in use/production:
The following are to be evaluated and completed under the guidance of Support:
NOTE: All corrective actions below are disruptive to
production and require an outage. No exceptions. Arrays will have to
be re-registered into CAM.
It is recommended that an escalation be opened for assistance with the actions below.
1. Incorrect NVSRAM. Download the correct NVSRAM, using "csmservice
-s" in CAM or SANtricity
2. Incorrect Product ID - Download the correct NVSRAM.
The incorrect Product ID presented to the host will be corrected once
the proper NVSRAM is loaded. This requires that the LUNs be un-mounted
at the Host and re-presented once the Host Type(s) has been corrected,
a reboot of certain hosts may be required.
3. Incorrect Network Information. Manual correction to proper values
using CAM or SANtricity.
4. Incorrect Host Types installed. Download the correct NVSRAM
With the improper NVSRAM in place, the Host Types will be located in
different regions. This will require a change to the configured
Types to properly select the correct Host region required for use.
5. Incorrect Feature ID. Run a safeSysWipe on both controllers and
reboot. This will create a new feature ID.
Customers with existing volumes may not be able to correct this
without wiping the configuration and recovering it. Customers found
with this situation should have a case opened and escalated
immediately for corrective action.
6. Duplicate Array WWN
NOTE: This only needs to be done if the 2 arrays have the same WWN in the SAN.
A. From the shell of one of the controllers, first enter
-> VKI_KMZALLOC 16,
This will return a random decimal value (and its hex equivalent) that
is a pointer to 16 bytes of zeroed out memory.
B. Use this returned address in the following command:
-> isp cfgWriteSAWWN,0x
C. Reboot both controllers. This part will affect host I/O. This will
generate a new saID on the module, and change the WWN. This

affects
switch zoning, and persistent binding on all attached hosts and switches.
If a specific saID is required, the array system memory must be hand-edited.
7. System is in a boot loop
Customers with existing volumes may not be able to correct this
without wiping the configuration and recovering it. Customers found
with this situation should have a case opened and escalated
immediately for corrective action.
Internal Contributor/submitter
curtis.decotis@sun.com
Internal Eng Responsible Engineer
brian.hanvey@sun.com

Attachments
This solution has no attachment
  Copyright © 2011 Sun Microsystems, Inc.  All rights reserved.
 Feedback