Sun Storage T3 Array
 
Sun Storage T3+ Array
 
Sun Storage 3910 Array
 

GCS>Sun Microsystems>Sun Alert>Criteria Category>Availability
 
GCS>Sun Microsystems>Sun Alert>Release Phase>Resolved
 
GCS>Sun Microsystems>Sun Alert>Criteria Category>Data Loss
 

Sun StorEdge T3 and T3+ arrays (including Sun StorEdge T3+ arrays contained in the Sun StorEdge 3900 and Sun StorEdge 6900 Series) may reset and/or lose host connectivity for 2-3 minutes if it has been running continuously for exactly 497 days and if I/O operations are in progress at that time.

Data may become unavailable or may get lost permanently, depending on the system configuration and how applications react to the arrays resetting or temporarily losing host connectivity.

This issue can occur with the following configurations:

• Sun StorEdge T3 array with firmware 1.18.01 or earlier
• Sun StorEdge T3+ array with firmware 2.01.03 or earlier
• Sun StorEdge 3900 Series (SE39xx) containing Sun StorEdge T3+ arrays with firmware 2.01.03 or earlier
• Sun StorEdge 6900 Series (SE69xx) containing Sun StorEdge T3+ arrays with firmware 2.01.03 or earlier

This issue will only occur if I/O operations are executed on the array at the time it has been running continuously for exactly 497 days. If the array is idle at that time, this issue will not occur.

Note: There is no "uptime" or similar command on the T3/T3+. To identify how long the T3/T3+ has been running it is necessary to review the T3/T3+ syslog (or remote logging file), or to review the change logs possibly kept by the system administrator to find the date of the last T3/T3+ boot. Applying new firmware to a T3/T3+ requires an array reboot. Therefore, T3/T3+ arrays whose firmware has been kept updated with firmware releases will have been rebooted during the update process and hence are less likely to be impacted by this issue.

1. Messages similar to the following may be seen in the "/var/adm/messages" file:

    unix: ID[SUNWssa.socal.link.5010] socal3: port 0: Fibre Channel is OFFLINE
unix: WARNING: /sbus@49,0/SUNW,socal@1,0/sf@0,0 (sf6):
unix:        Offline Timeout
unix: sf6:   target 0x2 al_pa 0xe4 offlined
unix: WARNING: /sbus@49,0/SUNW,socal@1,0/sf@0,0/ssd@w50020f2300007193,0 (ssd3):
unix:        SCSI transport failed: reason 'tran_err': giving up

The above messages indicate a loss of host connectivity to a T3/T3+ array and may occur for different reasons. Should the issue described in this Sun Alert document occur, one of the sets of T3/T3+ messages listed below will also be seen:

2. As it restarts, the T3/T3+ syslog may record the following reason for the T3/T3+ resetting:

    ROOT[1]: W: u1ctr Assertion Reset (3000) was initiated at yyyymmdd
hhmmss ../../common/bss/qlcf.c line xxxx, Assert(cmd->cmd_deadline
!= CAM_TIME_INFINITY) => 0 BOOT

3. The T3/T3+ syslog may record messages similar to the following, at the time that host connectivity is lost:

    ISR1[1]: N: u1ctr ISP2100[2] Fatal timeout on host 125
ISR1[1]: N: u1ctr ISP2100[2] Received LIP(f0,f0) async event
ISR1[1]: N: u1ctr ISP2100[2] qlcf_invalidate_pdb: PDB Invalidate (host id 125)
FCC0[1]: N: u1ctr Port event received on port 0, abort 0 (id 125)
ISR1[1]: N: u1ctr ISP2100[2] qlcf_sync_pdb: PDB Sync Initiated (host id 125)
ISR1[1]: N: u1ctr ISP2100[2] qlcf_update_pdb: PDB Sync Done (host id 125)
ISR1[1]: N: u1ctr ISP2100[2] PDB Sync Done (host id 125,
host WWN 2004020000101a00)
FCC0[1]: N: u1ctr PDB Changed on port 0 (id 125)
[...]

10 seconds later:

    ISR1[1]: N: u1ctr ISP2100[2] Fatal timeout on host 125
ISR1[1]: N: u1ctr ISP2100[2] qlcf_i_watch_host_port: Debug Code - ISP2100 Hang
Detected
ISR1[1]: N: u1ctr ISP2100[2] interface going offline
ISR1[1]: N: u1ctr ISP2100[2] qlcf_init_pdb: PDB Initialize
ISR1[1]: N: u1ctr ISP2100[2] QLCF_I_ABORT_ALL_TM_CMDS: Target-mode Flush Started
(lun = 0x0)
ISR1[1]: N: u1ctr ISP2100[2] interface going online
[...]
SIMT[1]: N: u1ctr Initializing host port u1p1 ISP2100 ... firmware status = 7
[...]
DUMP[1]: N: u1ctr ISP2100[2] [==>BEG]ISPDEBUGDUMP:
DUMP[1]: N: u1ctr ISP2100[2]  PBIU REGISTERS (OFFSET 00H, 8):
DUMP[1]: N: u1ctr ISP2100[2]        0000 0001 0002 0003 0004 0005 0006 0007
DUMP[1]: N: u1ctr ISP2100[2]        ---- ---- ---- ---- ---- ---- ---- ----
[... followed by lines of hex data ...]

To prevent the described issue, the Sun StorEdge T3/T3+ array needs to be rebooted before it has been running for 497 days.

This issue is addressed in the following releases:

• Sun StorEdge T3 array with firmware 1.18.02 or later (firmware 1.18.02 is available as patch 109115-13)
• Sun StorEdge T3+ array with firmware 2.01.04 or later (firmware 2.01.04 is available as patch 112276-07)
• Sun StorEdge 3900 Series (SE39xx) containing Sun StorEdge T3+ arrays with firmware 2.01.04 or later (firmware 2.01.04 is available as patch 112276-07)
• Sun StorEdge 6900 Series (SE69xx) containing Sun StorEdge T3+ arrays with firmware 2.01.04 or later (firmware 2.01.04 is available as patch 112276-07)

• Sun StorEdge T3 array firmware 1.18.02 is available as patch 109115-13

• Sun StorEdge T3+ array firmware 2.01.04 is available as patch 112276-07

References

A scheduled reboot, as described in the "Relief/Workaround" section above, may also be used to update arrays to the latest firmware revision (although currently available firmware does not address the issue described in this document).

References

This solution has no attachment