Sun Storage 3510 FC Array
 
Sun Storage 3511 SATA Array
 

GCS>Sun Microsystems>Storage - Disk>Modular Disk - 3xxx Arrays
 

Applies to:

Goal

- Attached via a storage area network (SAN) with STMS enabled or disabled

Determination from a Symantec Volume Manager subdisk will also be explained.

Solution

Steps to Follow:

If all of the above are true, /usr/sbin/format (herein referred to as format) will display the device and /usr/sbin/luxadm will recognize the local host bus adapters (HBA) that are connected:

    # /usr/sbin/luxadm -e port

    Found path to 1 HBA port

    /devices/ssm@0,0/pci@1d,700000/pci@2/SUNW,qlc@4/fp@0,0:devctl      CONNECTED

NOTE: If the HBA is "NOT CONNECTED", further troubleshooting is required. Possible causes are hardware (eg HBA, cables or switch port), software (eg SAN package is not installed or installed correctly or HBA driver) or the switch (eg zoning). Knowledge Article 1009557.1 provides useful Fibre Channel troubleshooting tips that can be applied to resolve many issues.

Array Directly Connected or Connected Via a SAN with STMS Disabled

On the Solaris host, the format disk listing will look similar to the following:

    AVAILABLE DISK SELECTIONS:
0. c0t8d0 
/ssm@0,0/pci@1c,700000/scsi@3/sd@8,0
1. c0t9d0 
/ssm@0,0/pci@1c,700000/scsi@3/sd@9,0
3. c3t216000C0FF87D15Ad0 
/ssm@0,0/pci@1d,700000/pci@2/SUNW,qlc@4/fp@0,0/ssd@w216000c0ff87d15a,0
4. c3t216000C0FF87D15Ad1 
/ssm@0,0/pci@1d,700000/pci@2/SUNW,qlc@4/fp@0,0/ssd@w216000c0ff87d15a,1
5. c3t216000C0FF87D15Ad2 
/ssm@0,0/pci@1d,700000/pci@2/SUNW,qlc@4/fp@0,0/ssd@w216000c0ff87d15a,2
Specify disk (enter its number):

1.) Take disk 3 for example:

           3. c3t216000C0FF87D15Ad0 
/ssm@0,0/pci@1d,700000/pci@2/SUNW,qlc@4/fp@0,0/ssd@w216000c0ff87d15a,0

...the device path can be broken down to determine exactly which array, logical disk and LUN this device is referring to.

From the output above, the item of interest is the last bit of the the device path, namely 216000c0ff87d15a,0.

This can be broken down into individual components as follows:

216000c0ff87d15a,0
|--------------| ^
^                |
|                 -> 0 equates to LUN 0
|
 -> 216000C0FF87D15A is the World Wide Port Name (WWPN) of the SE3510.  

The WWPN, 216000c0ff87d15a is further broken down into individual compontents as follows:

(spaced for readability)

2 1 6000c0ff 8 7d15a

-|-|--------|-|-----
^ ^     ^    ^   ^
| |     |    |   |
| |     |    |    -> The last 5 digits (7d15a) make up the SE3510's unique-identifier.
| |     |    |       The unique-identifier is unique for each SE3510.  If the chassis
| |     |    |       is replaced, the unique-identifier will change and show differently
| |     |    |       in the WWN of subsequently (re-)created LD's.
| |     |    |       The unique-identifier number is located on the front plate
| |     |    |       (bottom) when the front cover is opened and may be preceeded by a 0.
| |     |    |
| |     |     -> This is the least-significant nibble of the internal AL_PA number assigned to
| |     |        the port on the controller in hexidecimal (HEX), ie 0x?8 where ? is the
| |     |        most-significant nibble and 8 is the least-significant nibble. The internal AL_PA
| |     |        is a HEX representation of the target ID assigned to the port within the
| |     |        "view and edit Scsi channels" menu in the controller firmware. For example a target
| |     |        ID of 40 equates to an internal AL_PA of 0x28 which fits this example, as would an
| |     |        ID of 24 (0x18) or 56 (0x38), however there is no way to determine the most-significant nibble
| |     |        of the HEX AL_PA number from the WWN. This is not important anyway as
| |     |        the primary and secondary controllers are internally linked within the array
| |     |        thus providing seamless failover between the controllers.
| |     |
| |      -> These 8 digits (6000c0ff) can be explained as follows: .
| |         60 is a product identifier for DotHill.
| |         00c0ff is vendor unique ID from IEEE assign to DotHill.
| |
|  -> Indicates the channel (n) that this connection is made to on the
|     array in the form of (n+1). This example illustrates a connection on chl 0.
|     This will change according to the channel involved.
|
-> Indicates the Network Address Authority (NAA) format of the World Wide Number (WWN)
1 = IEEE, 2 = IEEE Extended, 5 = IEEE Registered, 6 = IEEE Registered extended 128-bit

From the information obtained above, it can be summarised that disk 3

  - is mapped to channel 0
- on a SE3510 array
- with unique identifier 7D15A

2.) The physical location of the HBA can be determined by referencing the relevant article in the Troubleshooting section of the machine in question, in the Sun[TM] System Handbook.

In this case, the device path /devices/ssm@0,0/pci@1d,700000/pci@2/SUNW,qlc@4/fp@0,0:devctl correlates to slot 5 on interface board 8 on a Sun Fire 6800.

3.) If the host is accessing the array via a SAN, the switch ports involved can be determined by displaying the World Wide Name information map for each device path:

    # /usr/sbin/luxadm -e dump_map /devices/ssm@0,0/pci@1d,700000/pci@2/SUNW,qlc@4/fp@0,0:devctl
Pos  Port_ID Hard_Addr Port WWN         Node WWN         Type
0    10100   0         216000c0ff87d15a 206000c0ff07d15a 0x0  (Disk device) <- Connection to array
1    10200   0         210100e08b2421b3 200100e08b2421b3 0x1f (Unknown Type)
2    10000   0         210100e08b2424b3 200100e08b2424b3 0x1f (Unknown Type,Host Bus Adapter) <- Connection to host

The "Port_ID" column can be broken down into individual components. The Port ID should be 6 digits, however, if the first digit is a 0, it may be removed. Thus, in this example, the full Port ID is 010000 and is broken down as follows:

    01 00 00      01 = is the switch domain ID
00 = is the physical port number
00 = is the AL_PA address.  This is not used by the SE3510,
so will always be 00

Note: All numbers are hexadecimal (hex) values and are subject to the appropriate port numbering rules of the switch in question.

With all the information listed above, one can map the disk from the host to
to the switch and then to the SE3510. In this example, we have determined that c3t216000C0FF87D15Ad0 maps to:

  - LUN 0
- that is mapped to channel 0
- on a SE3510 array with unique identifier 7D15A
- which is connected to port 1
- on the switch with domain ID 01.

The connection from the host to the switch is from:

  - the HBA in slot 5
- on interface board 8 of our SunFire 6800
- to port 0
- on the switch with domain ID 01

Array Directly Connected or Connected Via a SAN with STMS Enabled

The procedure for mapping the disk device to the array and LUN with STMS enabled is very similar to the procedure above. However there is an extra step that needs to be taken first, in order to determine the underlying devices. These steps are detailed using a different example from a SunFire[TM] V880.

When STMS is enabled the format disk listing will display STMS enabled devices similar to the following:

         4. c4t600C0FF0000000000034AA0717A93C02d0 
/scsi_vhci/ssd@g600c0ff0000000000034aa0717a93c02

Notice how the device path starts "/scsi_vhci/". This indicates the device is a STMS device.

The last part of the device path, 600c0ff0000000000034aa0717a93c02, provides a lot more information when STMS is enabled than when it's disabled.

This device name can be broken down into individual components as follows: (spaced for readability)

6 00c0ff 0000000000 034aa 0717a93c 02
-|------|----------|-----|--------|--
^   ^         ^       ^       ^    ^
|   |         |       |       |    |
|   |         |       |       |     -> 02 indicates the Partition number on the logical disk.
|   |         |       |       |        If the logical disk has not been partitioned, then
|   |         |       |       |        this value will be 00
|   |         |       |       |
|   |         |       |        -> This indicates the Logical Disk ID for the logical disk on
|   |         |       |           the SE3510.
|   |         |       |
|   |         |        -> The SE3510's unique-identifier.
|   |         |
|   |          -> Padding introduced by STMS in order for the WWN to comply with the IEEE standard
for this type of WWN (128 bits).
|   |
|    -> The unique IEEE vendor ID as outlined earlier.
|
-> Indicates the Network Address Authority (NAA) format of the World Wide Number (WWN) as outlined earlier.

Thus, from the device name alone, the device can be mapped to:

  - a LUN
- mapped from partition 2
- on a logical disk with ID 0717A93C
- on a SE3510
- with unique identifier of 034AA.

In order to map the logical device to the two (or more) paths to the LUN and determine the LUN number, extract the individual (physical) device addresses from the luxadm display output for this STMS device:

   # luxadm display /dev/rdsk/c4t600C0FF0000000000034AA0717A93C02d0s2
Vendor:		        SUN
Product ID:	        StorEdge 3510
Revision:		        327P
Serial Num:	        0034AA0717A9
Unformatted capacity:	377487.375 MBytes
Read Cache:		Enabled
Minimum prefetch:	0x0
Maximum prefetch:	0xffff
Device Type:		Disk device
Path(s):
/dev/rdsk/c4t600C0FF0000000000034AA0717A93C02d0s2
/devices/scsi_vhci/ssd@g600c0ff0000000000034aa0717a93c02:c,raw
Controller      	        /devices/pci@8,700000/SUNW,qlc@2/fp@0,0
Device Address		        256000c0ffc034aa,5
Host controller port WWN	        210000e08b0e5e46
Class			        primary
State			        ONLINE
Controller      	        /devices/pci@9,700000/SUNW,qlc@4/fp@0,0
Device Address		        216000c0ff8034aa,5
Host controller port WWN	        210000e08b0e502d
Class			        primary
State			        ONLINE

This can be shortened into one command:

     # luxadm display /dev/rdsk/c4t600C0FF0000000000034AA0717A93C02d0s2 | grep "Device Address"
Device Address		        256000c0ffc034aa,5
Device Address		        216000c0ff8034aa,5

Then, map each device address to the SE3510 and switch ports as we did in Step 1 above.

• Determine array and LUN ( Step 1)

    256000c0ffc034aa,5 -> 2 = IEEE Extended WWN type
-> 5 = Channel 4 on the array
-> 6000c0ff = Identifies array as SE3510
-> c = Least significant bit of internal AL_PA
-> 034aa = SE3510 unique identifier
-> 5 = LUN 5

    216000c0ff8034aa,5 -> 2 = IEEE Extended WWN type
-> 1 = Channel 0 on the array
-> 6000c0ff = Identifies array as SE3510
-> 8 = Least significant bit of internal AL_PA
-> 034aa = SE3510 unique identifier
-> 5 = LUN 5

• Determine the physical locations of the HBA's ( Step 2)

    /devices/pci@8,700000/SUNW,qlc@2/fp@0,0 = PCI Slot 3 on SunFire V880

    /devices/pci@8,700000/SUNW,qlc@2/fp@0,0 = PCI Slot 1 on SunFire V880

• Determine the switch connections (if connected via a SAN) ( Step 3)

     # luxadm -e dump_map /devices/pci@8,700000/SUNW,qlc@2/fp@0,0

      Pos  Port_ID Hard_Addr Port WWN         Node WWN         Type
0    20000   0         256000c0ffc034aa 206000c0ff0034aa 0x0  (Disk device) <- Connection to chl 4 on array
1    20200   0         266000c0ffe034aa 206000c0ff0034aa 0x0  (Disk device)
2    20700   0         210000e08b0ecf48 200000e08b0ecf48 0x1f (Unknown Type)
3    20900   0         266000c0ffe034d3 206000c0ff0034d3 0x0  (Disk device)
4    20a00   0         210100e08b2bacf7 200100e08b2bacf7 0x1f (Unknown Type)
5    20b00   0         256000c0ffc034d3 206000c0ff0034d3 0x0  (Disk device)
6    20e00   0         210000e08b0e5e46 200000e08b0e5e46 0x1f (Unknown Type,Host Bus Adapter) <- Connection to first HBA on host

    # luxadm -e dump_map /devices/pci@9,700000/SUNW,qlc@4/fp@0,0

      Pos  Port_ID Hard_Addr Port WWN         Node WWN         Type
0    10000   0         216000c0ff8034aa 206000c0ff0034aa 0x0  (Disk device)    <--- Channel 0 on array
1    10200   0         226000c0ffa034aa 206000c0ff0034aa 0x0  (Disk device)
2    10700   0         210000e08b0e402d 200000e08b0e402d 0x1f (Unknown Type)
3    10900   0         216000c0ff8034d3 206000c0ff0034d3 0x0  (Disk device)
4    10a00   0         210000e08b0bacf7 200000e08b0bacf7 0x1f (Unknown Type)
5    10b00   0         226000c0ffa034d3 206000c0ff0034d3 0x0  (Disk device)
6    10e00   0         210000e08b0e502d 200000e08b0e502d 0x1f (Unknown Type,Host Bus Adapter)  <--- Connection to second HBA on host

To Summarize this last example, it can be confirmed that c4t600C0FF0000000000034AA0717A93C02d0s2 is mapped to:

  - LUN 5
- which is mapped to channels 0 and 4
- of a SE3510
- with unique ID 034AA

and is accessible via two paths, one:

  - via the HBA in PCI slot 3
- which is connected to port 0
- on a switch with domain ID 02

and the the other:

  - via the HBA in PCI slot 1
- which is connected to port 14
- on a switch with domain ID 01

Determine Path for Veritas Volume Manager Device

If the LUN is under the control of Veritas Volume Manager and disk naming is in use, you can determine the underlying physical device by running "vxdisk list "

eg:

     # vxdisk list Bottom-3510_9
Device:    Bottom-3510_9
devicetag: Bottom-3510_9
type:      sliced
hostid:    {hostname}
disk:      name= id=1069863671.1825.testsys
group:     name=ib2dg id=1069863730.1834.testsys
info:      privoffset=1
flags:     online ready private autoconfig autoimport
pubpaths:  block=/dev/vx/dmp/Bottom-3510_9s4
char=/dev/vx/rdmp/Bottom-3510_9s4
privpaths: block=/dev/vx/dmp/Bottom-3510_9s3
char=/dev/vx/rdmp/Bottom-3510_9s3
version:   2.2
iosize:    min=512 (bytes) max=2048 (blocks)
public:    slice=4 offset=0 len=41860800
private:   slice=3 offset=1 len=32383
update:    time=1073385535 seqno=0.19
headers:   0 248
configs:   count=1 len=23884
logs:      count=1 len=3618
Defined regions:
config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
config   priv 000249-023901[023653]: copy=01 offset=000231 enabled
log      priv 023902-027519[003618]: copy=01 offset=000000 enabled
Multipathing information:
numpaths:   1
c8t600C0FF000000000001504755999EF09d0s2 state=enabled

The underlying devices are in the "Multipathing information" section at the end. The process of determining the mapping onto the SE3510 is then dependent on the output. The example above indicates STMS has been enabled, thus we follow the STMS Enabled method. If STMS has not been enabled, the device names will be similar to that listed in the output of format in the STMS Disabled section of this document.

This solution has no attachment