Sun Fire E6900 Server
 
Sun Fire 6800 Server
 
Sun Fire E4900 Server
 
Sun Fire 4800 Server
 
Sun Fire V1280 Server
 
Sun Fire E2900 Server
 

GCS>Sun Microsystems>Sun FAB>Standard>Reactive
 

• Part No: 540-6509
• Part Description: PCI-X IO Board

• Part No: 541-1486
• Part Description: PCI-X IB_SSC Board

• Part No: 541-0382
• Part Description: PCI+ IB_SSC Board

Storage devices connected to Emulex LP10000 PCI-X HBA (Sun PN# 375-3305 and PN# 375-3304) will not be accessible due to PCI bus errors.  In some cases, it may not be possible to install Solaris on systems/domains with Emulex LP10000 HBAs running in 33MHz mode because of PCI bus errors.  

This issue is seen in the following configurations:

• Emulex LP10000 cards that are installed in 33MHz-only slots of a PCI-X IO Assy.
• Emulex LP10000 cards that are installed in a slot that shares a PCI bus with a slot that has a 33MHz card in a PCI-X IO Assy and PCI-X IB_SSC Assy. For example, a 33MHz card installed in slot 6, and a LP10000 card installed in slot 7 of a PCI-X IO Assembly.

Note: This issue is NOT seen with Emulex LP10000 HBAs installed in the older PCI IO Assy or the PCI IB_SSC Assy.

Please refer to the Sun System Handbook and related Info Docs for the detailed IO PCI bus layout of the various affected IO Assemblies.  

From "prtdiag" output of an affected system/domain:

/N0/IB6/P1  PCI   25   B    5    100  100  2,0  ok    SUNW,emlxs-pci10df,fc00/fp (fp)   LP10000DC-S
/N0/IB6/P1  PCI   25   B    5    100  100  2,1  ok    SUNW,emlxs-pci10df,fc00/fp (fp)   LP10000DC-S
/N0/IB6/P1  PCI   25   A    6    33   100  2,0  ok    pci-pci1011,24.3/pci108e,1000     pci-bridge
/N0/IB6/P1  PCI   25   A    6    33   100  0,0  ok    pci108e,1000-pci108e,1000.1
/N0/IB6/P1  PCI   25   A    6    33   100  0,1  ok    SUNW,hme-pci108e,1001             SUNW,cheerio
/N0/IB6/P1  PCI   25   A    6    33   100  4,0  ok    SUNW,isptwo-pci1077,1020/sd (blo+ QLGC,ISP1040B
-> /N0/IB6/P1  PCI   25   A    7    33   100  1,0  ok    SUNW,emlxs-pci10df,fc00/fp (fp)   LP10000DC-S  <---
-> /N0/IB6/P1  PCI   25   A    7    33   100  1,1  ok    SUNW,emlxs-pci10df,fc00/fp (fp)   LP10000DC-S  <---
========================= Active Boards for Domain ===========================
Board        Receptacle    Occupant
FRU Name   Type         Status        Status        Condition Info
---------  -----------  -----------   ------------  --------- ----------------------------------------
/N0/SB2    CPU_V3       connected     configured    ok        powered-on, assigned
/N0/SB4    CPU_V2       connected     configured    ok        powered-on, assigned
-> /N0/IB6    PCI-X_I/O_B  connected     configured    ok        powered-on, assigned <--

Note that the LP10000 card installed in slot 7 is running at 33MHz (In the above scenario, the PCI bus is shared by a 33MHz Sun Swift card in slot 6 and LP10000 card in slot 7, hence the LP10000 card also runs at 33MHz as observed in the above "prtdiag" output). Also note that the IO Assembly in slot IB6 is a PCI-X IO Assembly.

When this issue occurs, the following type of errors are seen on the console (also recorded in /var/adm/messages):

Aug 23 11:34:36 qame10-a scsi: WARNING: /ssm@0,0/pci@18,700000/SUNW,emlxs@2/fp@0,0/ssd@w500000e010763ff1,0 (ssd24):
Aug 23 11:34:36 qame10-a        SCSI transport failed: reason 'timeout': retrying command
Aug 23 11:34:36 qame10-a pcisch: WARNING: pcisch0 (pci@18,700000): PCI fault log start:
Aug 23 11:34:36 qame10-a pcisch: PCI iommu error
Aug 23 11:34:36 qame10-a pcisch: pcisch0: Error 1 on IOMMU TLB entry 2:
Aug 23 11:34:36 qame10-a        Context=0 not Writable not Streamable
Aug 23 11:34:36 qame10-a        PCI Page Size=8k Address in page c1320000
Aug 23 11:34:36 qame10-a pcisch: Memory: Valid not Cacheable Page Frame=0
Aug 23 11:34:36 qame10-a pcisch: pcisch0 (pci@18,700000): PBM AFSR=0x0.00000000
Aug 23 11:34:36 qame10-a pcisch: dwordmask=0 bytemask=0
Aug 23 11:34:36 qame10-a pcisch: pcisch0 (pci@18,700000): PCI primary error (0):
Aug 23 11:34:36 qame10-a pcisch: pcisch0 (pci@18,700000): PCI secondary error (0):
Aug 23 11:34:36 qame10-a pcisch: pcisch0 (pci@18,700000): PBM AFAR 0.00000000:
Aug 23 11:34:36 qame10-a pcisch: WARNING: pcisch0: PCI config space CSR=0xaa0<signaled-target-abort>Aug 23 11:34:36 qame10-a pcisch: pcisch0 (pci@18,700000): PCI fault log end.
Aug 23 11:34:36 qame10-a pcisch: Scrubbing PCI iommu TLB entries
Aug 23 11:34:36 qame10-a pcisch: No fatal PCI bus error(s)
Aug 23 11:39:07 qame10-a scsi: WARNING: /ssm@0,0/pci@19,600000/SUNW,emlxs@1/fp@0,0/ssd@w2100000c509f1d80,0 (ssd59):
Aug 23 11:39:07 qame10-a        offline
Aug 23 11:39:13 qame10-a scsi: WARNING: /ssm@0,0/pci@18,700000/SUNW,emlxs@2/fp@0,0/ssd@w500000e010763ff1,0 (ssd24):
Aug 23 11:39:13 qame10-a        offline

Aug 23 11:39:13 qame10-a scsi: WARNING: /ssm@0,0/pci@18,700000/SUNW,emlxs@2/fp@0,0/ssd@w500000e010763ff1,0 (ssd24):

The current PCI nexus driver is incompatible in a configuration where the Emulex LP10000 HBA runs at 33MHz.

Avoid installing an Emulex LP10000 HBA in:

• 33MHz slots (Slot 0 and Slot 1 of PCI-X IO Assy)
• Slots where the PCI bus is shared by another slot that has a 33MHz HBA installed

For Solaris 10, install Kernel Update 118833-23 (or later) to fix this issue.
For Solaris 8 and Solaris 9, the fix is not available yet, but will be included in a future release of the "pcipsy" driver patch for Solaris 8 and Solaris 9.

If a 33Mhz PCI card is installed in a slot that is 33Mhz/66Mhz/100Mhz compatible, and if an Emulex LP10000 PCI-X card is installed in a slot that shares the same PCI bus as the 33Mhz card, then the LP10000 card is also forced to run at 33Mhz.

Refer to the following Infodoc for a detailed view of the PCI IO slot layout for various IO Assemblies:

• Infodoc 86460 for IO layout on Midrange systems

In any case, it is not recommended to install a 100Mhz PCI-X HBA in a 33Mhz slot or install in a slot that shares the PCI bus with another 33Mhz slot for obvious performance reasons.

• Other: Info Doc 86460

This solution has no attachment