9206-16e and disk errors


Recommended Posts

1 hour ago, icemansid said:

old drives connected to this controller will cause drive speed issues across all of the drives.

I don't think so, but you can troubleshoot in this direction.

 

As mention before, HBA should have PCIe switch and your mainboard also have PCIe switch, those were in cascade and may have compatible problem.

 

From Syslog, this have a sign, LSI HBA only running in PCIe Gen 1 speed, but bandwidth still have 16Gb/s. ( Even all problem solve, throughput will ceiling at this too )


Dec 30 20:01:55 UnRAID-Backup kernel: pci 0000:08:00.0: 16.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s x8 link at 0000:00:01.0 (capable of 63.008 Gb/s with 8 GT/s x8 link)

 

I also notice you have P420i HBA, 1 or 2 disk connect, could you unplug it and try.

 

Which Expander you got ??
 

 

Edited by Benson
Link to comment

I am using the HP 468405-002 and using a PCI-e to USB3 adpater to power it.  The p420i is the onboard running the HP OEM dirves in a raid array acting as a cache drive.  Ive rebooted and installed a second SAS2308 card for testing as well.  Ive also attached a fresh diagnostic report.

 

 

I dont exactly follow what you mean by PCIe switch.

unraid-backup-diagnostics-20200108-1436.zip

Link to comment

From new Syslog, the second SAS2308 card also run at PCIe Gen 1 speed. In fact all device were in Gen 1, but machine support PCIe Gen 3. Does you wrongly set the limit in BIOS ?

 

25 minutes ago, icemansid said:

I dont exactly follow what you mean by PCIe switch.

It like expander to provide a function to let you connect more device but share same bandwidth, just expander service for SAS device.
 

 

How about the result on 2nd 2308 HBA ??

Edited by Benson
Link to comment
3 hours ago, Benson said:

From new Syslog, the second SAS2308 card also run at PCIe Gen 1 speed. In fact all device were in Gen 1, but machine support PCIe Gen 3. Does you wrongly set the limit in BIOS ?

 

It like expander to provide a function to let you connect more device but share same bandwidth, just expander service for SAS device.
 

 

How about the result on 2nd 2308 HBA ??

2nd 2308 gives very similar results at this point.  Nothing in the BIOS stands out but I have an HPE storage engineer looking at it as well. Hopefully he can assist with the PCI negotiations..

Link to comment
11 hours ago, Benson said:

Dec 30 20:01:55 UnRAID-Backup kernel: pci 0000:08:00.0: 16.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s x8 link at 0000:00:01.0 (capable of 63.008 Gb/s with 8 GT/s x8 link)

I didn't even look at the diags for this since diskspeed docker should report the correct link speed and it was reporting PCIe 3.0, @jbartlettmight want to take a look at this.

Link to comment

The DiskSpeed app simply reports what the dd command is outputting. If you're having issues reading all the drives at once, try removing half of the drives and then run a controller benchmark to see if you get the same results or better. If the same, remove the existing and hook the others back up. If better, then add a couple drives and do another benchmark. Rinse & repeat. See if there's a magic number or a given drive that's causing the issue.

Link to comment
1 hour ago, jbartlett said:

The DiskSpeed app simply reports what the dd command is outputting.

I meant the link speed:

 

image.png.23795c099e2b92b3e53f25097f8cba6b.png

 

13 hours ago, Benson said:

Dec 30 20:01:55 UnRAID-Backup kernel: pci 0000:08:00.0: 16.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s x8 link at 0000:00:01.0 (capable of 63.008 Gb/s with 8 GT/s x8 link)

 

Link to comment
44 minutes ago, jbartlett said:

@icemansid - Please upload/email a debug file from the DiskSpeed app (link at the bottom of the page). Use the left button to create a regular/smaller report.

 

The link speed uses the results from a lspci -vmm command and the debug file will have what it returned.

File as requested.

 

file_20200109_123133.tar.gz

Edited by icemansid
err small file.
Link to comment
On 1/8/2020 at 12:33 PM, Benson said:

Dec 30 20:01:55 UnRAID-Backup kernel: pci 0000:08:00.0: 16.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s x8 link at 0000:00:01.0 (capable of 63.008 Gb/s with 8 GT/s x8 link)

Here's the controller link report for the 0000:08:00.0 device

08:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS2308 PCI-Express Fusion-MPT SAS-2 (rev 05)
	Capabilities: [68] Express (v2) Endpoint, MSI 00
		LnkCap:	Port #0, Speed 8GT/s, Width x8, ASPM L0s, Exit Latency L0s <64ns, L1 <1us
			ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
		LnkSta:	Speed 8GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

I have a lookup table that for PCI3 with a transfer rate of 8 and a width of 8 returns "7.88 GB/s". It looks like this reference table is in error.

 

Based on information from https://paolozaino.wordpress.com/2013/05/21/converting-gts-to-gbps/ I should be able to compute it. The transfer rate of 8 GT/s identifies it as a PCI 3 controller. 8 GT/s multiplied by 8 lanes equals 64 GT/s, multiplied by the overhead (LineCodeL/LineCodeH) of 128b/130b equals 63.02 GB/s.

 

Edited by jbartlett
Link to comment
  • 4 months later...

I just wanted to provide an update to this posting - as it turns out, the issue was with SMR drives being used in the system. While we removed the 9206 cards, it wasn't until we moved the SMR drives out of the parity that we were able to clear the issue.  

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.