DiskSpeed, hdd/ssd benchmarking (unRAID 6+), version 2.10.7


Recommended Posts

  • 2 weeks later...

Hi John,

 

If you have USBIP module vhci_hcd loaded then your scan fails for USBs.

 

I have attached the log here, let me know if you need any additional info or need me to test?

Spoiler

Scanning Hardware
14:14:51 Spinning up hard drives
14:14:51 Scanning system storage
14:14:51 Scanning USB Bus

Lucee 5.2.9.31 Error (expression)

MessageInvalid array index [0], arrays start with index [1]

StacktraceThe Error Occurred in
/var/www/ScanControllers.cfm: line 418

416: <CFSET USBTree[i]=ParseUSB(CurrLine)>
417: <CFSET Tmp=REMatchNoCase("[\da-f]{4}:[\da-f]{2}:[\da-f]{2}.[\da-f]{1}",CurrLine)>
418: <CFSET USBTree[i].Bus=Tmp[ArrayLen(Tmp)]>
419: </CFLOOP>
420:
 

called from /var/www/ScanControllers.cfm: line 372

370: <CFSET OK=1>
371: </CFIF>
372: </CFLOOP>
373: <!--- Set top level ChildDrives --->
374: <CFLOOP index="i" from="1" to="#ArrayLen(HWTree)#">
 

Java Stacktracelucee.runtime.exp.ExpressionException: Invalid array index [0], arrays start with index [1]
  at lucee.runtime.type.wrap.ListAsArray.getE(ListAsArray.java:110)
  at lucee.runtime.type.wrap.ListAsArray.get(ListAsArray.java:284)
  at lucee.runtime.type.wrap.ListAsArray.get(ListAsArray.java:289)
  at lucee.runtime.type.util.ArraySupport.get(ArraySupport.java:326)
  at lucee.runtime.util.VariableUtilImpl.get(VariableUtilImpl.java:263)
  at lucee.runtime.PageContextImpl.get(PageContextImpl.java:1516)
  at scancontrollers_cfm$cf.call_000027(/ScanControllers.cfm:418)
  at scancontrollers_cfm$cf.call(/ScanControllers.cfm:372)
  at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:933)
  at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:823)
  at lucee.runtime.listener.ClassicAppListener._onRequest(ClassicAppListener.java:66)
  at lucee.runtime.listener.MixedAppListener.onRequest(MixedAppListener.java:45)
  at lucee.runtime.PageContextImpl.execute(PageContextImpl.java:2464)
  at lucee.runtime.PageContextImpl._execute(PageContextImpl.java:2454)
  at lucee.runtime.PageContextImpl.executeCFML(PageContextImpl.java:2427)
  at lucee.runtime.engine.Request.exe(Request.java:44)
  at lucee.runtime.engine.CFMLEngineImpl._service(CFMLEngineImpl.java:1090)
  at lucee.runtime.engine.CFMLEngineImpl.serviceCFML(CFMLEngineImpl.java:1038)
  at lucee.loader.engine.CFMLEngineWrapper.serviceCFML(CFMLEngineWrapper.java:102)
  at lucee.loader.servlet.CFMLServlet.service(CFMLServlet.java:51)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
  at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
  at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
  at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:94)
  at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:492)
  at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
  at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
  at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
  at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:684)
  at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502)
  at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1152)
  at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684)
  at org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run(AprEndpoint.java:2464)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
  at java.lang.Thread.run(Thread.java:748)
 

Timestamp4/16/21 2:14:51 PM BST

 

Link to comment
On 4/16/2021 at 6:21 AM, SimonF said:

Hi John,

 

If you have USBIP module vhci_hcd loaded then your scan fails for USBs.

 

I have attached the log here, let me know if you need any additional info or need me to test?

 

Can you show me how you have the USBIP module vhci_hcd set up so I can duplicate the issue as you have it configured?

Link to comment
1 hour ago, jbartlett said:

 

Can you show me how you have the USBIP module vhci_hcd set up so I can duplicate the issue as you have it configured?

You just need to modprobe vhci_hcd this will then create two virtual usb hubs

 

Apr 18 06:48:19 computenode kernel: vhci_hcd vhci_hcd.0: USB/IP Virtual Host Controller
Apr 18 06:48:19 computenode kernel: vhci_hcd vhci_hcd.0: new USB bus registered, assigned bus number 3
Apr 18 06:48:19 computenode kernel: vhci_hcd: created sysfs vhci_hcd.0
Apr 18 06:48:19 computenode kernel: hub 3-0:1.0: USB hub found
Apr 18 06:48:19 computenode kernel: hub 3-0:1.0: 8 ports detected
Apr 18 06:48:19 computenode kernel: vhci_hcd vhci_hcd.0: USB/IP Virtual Host Controller
Apr 18 06:48:19 computenode kernel: vhci_hcd vhci_hcd.0: new USB bus registered, assigned bus number 4
Apr 18 06:48:19 computenode kernel: usb usb4: We don't know the algorithms for LPM for this host, disabling LPM.
Apr 18 06:48:19 computenode kernel: hub 4-0:1.0: USB hub found
Apr 18 06:48:19 computenode kernel: hub 4-0:1.0: 8 ports detected

Link to comment
  • jbartlett changed the title to DiskSpeed, hard drive benchmarking (unRAID 6+), version 2.9.2
  • 2 weeks later...

Hi John,

 

Am I going insane, or did there used to be a "Benchmark all" button or something to that extent? The only 'multiple benchmark' button I can find is under the controller info, and that seems to just check for a controller bottleneck.

 

Also, when I look at the "USB Bus Tree" page, I see this error.

 

Thanks for all your work!

 

image.thumb.png.cbfca52f9a81b3be46f29d888824c42a.png

Link to comment
5 hours ago, Alexstrasza said:

Hi John,

 

Am I going insane, or did there used to be a "Benchmark all" button or something to that extent? The only 'multiple benchmark' button I can find is under the controller info, and that seems to just check for a controller bottleneck.

 

Also, when I look at the "USB Bus Tree" page, I see this error.

 

Thanks for all your work!

 

The error in the USB Bus tree stopped the rest of the page from displaying which prevented you from seeing the Benchmark All button. I pushed version 2.9.3 which checks for the existence of the idProduct variable and sets it if not created by the USB device.

  • Thanks 1
Link to comment

...it's still strange...

Dont know if unraid or something else is the culprit...

But what i can say is this:

I can remember that an older version of this docker started "all" disk and "then" did the testing.

This version start a disk, do a test, start the next disk, do a test and so on...

 

unraid weird DiskSpeeds 2021-05-04 182642.jpg

Edited by Zonediver
Link to comment
4 hours ago, Zonediver said:

...it's still strange...

Dont know if unraid or something else is the culprit...

But what i can say is this:

I can remember that an older version of this docker started "all" disk and "then" did the testing.

This version start a disk, do a test, start the next disk, do a test and so on...

 

Clicking on the "Benchmark Drives" button on the home page (click on the "DiskSpeed" header) allows you to benchmark all controllers at the same time with controller testing each drive attached to it in sequence.

Link to comment
11 hours ago, jbartlett said:

Clicking on the "Benchmark Drives" button on the home page (click on the "DiskSpeed" header) allows you to benchmark all controllers at the same time with controller testing each drive attached to it in sequence.

 

I know this option but thats not the point.

The strange thing is, if all disk are in sleep and i start the benchmark, almost all disk are too slow and i dont know why...

If all of them are spun up, all is fine and running normal.

Link to comment
6 hours ago, Zonediver said:

I know this option but thats not the point.

The strange thing is, if all disk are in sleep and i start the benchmark, almost all disk are too slow and i dont know why...

If all of them are spun up, all is fine and running normal.

The average speed logic discards the first read which includes any potential spinup / drive head placement. It might not be dropping enough in your case. Does your SMART data include spin up time?

Link to comment
10 hours ago, Zonediver said:

Yes - do you need this data for all drives or only for that on the HBA?

Anyway, i will create a list of all disks containing this information.

All would be telling. Notably if the drives on one controller is higher than drives on another. In theory, they should all be roughly the same. If it looks like they are, just a number. It should be given in milliseconds or 1000=1 second.

 

The dd command outputs a status every second so if it takes longer than I accounted for, I may need to increase the number of seconds to drop from the start.

Link to comment

Good news: It's not your system! I spun down my drives and ran a controller benchmark and I got the same results.

 

image.png.b41d5ab8183c7d72aadb14d2942caecf.png

 

The drive benchmark performs a couple seek tests & a latency test on the drive prior to benchmarking so it's pretty much assured to have the drives spun up when it actually starts reading the drive via the dd command. The controller benchmark doesn't do those tests prior. I need to revisit the spin-up logic.

 

 

Link to comment
On 5/3/2021 at 6:10 PM, jbartlett said:

 

The error in the USB Bus tree stopped the rest of the page from displaying which prevented you from seeing the Benchmark All button. I pushed version 2.9.3 which checks for the existence of the idProduct variable and sets it if not created by the USB device.

 

Good to hear! I'm glad I'm not going insane :).

 

Unfortunately I still can't see the button because of another error:

key [iManufacturer] doesn't exist

 

Sounds like it's the same thing again but for this key 😅. I wonder what it's trying to read that lacks these values?

Link to comment
5 hours ago, Alexstrasza said:

 

Good to hear! I'm glad I'm not going insane :).

 

Unfortunately I still can't see the button because of another error:


key [iManufacturer] doesn't exist

 

Sounds like it's the same thing again but for this key 😅. I wonder what it's trying to read that lacks these values?

 

Your USB device is annoying. 😁

 

I added an existence check for that variable and looked for other potential misses and repushed version 2.9.3

  • Thanks 1
Link to comment
  • 2 weeks later...

Hi,

 

Trying to find why Unraid feels slow. BlackMagic Disk Speed on a Mac reports 110MB/s to a empty share, which is very reasonable (on Gigabit connection).

 

I've run DiskSpeed. My Parity disk comes to 265MB/s and my data disk to 175MB/s.

 

My question: is there an optimal placement of disks on ports? Right now, I have cache (SSD) on port 1, data disk on port 2 and parity on port 5 (see below full output)

 


 

Output from DiskSpeed:

400 Series Chipset SATA Controller


ASMedia Technology Inc. (Advanced Micro Devices, Inc. [AMD])
SATA controller

Type: Onboard Controller
Current & Maximum Link Speed: 8GT/s width x4 (3.94 GB/s max throughput)
Capabilities: storage msi pm pciexpress ahci_1.0 bus_master cap_list rom

Port 1: 

sdb 

1TB 

Samsung SSD 860 EVO Rev RVT01B6Q Serial: S3Z9NB0K616565P (Cache)

Port 2: 

sdc 

6TB 

Seagate ST6000DM003 Rev 0001 Serial: ZF300PP8 (Disk 1)

Port 3: 

N/A

Port 4: 

N/A

Port 5: 

sdd 

8TB 

Seagate ST8000VN004 Rev SC60 Serial: WSD133GD (Parity)

Port 6: 

N/A

Port 7: 

N/A

Port 8: 

N/A

 

Link to comment
On 5/20/2021 at 3:04 AM, Sander de Ruiter said:

Hi,

 

Trying to find why Unraid feels slow. BlackMagic Disk Speed on a Mac reports 110MB/s to a empty share, which is very reasonable (on Gigabit connection).

 

I've run DiskSpeed. My Parity disk comes to 265MB/s and my data disk to 175MB/s.

 

My question: is there an optimal placement of disks on ports? Right now, I have cache (SSD) on port 1, data disk on port 2 and parity on port 5 (see below full output)

 

Wondering what you mean by "feel slow". But those are good speeds. Try running a controller benchmark to see if a controller isn't able to handle all the drives active at the same time. I'll get a read speed from each drive one after the other and again all at the same time and compare the two.

 

From what I see, you're using an onboard controller with three drives out of a possible six. It looks like it should be able to handle it fine. You can also try running a disk benchmark at with a 1% interval scan (101 scan points vs the default 11) which might help locate questionable areas.

 

The next major release of DiskSpeed will add support for full surface level testing but I don't have a timeline for that release. I'm currently slammed at my job and I haven't had much time of late for hobby programming.

Link to comment
  • 3 weeks later...

I felt compelled to post here and give you a HUGE thank you for putting this utility together. It is exactly what I didn't know I needed! I have a lot of 2TB drives of various ages, some over 7 years, and while they pass their monthly parity check and show no issues in SMART, your utility will help me monitor the actual performance of the drives and help catch any issues. I imagine it's only a matter of time before I'll need to replace a couple of the drives.

 

Thank you!

  • Thanks 1
Link to comment
  • jbartlett changed the title to DiskSpeed, hdd/ssd benchmarking (unRAID 6+), version 2.10.7

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.