DiskSpeed, hard drive benchmarking (unRAID 6+), version 2.9.2


601 posts in this topic Last Reply

Recommended Posts

15 hours ago, juan11perez said:

Good day, so I ended up replacing the drive because it continued to just hang everything.

I plecleared it and in theory it "passed" after 56 hours. The diskspeed tests, continue to be very poor. 

So I think it's failed.

Before dispossing of it, i want to confirm my assumptions.

Any advise is appreciated.

Thank you

 

Are the most recent tests reflecting a line near the bottom of the graph? If so, then yes, I'd say it's likely a failed drive. Interesting enough, the SMART report doesn't show any bad sectors - which is pretty much the reason I created this app.

 

If the more recent drives show the reads near the top of the graph but with a dip in the middle, I'd be interested in taking this drive off your hands to use for testing version 3 of the app. If it's all flat towards the bottom, I've already have a drive doing that.

Link to post
  • Replies 600
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

This Docker Application will let you view your storage controllers & the drives attached to them and perform Benchmarks on both. Controller Benchmarks helps to identify if the drives attached to i

I'm taking this application out of BETA status. Version 2.0 has been released.   Release 2.0 Added progress bars to the drive benchmarking Rewrote the Controller Benchmark to be

I just noticed a trend in the file fragmentation. It seems the OS has the tendency to break the file up into chunks of a set size. I found this really strange because, well, why do it at all? Noticing

Posted Images

Thank you for your response. In latter tests the drive starts at over 100M and after reaching circa 10% it dives to around 50M and stays there to the end.

 

The middle dip results were produced until about 10 days ago.

Link to post
  • 2 weeks later...

Hi John,

 

If you have USBIP module vhci_hcd loaded then your scan fails for USBs.

 

I have attached the log here, let me know if you need any additional info or need me to test?

Spoiler

Scanning Hardware
14:14:51 Spinning up hard drives
14:14:51 Scanning system storage
14:14:51 Scanning USB Bus

Lucee 5.2.9.31 Error (expression)

MessageInvalid array index [0], arrays start with index [1]

StacktraceThe Error Occurred in
/var/www/ScanControllers.cfm: line 418

416: <CFSET USBTree[i]=ParseUSB(CurrLine)>
417: <CFSET Tmp=REMatchNoCase("[\da-f]{4}:[\da-f]{2}:[\da-f]{2}.[\da-f]{1}",CurrLine)>
418: <CFSET USBTree[i].Bus=Tmp[ArrayLen(Tmp)]>
419: </CFLOOP>
420:
 

called from /var/www/ScanControllers.cfm: line 372

370: <CFSET OK=1>
371: </CFIF>
372: </CFLOOP>
373: <!--- Set top level ChildDrives --->
374: <CFLOOP index="i" from="1" to="#ArrayLen(HWTree)#">
 

Java Stacktracelucee.runtime.exp.ExpressionException: Invalid array index [0], arrays start with index [1]
  at lucee.runtime.type.wrap.ListAsArray.getE(ListAsArray.java:110)
  at lucee.runtime.type.wrap.ListAsArray.get(ListAsArray.java:284)
  at lucee.runtime.type.wrap.ListAsArray.get(ListAsArray.java:289)
  at lucee.runtime.type.util.ArraySupport.get(ArraySupport.java:326)
  at lucee.runtime.util.VariableUtilImpl.get(VariableUtilImpl.java:263)
  at lucee.runtime.PageContextImpl.get(PageContextImpl.java:1516)
  at scancontrollers_cfm$cf.call_000027(/ScanControllers.cfm:418)
  at scancontrollers_cfm$cf.call(/ScanControllers.cfm:372)
  at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:933)
  at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:823)
  at lucee.runtime.listener.ClassicAppListener._onRequest(ClassicAppListener.java:66)
  at lucee.runtime.listener.MixedAppListener.onRequest(MixedAppListener.java:45)
  at lucee.runtime.PageContextImpl.execute(PageContextImpl.java:2464)
  at lucee.runtime.PageContextImpl._execute(PageContextImpl.java:2454)
  at lucee.runtime.PageContextImpl.executeCFML(PageContextImpl.java:2427)
  at lucee.runtime.engine.Request.exe(Request.java:44)
  at lucee.runtime.engine.CFMLEngineImpl._service(CFMLEngineImpl.java:1090)
  at lucee.runtime.engine.CFMLEngineImpl.serviceCFML(CFMLEngineImpl.java:1038)
  at lucee.loader.engine.CFMLEngineWrapper.serviceCFML(CFMLEngineWrapper.java:102)
  at lucee.loader.servlet.CFMLServlet.service(CFMLServlet.java:51)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
  at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
  at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
  at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:94)
  at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:492)
  at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
  at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
  at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
  at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:684)
  at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502)
  at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1152)
  at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684)
  at org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run(AprEndpoint.java:2464)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
  at java.lang.Thread.run(Thread.java:748)
 

Timestamp4/16/21 2:14:51 PM BST

 

Link to post
On 4/16/2021 at 6:21 AM, SimonF said:

Hi John,

 

If you have USBIP module vhci_hcd loaded then your scan fails for USBs.

 

I have attached the log here, let me know if you need any additional info or need me to test?

 

Can you show me how you have the USBIP module vhci_hcd set up so I can duplicate the issue as you have it configured?

Link to post
1 hour ago, jbartlett said:

 

Can you show me how you have the USBIP module vhci_hcd set up so I can duplicate the issue as you have it configured?

You just need to modprobe vhci_hcd this will then create two virtual usb hubs

 

Apr 18 06:48:19 computenode kernel: vhci_hcd vhci_hcd.0: USB/IP Virtual Host Controller
Apr 18 06:48:19 computenode kernel: vhci_hcd vhci_hcd.0: new USB bus registered, assigned bus number 3
Apr 18 06:48:19 computenode kernel: vhci_hcd: created sysfs vhci_hcd.0
Apr 18 06:48:19 computenode kernel: hub 3-0:1.0: USB hub found
Apr 18 06:48:19 computenode kernel: hub 3-0:1.0: 8 ports detected
Apr 18 06:48:19 computenode kernel: vhci_hcd vhci_hcd.0: USB/IP Virtual Host Controller
Apr 18 06:48:19 computenode kernel: vhci_hcd vhci_hcd.0: new USB bus registered, assigned bus number 4
Apr 18 06:48:19 computenode kernel: usb usb4: We don't know the algorithms for LPM for this host, disabling LPM.
Apr 18 06:48:19 computenode kernel: hub 4-0:1.0: USB hub found
Apr 18 06:48:19 computenode kernel: hub 4-0:1.0: 8 ports detected

Link to post
  • jbartlett changed the title to DiskSpeed, hard drive benchmarking (unRAID 6+), version 2.9.2
  • 2 weeks later...

Hi John,

 

Am I going insane, or did there used to be a "Benchmark all" button or something to that extent? The only 'multiple benchmark' button I can find is under the controller info, and that seems to just check for a controller bottleneck.

 

Also, when I look at the "USB Bus Tree" page, I see this error.

 

Thanks for all your work!

 

image.thumb.png.cbfca52f9a81b3be46f29d888824c42a.png

Link to post
5 hours ago, Alexstrasza said:

Hi John,

 

Am I going insane, or did there used to be a "Benchmark all" button or something to that extent? The only 'multiple benchmark' button I can find is under the controller info, and that seems to just check for a controller bottleneck.

 

Also, when I look at the "USB Bus Tree" page, I see this error.

 

Thanks for all your work!

 

The error in the USB Bus tree stopped the rest of the page from displaying which prevented you from seeing the Benchmark All button. I pushed version 2.9.3 which checks for the existence of the idProduct variable and sets it if not created by the USB device.

Link to post
Posted (edited)

...it's still strange...

Dont know if unraid or something else is the culprit...

But what i can say is this:

I can remember that an older version of this docker started "all" disk and "then" did the testing.

This version start a disk, do a test, start the next disk, do a test and so on...

 

unraid weird DiskSpeeds 2021-05-04 182642.jpg

Edited by Zonediver
Link to post
4 hours ago, Zonediver said:

...it's still strange...

Dont know if unraid or something else is the culprit...

But what i can say is this:

I can remember that an older version of this docker started "all" disk and "then" did the testing.

This version start a disk, do a test, start the next disk, do a test and so on...

 

Clicking on the "Benchmark Drives" button on the home page (click on the "DiskSpeed" header) allows you to benchmark all controllers at the same time with controller testing each drive attached to it in sequence.

Link to post
11 hours ago, jbartlett said:

Clicking on the "Benchmark Drives" button on the home page (click on the "DiskSpeed" header) allows you to benchmark all controllers at the same time with controller testing each drive attached to it in sequence.

 

I know this option but thats not the point.

The strange thing is, if all disk are in sleep and i start the benchmark, almost all disk are too slow and i dont know why...

If all of them are spun up, all is fine and running normal.

Link to post
6 hours ago, Zonediver said:

I know this option but thats not the point.

The strange thing is, if all disk are in sleep and i start the benchmark, almost all disk are too slow and i dont know why...

If all of them are spun up, all is fine and running normal.

The average speed logic discards the first read which includes any potential spinup / drive head placement. It might not be dropping enough in your case. Does your SMART data include spin up time?

Link to post
Posted (edited)
6 hours ago, jbartlett said:

Can you provide that information?

Yes - do you need this data for all drives or only for that on the HBA?

Anyway, i will create a list of all disks containing this information.

Edited by Zonediver
Link to post
10 hours ago, Zonediver said:

Yes - do you need this data for all drives or only for that on the HBA?

Anyway, i will create a list of all disks containing this information.

All would be telling. Notably if the drives on one controller is higher than drives on another. In theory, they should all be roughly the same. If it looks like they are, just a number. It should be given in milliseconds or 1000=1 second.

 

The dd command outputs a status every second so if it takes longer than I accounted for, I may need to increase the number of seconds to drop from the start.

Link to post
Posted (edited)
20 minutes ago, jbartlett said:

In theory, they should all be roughly the same.

...then this is strange...

The 8TB-disks are much faster then the 4TBs...

grafik.png.92195c44df1dec2eb67e2e4bb4158e42.png

 

Edited by Zonediver
Link to post

Good news: It's not your system! I spun down my drives and ran a controller benchmark and I got the same results.

 

image.png.b41d5ab8183c7d72aadb14d2942caecf.png

 

The drive benchmark performs a couple seek tests & a latency test on the drive prior to benchmarking so it's pretty much assured to have the drives spun up when it actually starts reading the drive via the dd command. The controller benchmark doesn't do those tests prior. I need to revisit the spin-up logic.

 

 

Link to post
Posted (edited)
12 minutes ago, jbartlett said:

The controller benchmark doesn't do those tests prior. I need to revisit the spin-up logic.

Interesting and good tho know that my hardware is not the culprit - thanks for this Info 👍

Edited by Zonediver
Link to post
On 5/3/2021 at 6:10 PM, jbartlett said:

 

The error in the USB Bus tree stopped the rest of the page from displaying which prevented you from seeing the Benchmark All button. I pushed version 2.9.3 which checks for the existence of the idProduct variable and sets it if not created by the USB device.

 

Good to hear! I'm glad I'm not going insane :).

 

Unfortunately I still can't see the button because of another error:

key [iManufacturer] doesn't exist

 

Sounds like it's the same thing again but for this key 😅. I wonder what it's trying to read that lacks these values?

Link to post
5 hours ago, Alexstrasza said:

 

Good to hear! I'm glad I'm not going insane :).

 

Unfortunately I still can't see the button because of another error:


key [iManufacturer] doesn't exist

 

Sounds like it's the same thing again but for this key 😅. I wonder what it's trying to read that lacks these values?

 

Your USB device is annoying. 😁

 

I added an existence check for that variable and looked for other potential misses and repushed version 2.9.3

Link to post
49 minutes ago, jbartlett said:

 

Your USB device is annoying. 😁

 

I added an existence check for that variable and looked for other potential misses and repushed version 2.9.3

 

It's working now 🎉! . Thanks John.

Link to post
  • 2 weeks later...

Hi,

 

Trying to find why Unraid feels slow. BlackMagic Disk Speed on a Mac reports 110MB/s to a empty share, which is very reasonable (on Gigabit connection).

 

I've run DiskSpeed. My Parity disk comes to 265MB/s and my data disk to 175MB/s.

 

My question: is there an optimal placement of disks on ports? Right now, I have cache (SSD) on port 1, data disk on port 2 and parity on port 5 (see below full output)

 


 

Output from DiskSpeed:

400 Series Chipset SATA Controller


ASMedia Technology Inc. (Advanced Micro Devices, Inc. [AMD])
SATA controller

Type: Onboard Controller
Current & Maximum Link Speed: 8GT/s width x4 (3.94 GB/s max throughput)
Capabilities: storage msi pm pciexpress ahci_1.0 bus_master cap_list rom

Port 1: 

sdb 

1TB 

Samsung SSD 860 EVO Rev RVT01B6Q Serial: S3Z9NB0K616565P (Cache)

Port 2: 

sdc 

6TB 

Seagate ST6000DM003 Rev 0001 Serial: ZF300PP8 (Disk 1)

Port 3: 

N/A

Port 4: 

N/A

Port 5: 

sdd 

8TB 

Seagate ST8000VN004 Rev SC60 Serial: WSD133GD (Parity)

Port 6: 

N/A

Port 7: 

N/A

Port 8: 

N/A

 

Link to post
On 5/20/2021 at 3:04 AM, Sander de Ruiter said:

Hi,

 

Trying to find why Unraid feels slow. BlackMagic Disk Speed on a Mac reports 110MB/s to a empty share, which is very reasonable (on Gigabit connection).

 

I've run DiskSpeed. My Parity disk comes to 265MB/s and my data disk to 175MB/s.

 

My question: is there an optimal placement of disks on ports? Right now, I have cache (SSD) on port 1, data disk on port 2 and parity on port 5 (see below full output)

 

Wondering what you mean by "feel slow". But those are good speeds. Try running a controller benchmark to see if a controller isn't able to handle all the drives active at the same time. I'll get a read speed from each drive one after the other and again all at the same time and compare the two.

 

From what I see, you're using an onboard controller with three drives out of a possible six. It looks like it should be able to handle it fine. You can also try running a disk benchmark at with a 1% interval scan (101 scan points vs the default 11) which might help locate questionable areas.

 

The next major release of DiskSpeed will add support for full surface level testing but I don't have a timeline for that release. I'm currently slammed at my job and I haven't had much time of late for hobby programming.

Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.