April 4, 20215 yr Thank you for your response. In latter tests the drive starts at over 100M and after reaching circa 10% it dives to around 50M and stays there to the end. The middle dip results were produced until about 10 days ago.
April 16, 20215 yr Hi John, If you have USBIP module vhci_hcd loaded then your scan fails for USBs. I have attached the log here, let me know if you need any additional info or need me to test? Spoiler Scanning Hardware 14:14:51 Spinning up hard drives 14:14:51 Scanning system storage 14:14:51 Scanning USB Bus Lucee 5.2.9.31 Error (expression) MessageInvalid array index [0], arrays start with index [1] StacktraceThe Error Occurred in /var/www/ScanControllers.cfm: line 418 416: <CFSET USBTree[i]=ParseUSB(CurrLine)> 417: <CFSET Tmp=REMatchNoCase("[\da-f]{4}:[\da-f]{2}:[\da-f]{2}.[\da-f]{1}",CurrLine)> 418: <CFSET USBTree[i].Bus=Tmp[ArrayLen(Tmp)]> 419: </CFLOOP> 420: called from /var/www/ScanControllers.cfm: line 372 370: <CFSET OK=1> 371: </CFIF> 372: </CFLOOP> 373: <!--- Set top level ChildDrives ---> 374: <CFLOOP index="i" from="1" to="#ArrayLen(HWTree)#"> Java Stacktracelucee.runtime.exp.ExpressionException: Invalid array index [0], arrays start with index [1] at lucee.runtime.type.wrap.ListAsArray.getE(ListAsArray.java:110) at lucee.runtime.type.wrap.ListAsArray.get(ListAsArray.java:284) at lucee.runtime.type.wrap.ListAsArray.get(ListAsArray.java:289) at lucee.runtime.type.util.ArraySupport.get(ArraySupport.java:326) at lucee.runtime.util.VariableUtilImpl.get(VariableUtilImpl.java:263) at lucee.runtime.PageContextImpl.get(PageContextImpl.java:1516) at scancontrollers_cfm$cf.call_000027(/ScanControllers.cfm:418) at scancontrollers_cfm$cf.call(/ScanControllers.cfm:372) at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:933) at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:823) at lucee.runtime.listener.ClassicAppListener._onRequest(ClassicAppListener.java:66) at lucee.runtime.listener.MixedAppListener.onRequest(MixedAppListener.java:45) at lucee.runtime.PageContextImpl.execute(PageContextImpl.java:2464) at lucee.runtime.PageContextImpl._execute(PageContextImpl.java:2454) at lucee.runtime.PageContextImpl.executeCFML(PageContextImpl.java:2427) at lucee.runtime.engine.Request.exe(Request.java:44) at lucee.runtime.engine.CFMLEngineImpl._service(CFMLEngineImpl.java:1090) at lucee.runtime.engine.CFMLEngineImpl.serviceCFML(CFMLEngineImpl.java:1038) at lucee.loader.engine.CFMLEngineWrapper.serviceCFML(CFMLEngineWrapper.java:102) at lucee.loader.servlet.CFMLServlet.service(CFMLServlet.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:94) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:492) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:684) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1152) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684) at org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run(AprEndpoint.java:2464) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Timestamp4/16/21 2:14:51 PM BST
April 18, 20215 yr Author On 4/16/2021 at 6:21 AM, SimonF said: Hi John, If you have USBIP module vhci_hcd loaded then your scan fails for USBs. I have attached the log here, let me know if you need any additional info or need me to test? Can you show me how you have the USBIP module vhci_hcd set up so I can duplicate the issue as you have it configured?
April 18, 20215 yr 1 hour ago, jbartlett said: Can you show me how you have the USBIP module vhci_hcd set up so I can duplicate the issue as you have it configured? You just need to modprobe vhci_hcd this will then create two virtual usb hubs Apr 18 06:48:19 computenode kernel: vhci_hcd vhci_hcd.0: USB/IP Virtual Host Controller Apr 18 06:48:19 computenode kernel: vhci_hcd vhci_hcd.0: new USB bus registered, assigned bus number 3 Apr 18 06:48:19 computenode kernel: vhci_hcd: created sysfs vhci_hcd.0 Apr 18 06:48:19 computenode kernel: hub 3-0:1.0: USB hub found Apr 18 06:48:19 computenode kernel: hub 3-0:1.0: 8 ports detected Apr 18 06:48:19 computenode kernel: vhci_hcd vhci_hcd.0: USB/IP Virtual Host Controller Apr 18 06:48:19 computenode kernel: vhci_hcd vhci_hcd.0: new USB bus registered, assigned bus number 4 Apr 18 06:48:19 computenode kernel: usb usb4: We don't know the algorithms for LPM for this host, disabling LPM. Apr 18 06:48:19 computenode kernel: hub 4-0:1.0: USB hub found Apr 18 06:48:19 computenode kernel: hub 4-0:1.0: 8 ports detected
May 3, 20215 yr Hi John, Am I going insane, or did there used to be a "Benchmark all" button or something to that extent? The only 'multiple benchmark' button I can find is under the controller info, and that seems to just check for a controller bottleneck. Also, when I look at the "USB Bus Tree" page, I see this error. Thanks for all your work!
May 3, 20215 yr Author 5 hours ago, Alexstrasza said: Hi John, Am I going insane, or did there used to be a "Benchmark all" button or something to that extent? The only 'multiple benchmark' button I can find is under the controller info, and that seems to just check for a controller bottleneck. Also, when I look at the "USB Bus Tree" page, I see this error. Thanks for all your work! The error in the USB Bus tree stopped the rest of the page from displaying which prevented you from seeing the Benchmark All button. I pushed version 2.9.3 which checks for the existence of the idProduct variable and sets it if not created by the USB device.
May 4, 20215 yr ...it's still strange... Dont know if unraid or something else is the culprit... But what i can say is this: I can remember that an older version of this docker started "all" disk and "then" did the testing. This version start a disk, do a test, start the next disk, do a test and so on... Edited May 4, 20215 yr by Zonediver
May 4, 20215 yr Author 4 hours ago, Zonediver said: ...it's still strange... Dont know if unraid or something else is the culprit... But what i can say is this: I can remember that an older version of this docker started "all" disk and "then" did the testing. This version start a disk, do a test, start the next disk, do a test and so on... Clicking on the "Benchmark Drives" button on the home page (click on the "DiskSpeed" header) allows you to benchmark all controllers at the same time with controller testing each drive attached to it in sequence.
May 5, 20215 yr 11 hours ago, jbartlett said: Clicking on the "Benchmark Drives" button on the home page (click on the "DiskSpeed" header) allows you to benchmark all controllers at the same time with controller testing each drive attached to it in sequence. I know this option but thats not the point. The strange thing is, if all disk are in sleep and i start the benchmark, almost all disk are too slow and i dont know why... If all of them are spun up, all is fine and running normal.
May 5, 20215 yr Author 6 hours ago, Zonediver said: I know this option but thats not the point. The strange thing is, if all disk are in sleep and i start the benchmark, almost all disk are too slow and i dont know why... If all of them are spun up, all is fine and running normal. The average speed logic discards the first read which includes any potential spinup / drive head placement. It might not be dropping enough in your case. Does your SMART data include spin up time?
May 6, 20215 yr 6 hours ago, jbartlett said: Can you provide that information? Yes - do you need this data for all drives or only for that on the HBA? Anyway, i will create a list of all disks containing this information. Edited May 6, 20215 yr by Zonediver
May 6, 20215 yr Author 10 hours ago, Zonediver said: Yes - do you need this data for all drives or only for that on the HBA? Anyway, i will create a list of all disks containing this information. All would be telling. Notably if the drives on one controller is higher than drives on another. In theory, they should all be roughly the same. If it looks like they are, just a number. It should be given in milliseconds or 1000=1 second. The dd command outputs a status every second so if it takes longer than I accounted for, I may need to increase the number of seconds to drop from the start.
May 6, 20215 yr 20 minutes ago, jbartlett said: In theory, they should all be roughly the same. ...then this is strange... The 8TB-disks are much faster then the 4TBs... Edited May 6, 20215 yr by Zonediver
May 6, 20215 yr Author Good news: It's not your system! I spun down my drives and ran a controller benchmark and I got the same results. The drive benchmark performs a couple seek tests & a latency test on the drive prior to benchmarking so it's pretty much assured to have the drives spun up when it actually starts reading the drive via the dd command. The controller benchmark doesn't do those tests prior. I need to revisit the spin-up logic.
May 6, 20215 yr 12 minutes ago, jbartlett said: The controller benchmark doesn't do those tests prior. I need to revisit the spin-up logic. Interesting and good tho know that my hardware is not the culprit - thanks for this Info 👍 Edited May 6, 20215 yr by Zonediver
May 7, 20215 yr On 5/3/2021 at 6:10 PM, jbartlett said: The error in the USB Bus tree stopped the rest of the page from displaying which prevented you from seeing the Benchmark All button. I pushed version 2.9.3 which checks for the existence of the idProduct variable and sets it if not created by the USB device. Good to hear! I'm glad I'm not going insane :). Unfortunately I still can't see the button because of another error: key [iManufacturer] doesn't exist Sounds like it's the same thing again but for this key 😅. I wonder what it's trying to read that lacks these values?
May 7, 20215 yr Author 5 hours ago, Alexstrasza said: Good to hear! I'm glad I'm not going insane :). Unfortunately I still can't see the button because of another error: key [iManufacturer] doesn't exist Sounds like it's the same thing again but for this key 😅. I wonder what it's trying to read that lacks these values? Your USB device is annoying. 😁 I added an existence check for that variable and looked for other potential misses and repushed version 2.9.3
May 7, 20215 yr 49 minutes ago, jbartlett said: Your USB device is annoying. 😁 I added an existence check for that variable and looked for other potential misses and repushed version 2.9.3 It's working now 🎉! . Thanks John.
May 20, 20215 yr Hi, Trying to find why Unraid feels slow. BlackMagic Disk Speed on a Mac reports 110MB/s to a empty share, which is very reasonable (on Gigabit connection). I've run DiskSpeed. My Parity disk comes to 265MB/s and my data disk to 175MB/s. My question: is there an optimal placement of disks on ports? Right now, I have cache (SSD) on port 1, data disk on port 2 and parity on port 5 (see below full output) Output from DiskSpeed: 400 Series Chipset SATA Controller ASMedia Technology Inc. (Advanced Micro Devices, Inc. [AMD]) SATA controller Type: Onboard Controller Current & Maximum Link Speed: 8GT/s width x4 (3.94 GB/s max throughput) Capabilities: storage msi pm pciexpress ahci_1.0 bus_master cap_list rom Port 1: sdb 1TB Samsung SSD 860 EVO Rev RVT01B6Q Serial: S3Z9NB0K616565P (Cache) Port 2: sdc 6TB Seagate ST6000DM003 Rev 0001 Serial: ZF300PP8 (Disk 1) Port 3: N/A Port 4: N/A Port 5: sdd 8TB Seagate ST8000VN004 Rev SC60 Serial: WSD133GD (Parity) Port 6: N/A Port 7: N/A Port 8: N/A
May 21, 20215 yr Author On 5/20/2021 at 3:04 AM, Sander de Ruiter said: Hi, Trying to find why Unraid feels slow. BlackMagic Disk Speed on a Mac reports 110MB/s to a empty share, which is very reasonable (on Gigabit connection). I've run DiskSpeed. My Parity disk comes to 265MB/s and my data disk to 175MB/s. My question: is there an optimal placement of disks on ports? Right now, I have cache (SSD) on port 1, data disk on port 2 and parity on port 5 (see below full output) Wondering what you mean by "feel slow". But those are good speeds. Try running a controller benchmark to see if a controller isn't able to handle all the drives active at the same time. I'll get a read speed from each drive one after the other and again all at the same time and compare the two. From what I see, you're using an onboard controller with three drives out of a possible six. It looks like it should be able to handle it fine. You can also try running a disk benchmark at with a 1% interval scan (101 scan points vs the default 11) which might help locate questionable areas. The next major release of DiskSpeed will add support for full surface level testing but I don't have a timeline for that release. I'm currently slammed at my job and I haven't had much time of late for hobby programming.
June 5, 20215 yr I felt compelled to post here and give you a HUGE thank you for putting this utility together. It is exactly what I didn't know I needed! I have a lot of 2TB drives of various ages, some over 7 years, and while they pass their monthly parity check and show no issues in SMART, your utility will help me monitor the actual performance of the drives and help catch any issues. I imagine it's only a matter of time before I'll need to replace a couple of the drives. Thank you!
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.