juan11perez Posted April 4, 2021 Share Posted April 4, 2021 Thank you for your response. In latter tests the drive starts at over 100M and after reaching circa 10% it dives to around 50M and stays there to the end. The middle dip results were produced until about 10 days ago. Quote Link to comment
SimonF Posted April 16, 2021 Share Posted April 16, 2021 Hi John, If you have USBIP module vhci_hcd loaded then your scan fails for USBs. I have attached the log here, let me know if you need any additional info or need me to test? Spoiler Scanning Hardware 14:14:51 Spinning up hard drives 14:14:51 Scanning system storage 14:14:51 Scanning USB Bus Lucee 5.2.9.31 Error (expression) MessageInvalid array index [0], arrays start with index [1] StacktraceThe Error Occurred in /var/www/ScanControllers.cfm: line 418 416: <CFSET USBTree[i]=ParseUSB(CurrLine)> 417: <CFSET Tmp=REMatchNoCase("[\da-f]{4}:[\da-f]{2}:[\da-f]{2}.[\da-f]{1}",CurrLine)> 418: <CFSET USBTree[i].Bus=Tmp[ArrayLen(Tmp)]> 419: </CFLOOP> 420: called from /var/www/ScanControllers.cfm: line 372 370: <CFSET OK=1> 371: </CFIF> 372: </CFLOOP> 373: <!--- Set top level ChildDrives ---> 374: <CFLOOP index="i" from="1" to="#ArrayLen(HWTree)#"> Java Stacktracelucee.runtime.exp.ExpressionException: Invalid array index [0], arrays start with index [1] at lucee.runtime.type.wrap.ListAsArray.getE(ListAsArray.java:110) at lucee.runtime.type.wrap.ListAsArray.get(ListAsArray.java:284) at lucee.runtime.type.wrap.ListAsArray.get(ListAsArray.java:289) at lucee.runtime.type.util.ArraySupport.get(ArraySupport.java:326) at lucee.runtime.util.VariableUtilImpl.get(VariableUtilImpl.java:263) at lucee.runtime.PageContextImpl.get(PageContextImpl.java:1516) at scancontrollers_cfm$cf.call_000027(/ScanControllers.cfm:418) at scancontrollers_cfm$cf.call(/ScanControllers.cfm:372) at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:933) at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:823) at lucee.runtime.listener.ClassicAppListener._onRequest(ClassicAppListener.java:66) at lucee.runtime.listener.MixedAppListener.onRequest(MixedAppListener.java:45) at lucee.runtime.PageContextImpl.execute(PageContextImpl.java:2464) at lucee.runtime.PageContextImpl._execute(PageContextImpl.java:2454) at lucee.runtime.PageContextImpl.executeCFML(PageContextImpl.java:2427) at lucee.runtime.engine.Request.exe(Request.java:44) at lucee.runtime.engine.CFMLEngineImpl._service(CFMLEngineImpl.java:1090) at lucee.runtime.engine.CFMLEngineImpl.serviceCFML(CFMLEngineImpl.java:1038) at lucee.loader.engine.CFMLEngineWrapper.serviceCFML(CFMLEngineWrapper.java:102) at lucee.loader.servlet.CFMLServlet.service(CFMLServlet.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:94) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:492) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:684) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1152) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684) at org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run(AprEndpoint.java:2464) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Timestamp4/16/21 2:14:51 PM BST Quote Link to comment
jbartlett Posted April 18, 2021 Author Share Posted April 18, 2021 On 4/16/2021 at 6:21 AM, SimonF said: Hi John, If you have USBIP module vhci_hcd loaded then your scan fails for USBs. I have attached the log here, let me know if you need any additional info or need me to test? Can you show me how you have the USBIP module vhci_hcd set up so I can duplicate the issue as you have it configured? Quote Link to comment
SimonF Posted April 18, 2021 Share Posted April 18, 2021 1 hour ago, jbartlett said: Can you show me how you have the USBIP module vhci_hcd set up so I can duplicate the issue as you have it configured? You just need to modprobe vhci_hcd this will then create two virtual usb hubs Apr 18 06:48:19 computenode kernel: vhci_hcd vhci_hcd.0: USB/IP Virtual Host Controller Apr 18 06:48:19 computenode kernel: vhci_hcd vhci_hcd.0: new USB bus registered, assigned bus number 3 Apr 18 06:48:19 computenode kernel: vhci_hcd: created sysfs vhci_hcd.0 Apr 18 06:48:19 computenode kernel: hub 3-0:1.0: USB hub found Apr 18 06:48:19 computenode kernel: hub 3-0:1.0: 8 ports detected Apr 18 06:48:19 computenode kernel: vhci_hcd vhci_hcd.0: USB/IP Virtual Host Controller Apr 18 06:48:19 computenode kernel: vhci_hcd vhci_hcd.0: new USB bus registered, assigned bus number 4 Apr 18 06:48:19 computenode kernel: usb usb4: We don't know the algorithms for LPM for this host, disabling LPM. Apr 18 06:48:19 computenode kernel: hub 4-0:1.0: USB hub found Apr 18 06:48:19 computenode kernel: hub 4-0:1.0: 8 ports detected Quote Link to comment
jbartlett Posted April 19, 2021 Author Share Posted April 19, 2021 Deployed version 2.9.2 which will ignore virtual USB hubs 1 Quote Link to comment
Alexstrasza Posted May 3, 2021 Share Posted May 3, 2021 Hi John, Am I going insane, or did there used to be a "Benchmark all" button or something to that extent? The only 'multiple benchmark' button I can find is under the controller info, and that seems to just check for a controller bottleneck. Also, when I look at the "USB Bus Tree" page, I see this error. Thanks for all your work! Quote Link to comment
jbartlett Posted May 3, 2021 Author Share Posted May 3, 2021 5 hours ago, Alexstrasza said: Hi John, Am I going insane, or did there used to be a "Benchmark all" button or something to that extent? The only 'multiple benchmark' button I can find is under the controller info, and that seems to just check for a controller bottleneck. Also, when I look at the "USB Bus Tree" page, I see this error. Thanks for all your work! The error in the USB Bus tree stopped the rest of the page from displaying which prevented you from seeing the Benchmark All button. I pushed version 2.9.3 which checks for the existence of the idProduct variable and sets it if not created by the USB device. 1 Quote Link to comment
Zonediver Posted May 4, 2021 Share Posted May 4, 2021 (edited) ...it's still strange... Dont know if unraid or something else is the culprit... But what i can say is this: I can remember that an older version of this docker started "all" disk and "then" did the testing. This version start a disk, do a test, start the next disk, do a test and so on... Edited May 4, 2021 by Zonediver Quote Link to comment
jbartlett Posted May 4, 2021 Author Share Posted May 4, 2021 4 hours ago, Zonediver said: ...it's still strange... Dont know if unraid or something else is the culprit... But what i can say is this: I can remember that an older version of this docker started "all" disk and "then" did the testing. This version start a disk, do a test, start the next disk, do a test and so on... Clicking on the "Benchmark Drives" button on the home page (click on the "DiskSpeed" header) allows you to benchmark all controllers at the same time with controller testing each drive attached to it in sequence. Quote Link to comment
Zonediver Posted May 5, 2021 Share Posted May 5, 2021 11 hours ago, jbartlett said: Clicking on the "Benchmark Drives" button on the home page (click on the "DiskSpeed" header) allows you to benchmark all controllers at the same time with controller testing each drive attached to it in sequence. I know this option but thats not the point. The strange thing is, if all disk are in sleep and i start the benchmark, almost all disk are too slow and i dont know why... If all of them are spun up, all is fine and running normal. Quote Link to comment
jbartlett Posted May 5, 2021 Author Share Posted May 5, 2021 6 hours ago, Zonediver said: I know this option but thats not the point. The strange thing is, if all disk are in sleep and i start the benchmark, almost all disk are too slow and i dont know why... If all of them are spun up, all is fine and running normal. The average speed logic discards the first read which includes any potential spinup / drive head placement. It might not be dropping enough in your case. Does your SMART data include spin up time? Quote Link to comment
Zonediver Posted May 5, 2021 Share Posted May 5, 2021 6 hours ago, jbartlett said: Does your SMART data include spin up time? Yep Quote Link to comment
jbartlett Posted May 5, 2021 Author Share Posted May 5, 2021 1 hour ago, Zonediver said: Yep Can you provide that information? Quote Link to comment
Zonediver Posted May 6, 2021 Share Posted May 6, 2021 (edited) 6 hours ago, jbartlett said: Can you provide that information? Yes - do you need this data for all drives or only for that on the HBA? Anyway, i will create a list of all disks containing this information. Edited May 6, 2021 by Zonediver Quote Link to comment
jbartlett Posted May 6, 2021 Author Share Posted May 6, 2021 10 hours ago, Zonediver said: Yes - do you need this data for all drives or only for that on the HBA? Anyway, i will create a list of all disks containing this information. All would be telling. Notably if the drives on one controller is higher than drives on another. In theory, they should all be roughly the same. If it looks like they are, just a number. It should be given in milliseconds or 1000=1 second. The dd command outputs a status every second so if it takes longer than I accounted for, I may need to increase the number of seconds to drop from the start. Quote Link to comment
Zonediver Posted May 6, 2021 Share Posted May 6, 2021 (edited) 20 minutes ago, jbartlett said: In theory, they should all be roughly the same. ...then this is strange... The 8TB-disks are much faster then the 4TBs... Edited May 6, 2021 by Zonediver Quote Link to comment
jbartlett Posted May 6, 2021 Author Share Posted May 6, 2021 Good news: It's not your system! I spun down my drives and ran a controller benchmark and I got the same results. The drive benchmark performs a couple seek tests & a latency test on the drive prior to benchmarking so it's pretty much assured to have the drives spun up when it actually starts reading the drive via the dd command. The controller benchmark doesn't do those tests prior. I need to revisit the spin-up logic. Quote Link to comment
Zonediver Posted May 6, 2021 Share Posted May 6, 2021 (edited) 12 minutes ago, jbartlett said: The controller benchmark doesn't do those tests prior. I need to revisit the spin-up logic. Interesting and good tho know that my hardware is not the culprit - thanks for this Info 👍 Edited May 6, 2021 by Zonediver Quote Link to comment
Alexstrasza Posted May 7, 2021 Share Posted May 7, 2021 On 5/3/2021 at 6:10 PM, jbartlett said: The error in the USB Bus tree stopped the rest of the page from displaying which prevented you from seeing the Benchmark All button. I pushed version 2.9.3 which checks for the existence of the idProduct variable and sets it if not created by the USB device. Good to hear! I'm glad I'm not going insane :). Unfortunately I still can't see the button because of another error: key [iManufacturer] doesn't exist Sounds like it's the same thing again but for this key 😅. I wonder what it's trying to read that lacks these values? Quote Link to comment
jbartlett Posted May 7, 2021 Author Share Posted May 7, 2021 5 hours ago, Alexstrasza said: Good to hear! I'm glad I'm not going insane :). Unfortunately I still can't see the button because of another error: key [iManufacturer] doesn't exist Sounds like it's the same thing again but for this key 😅. I wonder what it's trying to read that lacks these values? Your USB device is annoying. 😁 I added an existence check for that variable and looked for other potential misses and repushed version 2.9.3 1 Quote Link to comment
Alexstrasza Posted May 7, 2021 Share Posted May 7, 2021 49 minutes ago, jbartlett said: Your USB device is annoying. 😁 I added an existence check for that variable and looked for other potential misses and repushed version 2.9.3 It's working now 🎉! . Thanks John. 1 Quote Link to comment
Sander de Ruiter Posted May 20, 2021 Share Posted May 20, 2021 Hi, Trying to find why Unraid feels slow. BlackMagic Disk Speed on a Mac reports 110MB/s to a empty share, which is very reasonable (on Gigabit connection). I've run DiskSpeed. My Parity disk comes to 265MB/s and my data disk to 175MB/s. My question: is there an optimal placement of disks on ports? Right now, I have cache (SSD) on port 1, data disk on port 2 and parity on port 5 (see below full output) Output from DiskSpeed: 400 Series Chipset SATA Controller ASMedia Technology Inc. (Advanced Micro Devices, Inc. [AMD]) SATA controller Type: Onboard Controller Current & Maximum Link Speed: 8GT/s width x4 (3.94 GB/s max throughput) Capabilities: storage msi pm pciexpress ahci_1.0 bus_master cap_list rom Port 1: sdb 1TB Samsung SSD 860 EVO Rev RVT01B6Q Serial: S3Z9NB0K616565P (Cache) Port 2: sdc 6TB Seagate ST6000DM003 Rev 0001 Serial: ZF300PP8 (Disk 1) Port 3: N/A Port 4: N/A Port 5: sdd 8TB Seagate ST8000VN004 Rev SC60 Serial: WSD133GD (Parity) Port 6: N/A Port 7: N/A Port 8: N/A Quote Link to comment
jbartlett Posted May 21, 2021 Author Share Posted May 21, 2021 On 5/20/2021 at 3:04 AM, Sander de Ruiter said: Hi, Trying to find why Unraid feels slow. BlackMagic Disk Speed on a Mac reports 110MB/s to a empty share, which is very reasonable (on Gigabit connection). I've run DiskSpeed. My Parity disk comes to 265MB/s and my data disk to 175MB/s. My question: is there an optimal placement of disks on ports? Right now, I have cache (SSD) on port 1, data disk on port 2 and parity on port 5 (see below full output) Wondering what you mean by "feel slow". But those are good speeds. Try running a controller benchmark to see if a controller isn't able to handle all the drives active at the same time. I'll get a read speed from each drive one after the other and again all at the same time and compare the two. From what I see, you're using an onboard controller with three drives out of a possible six. It looks like it should be able to handle it fine. You can also try running a disk benchmark at with a 1% interval scan (101 scan points vs the default 11) which might help locate questionable areas. The next major release of DiskSpeed will add support for full surface level testing but I don't have a timeline for that release. I'm currently slammed at my job and I haven't had much time of late for hobby programming. Quote Link to comment
pantheis Posted June 5, 2021 Share Posted June 5, 2021 I felt compelled to post here and give you a HUGE thank you for putting this utility together. It is exactly what I didn't know I needed! I have a lot of 2TB drives of various ages, some over 7 years, and while they pass their monthly parity check and show no issues in SMART, your utility will help me monitor the actual performance of the drives and help catch any issues. I imagine it's only a matter of time before I'll need to replace a couple of the drives. Thank you! 1 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.