DiskSpeed, hard drive benchmarking (unRAID 6+), version 2.9.2


Recommended Posts

On 9/29/2018 at 12:32 PM, interwebtech said:

Taking drive images requests?

 

 

That's a good image. I'll save it to add but you can also add & submit images too 👍

 

If you haven't already done so and you'd like to check out the feature, view the drive with the image you want to update and click on "Edit Drive", then "Upload New Image". Follow the instructions and you're good to go.

 

Since the capacity is on the image, you don't have to worry about the text overlay to add it.

Edited by jbartlett
Link to comment
  • 1 month later...

Pretty significant changes come with beta 6a

  • Updated to support in-window vertical scrolling of both the left side drive list and the right side informational display.
  • Viewing the controller information, display if the drive controller is onboard or what PCI/PCI-e slot the controller card is in on the motherboard
  • Changed how the Benchmark abort button works to accommodate how different browsers render frames
  • Redesign the Rotate option on the Drive Edit text overlay from free-form rotation to 90 degree only
  • Trap errors from lsusb & lspci during hardware scanning
  • Fixed line wrapping of drive selection on the Benchmark Drives screen when the All Drives checkbox is unset
  • Modify the Benchmark submission to pass a SHA1 (non-reverseable encryption) hash of the app's unique installation ID and a compressed MD5 hash of the drive's vendor, model, & serial number to support benchmark recovery and allow your benchmark to compare with others.
    The Unique Installation ID is derived by taking a SHA1 of the unRAID registration name and a SHA1 of the unRAID registration ID and generating a SHA1 of those two SHA1's
    For non-unRAID installations, a SHA1 of the Motherboard's Serial ID is used.
    The compressed MD5 takes each 4 byte block of the MD5 of the drive's Serial Number, converts it from Hex to Dec, and adds all the blocks together. The resulting sum is converted to Base36. It is impossible to derive the original values.
    Example Installation ID: 1946C31AEEF030DB495B0E13AD1403E9CB0AA7A2
    Example Drive ID: 4QJ3
  • On the first run after upgrading to Beta 6a, all benchmarks will be flagged as not being submitted - so you will get the button to submit them again. This is to allow your Benchmark information to be saved in the new database layout. Benchmark submissions using a version prior to 6a are disabled.

 

Controller information may not display the PCIe slot information on Ryzen systems and report it is an onboard controller. Under investigation.

Edited by jbartlett
Link to comment

Hello,

 

on a fresh install on the docker and appdata. I get this issue

Scanning Hardware
18:32:51 Spinning up hard drives
18:32:51 Scanning system storage
18:33:04 Scanning USB Bus
18:33:05 Scanning hard drives

Lucee 5.2.9.31 Error (application)

MessageError invoking external process

Detail/usr/bin/lspci: option requires an argument -- 's'
Usage: lspci [<switches>]

Basic display modes:
-mm Produce machine-readable output (single -m for an obsolete format)
-t Show bus tree

Display options:
-v Be verbose (-vv for very verbose)
-k Show kernel drivers handling each device
-x Show hex-dump of the standard part of the config space
-xxx Show hex-dump of the whole config space (dangerous; root only)
-xxxx Show hex-dump of the 4096-byte extended config space (root only)
-b Bus-centric view (addresses and IRQ's as seen by the bus)
-D Always show domain numbers

Resolving of device ID's to names:
-n Show numeric ID's
-nn Show both textual and numeric ID's (names & numbers)
-q Query the PCI ID database for unknown ID's via DNS
-qq As above, but re-query locally cached entries
-Q Query the PCI ID database for all ID's via DNS

Selection of devices:
-s [[[[<domain>]:]<bus>]:][<slot>][.[<func>]] Show only devices in selected slots
-d [<vendor>]:[<device>][:<class>] Show only devices with specified ID's

Other options:
-i <file> Use specified ID database instead of /usr/share/misc/pci.ids.gz
-p <file> Look up kernel modules in a given file instead of default modules.pcimap
-M Enable `bus mapping' mode (dangerous; root only)

PCI access options:
-A <method> Use the specified PCI access method (see `-A help' for a list)
-O <par>=<val> Set PCI access parameter (see `-O help' for a list)
-G Enable PCI access debugging
-H <mode> Use direct hardware access (<mode> = 1 or 2)
-F <file> Read PCI configuration dump from a given file

StacktraceThe Error Occurred in
/var/www/ScanControllers.cfm: line 449 

447: <CFSET HW[Key].Path=ControllerPath>
448: <!--- Get the controller information --->
449: <cfexecute name="/usr/bin/lspci" arguments="-vmm -s #Key#" timeout="300" variable="lspci" />
450: <CFSET tmpbus=Replace(Key,":","-","ALL")>
451: <CFFILE action="write" file="#PersistDir#/lspci-vmm_#tmpbus#.txt" output="#lspci#" addnewline="NO" mode="666">


called from /var/www/ScanControllers.cfm: line 445 

443: </CFIF>
444: </CFLOOP>
445: <CFIF StructKeyExists(HW,Key) EQ "NO">
446: <CFSET HW[Key]=Duplicate(BlankController)>
447: <CFSET HW[Key].Path=ControllerPath>


Java Stacktracelucee.runtime.exp.ApplicationException: Error invoking external process
  at lucee.runtime.tag.Execute.doEndTag(Execute.java:258)
  at scancontrollers_cfm$cf.call_000040(/ScanControllers.cfm:449)
  at scancontrollers_cfm$cf.call(/ScanControllers.cfm:445)
  at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:933)
  at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:823)
  at lucee.runtime.listener.ClassicAppListener._onRequest(ClassicAppListener.java:66)
  at lucee.runtime.listener.MixedAppListener.onRequest(MixedAppListener.java:45)
  at lucee.runtime.PageContextImpl.execute(PageContextImpl.java:2464)
  at lucee.runtime.PageContextImpl._execute(PageContextImpl.java:2454)
  at lucee.runtime.PageContextImpl.executeCFML(PageContextImpl.java:2427)
  at lucee.runtime.engine.Request.exe(Request.java:44)
  at lucee.runtime.engine.CFMLEngineImpl._service(CFMLEngineImpl.java:1090)
  at lucee.runtime.engine.CFMLEngineImpl.serviceCFML(CFMLEngineImpl.java:1038)
  at lucee.loader.engine.CFMLEngineWrapper.serviceCFML(CFMLEngineWrapper.java:102)
  at lucee.loader.servlet.CFMLServlet.service(CFMLServlet.java:51)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
  at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
  at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
  at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:94)
  at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:492)
  at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
  at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
  at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
  at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:684)
  at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502)
  at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1152)
  at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684)
  at org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run(AprEndpoint.java:2464)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
  at java.lang.Thread.run(Thread.java:748)


Timestamp11/21/18 6:33:05 PM EST

Any help would be greatly appreciated

Link to comment
4 hours ago, eurlin said:

Hello,

 

on a fresh install on the docker and appdata. I get this issue


Scanning Hardware
18:32:51 Spinning up hard drives
18:32:51 Scanning system storage
18:33:04 Scanning USB Bus
18:33:05 Scanning hard drives

Lucee 5.2.9.31 Error (application)

MessageError invoking external process

Detail/usr/bin/lspci: option requires an argument -- 's'
Usage: lspci [<switches>]


StacktraceThe Error Occurred in
/var/www/ScanControllers.cfm: line 449 

447: <CFSET HW[Key].Path=ControllerPath>
448: <!--- Get the controller information --->
449: <cfexecute name="/usr/bin/lspci" arguments="-vmm -s #Key#" timeout="300" variable="lspci" />
450: <CFSET tmpbus=Replace(Key,":","-","ALL")>
451: <CFFILE action="write" file="#PersistDir#/lspci-vmm_#tmpbus#.txt" output="#lspci#" addnewline="NO" mode="666">

Any help would be greatly appreciated

 

That's really odd. I can't duplicate and that shouldn't happen. You've got something weird going on with your setup. if you could, please use update the Docker and run again. When you get the error, replace the "/ScanControllers.cfm" in the URL with "/isolated/CreateDebugInfo.cfm" and click on the "Create debug file with Controller Info". Email the resulting file to hddb@strangejourney.net

 

I added some debug code to create an empty file with the command about to be executed and then deletes it after executing it.

Edited by jbartlett
Link to comment

Turned this on today to bench 2 new WD drives added to the array. Getting the following error when trying to open the WebGUI. I uninstalled and reinstalled via user templates and again straight from CA search results. No change, error persists. I even tried creating the "submitted.txt" it is complaining about in the location specified and while it did get deleted, the error still remains.

 

Lucee 5.2.9.31 Error (application)
Message	source file [/tmp/DiskSpeed/Instances/local/driveinfo/ssd_970_evo_1b2qexe7_s467nf0k603458f_1tb/benchmark/1538249704071/submitted.txt] is not a file
Stacktrace	The Error Occurred in
/var/www/Upgrade.cfm: line 9 
7: <CFLOOP index="BenchIdx" from="1" to="#BenchData.RecordCount#">
8: <CFIF FileExists("#BenchDir#/#BenchData.Name[BenchIdx]#/version.txt") EQ "NO">
9: <CFFILE action="delete" file="#BenchDir#/#BenchData.Name[BenchIdx]#/submitted.txt">
10: <CFFILE action="write" file="#BenchDir#/#BenchData.Name[BenchIdx]#/version.txt" output="#Version#" addnewline="NO">
11: </CFIF>

called from /var/www/index.cfm: line 65 
63: 
64: <CFIF FileExists("/tmp/DiskSpeedTmp/firstrun.txt") EQ "NO">
65: <CFINCLUDE TEMPLATE="Upgrade.cfm">
66: <CFLOCATION URL="ScanControllers.cfm" AddToken="NO">
67: </CFIF>

Java Stacktrace	lucee.runtime.exp.ApplicationException: source file [/tmp/DiskSpeed/Instances/local/driveinfo/ssd_970_evo_1b2qexe7_s467nf0k603458f_1tb/benchmark/1538249704071/submitted.txt] is not a file
 	at lucee.runtime.tag.FileTag.checkFile(FileTag.java:1159)
 	at lucee.runtime.tag.FileTag.actionDelete(FileTag.java:579)
 	at lucee.runtime.tag.FileTag.doStartTag(FileTag.java:381)
 	at upgrade_cfm$cf.call(/Upgrade.cfm:9)
 	at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:933)
 	at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:823)
 	at lucee.runtime.PageContextImpl.doInclude(PageContextImpl.java:805)
 	at index_cfm$cf.call(/index.cfm:65)
 	at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:933)
 	at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:823)
 	at lucee.runtime.listener.ClassicAppListener._onRequest(ClassicAppListener.java:66)
 	at lucee.runtime.listener.MixedAppListener.onRequest(MixedAppListener.java:45)
 	at lucee.runtime.PageContextImpl.execute(PageContextImpl.java:2464)
 	at lucee.runtime.PageContextImpl._execute(PageContextImpl.java:2454)
 	at lucee.runtime.PageContextImpl.executeCFML(PageContextImpl.java:2427)
 	at lucee.runtime.engine.Request.exe(Request.java:44)
 	at lucee.runtime.engine.CFMLEngineImpl._service(CFMLEngineImpl.java:1090)
 	at lucee.runtime.engine.CFMLEngineImpl.serviceCFML(CFMLEngineImpl.java:1038)
 	at lucee.loader.engine.CFMLEngineWrapper.serviceCFML(CFMLEngineWrapper.java:102)
 	at lucee.loader.servlet.CFMLServlet.service(CFMLServlet.java:51)
 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292)
 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
 	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
 	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
 	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:94)
 	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:492)
 	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
 	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
 	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
 	at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:684)
 	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
 	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502)
 	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1152)
 	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684)
 	at org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run(AprEndpoint.java:2464)
 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
 	at java.lang.Thread.run(Thread.java:748)
 
Timestamp	11/25/18 9:18:13 AM PST

 

Edited by interwebtech
Link to comment
4 hours ago, interwebtech said:

Turned this on today to bench 2 new WD drives added to the array. Getting the following error when trying to open the WebGUI. I uninstalled and reinstalled via user templates and again straight from CA search results. No change, error persists. I even tried creating the "submitted.txt" it is complaining about in the location specified and while it did get deleted, the error still remains.

Please update the app and try again. I put an existence check on the delete of submitted.txt which caught the scenario where a benchmark was done but wasn't previously uploaded.

 

eurlin, still looking into your issue.

Link to comment

Unable to upload new drive image. Pressing button has no effect. I am able to browse and download images from your library but cannot apply them to a drive.

 

 

Started a new benchmark all drives (except SSDs) session (windows 10, chrome browser), then walked away to go about my day. Came back an hour or so later and it was still on the first drive with no apparent activity. I did a "purge and start over" (thus attempt to edit images of my M.2 drives). I have not rerun it again yet as I have a viewer on Plex and don't want to skew results. Attached debug if it might be helpful.

ScottMarch_20181126_120701.tar.gz

Link to comment

The unable to upload images is a known issue that's been annoying the heck out of me. The right side is in an iframe and if it launches the upload dialog there, it's limited to that frame and not the entire browser window. The code running in the iframe updates a div block in the main window with a button to launch it but nothing I do activates the click event on it, even though every resource I've found says it should.

 

I decided to release it with that issue so everything else could be tested while I applied percussive techniques to the issue.

Link to comment
  • 2 weeks later...

Just wanted to thank John for this awesome application. I think I have used it before but haven't in a while and used it again today. Just simply incredible and nice to use.

 

About the only thing I am looking forward to is when its able to test all drives at the same time to test out bus limitations. I see that's planned for the future. 

 

Great work John!

  • Like 1
Link to comment
  • 1 month later...

First off, thanks for creating this. The data it provides is fantastic. 

 

So now here's my issue. I installed an LSI 9211 SAS controller flashed to IT mode. I have one drive connected to it right now and the drive is not assigned to the array yet. When I run the "Benchmark drives" (or the individual drive) it finished the scan, but then reports the speed gap is too large and retries. I've let it go up to 100 retries before aborting. I have tried checking the box to "disable speed gap detection" but it seems to have no affect. I'm using Google Chrome. I read that the speed gap is supposed to increase by 5 MB each try, but mine stays the same (45 MB).

 

Am I doing something wrong?

 

SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]: Scanning sdf at 8 TB (100%) - Speed Gap of 79.95 MB (max allowed is 45 MB), retrying (22)

Link to comment
On 2/2/2019 at 7:03 AM, Wayne66 said:

First off, thanks for creating this. The data it provides is fantastic. 

 

So now here's my issue. I installed an LSI 9211 SAS controller flashed to IT mode. I have one drive connected to it right now and the drive is not assigned to the array yet. When I run the "Benchmark drives" (or the individual drive) it finished the scan, but then reports the speed gap is too large and retries. I've let it go up to 100 retries before aborting. I have tried checking the box to "disable speed gap detection" but it seems to have no affect. I'm using Google Chrome. I read that the speed gap is supposed to increase by 5 MB each try, but mine stays the same (45 MB).

 

Am I doing something wrong?

 

SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]: Scanning sdf at 8 TB (100%) - Speed Gap of 79.95 MB (max allowed is 45 MB), retrying (22)

 

When it's testing a portion of the drive, it does a balls-to-the-wall straight read using the dd utility which is instructed to log a progress report every second. The read speeds should be pretty consistent from one second to the next. If it detects a large difference in the read speed from one spot to the next, that is typically a sign that something is reading from the drive so it tries again at that same location. It should also be increasing the gap by 5MB each time it retries a spot. Sometimes though, there could be a bad spot there or something else funky going on causing it so I added the option to disable that re-check. It should just keep going and not retry. From what you're saying, it sounds like it's not honoring the disable option and the gap allowed should be a lot more than 45MB after 22 retries. I'll look into this.

Link to comment
  • jbartlett changed the title to DiskSpeed, hard drive benchmarking (unRAID 6+), version 2.9.2

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.