Jump to content
jbartlett

DiskSpeed, hard drive benchmarking (unRAID 6+), version 2.4

380 posts in this topic Last Reply

Recommended Posts

This Docker Application will let you view your storage controllers & the drives attached to them and perform Benchmarks on both. Controller Benchmarks helps to identify if the drives attached to it will potentially exceed the capacity of the controller if they were all fully being read from at the same time (such as during a Parity check). Drive Benchmarks lets you monitor the performance over time to look for desegregation or unexpected slow areas while getting a clean SMART report.

 

Installation

Via the Community Application: Search for "DiskSpeed"

Manual Installation

    (The Community Applications plugin is having issues currently, here's a work around for now)

    Save the attached "my-DiskSpeed.xml" file to your NAS under \\tower\flash\config\plugins\dockerMan\templates-user

    View the Docker tab in your unRAID Administrator , click on "Add Container"

    Under "Select a template", pick "my-DiskSpeed"

    The defaults should work as-is unless you have port 18888 already in use. If so, change the Web Port & WebUI settings to a new port number. The Docker will create a directory called "DiskSpeed" in your appdata directory to hold persistent data.

    Note: Privileged mode is required so that the application can see the controllers & drives on the host OS.

    This docker will use up to 512MB of RAM. RAM optimization will happen in a later BETA.

 

Running

View the Docker tab in your unRAID Administrator and click on the icon next to "DiskSpeed" and select WebUI. A new window will open.

On first time run (or after the Docker app is updated or you select to rescan hardware), the application will scan your system to locate drive controllers & the hard drives attached to them.

 

Drive Images

As of this post, the Hard Drive Database (HDDB) has 900+ drive models in 20+ brands. If you have one or more drives that do not have a predefined image in the HDDB, you have a couple options available - wait for me to add the image which will be displayed after you click "Rescan Controllers" or you can add the drive yourself by editing it and uploading a drive image for it. You can view drive images in the HDDB to see if there's an image that'll fit your drive and optionally upload it so others can benefit.

 

Controller & Drive Identification Issues

Some drives, notably SSD's, do not reveal the Vendor correctly or at all. If you view the Drive information and it has the same value for the vendor as the model or an incorrect or missing Vendor, please inform me so that I can manually add the drive to the database or add code to handle it. If you have a controller that is not detected, please notify me.

 

Benchmarking Drives

The current method of benchmarking the hard drives is to read the hard drive at certain percentages for 15 seconds and takes the average speed over each of those seconds except for the first 2 seconds which tend to trend high. Hard Drives report an optimal block size to use while reading but if not, a block size of 128K is used. Since Docker under unRAID requires the array to be running, SpeedGap detection was added to detect disk drive activity during the test by comparing the smallest & largest amount of bytes read over the 15 seconds. If a gap is detected over a given size which starts at 45MB, the gap allowed is increased by 5MB and the spot retested. If a drive keeps triggering the SpeedGap detection over each spot, you may need to disable the SpeedGap detection as the drive is very erratic in its read speeds.

 

One drive per controller is tested at the same time. In the future, each drive will be tested to see it's maximum transfer rate and the system will be tested to see how much data each controller can transfer and how much data the entire system bus can handle. Then multiple drives over multiple controllers will be read simultaneously while keeping the overall bandwidth by controller & system under the maximum transfer rate.

 

Contributing to the Hard Drive Database

If you have a drive that doesn't have information in the Hard Drive Database other than the model or you've performed benchmark tests, a button will be displayed at the bottom of the page labeled "Upload Drive & Benchmark Data to the Hard Drive Database". The HDDB will display information given up by the OS for the drives and the average speed graphs for comparison.

 

Application Errors

If you get an error message, please post the error here and the steps you took to cause it to happen. There will be a long string of java diagnostics after the error message (java stack) that you do not need to include, just the error message details.

If you can't get past the Scanning Hardware screen, change the URL from http://[ip]:[port]/ScanControllers.cfm to http://[ip]:[port]/isolated/CreateDebugInfo.cfm and hit enter.

 

Note: The unRAID diagnostic file doesn't provide any help. If submitting a diagnostic file, please use the link at the bottom of the controllers in the Diskspeed GUI.

 

Home Screen (click top label to return to this screen)

DiskSpeedDockerHome.thumb.jpg.041ec71e0f665b2d2fa43cdad7c52561.jpg

 

 

 

Controller Information

ControllerInfo2.thumb.png.f76e947529cdd73cea31feb583f6eb41.png

 

 

 

Drive Information

DriveInfo.png.07842dd00917e01e406e342ecacea844.png

 

Drive Editor

 

DriveEditor.png.d8715c6494f9c048d73f9baac7dc78d1.png

 

my-DiskSpeed.xml

Edited by jbartlett

Share this post


Link to post

When I tested this utility against my production server, I noticed that I had a drive going wonky on me. I couldn't even test it at first because it kept tripping the SpeedGap detection in which the minimum & maximum speed over 15 seconds was too great - a sign of disk activity or in my case, a drive giving very inconsistent read speeds. I had to add logic to disable the SpeedGap detection to even be able to fully test the drive.

 

In this case, the drives are all the same make & revision and their curve should be nearly identical but Disk 5 stands out.

 

baddrive.png.044505bc32bd13dee1ec79674beb365f.png

 

Viewing Drive 5 by itself, I can see it's curve is not normal. Spinners should have a steady decline over the entire range of the drive. I'll be retiring this drive from my main server and using it for platter heat map test where the entire drive is read and a heatmap of the read speeds given.

 

baddrive2.png.6a323cbce4da0f25d21e58264a236845.png

Edited by jbartlett

Share this post


Link to post

VERY NICE

 

I notice I've got several missing models/images though. I see that you show that "they are reported for my review". Does that mean you don't need any more info from me regarding them?

Share this post


Link to post

Also, i see that my Mushkin MKNSSDRE240GB-LT SSD seems have been improperly identified as a Toshiba drive.

 

SafariScreenSnapz088.jpg.26e94b134dad6d09097d13f8c82427db.jpg

 

I added the image manually from your database.

Edited by wgstarks

Share this post


Link to post

WD80EMAZ was missing from the database as far as I could see. I uploaded a cropped image I stole off the internet.

Share this post


Link to post
3 hours ago, wgstarks said:

VERY NICE

 

I notice I've got several missing models/images though. I see that you show that "they are reported for my review". Does that mean you don't need any more info from me regarding them?

 

Correct. I have 16 drive models identified that aren't in my database.

Share this post


Link to post
3 hours ago, wgstarks said:

Mushkin MKNSSDRE240GB-LT SSD seems have been improperly identified as a Toshiba

 

That seems to be the case for all Mushkin drives. I added logic to beta 2a (pushed) to change the Vendor to Mushkin if the model starts with MKN.

Share this post


Link to post
1 hour ago, jonathanm said:

WD80EMAZ was missing from the database as far as I could see. I uploaded a cropped image I stole off the internet.

 

All of the images came from the same source ;) Though some vendors were nice enough to have presentation media to use.

Share this post


Link to post

Will the docker still auto-update if I installed it manually? CA server still appears to be tits up. Not sure if that also effects updates.

Share this post


Link to post
2 minutes ago, wgstarks said:

Will the docker still auto-update if I installed it manually?

Yes

Share this post


Link to post
1 hour ago, jbartlett said:

 

That seems to be the case for all Mushkin drives. I added logic to beta 2a (pushed) to change the Vendor to Mushkin if the model starts with MKN.

After updating to beta 2a I see that changes that I made to the drive capacity display aren’t being saved. Would it be possible to save these changes across docker updates?

 

Also, the Mushkin is now detected correctly. Thanks.

Edited by wgstarks

Share this post


Link to post

Beta 2b pushed

  • Change Vendor to Muskin if the model starts with "MKN" - Muskin drives identify as Toshiba.
  • Change Vendor to Crucial if the model starts with "MTFD" - drive Vendor not given
  • Change Vendor to OCZ if the vendor starts with "OCZ-"
  • Change Vendor to "Samsung" if model string starts with "Samsung" - some drives report as "Western Digital"
  • Added Model RegEx "CT[0-9]{3,}(B|M)[A-Z]?[0-9]{2,}SSD" to identify Crucial drives
  • Added Mushkin model cleanup
  • Added Plextor model cleanup

Share this post


Link to post

Not seeing the drives on my Areca controller.

ARC-1280/1280ML 24-Port PCI-Express to SATA II RAID Controller
Areca Technology Corp. 
RAID bus controller 

No drives detected

Current Link Speed: 2.5GT/s width x8 (2 GB/s max throughput)
Maximum Link Speed: 2.5GT/s width x8 (2 GB/s max throughput)
Bus: 32 bits @ 33MHz
Driver: arcmsr
Capabilities: storage pm msi pciexpress bus_master cap_list rom

ps. just noticed it misidentifying the model number. I have Areca ARC-1231ML

Edited by interwebtech

Share this post


Link to post
5 minutes ago, interwebtech said:

just noticed it misidentifying the model number. I have Areca ARC-1231ML

 

I won't be able to correct these. It's displaying what the OS is reporting. Likely the PCI ID database for the the controller is wrong in the master PCI ID list.

Share this post


Link to post
1 hour ago, wgstarks said:

After updating to beta 2a I see that changes that I made to the drive capacity display aren’t being saved. Would it be possible to save these changes across docker updates?

 

Also, the Mushkin is now detected correctly. Thanks.

 

15 minutes ago, jbartlett said:

 

Beta 2b pushed

 

Same issue with display customizations not being saved across docker updates.

Pretty minor issue though.

Share this post


Link to post
1 minute ago, wgstarks said:

Same issue with display customizations not being saved across docker updates.

 

Check your Docker setup to see if Local Storage is defined and the directory location contains files. This is a sign that it's not saving data to the appdata share and the changes are being stored in the Docker container itself which is lost whenever you update the application via Docker Update.

Share this post


Link to post
38 minutes ago, jbartlett said:

 

You mean the drive editor? I haven't made any changes to the code behind it. Try clicking the "Rescan Controllers"  button to see if that helps.

Rescanning Controllers seems to be the cause of the problem.

 

It looks like edits to the drives are saved in storage.json and whenever I rescan controllers storage.json gets overwritten (actually it’s renamed to oldstorage.json) so any edits that were made to it are lost.

 

This is just what I’m seeing on my system. Maybe this isn’t proper behavior?

Share this post


Link to post

Installed

ooYPseY.png

 

this is what I have.

 

and this is what is "found" even after re-scans.

cAXAgiE.png

 

Missing a few drives there. (3 total)

 

1 3tb WD

1 2TB WD

1 5TB Seagate

 

any advice?

Share this post


Link to post
29 minutes ago, wgstarks said:

Rescanning Controllers seems to be the cause of the problem.

 

It looks like edits to the drives are saved in storage.json and whenever I rescan controllers storage.json gets overwritten (actually it’s renamed to oldstorage.json) so any edits that were made to it are lost.

 

This is just what I’m seeing on my system. Maybe this isn’t proper behavior?

 

Nope, sure isn't proper behavior. The storage.json file is recreated every time the system is scanned and oldstorage.json is created if I need to preserve any data but it should be saving the drive configuration in a save directory under <appdata>/DiskSpeed/Instances/local/driveinfo for the drive in a file named "config.json" which doesn't seem to be always created. I'll look into it.

Share this post


Link to post
20 minutes ago, Nyghthawk said:

Missing a few drives there. (3 total)

 

Do you have a 3rd controller that the three missing drives are attached to? If so, what is it?

Share this post


Link to post

Not that I am aware of. 

 

14 SATA connectors on the Motherboard is all I have. I think its split up in 10 and then 4

 

1 minute ago, jbartlett said:

 

Do you have a 3rd controller that the three missing drives are attached to? If so, what is it?

 

Edited by Nyghthawk

Share this post


Link to post

I have a similar issue in that one of my drives is not shown but the other 5 are. Also on the right is says 1 new drive detected (SDF) which is also the one not shown.

 

image.thumb.png.73a5e7b9c2819fd83405138ccdb79b12.png

Share this post


Link to post
5 minutes ago, jbartlett said:

 

Nope, sure isn't proper behavior. The storage.json file is recreated every time the system is scanned and oldstorage.json is created if I need to preserve any data but it should be saving the drive configuration in a save directory under <appdata>/DiskSpeed/Instances/local/driveinfo for the drive in a file named "config.json" which doesn't seem to be always created. I'll look into it.

I’m getting <appdata>/DiskSpeed/Instances/local/<diskID>/driveinfo/config.json but looks like no edits are written to this file.

Share this post


Link to post
33 minutes ago, Nyghthawk said:

14 SATA connectors on the Motherboard is all I have. I think its split up in 10 and then 4

 

27 minutes ago, MMW said:

I have a similar issue in that one of my drives is not shown but the other 5 are. Also on the right is says 1 new drive detected (SDF) which is also the one not shown.

 

Onboard SATA controllers aren't being detected. I'll add code to pack up the saved data files so I can see exactly what you see and troubleshoot.

Share this post


Link to post

Beta 2c pushed

  • Added additional Mushkin model cleanup
  • Added Port Number for the drives on the Home Screen
  • Added troubleshooting debug file logic
  • Keep drive edits after rescanning the controllers

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.