How exactly are drives utilized in unRAID?


rmp5s

Recommended Posts

My array is...BRUTALLY...slow and I've been trying to think of a way to speed things up beyond adding a cache drive, which I already have.

 

I was thinking slapping a fast(ish) SATA SSD in with all my slow spinners wouldn't do anything but...I dunno...Will it?

 

I was watching this:

 

 

...and it made me wonder...maybe unRAID CAN utilize faster storage in the array to speed things along. 

 

So...what's the official verdict? Toss a few Samsung SSDs in with the spinners or no?

 

Link to comment

And, by "brutally slow", I mean EVERYTHING is drive-speed limited. The CPUs never even really do anything. Drive speed is by FAR the bottleneck no matter what I try to do on the thing. I noticed this when I was trying to render a video in my "Adobe render machine VM" and it was taking FOR. EV. ER. (Cue Sandlot clip.)

 

I'd really like to use my server to render videos as that's what I actually got the thing for. lol I don't use it for that though because my weak ass hyper-threaded dual core i7 powered laptop will crank out a video in LITERALLY a tenth of the time...

Link to comment

Array drives are always limited by that single device speed for reads and the slowest device in the array for writes (using turbo write), you can add SSDs to the array but only reads will be fast, assuming parity is a hard drive, also array devices can't be trimmed for now, so write performance might degrade with use.

Link to comment
On 12/4/2019 at 1:03 AM, johnnie.black said:

Array drives are always limited by that single device speed for reads and the slowest device in the array for writes (using turbo write), you can add SSDs to the array but only reads will be fast, assuming parity is a hard drive, also array devices can't be trimmed for now, so write performance might degrade with use.

Ah, ok. Doesn't sound like slapping SSDs in would help much. That sucks. 

 

Oh well. Thanks!

Link to comment
On 12/3/2019 at 10:53 PM, rmp5s said:

And, by "brutally slow", I mean EVERYTHING is drive-speed limited. The CPUs never even really do anything. Drive speed is by FAR the bottleneck no matter what I try to do on the thing. I noticed this when I was trying to render a video in my "Adobe render machine VM" and it was taking FOR. EV. ER. (Cue Sandlot clip.)

 

I'd really like to use my server to render videos as that's what I actually got the thing for. lol I don't use it for that though because my weak ass hyper-threaded dual core i7 powered laptop will crank out a video in LITERALLY a tenth of the time...

That description of "brutally slow" means nothing. What is the actual transfer speed?

 

From my own experience, only Linus-level stuff can make video rendering reach disk limit.

Even 125MB/s (network-limited speed) is 1 gigabit / s <-- that is a ridiculous bit rate.

 

Link to comment
11 minutes ago, testdasi said:

That description of "brutally slow" means nothing. What is the actual transfer speed?

 

From my own experience, only Linus-level stuff can make video rendering reach disk limit.

Even 125MB/s (network-limited speed) is 1 gigabit / s <-- that is a ridiculous bit rate.

 

"Brutally slow" as in, if I'm seeing 4.5-5 megs, I'm stoked.
1063132603_2019-12-0510_19_49-QEMU(Controller)-TightVNCViewer.thumb.png.c3f2115d59f8b0618dc75b9b595ffa17.png

 

This is transferring a file from an unRAID share to the desktop of a VM.  Transferring from my laptop to a share is lucky to break a meg.

Link to comment
On 12/5/2019 at 10:36 AM, johnnie.black said:

That's far from normal, you should get 40/50MB/s with default writing mode and line speed with turbo write, post the diagnostics, might provide some clues.

Good to hear something isn't right.  Because, if this WAS normal, I don't know if I'd be able to continue to use unRAID.  It's SO slow.

Diagnostics attached.  Thanks so much for the assistance.  

tower-diagnostics-20191206-2141.zip

Link to comment
Dec  6 07:54:55 Tower emhttpd: unclean shutdown detected
...
Dec  6 07:55:05 Tower kernel: mdcmd (40): check nocorrect
Dec  6 07:55:05 Tower kernel: md: recovery thread: check P ...
...
Dec  6 09:23:33 Tower kernel: md: recovery thread: P incorrect, sector=1586007976
Dec  6 09:23:33 Tower kernel: md: recovery thread: P incorrect, sector=1586008008
Dec  6 09:23:39 Tower kernel: md: recovery thread: P incorrect, sector=1587187688
Dec  6 09:23:39 Tower kernel: md: recovery thread: P incorrect, sector=1587187696
Dec  6 09:23:39 Tower kernel: md: recovery thread: P incorrect, sector=1587187704
Dec  6 09:23:42 Tower kernel: md: recovery thread: P incorrect, sector=1587880552
Dec  6 09:23:43 Tower kernel: md: recovery thread: P incorrect, sector=1588069608
Dec  6 09:23:43 Tower kernel: md: recovery thread: P incorrect, sector=1588069624
Dec  6 09:23:52 Tower kernel: md: recovery thread: P incorrect, sector=1589865904
Dec  6 09:23:52 Tower kernel: md: recovery thread: P incorrect, sector=1589865912
Dec  6 09:23:52 Tower kernel: md: recovery thread: P incorrect, sector=1589865920
Dec  6 09:23:52 Tower kernel: md: recovery thread: P incorrect, sector=1589865928
Dec  6 09:23:52 Tower kernel: md: recovery thread: P incorrect, sector=1589866056
Dec  6 09:23:53 Tower kernel: md: recovery thread: P incorrect, sector=1589997040
Dec  6 09:23:53 Tower kernel: md: recovery thread: P incorrect, sector=1589997088
Dec  6 09:23:53 Tower kernel: md: recovery thread: P incorrect, sector=1589997104
Dec  6 09:23:53 Tower kernel: md: recovery thread: P incorrect, sector=1589997296
Dec  6 09:23:54 Tower kernel: md: recovery thread: P incorrect, sector=1590007120
Dec  6 09:23:56 Tower kernel: md: recovery thread: P incorrect, sector=1590090488
Dec  6 09:23:57 Tower kernel: md: recovery thread: P incorrect, sector=1590134664
Dec  6 09:23:58 Tower kernel: md: recovery thread: P incorrect, sector=1590165472
Dec  6 09:24:12 Tower kernel: md: recovery thread: P incorrect, sector=1593228992
Dec  6 09:24:18 Tower kernel: md: recovery thread: P incorrect, sector=1594907032
Dec  6 09:24:27 Tower kernel: md: recovery thread: P incorrect, sector=1597095944
Dec  6 09:26:37 Tower kernel: md: recovery thread: P incorrect, sector=1636990432
Dec  6 09:26:52 Tower kernel: md: recovery thread: P incorrect, sector=1640489872
Dec  6 09:27:21 Tower kernel: md: recovery thread: P incorrect, sector=1647481728
Dec  6 09:28:18 Tower kernel: md: recovery thread: P incorrect, sector=1662632216

Have you been doing all your testing with a parity check underway? And obviously your parity needs correcting.

 

Why did you have an unclean shutdown? You must not power down or reboot with the array started. Only shutdown or reboot from the webUI so Unraid can stop the array cleanly.

Link to comment
On 12/6/2019 at 10:56 PM, trurl said:

Have you been doing all your testing with a parity check underway? And obviously your parity needs correcting.

 

Why did you have an unclean shutdown? You must not power down or reboot with the array started. Only shutdown or reboot from the webUI so Unraid can stop the array cleanly.

No, not all testing is with a parity check underway.

My parity needs correcting?

I haven't had an unclean shutdown.  Not that I know of.  The thing is in a datacenter...it should never be shutting down.  Ever.  

 

Link to comment
4 hours ago, rmp5s said:

I haven't had an unclean shutdown.  Not that I know of.  The thing is in a datacenter...it should never be shutting down.  Ever.  

Never say never.

Ask your datacenter provided what happened on Dec 6 at 07:54:55.

Even if nobody accidentally turns it off, your server (or server host) could have crashed - that counts as an unclean shutdown.

Dec  6 07:54:55 Tower emhttpd: unclean shutdown detected

 

Link to comment
8 hours ago, rmp5s said:

No, not all testing is with a parity check underway.

My parity needs correcting?

I haven't had an unclean shutdown.  Not that I know of.  The thing is in a datacenter...it should never be shutting down.  Ever.  

Why do you think I posted that snippet from your syslog? You didn't know a parity check was underway?

 

You were in a non-correcting parity check due to unclean shutdown, and it found parity errors, so your parity needs correcting.

 

3 hours ago, testdasi said:

Even if nobody accidentally turns it off, your server (or server host) could have crashed - that counts as an unclean shutdown.

Setup Sylog Server so your syslog can be retained after a crash.

Link to comment
8 hours ago, rmp5s said:

The thing is in a datacenter...it should never be shutting down.  Ever.  

Another snippet, the first few lines of your syslog:

Dec  6 07:54:30 Tower kernel: microcode: microcode updated early to revision 0x714, date = 2018-05-08
Dec  6 07:54:30 Tower kernel: Linux version 4.18.20-unRAID (root@develop64) (gcc version 8.1.1 20180626 (GCC)) #1 SMP Fri Nov 23 11:38:16 PST 2018
Dec  6 07:54:30 Tower kernel: Command line: BOOT_IMAGE=/bzimage initrd=/bzroot

So as you can see, your server had rebooted

Link to comment
On 12/11/2019 at 6:23 AM, trurl said:

Another snippet, the first few lines of your syslog:


Dec  6 07:54:30 Tower kernel: microcode: microcode updated early to revision 0x714, date = 2018-05-08
Dec  6 07:54:30 Tower kernel: Linux version 4.18.20-unRAID (root@develop64) (gcc version 8.1.1 20180626 (GCC)) #1 SMP Fri Nov 23 11:38:16 PST 2018
Dec  6 07:54:30 Tower kernel: Command line: BOOT_IMAGE=/bzimage initrd=/bzroot

So as you can see, your server had rebooted

Hmm...well, wtf...

How is it rebooting but looking exactly as I left it when I log in?  My unRAID server at home just got unplugged on accident and it was completely dead until I went back in and restarted it.  Had to go back in and restart the array and all that.

 

 

Link to comment
1 hour ago, trurl said:

Likely there is a setting in the BIOS that tells it to reboot after a crash or when power returns.

Right, but the array starts back up, all the VMs start back up...all that.

It's just really weird.

Not to mention, why tf is it rebooting at all...

Weird.

Link to comment
On 12/11/2019 at 6:19 AM, trurl said:

Setup Sylog Server so your syslog can be retained after a crash.

Did some poking around and apparently integrated syslog functionality rolled out in version 6.7.0 and I was still on 6.6.6 on that server.  So, I updated it.  

 

No problem, right?

 

WRONG!!  Now I can't log in.

 

*sigh*  lol

 

Nothing can be easy.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.