unRAID Server Release 5.0-rc11 Available

[jowi]

I give up. I'm back on rc5, started a parcheck, and i got even worse speeds on rc5 then i ever had on rc11... rc5 is now starting the check at 28MB/s... wtf

[Helmonder]

Well... have you made sure to remove everything in the packages directory so you are running pure vanilla stock ?  If the unraid distro is not the difference then a plugin might be..

[jowi]

I dont want to re-install everything i so carefully built, so i just disable the plugins (sab/sick/dropbox/apcups/slimserver)

Anyway, i've ordered an IBM M1015 card, to find out if the SAS2LP is the problem.

[Helmonder]

I understand... However the way unraid is setup it is real easy to make a copy of your flashdrive, then throw everything out of the packages directory, remove plugins and reboot..

 

check it out.

 

Then copy your backup back to the flashdrive and reboot to in the same mode as before.

[jowi]

Ok, i made a backup and started rc11 from scratch. Parcheck was slower then under rc5, but with a pretty steady 95MB/s acceptable.

After that i added my plugins one by one, and tested the parcheck speed everytime. The speed kept at an acceptable 90-95MB/s.

 

i am now running rc11 with sickbeard, sabnzbd, dropbox, slimserver, apcupsd and subversion.

 

So, i am now running a 'fresh' rc11 install, with reasonable parcheck speeds. Its not what i was used to on rc5, but it's acceptable. Let's see how long this lasts. I have not yet installed simplefeatures, will do that as well.

[jowi]

... and after a reboot the bloody thing is back doing parchecks at 20MB/s. Nice

 

[mr-hexen]

Ok, i made a backup and started rc11 from scratch. Parcheck was slower then under rc5, but with a pretty steady 95MB/s acceptable.

After that i added my plugins one by one, and tested the parcheck speed everytime. The speed kept at an acceptable 90-95MB/s.

 

i am now running rc11 with sickbeard, sabnzbd, dropbox, slimserver, apcupsd and subversion.

 

So, i am now running a 'fresh' rc11 install, with reasonable parcheck speeds. Its not what i was used to on rc5, but it's acceptable. Let's see how long this lasts. I have not yet installed simplefeatures, will do that as well.

 

did you reboot between adding each plugin one by one, or did you add a plugin, parcheck, add another, parcheck. . . .

 

 

id suggest adding one at a time and rebooting between (since this is what caused your sys to parcheck slow again).

[Frank1940]

... and after a reboot the bloody thing is back doing parchecks at 20MB/s. Nice

 

My experience indicates that 'a watched pot boils slowly'.  You have to wait an hour or two before things settle down to get an reasonable indication of parity check speeds.  The best test is to walk away and until it finishes.  Constant refreshes of the GUI to observe progress seems to slow things down--- at least on my system. 

 

Total parity check time on mine is about seven hours.  At the beginning, the estimate initially is eight days.  After a few minutes, the estimate has come down to between 12 and 13 hours-- The estimated finish time keeps varying.

[jowi]

No i did not reboot in between each plugin.

 

I did reboot after installing simplefeatures 1.0.11, and after that things went bad again.

After removing simplefeatures, rebooted, speed was back at 90MB/s...

So i installed SF again, thinking i found the cause, rebooted, and now the speed is about 80-90MB/s... a bit slower but not extremely slow

 

All in all, VERY inconsistent and unstable.

 

@frank1940 my experience with rc5 is that it shows a trustworthy average after just a few minutes.

But i will start it and let it run for about an hour or so, to see what it does.

[mr-hexen]

id leave SF out for a while and test periodically. sounds like it may be the cause or at least contributing. it did receive a major update recently so all the bugs might not be worked out.

[jowi]

id leave SF out for a while and test periodically. sounds like it may be the cause or at least contributing. it did receive a major update recently so all the bugs might not be worked out.

My former tests where with SF 1.0.5, so not the recently updated one.

So if its SF related, it is not bound to the latest update.

 

It looks like after each reboot, unraid (or the hardware) is flipping a coin to determine if this time it will work or not...

[madburg]

My measurements (RC5 is faster than RC11 and RC8)...

 

RC11 (linux 3.4.26)

2TB disk

1048576000 bytes (1.0 GB) copied, 31.9798 s, 32.8 MB/s

1048576000 bytes (1.0 GB) copied, 24.0046 s, 43.7 MB/s

1048576000 bytes (1.0 GB) copied, 25.8743 s, 40.5 MB/s

1048576000 bytes (1.0 GB) copied, 23.9629 s, 43.8 MB/s

1048576000 bytes (1.0 GB) copied, 26.0995 s, 40.2 MB/s

Average: 40.2 MB/s

 

3TB disk

1048576000 bytes (1.0 GB) copied, 23.3049 s, 45.0 MB/s

1048576000 bytes (1.0 GB) copied, 21.9795 s, 47.7 MB/s

1048576000 bytes (1.0 GB) copied, 24.5101 s, 42.8 MB/s

1048576000 bytes (1.0 GB) copied, 23.0835 s, 45.4 MB/s

1048576000 bytes (1.0 GB) copied, 24.0225 s, 43.6 MB/s

Average: 44.9 MB/s

 

RC5 (linux 3.0.35)

2TB disk

1048576000 bytes (1.0 GB) copied, 21.7467 s, 48.2 MB/s

1048576000 bytes (1.0 GB) copied, 23.7081 s, 44.2 MB/s

1048576000 bytes (1.0 GB) copied, 21.8871 s, 47.9 MB/s

1048576000 bytes (1.0 GB) copied, 23.6609 s, 44.3 MB/s

1048576000 bytes (1.0 GB) copied, 21.0582 s, 49.8 MB/s

Average: 46.9 MB/s (17% faster)

 

3TB disk

1048576000 bytes (1.0 GB) copied, 19.8749 s, 52.8 MB/s

1048576000 bytes (1.0 GB) copied, 21.2992 s, 49.2 MB/s

1048576000 bytes (1.0 GB) copied, 17.6082 s, 59.6 MB/s

1048576000 bytes (1.0 GB) copied, 20.5398 s, 51.1 MB/s

1048576000 bytes (1.0 GB) copied, 18.7175 s, 56.0 MB/s

Average: 53.7 MB/s (20% faster)

 

RC8 (linux 3.4.11)

2TB disk

1048576000 bytes (1.0 GB) copied, 25.8166 s, 40.6 MB/s

1048576000 bytes (1.0 GB) copied, 23.4188 s, 44.8 MB/s

1048576000 bytes (1.0 GB) copied, 26.2175 s, 40.0 MB/s

1048576000 bytes (1.0 GB) copied, 24.1458 s, 43.4 MB/s

1048576000 bytes (1.0 GB) copied, 26.4477 s, 39.6 MB/s

Average: 41.7 MB/s

 

3TB disk

1048576000 bytes (1.0 GB) copied, 23.7752 s, 44.1 MB/s

1048576000 bytes (1.0 GB) copied, 22.1375 s, 47.4 MB/s

1048576000 bytes (1.0 GB) copied, 23.5523 s, 44.5 MB/s

1048576000 bytes (1.0 GB) copied, 22.8074 s, 46.0 MB/s

1048576000 bytes (1.0 GB) copied, 24.4973 s, 42.8 MB/s

Average: 45.0 MB/s

Was this tested with stock/original disk tunable parameters with each unRaid versions?

 

[jowi]

Ok, i think i've reached some sort of conclusion. After a lot of testing, i think there are 2 'problems' here.

 

First thing is that after a reboot, sometimes the parcheck speed is way to slow (about 18-28MB/s MAX...) This happens on rc11, but also on rc5. Sometimes i have to reboot a couple of times to get it 'right'. I suspect the sas2lp card. I've ordered an IBM 1015 to see if that solves this erratic boot behaviour.

 

Once booted in a 'good mode', it looks like the overall transfer speed of rc5 is about 10-20% faster then rc11. This is confirmed by others in this thread as well. The parcheck speed however, has a bigger difference, at least in my system. I've ran both versions for over 1 hour (stock unraid, no plugins, no sf), checking parcheck speed every 10/15 minutes, and both versions reached their 'constant' speed in about a minute and stayed there, rc5 ran at 120-130MB/s, while rc11 ran at 70-80MB/s. That is about 40% slower.

 

For now, i'm back at rc5. Once i get the M1015 card, i'll do another session like this.

[mr-hexen]

i did a check on rc11 and it started @ 113MB/s and 3 hours later was still high 90's.

 

i cancelled it as it was completely normal.

 

so we have the same MD driver, so its not an "unRAID" problem but likely a driver problem for your specific setup.

[dgaschk]

I've ran both versions for over 1 hour (stock unraid, no plugins, no sf), checking parcheck speed every 10/15 minutes, and both versions reached their 'constant' speed in about a minute and stayed there, rc5 ran at 120-130MB/s, while rc11 ran at 70-80MB/s. That is about 40% slower.

 

When you say you're checking the speed every 10-15 minutes, my guess is you're not calculating the speed yourself, you are just taking readings from the web guis of two different unraid versions. Right? That's like you're reading speeds from two (differently calibrated) tachometers: one (or both of them) may be miscalculating the actual speed. It would be good if you calculate those speeds yourself -- just divide the total number of megabytes processed by the number of seconds it took to process them.  And, give it some time after you start the parity check before you do it.  Say, do it once at the 30 min mark. (Best is, just let the whole thing finish).

 

The only accurate way to determine the parity check speed is to divide the size of the parity disk by the time reported in the syslog at the completion of a check.

[mr-hexen]

i think if you have a 3TB parity drive and only 2TB data drives unRAID will still perform a 3TB parity check. not sure how that last TB would do in terms of speed though.

[jowi]

The only accurate way to determine the parity check speed is to divide the size of the parity disk by the time reported in the syslog at the completion of a check.

So basically what you people are saying is that Tom can build a complex storage OS but cant manage to divide two numbers in the web interface? I trust the interface shows the correct speed.

[dgaschk]

 

The only accurate way to determine the parity check speed is to divide the size of the parity disk by the time reported in the syslog at the completion of a check.

 

 

No.

 

The parity disk can be larger than the data disks.

 

So, no.

 

But is's all still the simple matter of dividing two numbers.

 

Parity is computed over the size of the parity drive. The size of the data drives is not important; although, it will effect the speed. So, yes.

[dgaschk]

This should be easy to test. When does the parity check end? Does it stop after the biggest data disk is read or does the entire parity drive get read?

 

The remainder of the parity drive must be zero in order to allow the addition of cleared (zeroed) disks without effecting parity. The parity check reads the entire parity drive to ensure that parity is correct. It looks for zeros on the remainder of the disk. So yes, the time reported in the log includes the entire parity disk. Divide the size of the parity drive by the time recorded in the syslog to determine the parity check rate.

[itimpi]

Could you please put in your signature how to rename the /boot(flash) directory? Thanks.

Why would you want to do that?  You cannot rename the /boot directory unless you want to really mess things up.  Are you sure you are not getting confused with renaming things inside the /boot (flash) directory that you would do just like any other file or folder.

[Joe L.]

i think if you have a 3TB parity drive and only 2TB data drives unRAID will still perform a 3TB parity check. not sure how that last TB would do in terms of speed though.

The last 1TB would go very fast, as it would only be reading the parity disk and doing xor operations with the zeros simulated from where data disks are smaller than the parity disk.  I'll bet you could see parity check speeds well over 100MB/s at that point.

 

Joe L.

[cyrnel]

i think if you have a 3TB parity drive and only 2TB data drives unRAID will still perform a 3TB parity check. not sure how that last TB would do in terms of speed though.

The last 1TB would go very fast, as it would only be reading the parity disk and doing xor operations with the zeros simulated from where data disks are smaller than the parity disk.  I'll bet you could see parity check speeds well over 100MB/s at that point.

 

Joe L.

 

Yes, it does the whole thing. I just did one of these weirdo upgrades after updating to rc11. This system's new parity drive is a 3TB Seagate ST3000DM001, with the rest a mix of 2TB drives. That last non-existent TB was surprisingly slow. Surprising to my initial hope it might jump to the end. Really, it ran at the raw speed of the drive on a MB SATA port, starting in the 140MB/s range and dropping near 100MB/s toward the end.

[Joe L.]

i think if you have a 3TB parity drive and only 2TB data drives unRAID will still perform a 3TB parity check. not sure how that last TB would do in terms of speed though.

The last 1TB would go very fast, as it would only be reading the parity disk and doing xor operations with the zeros simulated from where data disks are smaller than the parity disk.  I'll bet you could see parity check speeds well over 100MB/s at that point.

 

Joe L.

 

Yes, it does the whole thing. I just did one of these weirdo upgrades after updating to rc11. This system's new parity drive is a 3TB Seagate ST3000DM001, with the rest a mix of 2TB drives. That last non-existent TB was surprisingly slow. Surprising to my initial hope it might jump to the end. Really, it ran at the raw speed of the drive on a MB SATA port, starting in the 140MB/s range and dropping near 100MB/s toward the end.

Well, it HAS to read the parity drive to ensure it has all zeros.  It would be able to read it as pretty must the full raw speed, exactly as you described, as long as your SATA disk controller and PCI/PCIe bus could keep up with that speed.  (Most should unless other activity was concurrently taking place on the server)

 

Joe L.

[garycase]

Remember -- all modern drives use zoned sectoring;  so the "end" of the drive is read much slower than the beginning of the drive as the head moves from the outer cylinders to the inner cylinders.

 

So a parity check will slow down significantly as it gets to the last 25% or so of any drive.  When you have a parity drive larger than any other drives, clearly the computational overhead of the check decreases once the check gets past all the data drives -- but at the same time it's getting towards the inner cylinders of the drive ... the slowest part.

 

If you carefully watch the speed variation as the parity check proceeds, you'll also see this slowdown as each of the data drives get towards the end ... so if you have a 3TB parity with all 2TB data drives you'll see the parity check slow down significantly as it gets to the last 20% or so of the 2TB drives;  then pick up speed;  and then slow down again as it gets towards the end of the 3TB drive.

 

[cyrnel]

Well, it HAS to read the parity drive to ensure it has all zeros.  It would be able to read it as pretty must the full raw speed, exactly as you described, as long as your SATA disk controller and PCI/PCIe bus could keep up with that speed.  (Most should unless other activity was concurrently taking place on the server)

 

Joe L.

 

Of course. I wasn't  questioning anything. I'd noticed the other posts, possibly mirroring my "wouldn't that be nice" optimism that the rebuild would somehow be smart about the last TB and only consider it once a data drive intruded into those sectors.