jbuszkie
-
Posts
688 -
Joined
-
Last visited
Content Type
Profiles
Forums
Downloads
Store
Gallery
Bug Reports
Documentation
Landing
Posts posted by jbuszkie
-
-
I"m not too concerned with the parity check speeds... I just didn't want it to be worse with the shingled drive...If you really want good parity check speeds, you simply need to eliminate your "bottleneck drives". Especially if you have any older drives with densities below 1TB/platter (I suspect some of your 2TB drives may be in that category).
But.. Can I tell from the smart data what my densities are? or do I have to look it up online by model number?
Jim
-
I'd love to have a pair of those for my cache drives! :-)
-
I do! It was bugging me!But I suspect you feel better having physically confirmed it
-
Just to throw my results in. I just did a pre-clear on a new 8T
~61 hours to preclear. I'm currently running a parity sync right now. It seems to be taking longer than my older 4T drive. Maybe I'll do a re-sync with the 4T drive for comparison??..
Call it 13.5 hours to get to my old 4T mark where it would have been done. Now it's just parity drive left as I have no other disks bigger then 4T.
So I went back and put my 4T back in as parity and rebuilt parity just so I could have a comparison.
13h 37m to build parity.
Since I didn't have an exact point where I crossed the 4T mark with the 8T parity build, I estimated 13h 30m.. I'll say +/- 15 minuted on that number.
So now I can conclude that the 8T Shingled did not take noticeable time longer or shorter. The preceived slowness WAS all in my head!
And now back to the 8T parity drive!
-
Considering it could be months between powercycles.. The syslogs could get big. I know the mover generates a lot of syslog entries.. If you've got a crapload of files that get moved.... But that might be a good idea anyway to do the diagnostic thing..In general, the only reason the syslog is rotated (making additional huge syslog files) is because something is seriously wrong, in which case the diagnostics zip would probably be a better choice, and it DOES include all syslog files. Perhaps if you are modifying Powerdown, execute 'diagnostics' if more than one syslog file exists.
Jim
-
hmm.. Maybe I'll have to try and hack it so it will save all of the syslogs.. Thanks for the confirmation.This might be a dumb question... But does this powerdown script handle multiple log files for archiving?
Doesn't unraid move the syslog file to something like syslog_0 (I can't remember what it actually is) when the file gets to a certain size?
does the power down script attempt to copy those files too to the log directory on the flash drive? I was looking at the code and it doesn't not appear to.. unless I'm missing something...
The only log archived on the flash drive is the latest.
Jim
-
Will that cover all open pages on all machines? I assume yes...I have done the nr_requests.. but how do you disable the gui updates? (I may have it open a on couple machines..)
Jim
Settings - Display Settings - Page Update Frequency (set to Disabled)
Jim
-
Yeah.. I'm aware of the inner cylinder slowness.. I should have put an "as expected" at the end of my last post! I'm not quite ready to get rid of my 2T drives yet. I'm eventually replacing them. Once they hit the 5 year mark they are put next on the list to be replaced.... the speed slows down every hour until it gets rid of a drive.. Then it briefly speeds up then slows down until the next drive(s) finish..
This is, of course, normal. As a drive moves to the inner cylinders, the transfer rate drops significantly. Since a parity check is always limited by the slowest drive current active in the check, when you have a mixed set of drives, each different size will result in another "inner cylinder slowdown". So, in your case, it will slow down as it gets close to the 2TB point; then speed up somewhat until the 3TB drives reach their inner cylinders; and then slow down again as the 4TB drives reach their inner cylinders; and finally slow down for the last time as the 8TB drive reaches its inner cylinders. As it passes each of those boundaries, the speed will bump up a bit.
Jim
-
Ah... I didn't save the old config.. I guess That is necessary to this. I'll have to remember that if I want to put my 4T back a parity to test the speed and be covered...
-
I have done the nr_requests.. but how do you disable the gui updates? (I may have it open a on couple machines..)
Jim
-
What OS is giving you that speed graph during the copy? Is that win10?As for normal usage write speed, I’ve found it to be between 45 and 60MB/s for large files, and as expect slower than normal drives for smaller files, see image 2, as these are a very small percentage of my files I’m very happy with the performance.
-
I do have some 4T drives. My whole thought process is that maybe the "shingled" Archive drive might have a negative effect in my case.and if you have any other 4TB drives (do you?) then they will limit the check speed through the 4TB point => so IF you have 4TB drives still in the system it's unlikely the 8TB unit will make ANY difference in the parity check speed through 4TB.
My guess is that it doesn't.. But I'd love to prove myself right or wrong by putting my old parity disk back in!
I can see in my hourly status reports that the speed slows down every hour until it gets rid of a drive.. Then it briefly speeds up then slows down until the next drive(s) finish.. Damn! I really wish I had an older parity check speed for comparison!
-
It's pretty simple to just not write to the array while you're doing a parity upgrade
Not really.. I have automatic backups that occur every night. I'd rather not have to remember to disable them... only to forget to enable them again.
The reason you're referring to (so your old parity is still valid) is so that if a drive during the new sync was to fail you could do a New Config with the old parity drive and still recover the data from the failed drive. Note that while conceptually this is true, at the moment it wouldn't work, because the "Trust Parity" option invokes an immediate parity check when the array is first started, and this can effectively invalidate your old parity drive because any bad sectors on the failed drive will cause invalid "corrections" on the parity drive if they're in the first part of the drive -- i.e. they may be done before you can CANCEL that check. Hopefully this will be fixed SOON (I've asked for it several times).
Yes this is what I'm referring to.. And I do know of the auto party sync when trust parity thing is invoked. But isn't the old way still valid with the setinvalid slot thing? Couldn't that still be used to recover the failed disk?
Jim
-
This might be a dumb question... But does this powerdown script handle multiple log files for archiving?
Doesn't unraid move the syslog file to something like syslog_0 (I can't remember what it actually is) when the file gets to a certain size?
does the power down script attempt to copy those files too to the log directory on the flash drive? I was looking at the code and it doesn't not appear to.. unless I'm missing something...
-
Motherboard port.I also noticed you're using the SASLP controller => is the new 8TB Seagate connected to a motherboard port or to that card ??
This perceived slower sync.. May all be in my head. I can't locate any logs that give my any insight to older parity check times.
I need to look at my log retention scheme...
-
Me too! +1 to that as well...Related to this would be an option in the New Config to leave all current assignments in place (as though one had done a New Config and then assigned all drives as they were before the New Config) so that one can now make any desired changes. As well as being a convenience, this would also dramatically reduce the chance of anyone accidentally assigning the drives incorrectly when doing the New Config. I even think it should be the default behaviour with a checkbox added labelled something like "Clear all current assignments"Something I grumble about every time I hit that "New Config" button.
But I agree that changing the default behavior needs to be don't do a correcting parity check is the first priority.
-
Yes.. I'm aware of that.. but I'm comparing to when it gets to the 4T point where my old parity sync would have finished.It takes longer to do a parity check because before it was only checking up to 4TB, and now it's checking up to 8TB. Even if you have no >4TB array drives it will still take longer. I have a 6TB parity and my largest array drive is 4TB and it still checks those extra 2TB on the parity drive and takes forever.
-
Was your old 4TB parity drive a 7200rpm drive?
... and how long did it take with the old drive to do a parity check? [Note that it's not unusual for a sync to take a bit longer than a check ... so it'll be a better comparison to do a check after the sync has completed and note when it gets to the 4TB mark then. You SHOULD do a check anyway, to confirm the sync was good.]
I think it was a 5900 drive. "ST4000VN000". I'll have to check to see if I have any data from a full parity check. I'm pretty sure I don't have any since we the nr_requests thing was discovered..
I'll dig through my posts and logs to see. Maybe the last monthly parity check will be in logs somewhere..
-
I'd like to see an easy way to make the array read only.. Solely for the times when I'm upgrading parity. Right now, I believe the procedure is to
not write to the array so your old parity is still valid. I did this by going into maintenance mode. I'd love an option to make the array RO so files can still be read... but no write will occur... One step further would be to allow new files to be written to the cache drive.. but not moved to the array until you made the array normal RW again.
Is there a way to do this already?
And sorry if this has already been suggested..
Jim
-
I mean this in the nicest possible way.. but you suck! I wish I had those speeds! lolI doubt that the 8TB Seagate is slower than the old 4TB disk, they are some of the fastest disks I used for parity check/sync, what’s probably limiting your speed are the older disks or a controller bottleneck, this was my parity sync, and this server still has one 3TB WD green, with all 8TB Seagates it would be slightly faster.
Subject: Notice [TOWER7] - Parity sync: finished (0 errors) Description: Duration: 15 hours, 47 minutes, 30 seconds. Average speed: 140.7 MB/sec
I'm really tempted to go back to the 4T parity drive just to see how long that would take.. then it would be an apples to apples comparison. But I don't know that I want to put that extra stress on my system and have my array be unavailable for another day! But I am dying to know!
I don't know what to do! *pout*
I do have some older 2T and 3T disks in my system and I only have the older saslp controller (3Gb/s only I believe or was it 1.5Gb/s)
-
Just to throw my results in. I just did a pre-clear on a new 8T
~61 hours to preclear. I'm currently running a parity sync right now. It seems to be taking longer than my older 4T drive. Maybe I'll do a re-sync with the 4T drive for comparison??..
Call it 13.5 hours to get to my old 4T mark where it would have been done. Now it's just parity drive left as I have no other disks bigger then 4T.
========================================================================1.16 == invoked as: /boot/config/plugins/preclear.disk/preclear_disk.sh -M 4 -o 2 -c 1 -f -J /dev/sdg == ST8000AS0002-1NA17Z Z840AB6F == Disk /dev/sdg has been successfully precleared == with a starting sector of 1 == Ran 1 cycle == == Using :Read block size = 1000448 Bytes == Last Cycle`s Pre Read Time : 19:16:58 (115 MB/s) == Last Cycle`s Zeroing time : 18:31:56 (119 MB/s) == Last Cycle`s Post Read Time : 23:09:17 (95 MB/s) == Last Cycle`s Total Time : 60:59:48 == == Total Elapsed Time 60:59:48
-
Ok.. Pre-clear took 60 hours! ouch!
The parity synch seems to be taking longer too. Maybe once this is done, I'll resynch my old parity drive for a time comparison.
It's almost at the 4T mark and it's about 14 hours into it.. I swore my old numbers were about 12 hours.. hmmm....
Jim
-
There's a faster pre-clear script floating around here that increases the post-read speed, but I haven't used it yet.
It works well!
-
This pre-clear is going to take a while..
137MB/s, 132MB/s, and 125MB/s for the 25% 50% and 75% points of the pre-read. It seems like it's going to be like 16-18 hours just for the pre-read!
I am going to use this archive drive as a parity drive justified by the link in the post above.
Jim
Seagate 8TB Shingled Drives in UnRAID
in Storage Devices and Controllers
Posted
To answer my own post I found this..
http://rml527.blogspot.com/2010/09/hdd-platter-capacity-database.html
Now there may be something better.. but this had the info..