jbuszkie

Members
  • Posts

    688
  • Joined

  • Last visited

Posts posted by jbuszkie

  1. If you really want good parity check speeds, you simply need to eliminate your "bottleneck drives".  Especially if you have any older drives with densities below 1TB/platter (I suspect some of your 2TB drives may be in that category).

    I"m not too concerned with the parity check speeds...  I just didn't want it to be worse with the shingled drive...

    But..  Can I tell from the smart data what my densities are?  or do I have to look it up online by model number?

     

    Jim

  2. Just to throw my results in.  I just did a pre-clear on a new 8T

     

    ~61 hours to preclear.  I'm currently running a parity sync right now.  It seems to be taking longer than my older 4T drive.  Maybe I'll do a re-sync with the 4T drive for comparison??..

     

    Call it 13.5 hours to get to my old 4T mark where it would have been done.  Now it's just parity drive left as I have no other disks bigger then 4T.

    So I went back and put my 4T back in as parity and rebuilt parity just so I could have a comparison.

    13h  37m  to build parity. 

    Since I didn't have an exact point where I crossed the 4T mark with the 8T parity build, I estimated 13h 30m..  I'll say +/- 15 minuted on that number. 

    So now I can conclude that the 8T Shingled did not take noticeable time longer or shorter.  The preceived slowness WAS all in my head!

    And now back to the 8T parity drive! :)

     

     

  3. In general, the only reason the syslog is rotated (making additional huge syslog files) is because something is seriously wrong, in which case the diagnostics zip would probably be a better choice, and it DOES include all syslog files.  Perhaps if you are modifying Powerdown, execute 'diagnostics' if more than one syslog file exists.

    Considering it could be months between powercycles..  The syslogs could get big.  I know the mover generates a lot of syslog entries..  If you've got a crapload of files that get moved....  But that might be a good idea anyway to do the diagnostic thing..

     

    Jim

  4. This might be a dumb question...  But does this powerdown script handle multiple log files for archiving?

    Doesn't unraid move the syslog file to something like syslog_0 (I can't remember what it actually is) when the file gets to a certain size?

    does the power down script attempt to copy those files too to the log directory on the flash drive? I was looking at the code and it doesn't not appear to.. unless I'm missing something...

     

    The only log archived on the flash drive is the latest.

    hmm..  Maybe I'll have to try and hack it so it will save all of the syslogs..  Thanks for the confirmation.

     

    Jim

  5. ... the speed slows down every hour until it gets rid of a drive..  Then it briefly speeds up then slows down until the next drive(s) finish..

     

    This is, of course, normal.    As a drive moves to the inner cylinders, the transfer rate drops significantly.  Since a parity check is always limited by the slowest drive current active in the check, when you have a mixed set of drives, each different size will result in another "inner cylinder slowdown".    So, in your case, it will slow down as it gets close to the 2TB point;  then speed up somewhat until the 3TB drives reach their inner cylinders; and then slow down again as the 4TB drives reach their inner cylinders; and finally slow down for the last time as the 8TB drive reaches its inner cylinders.    As it passes each of those boundaries, the speed will bump up a bit.

    Yeah..  I'm aware of the inner cylinder slowness..  I should have put an "as expected" at the end of my last post! :)  I'm not quite ready to get rid of my 2T drives yet.  I'm eventually replacing them.  Once they hit the 5 year mark they are put next on the list to be replaced.

     

    Jim

  6. and if you have any other 4TB drives (do you?) then they will limit the check speed through the 4TB point => so IF you have 4TB drives still in the system it's unlikely the 8TB unit will make ANY difference in the parity check speed through 4TB.   

    I do have some 4T drives.  My whole thought process is that maybe the "shingled" Archive drive might have a negative effect in my case.

    My guess is that it doesn't..  But I'd love to prove myself right or wrong by putting my old parity disk back in! :)

    I can see in my hourly status reports that the speed slows down every hour until it gets rid of a drive..  Then it briefly speeds up then slows down until the next drive(s) finish..  Damn!  I really wish I had an older parity check speed for comparison!

     

  7. It's pretty simple to just not write to the array while you're doing a parity upgrade  :)

    Not really..  I have automatic backups that occur every night.  I'd rather not have to remember to disable them... only to forget to enable them again.

     

    The reason you're referring to (so your old parity is still valid) is so that if a drive during the new sync was to fail you could do a New Config with the old parity drive and still recover the data from the failed drive.  Note that while conceptually this is true, at the moment it wouldn't work, because the "Trust Parity" option invokes an immediate parity check when the array is first started, and this can effectively invalidate your old parity drive because any bad sectors on the failed drive will cause invalid "corrections" on the parity drive if they're in the first part of the drive -- i.e. they may be done before you can CANCEL that check.    Hopefully this will be fixed SOON (I've asked for it several times).

    Yes this is what I'm referring to..    And I do know of the auto party sync when trust parity thing is invoked.  But isn't the old way still valid with the setinvalid slot thing? Couldn't that still be used to recover the failed disk?

     

    Jim

  8. This might be a dumb question...  But does this powerdown script handle multiple log files for archiving?

    Doesn't unraid move the syslog file to something like syslog_0 (I can't remember what it actually is) when the file gets to a certain size?

    does the power down script attempt to copy those files too to the log directory on the flash drive? I was looking at the code and it doesn't not appear to.. unless I'm missing something...

  9. Related to this would be an option in the New Config to leave all current assignments in place (as though one had done a New Config and then assigned all drives as they were before the New Config) so that one can now make any desired changes.  As well as being a convenience, this would also dramatically reduce the chance of anyone accidentally assigning the drives incorrectly when doing the New Config.  I even think it should be the default behaviour with a checkbox added labelled something like "Clear all current assignments"

     

    Something I grumble about every time I hit that "New Config" button.

    Me too! +1 to that as well...

    But I agree that changing the default behavior needs to be don't do a correcting parity check is the first priority.

  10. It takes longer to do a parity check because before it was only checking up to 4TB, and now it's checking up to 8TB. Even if you have no >4TB array drives it will still take longer. I have a 6TB parity and my largest array drive is 4TB and it still checks those extra 2TB on the parity drive and takes forever.

    Yes..  I'm aware of that..  but I'm comparing to when it gets to the 4T point where my old parity sync would have finished.
  11.  

    Was your old 4TB parity drive a 7200rpm drive?

     

    ... and how long did it take with the old drive to do a parity check?    [Note that it's not unusual for a sync to take a bit longer than a check ... so it'll be a better comparison to do a check after the sync has completed and note when it gets to the 4TB mark then.  You SHOULD do a check anyway, to confirm the sync was good.]

    I think it was a 5900 drive. "ST4000VN000".  I'll have to check to see if I have any data from a full parity check.  I'm pretty sure I don't have any since we the nr_requests thing was discovered..

    I'll dig through my posts and logs to see.  Maybe the last monthly parity check will be in logs somewhere..

     

     

  12. I'd like to see an easy way to make the array read only..  Solely for the times when I'm upgrading parity.  Right now, I believe the procedure is to

    not write to the array so your old parity is still valid.  I did this by going into maintenance mode.  I'd love an option to make the array RO so files can still be read...  but no write will occur...  One step further would be to allow new files to be written to the cache drive.. but not moved to the array until you made the array normal RW again.

     

    Is there a way to do this already?

     

    And sorry if this has already been suggested..

     

    Jim

  13.  

    I doubt that the 8TB Seagate is slower than the old 4TB disk, they are some of the fastest disks I used for parity check/sync, what’s probably limiting your speed are the older disks or a controller bottleneck, this was my parity sync, and this server still has one 3TB WD green, with all 8TB Seagates it would be slightly faster.

     

    Subject: Notice [TOWER7] - Parity sync: finished (0 errors)
    Description: Duration: 15 hours, 47 minutes, 30 seconds. Average speed: 140.7 MB/sec

    I mean this in the nicest possible way..  but you suck! :D  I wish I had those speeds! lol

     

    I'm really tempted to go back to the 4T parity drive just to see how long that would take..  then it would be an apples to apples comparison. But I don't know that I want to put that extra stress on my system and have my array be unavailable for another day! :(  But I am dying to know!

    I don't know what to do! *pout*

     

    I do have some older 2T and 3T disks in my system and I only have the older saslp controller (3Gb/s only I believe or was it 1.5Gb/s) 

  14. Just to throw my results in.  I just did a pre-clear on a new 8T

     

    ~61 hours to preclear.  I'm currently running a parity sync right now.  It seems to be taking longer than my older 4T drive.  Maybe I'll do a re-sync with the 4T drive for comparison??..

     

    Call it 13.5 hours to get to my old 4T mark where it would have been done.  Now it's just parity drive left as I have no other disks bigger then 4T.

     

     

    ========================================================================1.16
    == invoked as: /boot/config/plugins/preclear.disk/preclear_disk.sh -M 4 -o 2 -c 1 -f -J /dev/sdg
    == ST8000AS0002-1NA17Z   Z840AB6F
    == Disk /dev/sdg has been successfully precleared
    == with a starting sector of 1 
    == Ran 1 cycle
    ==
    == Using :Read block size = 1000448 Bytes
    == Last Cycle`s Pre Read Time  : 19:16:58 (115 MB/s)
    == Last Cycle`s Zeroing time   : 18:31:56 (119 MB/s)
    == Last Cycle`s Post Read Time : 23:09:17 (95 MB/s)
    == Last Cycle`s Total Time     : 60:59:48
    ==
    == Total Elapsed Time 60:59:48