Do the smart log checks, make sure you are working with drives you can trust.

 

I'll do that next, if parity fails now that I've moved them off the motherboard.  It's gonna take a couple hours for the first parity check... then I'll have to put some data on the array, and run another check...

 

So, here's hoping!!

 

Robbie

You are missing an important aspect. All the testing of parity may for naught if the smart log report shows many reallocated or pending sectors.

 

The grabbing of the smartlogs is only minutes vs the hours of testing being executed.

Okay cool.  Can I run that test WHILE parity is being checked?

Gosh, sorry for the n00b questions  LOL.

 

I didn't even think that through...

 

LOL.

 

So; next up.. I'm a debian guy, so I'm pretty used to just apt-get'ing anything I want.  In unRAID/slack, is there a repo app I can install smartctl?  Or is there a site I should go to to get the file, or how do I go about installing that?

PS - It's early, and I have not had my first coffee... so I'll blame my brain on that    haha!

So; next up.. I'm a debian guy, so I'm pretty used to just apt-get'ing anything I want.  In unRAID/slack, is there a repo app I can install smartctl?  Or is there a site I should go to to get the file, or how do I go about installing that?

 

See these posts.

 

http://lime-technology.com/forum/index.php?topic=2817.msg23548#msg23548

http://lime-technology.com/wiki/index.php/Troubleshooting#Obtaining_a_SMART_report

 

you can grab the smart logs while the parity test is in progress.

 

Wait until the drive is idle before doing a short (2 minutes) or long (200 minutes or so) test.

 

Okay cool.  Can I run that test WHILE parity is being checked?

Yes, you can run smartctl commands while parity is being checked.

 

We don't have apt-get, but unRAID has wget, (and it was just added in unRAID 4.4.2) so you can run

mkdir /boot/packages

cd /boot/packages

wget  http://slackware.cs.utah.edu/pub/slackware/slackware-12.0/slackware/a/cxxlibs-6.0.8-i486-4.tgz

installpkg cxxlibs-6.0.8-i486-4.tgz

smartctl -a -d ata /dev/sda >/boot/smart_sda.txt

smartctl -a -d ata /dev/sdb >/boot/smart_sdb.txt

smartctl -a -d ata /dev/sdc >/boot/smart_sdc.txt

smartctl -a -d ata /dev/sdd >/boot/smart_sdd.txt

 

You can add a line at the end of the config/go script to re-run installpkg command like this:

test ! -f /usr/lib/stdc++.so.6 && installpkg /boot/packages/cxxlibs-6.0.8-i486-4.tgz

 

The "test" will ensure you will not overwrite an existing, newer library with an older one as you upgrade unRAID in the future.

unRAID has wget, so you can run

 

Yep; that's how I did it... wget the file to my /boot folder and installed it; thanks!  Helped to have a url for the file.

 

I whipped out http://www.category5.tv/files/bash/smartlog to make it sexy for my system (so the logs are a bit more verbose by including the friendly unRAID drive assignment).

 

My log is attached.  The cache drive is not in use yet... so you can ignore the fact that SMART is disabled; I simply installed that drive physically (to save me time down the road), but as per your recommendation, I have not actually made it part of my array.

 

Robbie

The drives look good. No significant errors/reallocations.

When the machine is going to be idle run a short test... then after 10 minutes run a long test and leave the machine alone for about 2 hours.

You may need to stop the array before doing the long test so do it over night.

unRAID has wget, so you can run

 

Yep; that's how I did it... wget the file to my /boot folder and installed it; thanks!  Helped to have a url for the file.

 

I whipped out http://www.category5.tv/files/bash/smartlog to make it sexy for my system (so the logs are a bit more verbose by including the friendly unRAID drive assignment).

 

My log is attached.  The cache drive is not in use yet... so you can ignore the fact that SMART is disabled; I simply installed that drive physically (to save me time down the road), but as per your recommendation, I have not actually made it part of my array.

 

Robbie

You should create a "/boot/packages" folder to be consistent with the add-on packages standard that has been developed by us as users of unRAID and put the downloaded packages there to  be consistent with the rest of us. 

 

We actually have a "package manager" that can assist in much of this that I developed a year or so ago, but first get your array stable as I'm just about ready to release my next version of it with all the updates that have occurred in the past year... so hold off a day or two before installing it.  (it is named "unMENU" ... it is a complete web-server/plug-in system written in "awk".  One of the plug-ins is a package manager, using a "wget" equivalent written again in "awk".  unRAID did not have a web-server we could hook into, so I wrote one using what was available in unRAID to tie it all together.)

 

Joe L.

Awesome, Joe.  I can't wait to check it out.

 

Yes, I wget'd the package into the /boot/packages folder (as I saw that was where you were storing things, I wanted to be consistent).

 

Thanks!!  Can't wait to see consistently good parity so I can start using this puppy (and not fearing for the 300GB of data that is already on it).

 

Robbie

Awesome, Joe.  I can't wait to check it out.

 

Yes, I wget'd the package into the /boot/packages folder (as I saw that was where you were storing things, I wanted to be consistent).

 

Thanks!!  Can't wait to see consistently good parity so I can start using this puppy (and not fearing for the 300GB of data that is already on it).

 

Robbie

For now, you can use md5sum to check the copy of the files you move to the unRAID server...  At least you will know the copy to the data disk was accurate.

 

Joe L.

Now there's an idea...

 

I'll md5 a big movie file or something before the next transfer... I'll copy it over and run a new md5..

 

Thanks for the idea!

 

Robbie

Another 2 hours remaining to my parity check now that I'm on the PCI controllers.  No errors, but have not written to the array since last check.

 

But, I was encouraged that having copied a 500 MB video file to the array and md5'ing it, the strings matched.  So, I'm hopeful.

 

Thanks,

Robbie

Now that you are on the PCI card controllers with all your disks, what is the "Parity Check" speed looking like?

 

It is starting to look encouraging with matching md5 checksums...  It gives me more of a "warm-and-cuddly" feeling about your data's safety.

 

Each time the "parity check" process detects a difference, it corrects parity based on the data disks. (If the data disks have not failed, they are considered to be the "accurate" values to base the parity block.)  By the time you are aware of the parity sync errors, they have been corrected and the parity disk written with the updated values.

 

In the most recent two beta versions a "Parity Verify - but do not correct" feature was added to the underlying "md" device, but no external button on the web-interface exists to invoke it.  For now, it is a "command line" thing...    I know you do not like "beta" versions for your production data... but some day you might be able to use that feature to do some of these tests without modification of any disk, and re-test as needed without having to write new files.

 

Joe L.

44,660  KB/sec ... I had not monitored it previously, so I'm not sure if that's better or worse.

 

It seems to be taking longer than last time... but that could just be that I'm sitting here watching it  LOL

 

I will look forward to when the newest version of unRAID comes out of beta.  That'll be great!  For me, for now, all features are new features... so I'm perfectly happy on 4.4.2.  I'm sure once I know it inside-out, I'll want more features 

 

My parity check just started reporting errors; up to 5 now with 2 hours left to the test.  That could simply be because the parity was created on the old controller previously... but we'll know after I run it a second time after writing some data on here.

 

Robbie

The sync seems to only be accessing the parity drive at the moment, and speeds are at 81,062 KB/sec.

44,660  KB/sec ... I had not monitored it previously, so I'm not sure if that's better or worse.

 

It seems to be taking longer than last time... but that could just be that I'm sitting here watching it  LOL

 

I will look forward to when the newest version of unRAID comes out of beta.  That'll be great!  For me, for now, all features are new features... so I'm perfectly happy on 4.4.2.  I'm sure once I know it inside-out, I'll want more features 

 

My parity check just started reporting errors; up to 5 now with 2 hours left to the test.  That could simply be because the parity was created on the old controller previously... but we'll know after I run it a second time after writing some data on here.

 

Robbie

That is not a bad speed.   It will speed up as you get past the size of the smaller drives, since it does not have to read them to know they are "zero" past their native capacity.

 

As far as all the features being new...  it is OK... we know you can handle some of the more technical tasks...  all you need to remember is unRAID is built on a stripped down version of Slackware...

 

On another topic... what brand UPS do you typically use?

 

Joe L.

The sync seems to only be accessing the parity drive at the moment, and speeds are at 81,062 KB/sec.

The parity check is past the size of the data drives... now it is just making sure the balance of the parity drive is set to zero (for even parity).  The speed increase is quite normal... Odds are your 2 hours estimate to complete is now a bit less.

As far as all the features being new...  it is OK... we know you can handle some of the more technical tasks...  all you need to remember is unRAID is built on a stripped down version of Slackware...

 

Yep.  Love that fact.  Now, to inevitably setup SSH so I can get rid of telnet on here  

 

On another topic... what brand UPS do you typically use?

 

APC if I have a choice.  I also have a couple Ultra (off-brand, cheapo) and one Compaq server UPS.

 

Most of my UPS's are 1000 VA.  One of the Ultra units is 1500 VA, and a couple of the smaller APC units are 650 VA or thereabouts (for network switches, my audio rack, etc.)  But the PCs are all on 1000 VA units, one per PC.

 

Robbie

Odds are your 2 hours estimate to complete is now a bit less.

 

Yep; it jumped from a 134 minute estimate to 70 minutes in only 2 seconds    Now, about 10 minuets later, it says 54 minutes remaining.

 

Almost as accurate as a Windows file transfer 

 

Haha!

Our little secret; the key to how I'm able to get my wife to let me sit down here at the server on a Saturday:  "If I don't resolve the parity issues, we'll lose all our family photos".

 

And so, here I sit. 

As far as all the features being new...  it is OK... we know you can handle some of the more technical tasks...  all you need to remember is unRAID is built on a stripped down version of Slackware...

 

Yep.  Love that fact.  Now, to inevitably setup SSH so I can get rid of telnet on here  

 

On another topic... what brand UPS do you typically use?

 

APC if I have a choice.  I also have a couple Ultra (off-brand, cheapo) and one Compaq server UPS.

 

Most of my UPS's are 1000 VA.  One of the Ultra units is 1500 VA, and a couple of the smaller APC units are 650 VA or thereabouts (for network switches, my audio rack, etc.)  But the PCs are all on 1000 VA units, one per PC.

 

Robbie

Are your UPS's older style with serial connections to the PC, or newer style with USB connectivity? (for communications with the PC for controlled shutdown)

 

Mine is an APC unit with USB connectivity... works really well with unRAID.   (We've already figured out the hard stuff, so don't waste time re-engineering how to tie it in for a smooth shutdown in an extended outage... we already have the packages)

 

Yeah... the "estimate" is weighted average... takes a bit of time to catch up...   You should see it when all the disks are spun down, and you click the "Check" button.  When it averages zero bytes processed in the first 30 seconds or so, as it spins up all the drives in my array, the estimate is many many thousands of minutes remaining...

 

Joe L.

Our little secret; the key to how I'm able to get my wife to let me sit down here at the server on a Saturday:  "If I don't resolve the parity issues, we'll lose all our family photos".

 

And so, here I sit. 

I'm about ready to get busy on some errands and shopping with my wife...  Our photos are safe....

 

With any luck, now that you are not using the MB disk controllers, you'll be fine.  Those early nforce boards had the same issues when you were using them with windows... you just blamed Microsoft when the software crashed.... (and the guilty MB was never blamed)

 

Joe L.

My UPS's are USB.  To be honest, I've never really used the feature at home (I do at work, of course)... but now that I have an unRAID server, I will... to prevent a dirty shutdown (which has never happened here, as I'll just run down and safe shutdown everything if the power is out for more than a few minutes).

 

With any luck, now that you are not using the MB disk controllers, you'll be fine.

 

I'm extremely hopeful.  The intention with this build was to be able to use old junk hardware that was sitting in the storage room taking up space... so I really want this system to work.

 

Those early nforce boards had the same issues when you were using them with windows... you just blamed Microsoft when the software crashed.... (and the guilty MB was never blamed)

 

Unfortunately I cannot use that excuse, as I don't have Windows on my computers    Linux all the way.  All writing so far has been via smb:// under Ubuntu or directly cp'ing the files under a telnet connection.

 

HOWEVER... I'm sure I could think of some way to blame Microsoft... there's always a way.