Upgraded to v6 - now want to add bigger drives

[trurl]

What unRAID will do when you start after a New Config is rebuild parity against any drives you have assigned, regardless of what is on those disks. After the parity rebuild, if any of them need formatting then formatting a drive is just like any other write operation as far as parity goes just like garycase said.

 

The thing that I think confuses people about formatting drives is that is what they always do with a new disk when they put it in their Windows PC, or what they do when they want to get rid of everything on the disk and start over. But formatting is not the same as clearing a drive or deleting all the files on a drive. What formatting really means is creating an empty filesystem. That's what it has always meant on any OS you have ever used, but it was just never spelled out for you. An empty filesystem is not a clear disk because the filesystem itself contains data about the files. If you format a drive that had files on it, those files are not wiped out. You just get an empty filesystem that no longer refers to them. When format creates an empty filesystem, it does this by writing the filesystem data to the disk, which will be treated by unRAID parity just like any other write to a data drive.

[bonienl]

What unRAID will do when you start after a New Config is rebuild parity against any drives you have assigned, regardless of what is on those disks. After the parity rebuild, if any of them need formatting then formatting a drive is just like any other write operation as far as parity goes just like garycase said.

 

The thing that I think confuses people about formatting drives is that is what they always do with a new disk when they put it in their Windows PC, or what they do when they want to get rid of everything on the disk and start over. But formatting is not the same a clearing a drive or deleting all the files on a drive. What formatting really means is creating an empty filesystem. That's what it has always meant on any OS you have ever used, but it was just never spelled out for you. An empty filesystem is not a clear disk because the filesystem itself contains data about the files. If you format a drive that had files on it, those files are not wiped out. You just get an empty filesystem that no longer refers to them. When format creates an empty filesystem, it does this by writing the filesystem data to the disk, which will be treated by unRAID parity just like any other write to a data drive.

 

Very well explained

 

You should make this a sticky, it occurs to me that many new users are confused about this.

 

[walterg74]

Well, the thing is since I know my disks should be ok, I wanted to put things back as they were...

 

Followed the wiki about trusting parity, however when I ran initconfig I did not get the results shown there where it says all disks should habe blue balls ( no pun intended...).

 

Restored the file and O got back to where I was... Did new config and saw it pretty much is doing the same thing....

 

So I went ahead woth new config, added ONLY the previously existing drives (as it says in many places) and selected to start with trust parity.

 

I then launched a parity check ans it is running now. Also just to avoid a y wrotes -just in case- I also selected to NOT write corrections to parity.

 

Thoughts?

[garycase]

Not sure why you didn't include the 3TB drives, but you can indeed simply add them later.

 

IF you indeed had not made ANY writes to any of the drives in the process of trying things earlier, then your parity check should be "clean" (no sync errors) ... and IF that's the case then you'd have an option of rebuilding a drive if it turns out you indeed have had one fail [it sounds more like a poorly seated cable than a failed drive, however].

 

Keep us informed of the status when the parity check completes -- you might also post a picture of the Main page on the Web GUI when all is done.

 

[walterg74]

Not sure why you didn't include the 3TB drives, but you can indeed simply add them later.

 

IF you indeed had not made ANY writes to any of the drives in the process of trying things earlier, then your parity check should be "clean" (no sync errors) ... and IF that's the case then you'd have an option of rebuilding a drive if it turns out you indeed have had one fail [it sounds more like a poorly seated cable than a failed drive, however].

 

Keep us informed of the status when the parity check completes -- you might also post a picture of the Main page on the Web GUI when all is done.

 

The reason is because I read on a few oksts where a user did include new drives, and if the system takes them into account then the oarity will not match, as what's on that disk did nit include them?

 

As soon as I started the check (immediately I would say),  it did list under "sync errors detected" 263.

 

Right now it's at almost 50% and that count is at 267.

 

Will post screenshot.

[walterg74]

Ok, so not good...

 

Parity check finished, said parity was OK  (still had those sync errors I mentioned before), however, disk 4 gave error again  (I had just re-seated everything, didn't really change any cables yet).

 

Here are the screenshots and I also downloaded the syslog.

 

What do you recommend I try next? (For now, haven't even rebooted/stopped/shutdown anything as to preserve what is there, and as you can see it's rebuilding anything on that disk on the fly if I want to read from it).

 

 

[walterg74]

Splitting up for post size limitations...

[walterg74]

Last one plus syslog...

tower-syslog-20160101-1837.zip

[trurl]

Instead of posting just the syslog like on V5, on V6 you should always go to Tools - Diagnostics and post the complete diagnostics zip file. It contains the syslog, SMART of each drive, and a lot of other useful information.

[walterg74]

Instead of posting just the syslog like on V5, on V6 you should always go to Tools - Diagnostics and post the complete diagnostics zip file. It contains the syslog, SMART of each drive, and a lot of other useful information.

 

Thanks for the tip!

 

Here it is.

tower-diagnostics-20160101-2238.zip

[trurl]

Was hoping for SMART on disk4 but looks like it isn't responding. Have you tried checking power and sata connections, changing cables or ports?

[walterg74]

Was hoping for SMART on disk4 but looks like it isn't responding. Have you tried checking power and sata connections, changing cables or ports?

 

Hadn't yet. I will go ahead and change the cable now, however anything to consider on startup, since I will have to power down to do so?

[garycase]

You can try reseating and/or replacing the SATA cable and reseating the power cable for disk #4; but at this point your options are limited.

 

While parity is "good" ... there are also a modest # of sync errors, which, if they are actual parity errors you didn't allow to be corrected, will corrupt a disk #4 rebuild;  on the other hand, if they're reflecting errors in one of your disks (perhaps #4) then you may actually get a good rebuild.    But there's no way to know for sure which of those is the case unless you have either checksums of your files or backups you can compare them to.

 

I'd reseat the cables, then do a New Config with your current config and the "Trust Parity" option and see if you can access disk #4 okay => look at several of the files and see if they look okay and, better yet, copy the entire disk to another location so you have a current backup of its data.    Then I'd do yet-another New Config with ALL of your disks (including the new 3tB ones), and just let it do a new parity sync  [i.e. do NOT use the trust parity box].

 

[walterg74]

You can try reseating and/or replacing the SATA cable and reseating the power cable for disk #4; but at this point your options are limited.

 

While parity is "good" ... there are also a modest # of sync errors, which, if they are actual parity errors you didn't allow to be corrected, will corrupt a disk #4 rebuild;  on the other hand, if they're reflecting errors in one of your disks (perhaps #4) then you may actually get a good rebuild.    But there's no way to know for sure which of those is the case unless you have either checksums of your files or backups you can compare them to.

 

I'd reseat the cables, then do a New Config with your current config and the "Trust Parity" option and see if you can access disk #4 okay => look at several of the files and see if they look okay and, better yet, copy the entire disk to another location so you have a current backup of its data.    Then I'd do yet-another New Config with ALL of your disks (including the new 3tB ones), and just let it do a new parity sync  [i.e. do NOT use the trust parity box].

 

Ok. Just changed for a new SATA cable.

 

Poewered on, did the trust parity and as you recommended I am copying everything from disk 4 to a different location.

 

So far it's reading just fine from \\tower\disk4

 

When finished I will do the second "new config" with ALL disks.

 

It was all at once I could do it right? Assigning one of the 3TB ones as parity, rest of data as data en both the other 3TB and "old" parity as data?  It's smart enough to build parity on the new one and only afterwards I can format the other 2 disks? (old parity + new second 3TB one?)

[garycase]

... It was all at once I could do it right? Assigning one of the 3TB ones as parity, rest of data as data en both the other 3TB and "old" parity as data?  It's smart enough to build parity on the new one and only afterwards I can format the other 2 disks? (old parity + new second 3TB one?)

 

Yes, that's correct.  Waiting for the parity sync to finish before formatting the old parity drive preserves the ability to restore the original configuration "just in case" you were to have a drive fail during the parity sync => not very likely, but it's easy enough to keep the option open.    Once it's finished, formatting the two "unformatted" drives is just a matter of checking a box, clicking on the option, and waiting 2-3 minutes.

 

[walterg74]

... It was all at once I could do it right? Assigning one of the 3TB ones as parity, rest of data as data en both the other 3TB and "old" parity as data?  It's smart enough to build parity on the new one and only afterwards I can format the other 2 disks? (old parity + new second 3TB one?)

 

Yes, that's correct.  Waiting for the parity sync to finish before formatting the old parity drive preserves the ability to restore the original configuration "just in case" you were to have a drive fail during the parity sync => not very likely, but it's easy enough to keep the option open.    Once it's finished, formatting the two "unformatted" drives is just a matter of checking a box, clicking on the option, and waiting 2-3 minutes.

 

Ok... So I managed to copy almost all disk4  (Didn't have free space elsewhere for ALL of it, but what I left without copying was just movies that no big deal if something happened to them...), and thankfully it read the whole ting just fine, no errors, and no using the other drives either (it was the only disk spun up even)

 

I did the new config like you said, and selected the 3TB one connected to a motherboard port as parity, then the second 3TB one as data disk1 (so later when I start replacing disks that one is already the first "big one" -not that it really matters probably?-) and the rest of the disks as data too and in the same order I had them before  (just my OCD?).

 

Like you said it would, I got the 2 drives listed (second 3TB and old parity) showing up as un-mountable due to not being formatted and the option to format them.

 

Parity sync has started, so I will wait until that finishes (says about almost 24 hours...) and then I will do a parity check and let's hope it was in fact just the cable and I don't get any more errors...

 

For the last part after that, while you say it will only take 2-3 minutes after selecting to format those 2 drives (I assume due to them having been pre/cleared already and flagged as such?), wouldn't it have to do a new parity sync, and looking at another 24 hour process..?

 

Thanks!!

 

[trurl]

...For the last part after that, while you say it will only take 2-3 minutes after selecting to format those 2 drives (I assume due to them having been pre/cleared already and flagged as such?), wouldn't it have to do a new parity sync, and looking at another 24 hour process..?

... What formatting really means is creating an empty filesystem. That's what it has always meant on any OS you have ever used... When format creates an empty filesystem, it does this by writing the filesystem data to the disk, which will be treated by unRAID parity just like any other write to a data drive.

 

[walterg74]

...For the last part after that, while you say it will only take 2-3 minutes after selecting to format those 2 drives (I assume due to them having been pre/cleared already and flagged as such?), wouldn't it have to do a new parity sync, and looking at another 24 hour process..?

... What formatting really means is creating an empty filesystem. That's what it has always meant on any OS you have ever used... When format creates an empty filesystem, it does this by writing the filesystem data to the disk, which will be treated by unRAID parity just like any other write to a data drive.

 

Thought so, but.. You know...

 

Now to,wait...

 

Thanks

[garycase]

... For the last part after that, while you say it will only take 2-3 minutes after selecting to format those 2 drives (I assume due to them having been pre/cleared already and flagged as such?), wouldn't it have to do a new parity sync, and looking at another 24 hour process..?

 

As I noted earlier, NO -- it just takes a couple minutes.    Has nothing to do with whether or not they were pre-cleared (that's only a factor if you're adding a drive to a system that already has valid parity).    Once parity has been established (that's what the parity sync is doing now), any writes to ANY drive will automatically be done in a manner that keeps parity up-to-date.    Formatting a drive is simply initializing the file system on that drive -- it's doing very few writes to the drive ... that's why it only takes a couple minutes.

 

In other words, NO, there's no additional long process needed.    As I've said several times in this thread, it will only take 2-3 minutes 

[walterg74]

... For the last part after that, while you say it will only take 2-3 minutes after selecting to format those 2 drives (I assume due to them having been pre/cleared already and flagged as such?), wouldn't it have to do a new parity sync, and looking at another 24 hour process..?

 

As I noted earlier, NO -- it just takes a couple minutes.    Has nothing to do with whether or not they were pre-cleared (that's only a factor if you're adding a drive to a system that already has valid parity).    Once parity has been established (that's what the parity sync is doing now), any writes to ANY drive will automatically be done in a manner that keeps parity up-to-date.    Formatting a drive is simply initializing the file system on that drive -- it's doing very few writes to the drive ... that's why it only takes a couple minutes.

 

In other words, NO, there's no additional long process needed.    As I've said several times in this thread, it will only take 2-3 minutes 

 

Whew... Ok then got it

 

I think I will look into improving the interfaces though... During the parity sync the speed I was getting was around 35MB/sec.... (CPU was not an issue, it was at around 20%). Once it passed the 1TB mark (the size of the "old" data disks) it shot up to 92-95MB/sec....

 

Parity sync is finished, says parity is valid. Would the last steps now be format/parity check or parity check/format?

 

Also, do you think it's worthwhile to do a *long* S.M.A.R.T. test on that disk 4?

[garycase]

It doesn't really matter which order, but you should do both.    Personally, I'd do a parity check, then a format.  [That preserves the old parity drive until you've confirmed the new one is good.]

 

I think your disk #4 issue was the SATA cable, but it certainly wouldn't hurt to run a long SMART test AFTER you've done the rest of the initial configuration [parity check and format].

 

 

[walterg74]

It doesn't really matter which order, but you should do both.    Personally, I'd do a parity check, then a format.  [That preserves the old parity drive until you've confirmed the new one is good.]

 

I think your disk #4 issue was the SATA cable, but it certainly wouldn't hurt to run a long SMART test AFTER you've done the rest of the initial configuration [parity check and format].

 

Ok, running the parity check as I write this...

 

Will definitely look into,the speed issue, as with the check which (for now) is all reads, still gettin only 35MB/sec speeds...

[garycase]

Nothing to "look into" r.e. your speed issue.    It's simple:  You have 4 drives connected to a 4-port PCI card !!

 

When all 4 are active, that means you're sharing 133MB/s bandwidth among 4 drives ... so about 33Mb/s per drive is all that's available.    When you get past the 1TB point in your parity check (or sync; or drive rebuild), the 1TB drives that are connected to that card are no longer involved in the operation -- so the speed jumps significantly to reflect the performance of your other controller(s) and drives.

 

If you replace the PCI card with a PCIe card (preferably with more than 1 lane), you'll have far better bandwidth -- and the parity check speeds will reflect that.

 

Note that in "normal" use the card doesn't have much impact, since you typically only have one drive at a time in use from those connected to the card -- so it has the full 133MB/s bandwidth available to it.

 

[walterg74]

Nothing to "look into" r.e. your speed issue.    It's simple:  You have 4 drives connected to a 4-port PCI card !!

 

When all 4 are active, that means you're sharing 133MB/s bandwidth among 4 drives ... so about 33Mb/s per drive is all that's available.    When you get past the 1TB point in your parity check (or sync; or drive rebuild), the 1TB drives that are connected to that card are no longer involved in the operation -- so the speed jumps significantly to reflect the performance of your other controller(s) and drives.

 

If you replace the PCI card with a PCIe card (preferably with more than 1 lane), you'll have far better bandwidth -- and the parity check speeds will reflect that.

 

Note that in "normal" use the card doesn't have much impact, since you typically only have one drive at a time in use from those connected to the card -- so it has the full 133MB/s bandwidth available to it.

 

Ok... help...

 

While the parity sync finished just fine, the check parity about 1/3 in failed... seems the disk is once again not found...

 

At this point I would then say something is in fact wrong with the disk....

 

What should I do now?

 

Would I need to do a new config with just the original disks again, trust parity (from the old 1TB),  and continue to the "swap/disable" procedure..?

 

[trurl]

Could be a bad drive I guess, but I think it's at least as likely to be a problem with something else. How old is your power supply? Power connections good anywhere there is a connection, not just at the drive? Have you done a memtest? Reseated cables AND controller in its motherboard slot?

 

Post a new diagnostic that includes this latest failure.