Okay - so using the Mirror method, I was copying from a (nearly full) 3TB reiserfs drive to a 3TB xfs-enc drive. .... and it ran out of space at the end of the copy (It was copying the last 20gb file). 

 

I have a 4TB drive that I can move this stuff on to. What's the best way to format the 3TB destination drive? Just swap file systems again? Is there an easier/better way? I can skip this drive until I get to the 2tb reiserfs drives. 

53 minutes ago, axeman said:

Okay - so using the Mirror method, I was copying from a (nearly full) 3TB reiserfs drive to a 3TB xfs-enc drive. .... and it ran out of space at the end of the copy (It was copying the last 20gb file). 

 

I have a 4TB drive that I can move this stuff on to. What's the best way to format the 3TB destination drive? Just swap file systems again? Is there an easier/better way? I can skip this drive until I get to the 2tb reiserfs drives. 

Not entirely sure I understand your question or what you propose to do. You talk about a 4TB drive then you ask about formatting a 3TB drive.

 

In any case, the only way to format a drive that is already in the array is to let unRAID format it by changing its filesystem while it is still in the array. Any other method would obviously invalidate parity. 

17 minutes ago, trurl said:

Not entirely sure I understand your question or what you propose to do. You talk about a 4TB drive then you ask about formatting a 3TB drive.

 

In any case, the only way to format a drive that is already in the array is to let unRAID format it by changing its filesystem while it is still in the array. Any other method would obviously invalidate parity. 

 

Thanks - to clarify ...

 

i was copying from Disk 1(3tb ReiserFS) to Disk 14  (3 TB xfs-enc). Disk 14 ran out of space. 

Disk 13 is a 4tb (xfs-enc) . I'd like to completely abort DISK 1 to Disk 14 and just go from Disk 1 to Disk 13. 

 

At the end of that, I'd like to format Disk 13. Then again - what's the point...I guess I should just delete the files there.

 

For some reason after this copy round ended, UnRaid seems to be slow/unresponsive. I'm going to restart the array hopefully that helps

As you've noticed, it's possible that an XFS volume won't be able to hold as much as a Reiser volume of the same size due to file system overhead differences.    Assuming you did a COPY (not a MOVE) and all the data is still on disk #1, you can simply delete all of the data from #14 and then copy everything to disk #13 (which clearly won't run out of space since it's a full TB larger than #1.

 

Not sure why the array is sluggish -- a reboot should indeed resolve that.

 

I'd recommend producing a process listing before the reboot - just to see if some process is consuming lots of CPU or having eaten lots of RAM. If there is an issue, it can only be fixed if people help to collect evidence.

6 hours ago, garycase said:

As you've noticed, it's possible that an XFS volume won't be able to hold as much as a Reiser volume of the same size due to file system overhead differences.    Assuming you did a COPY (not a MOVE) and all the data is still on disk #1, you can simply delete all of the data from #14 and then copy everything to disk #13 (which clearly won't run out of space since it's a full TB larger than #1.

 

Not sure why the array is sluggish -- a reboot should indeed resolve that.

 

 

3 hours ago, pwm said:

I'd recommend producing a process listing before the reboot - just to see if some process is consuming lots of CPU or having eaten lots of RAM. If there is an issue, it can only be fixed if people help to collect evidence.

 

Thanks - sorry I updated (unraid was 6.4.0 to latest ) and rebooted before I saw your post... but here's where its weird - all my drives came back with high CRC errors. So perhaps i've got to go jiggle my SAS expander's cable. 

1 minute ago, axeman said:

 

 

Thanks - sorry I updated (unraid was 6.4.0 to latest ) and rebooted before I saw your post... but here's where its weird - all my drives came back with high CRC errors. So perhaps i've got to go jiggle my SAS expander's cable. 

Lots of transfer errors would produce the effect that the array becomes sluggish.

13 minutes ago, pwm said:

Lots of transfer errors would produce the effect that the array becomes sluggish.

 

 

Yeah, a second reboot cleared all that up - don't know what happened first time i've seen it on all drives. I'm running a Parity check w. corrections fingers crossed. 

17 minutes ago, axeman said:

all my drives came back with high CRC errors.

The CRC attribute monitoring was added on v6.4.1, those errors might be old since CRC attribute doesn't reset, as long as they don't keep increasing you're fine.

1 hour ago, johnnie.black said:

The CRC attribute monitoring was added on v6.4.1, those errors might be old since CRC attribute doesn't reset, as long as they don't keep increasing you're fine.

 

That's a relief! (I think). Hopefully it stays put.

CRC errors aren't all that unusual.   I have one drive with quite a few (number hasn't changed in years); and one other drive with 1 CRC error that has been there since I added the drive a few months ago (a new 4TB WD Red in one of my older arrays).     I don't like the "error indicator" on the Dashboard for these, so I simply went to disk settings and unchecked the "UDMA CRC error rate" attribute so it's not reported   ==>  easy enough to just look at SMART data periodically to see if anything's changed; and if there were real problems you'd almost certainly see other errors anyway (i.e. an uncorrectable error).

 

 

 

5 minutes ago, garycase said:

CRC errors aren't all that unusual.   I have one drive with quite a few (number hasn't changed in years); and one other drive with 1 CRC error that has been there since I added the drive a few months ago (a new 4TB WD Red in one of my older arrays).     I don't like the "error indicator" on the Dashboard for these, so I simply went to disk settings and unchecked the "UDMA CRC error rate" attribute so it's not reported   ==>  easy enough to just look at SMART data periodically to see if anything's changed; and if there were real problems you'd almost certainly see other errors anyway (i.e. an uncorrectable error).

 

 

 

 

Yeah - just might do that - after my parity check finishes, if nothing changed. 

So I'm about to add an 8TB drive to my system.  (Parity is already 8TB)

 

I had a 500GB drive I moved all the data to other drives.  I then removed that drive so I could put in the 8TB to do the pre-clear.

 

The parity was valid before I removed the drive.  (Well I think it was, I actually checked the parity before I deleted everything and removed the drive).

 

Before adding the 8TB to the array officially, I think I just need to update parity...  Which should essentially remove the 500GB from the array.

 

I can then format and add (as XFS) the new 8TB?

 

 

1 minute ago, JackBauer said:

So I'm about to add an 8TB drive to my system.  (Parity is already 8TB)

 

I had a 500GB drive I moved all the data to other drives.  I then removed that drive so I could put in the 8TB to do the pre-clear.

 

The parity was valid before I removed the drive.  (Well I think it was, I actually checked the parity before I deleted everything and removed the drive).

 

Before adding the 8TB to the array officially, I think I just need to update parity...  Which should essentially remove the 500GB from the array.

 

I can then format and add (as XFS) the new 8TB?

 

 

 

Have you precleared the 8TB?

 

Just to make sure I understand what you propose. You want to remove a 500GB disk from the array. That will require a New Config and parity resync. Then you want to add an 8TB disk to the array. If it isn't precleared unRAID will clear it for you so parity will remain valid. You can format the new 8TB as XFS.

 

That sounds OK. Another approach would be to rebuild the 500GB to the 8TB then format it.

 

So, it's either rebuild parity and clear/format the new disk or rebuild/upsize the empty data disk and format it. If you haven't precleared already rebuilding/upsizing the empty data disk is probably quicker. And parity would remain valid the whole time.

 

And another possibility would be to just New Config without the 500GB and with the 8TB then rebuild parity and format the 8TB. That is probably the simplest and quickest. Do you have good backups?

My 8TB is preclearing now, should wrap up tonight sometime I think.

 

The 500GB is out of the array, all data was removed and files deleted.  When you say "new config"...  What do you mean exactly?

 

My array is started, although as expected it is currently missing a disk.  Parity is valid, although Unraid says my array is unprotected...  (As it is missing a disk..  disk 5, not installed)

 

Right now it WILL let me do a parity check and write corrections...  Even though I am missing a disk.  So that was what I was thinking I should do.  Update parity, add then add the new 8TB drive.

 

Thanks for the help...

1 hour ago, JackBauer said:

My 8TB is preclearing now, should wrap up tonight sometime I think.

 

The 500GB is out of the array, all data was removed and files deleted.  When you say "new config"...  What do you mean exactly?

 

My array is started, although as expected it is currently missing a disk.  Parity is valid, although Unraid says my array is unprotected...  (As it is missing a disk..  disk 5, not installed)

 

Right now it WILL let me do a parity check and write corrections...  Even though I am missing a disk.  So that was what I was thinking I should do.  Update parity, add then add the new 8TB drive.

 

Thanks for the help...

No, there is no way to do a parity check with a missing disk. I think what it will let you do is called a "read check". It can't check or correct parity with a missing disk.

 

So, that brings us to the 1st scenario, the one I thought you had in mind before I proposed the others.

___________

Scenario 1:

In order to make parity valid without that disk you would have to go to Tools - New Config, set a new disk configuration without the disk, and resync (rebuild) parity.

 

Then after parity is valid (with one less disk in the configuration), you could ADD the precleared disk. When you ADD a disk to an array with valid parity, unRAID will clear it unless it is already clear. This is so parity will remain valid. This is technically the only time a clear disk is required. A clear disk is all zeros so has no effect on parity.

 

After the disk is added then unRAID will let you format it. Format is only writing an empty filesystem (just a top level directory with no folders or files in it). That write of an empty filesystem updates parity just like any other write operation, so formatting a disk that is in the array maintains parity.

 

In the end, Scenario 1 means you have to rebuild parity, clear a disk, and format it.

___________

Scenario 2:

You could replace the missing disk with the new 8TB disk and let it rebuild that missing data disk onto the new disk. Even though the missing disk doesn't have any files on it, it is not a clear disk. It has an empty filesystem, and it really still has a lot of data on it that isn't really part of the filesystem anymore. When you delete files they aren't overwritten with zeros (cleared), they are just marked as deleted in the filesytem. So all of that deleted data is still part of parity. If you rebuild onto the 8TB, you will end up with an upsized disk that has an empty filesystem on it (still the same type of filesystem as before), and parity will remain valid. Then you can change the disk's filesystem, formatting it. And as already mentioned, format is just a write operation and that will update parity so it is maintained.

 

In the end, Scenario 2 means you rebuild/upsize the missing data disk, then format it. Doesn't really matter if it was clear or not since it is completely overwritten during the rebuild.

___________

Scenario 3:

You could set a New Configuration with the 8TB disk in place of the missing disk, and resync (rebuild) parity. Then you can format the new disk.

 

In the end, Scenario 3 means you rebuild parity with a new larger disk in the array and format it. Doesn't really matter if it was clear or not since parity is going to be completely overwritten to conform to what was on all the disks anyway.

___________

Any of these ways will wind up at the same place, with valid parity and 8TB XFS disk instead of the old 500GB disk. Note however that Scenario 2 doesn't require rebuilding parity. That means that at any time, if there was a problem, you could put the original 500GB disk back in (New Config) and tell it parity is already valid. Because it would be. And if the problem was on another disk, you could rebuild that other disk from parity plus the original 500GB disk and all other disks.

 

And Scenario 2 is very similar to what the rest of this thread is about. You empty a disk so you can format it to XFS. Then repeat with your other disks. You don't have to rebuild parity at any time. The only difference in your case is you upsize the disk in the usual way before changing its filesystem.

 

I know this is kind of long, as is the rest of this thread, and there are links to procedures to follow that are kind of long. But if you understand how unRAID uses parity, all of these scenarios are sort of obvious.

 

9 hours ago, garycase said:

I don't like the "error indicator" on the Dashboard for these, so I simply went to disk settings and unchecked the "UDMA CRC error rate" attribute so it's not reported

 

9 hours ago, axeman said:

Yeah - just might do that - after my parity check finishes, if nothing changed. 

 

You can just acknowledge the current values on the dashboard page by clicking on the SMART status, this way, and if notifications are enable,  you'll get warned when they change.

5 hours ago, trurl said:

___________

Scenario 1:

In order to make parity valid without that disk you would have to go to Tools - New Config, set a new disk configuration without the disk, and resync (rebuild) parity.

 

Then after parity is valid (with one less disk in the configuration), you could ADD the precleared disk. When you ADD a disk to an array with valid parity, unRAID will clear it unless it is already clear. This is so parity will remain valid. This is technically the only time a clear disk is required. A clear disk is all zeros so has no effect on parity.

 

After the disk is added then unRAID will let you format it. Format is only writing an empty filesystem (just a top level directory with no folders or files in it). That write of an empty filesystem updates parity just like any other write operation, so formatting a disk that is in the array maintains parity.

 

In the end, Scenario 1 means you have to rebuild parity, clear a disk, and format it.

 

 

Ok so when I go to Tools:New Config...  There are four options under Retain Configuration.  I'm guessing I check parity and data (I have no cache)  Although I'm not sure as one of my data drives is missing, I'm not sure what the next step is after I click apply.

 

I'm being extra careful here (of course).

 

A quick search did not show me much information (at all) on the 'Retain Configuration" setting.

1 hour ago, JackBauer said:

 

Ok so when I go to Tools:New Config...  There are four options under Retain Configuration.  I'm guessing I check parity and data (I have no cache)  Although I'm not sure as one of my data drives is missing, I'm not sure what the next step is after I click apply.

 

I'm being extra careful here (of course).

 

A quick search did not show me much information (at all) on the 'Retain Configuration" setting.

Whatever you choose, at the next step unRAID will let you change the disk assignments as you choose before it actually does anything. We usually just tell people to choose All and go from there. After choosing All, at that next step, you would unassign the missing disk if it isn't already unassigned. Then, it will let you start the array and rebuild parity. You must NOT check the box that says parity is already valid, since it isn't without that original disk5.

Parity is rebuilding now.  Thanks much for the help!

Now I'm debating stopping the parity build...  As that parity disk is a shingled HD...

 

And swapping in my WD red as the new parity, pushing the shingled to be the replacement data disk instead.

 

Thoughts?

I don't have any experience with those but there are threads about them here.

 

Not a problem to stop and replace parity. You wouldn't even have to New Config. When it sees you have replaced parity it will automatically want to resync.

13 hours ago, johnnie.black said:

 

 

You can just acknowledge the current values on the dashboard page by clicking on the SMART status, this way, and if notifications are enable,  you'll get warned when they change.

 

Thanks Johnnie -- that's nice to know ... and much better than disabling the monitoring of the attribute.   In fact, I had done that quite some time ago for a disk with reallocated sectors, but had forgotten you could do it

14 hours ago, johnnie.black said:

 

 

You can just acknowledge the current values on the dashboard page by clicking on the SMART status, this way, and if notifications are enable,  you'll get warned when they change.

 

Thanks - that's what i'm going to do then!