Re: Format XFS on replacement drive / Convert from RFS to XFS (discussion only)

[PeteAron]

Now up to 27,000 reallocated sectors....

[Frank1940]

14 minutes ago, kimifelipe said:

Now up to 27,000 reallocated sectors....

I would think you would be better served to start a new thread about your problem.   While you might be using the method suggested in this thread, you issue is not about the actual process but more a hardware failure.  If you have only one disk with a problem, you have not lost any data at this point.  But you do need to seek some advice on how to proceed from here...

[trurl]

23 minutes ago, kimifelipe said:

I have a failing disk - it shows over 1000 reallocated sectors.  So instead of doing the usual rebuild of the disk by replacing it, I was trying this method to copy the files to the new xfs disc.  I had just completed a parity check with no errors, so I thought I would be ok.  I began the disk copy last night following the published procedure. about 2/3 the way through, the copy rate dropped to about 5-30 kb/sec, and I am seeing 36000 or so errors on the unraid display for this copy. 

 

should I abort the copy, remove the failing disk, and rebuild it to a new replacement, and after that work on the xfs conversion?  Or, should I just wait for the copy to complete?

 

if I need to abort, how do I do that? I am using a direct console command to do this.

I understand why you might want to do it this way, but trying to do extensive file operations with an unreliable disk in the array is definitely not recommended. If you were moving instead of copying, or if you were using turbo write, there is a very good chance that you would compromise parity and then rebuilding would be compromised.

 

What console command are you using? Have you tried just aborting it with control-c?

 

[trurl]

2 minutes ago, Frank1940 said:

If you have only one disk with a problem, you have not lost any data at this point. 

Unless he was using turbo write, in which case parity might be invalid since it would have been calculated by reading the bad disk.

[PeteAron]

trurl,

 

I used

rsync -avPX /mnt/disk10/ /mnt/disk11/

 

it looks like the array is copying to the xfs swap disk using the parity protection.  it is nearly complete - 2.54 out of 2.73 TB have been copied, and I am up to 43000 errors on the failing disk.  I have a new disk on the way - after this copy is complete, I will probably use the procedure to remove the failing disk.

[JorgeB]

30 minutes ago, kimifelipe said:

should I abort the copy, remove the failing disk, and rebuild it to a new replacement, and after that work on the xfs conversion?  Or, should I just wait for the copy to complete?

 

You should replace it, that's what parity is for.

[trurl]

Just now, kimifelipe said:

it looks like the array is copying to the xfs swap disk using the parity protection.

All writes to a disk in the parity array also updates the parity disk at the same time. This is a very basic principle of unRAID that you should always be aware of.

[PeteAron]

When a read error occurs, am I writing the bad data from the disk with the read error, or the good data from parity?  If the latter I should be ok.

 

is parity being updated when I get a read error and write the data to a new disk?  If so  then I am already screwed aren't I?

[Frank1940]

I believe it tries the failing disk first and then reconstructs the data from a parity calculation.  That is why the process is so slooowwww.

[trurl]

When you said the disk was failing, did you mean unRAID had disabled the disk (does it have a red X next to it)?

 

Where are you seeing 27000 reallocated? I'm not sure a number that large for that attribute is actually possible.

 

Is the disk with the read error the same disk with the large number of reallocations? Are any read/write errors being displayed for any other disks?

 

 

 

[PeteAron]

@ Frank - yes that is what I thought was happening.

 

@trurl  - The disk in "Dashboard" has a yellow triangle next to it, and the popup shows 3xxxx reallocated sectors.  In "Main" this disk shows 46xxx errors.  No other disks show errors.  the failing disk does not have a red X next to it. 

[PeteAron]

So, if you were in my position, what would you do?  I have about 300 gb left to copy, but it is moving at less than 50 kb/sec, with errors growing.  Would you stop the copy where it is, rebuild the failing disk with a new disk, then do the copy again?  I think if I continue as is it will take another day or so at this speed.  I'm not sure what the best course of action is.

[trurl]

I would

 

3 minutes ago, kimifelipe said:

stop the copy where it is, rebuild the failing disk with a new disk, then do the copy again

 

Any time you have an unreliable disk in the array you are effectively operating without protection, because ALL disks are required to rebuild a disabled or missing disk. Parity alone cannot reconstruct any data.

[Frank1940]

IF that disk were physically offline, the whole process could go much faster but I am unsure how to do that and not screw up something else. You could stop the whole process and physically remove the disk from the array and do the copy again using the virtual disk.  As a point of data security, do you have another disk that you could use to rebuilt the failed disk?  That would allow you to use that XFS formatted disk as a secondary backup which would protect the data already copied until you were sure that the rebuilt was successful.

 

I think you have two choices at this point. Which would be the "better" choice is a matter of conjecture at this point.

 

1---  Stop the process and physically remove the failing/failed disk (Giving you a missing disk) and restart the entire copy process to the XFS formatted disk.

 

2--- Use that current XFS disk to rebuilt the failing/failed disk onto. 

 EDIT:              This will not work if the XFS disk is a part of the array!    

Edited October 4, 2017 by Frank1940

[JorgeB]

16 minutes ago, kimifelipe said:

I have about 300 gb left to copy, ..., then do the copy again? 

 

Also note that if you're using rsync it will resume the copy and only transfer the missing files.

[ijuarez]

Just wanted to thank all who have posted on getting the conversion done

 

I was able to do a convert on a disk with out data loss. 

[PeteAron]

I don't have a disk to replace the failing disk - I bought a new disk and its on its way.  I can wait until I get it and replace the failed disk with the new disk using parity to rebuild it, or I could remove the failing disk, finish the rsync copy using parity for the data, then complete the xfs conversion process for the disk.

 

obviously the safest way is to rebuild the failed disk but since I'm only 300 gb away, would it make sense to remove the disk, finish the copy and then re-set the configuration and parity using the xfs replacement?

[Frank1940]

8 minutes ago, kimifelipe said:

obviously the safest way is to rebuild the failed disk but since I'm only 300 gb away, would it make sense to remove the disk, finish the copy and then re-set the configuration and parity using the xfs replacement?

 

Did you add that XFS disk as a part of the array?  If so, you can't use it to rebuilt the failed disk onto.  You will need that new disk to do a rebuilt. 

[trurl]

31 minutes ago, kimifelipe said:

obviously the safest way is to rebuild the failed disk but since I'm only 300 gb away, would it make sense to remove the disk, finish the copy and then re-set the configuration and parity using the xfs replacement?

 

In order to remove the failing disk, you are going to have to stop the copy and shutdown. You could then restart the copy with the missing disk emulated by the rest of the array. Then you could set a New Config without the missing disk and rebuild parity.

 

Is this what you had in mind?

 

If you are reasonably confident in the rest of the disks, and you have good backups of any irreplaceable files, then this might be OK. But you will be working all of the disks to emulate the missing disk, you will be operating without any protection, and it will still be slow since all disks must be read, then a data disk and parity disk must be written.

 

It shouldn't be as slow as what you are doing now, and it might even be more reliable. I think there is a good chance unRAID will disable the failing disk before it ever completes the copy anyway.

 

I often recommend not writing to any disk in the array when it is compromised, and instead copy data to another system or an unassigned disk. That way parity isn't written and there is less disk activity.

[PeteAron]

OK.  I have shut down the array waiting for the new drive.  I will pull the failing drive and let the system rebuild it onto the new drive.  it will take about a day and a half I think since it has to preclear the new drive first. 

[trurl]

unRAID only requires a clear disk when it is ADDED to a NEW slot in an array that already has parity. This is so parity will remain valid.

 

Replacement disks do not have to be clear since they will be completely overwritten from the parity calculation anyway.

 

People often preclear new disks just to test them even if they don't require a clear disk, but this testing can be done on another computer using other software. The disk manufacturers have free downloads for testing disks.

[ijuarez]

On 10/4/2017 at 10:28 AM, ijuarez said:

Just wanted to thank all who have posted on getting the conversion done

 

I was able to do a convert on a disk with out data loss. 

Well i take it back, i lost data but not all data.

 

What i did was copy all data from disk2 rieser to disk9 xfs. It copied 1.8TB (used krusader)

 

then stop the array, ran a new config, click preserve all , and all disk went blue when to disk2 changed the format to xfs, started the array, click on parity is valid, disk was unmounted, mounted it, of course it formatted to xfs.

 

user krudaser to copy back to data from disk9 to disk2 but it only copied 50GB (hmmm) everything seem to work until i need to watch a tv series, and then noticed that the folder for series was completely gone. Not sure how that happen. I was able to get the series back. At this point im not sure what else may be missing from the 1.8TB initial copy.

 

I will use the rsync on the next disk see of that has a different outcome.

 

EDIT:  So im guessing the difference between 1.8T and 50GB it definitely  cause for concern I am missing movies i am not entirely sure why it lost that much data.

Edited October 5, 2017 by ijuarez
add more info

[bombz]

-removed-

Edited October 5, 2017 by bombz

[trurl]

6 hours ago, ijuarez said:

Well i take it back, i lost data but not all data.

 

What i did was copy all data from disk2 rieser to disk9 xfs. It copied 1.8TB (used krusader)

 

then stop the array, ran a new config, click preserve all , and all disk went blue when to disk2 changed the format to xfs, started the array, click on parity is valid, disk was unmounted, mounted it, of course it formatted to xfs.

 

user krudaser to copy back to data from disk9 to disk2 but it only copied 50GB (hmmm) everything seem to work until i need to watch a tv series, and then noticed that the folder for series was completely gone. Not sure how that happen. I was able to get the series back. At this point im not sure what else may be missing from the 1.8TB initial copy.

 

I will use the rsync on the next disk see of that has a different outcome.

 

EDIT:  So im guessing the difference between 1.8T and 50GB it definitely  cause for concern I am missing movies i am not entirely sure why it lost that much data.

Wasn't really necessary to New Config, but that didn't hurt anything. Don't know about your missing data. My guess would be user error of some kind.

[ijuarez]

32 minutes ago, trurl said:

Wasn't really necessary to New Config, but that didn't hurt anything. Don't know about your missing data. My guess would be user error of some kind.

I dont disagree Trurl