I had an XFS drive get corrupted on me due to power loss.

Right now it 'can't read superblock on a xfs partition' so that's a fun problem

 

Since I'm likely going to have to replace the drive, I was wondering if BTRFS was less susceptible to disk corruption vs XFS on power loss?

Thoughts?

 

 

(Longer version. I was trying to make the array smaller. I removed the old drives, init'ed the array, and was assigning back the drives. It was only once the array booted up and the new parity check was started that Unraid told me that drive was 'unmountable'. At that point, the new parity check had wiped out the old version of the parity and so I was left with this drive situation. Since I'll need to 'start over' per se, now would be the time to change the filesystems if that's the way to go)

 

 

Have you tried putting the array into Maintenance mode and then running xfs_repair against the disk?    XFS has backup superblocks on the disk, and xfs_repair will span the disk to find them and will normally succeed. Depending on what other corruption has occurred you may find that the data is restored.

I had an XFS drive get corrupted on me due to power loss.

every server should have a UPS.  Actually, every computer should have one.

Since I'm likely going to have to replace the drive

Why?

 

Good thought. I ran a xfs_repair -n to check and that's when it replied 'Sorry, could not find valid secondary superblock option' (Which I found odd)

I'll likely have to blow it out with a xfs_repair -L to at least hope I can easily recover the data

Yes I caused it.

 

I'll need to replace it since I can't find a valid superblock for drive. I figure I'll put another drive in the array and run this corrupted drive outside the array and copy what I can off of it to the replacement drive.

 

 

If possible make sure you're running the commands on the unraid device (/dev/md#) otherwise your array parity will not be updated with changes and you will have to rebuild parity.

 

I dont know the answer to your question, but I have yet to see anyone recover from BTRFS errors while others have recovered from XFS errors.

Thanks BRIT good to hear about BTRFS (a lot less work for me too)

 

I'm running the commands on the /dev/sd# with the array not started

Since the drive is 'unmountable' it wouldn't be available on /dev/md#

 

Parity is already shot since I was trying to shrink the array at the time.

Thanks BRIT good to hear about BTRFS (a lot less work for me too)

 

I'm running the commands on the /dev/sd# with the array not started

Since the drive is 'unmountable' it wouldn't be available on /dev/md#

That would not work as using the /dev/sd? type devices you also need to include the partition number or you will get the message about a missing superblock.

 

Note however that it is not required for a disk to be mountable if you start the disk in maintenance mode.  All that does is map the /dev/md? type devices to the appropriate drive+partition on each disk.  It will work fine with any disk as long as it is partitioned - there is no need to have a valid file system in the partition.

Thanks itimpi

Perhaps that is actually my current issue with the lack of Superblock.

Let me jump to maintenance mode and give it another whirl.

As already noted, you should absolutely have a UPS on the system.    If you haven't yet rectified that oversight, it should be high on your list of "to do's".    EVERY PC should have one ... but especially servers.

 

Yup. There's a UPS. I forced the bad shutdown.

That said, my UPS is just a generic one. Is there one you recommend that can trigger a graceful shutdown?

 

 

(xfs_repair is still searching for a secondary superblock)

... as for your question => based on the experiences on this forum, you're better off with XFS than BTRFS.    BTRFS has a lot of promise, but is still relatively immature; and I've seen far more data loss with it than with XFS.

 

The best file system by far for recovery from corruption is Reiser => but that's fallen out of favor since it's been superseded by the newer files systems [Clearly it's not been as well maintained since Hans was sent to prison].

It would certainly be nice if there was a recovery tool for XFS that worked as well as reiserfsck.

 

 

... as for your question => based on the experiences on this forum, you're better off with XFS than BTRFS.    BTRFS has a lot of promise, but is still relatively immature; and I've seen far more data loss with it than with XFS.

 

The best file system by far for recovery from corruption is Reiser => but that's fallen out of favor since it's been superseded by the newer files systems [Clearly it's not been as well maintained since Hans was sent to prison].

It would certainly be nice if there was a recovery tool for XFS that worked as well as reiserfsck.

 

I realize Reiser is antiquated, but is there any value to going back to straight resier for Unraid purposes in your opinion?

Yup. There's a UPS. I forced the bad shutdown.

That said, my UPS is just a generic one. Is there one you recommend that can trigger a graceful shutdown?

 

 

(xfs_repair is still searching for a secondary superblock)

 

First, you don't want a UPS that doesn't have automatic voltage regulation (AVR) => many of the really inexpensive ones do not.

 

Second, any UPS that can be controlled by the APCUPSD code that's built-in to UnRAID v6 will automatically shut down the system gracefully.    Using a good APC unit is the "safest" approach.    This is a good choice for an inexpensive unit:  http://www.newegg.com/Product/Product.aspx?Item=N82E16842101381

 

CyberPower units also work well; and they have a very low cost unit with both AVR and true sine wave outputs (at least $100 less than an APC unit with true sine wave):  http://www.newegg.com/Product/Product.aspx?Item=N82E16842102131

 

Both of these are relatively low capacity units -- if you need higher output capacity there are higher capacity units for a few more $$

 

 

 

 

... as for your question => based on the experiences on this forum, you're better off with XFS than BTRFS.    BTRFS has a lot of promise, but is still relatively immature; and I've seen far more data loss with it than with XFS.

 

The best file system by far for recovery from corruption is Reiser => but that's fallen out of favor since it's been superseded by the newer files systems [Clearly it's not been as well maintained since Hans was sent to prison].

It would certainly be nice if there was a recovery tool for XFS that worked as well as reiserfsck.

 

I realize Reiser is antiquated, but is there any value to going back to straight resier for Unraid purposes in your opinion?

 

As long as you're well backed up, then I wouldn't think so.    I would NOT, however, convert all current Reiser disks to XFS, as many folks have been doing.    There's no compelling reason to do so; and if you do encounter corruption you may very well be able to recover with RFS whereas you wouldn't be able to with XFS.

 

Far more important than worrying about how you might recover from disk corruption is to be sure you have a good backup strategy and have a complete set of backups of anything you don't want to lose.    My thoughts on that are pretty well detailed here:  http://lime-technology.com/forum/index.php?topic=31020.0

 

 

Yup. I fell in the camp of converting ALL my disks over to XFS for unraid 6

I have backups, but the hardest part is figuring out what was on this particular disk in relation to the others to restore.

Since my backups aren't actively online it's harder to run a straight diff on the whole array with ViceVersa or the like.

 

Appreciate the UPS suggestions. What I've had had been good enough for a while.

IF your backup disks collectively contain the same set of files that are on your array, it's not too difficult to restore everything to the array when you lose a disk by doing something along the following lines ...

 

Suppose you have a large share called "Movies" and you have files from that share spread on multiple backup disks.    You can use SyncBack (the free version is fine) and set up a "Profile" to backup everything from Drive X: to \\Tower\Movies  ... assuming your UnRAID server is called "Tower".

 

Now, one-at-a-time, attach a backup disk to your PC and use Disk Management to assign it drive letter X:  [Note that once you've done this, every time you attach the drive it will be X: => if you've already done this with your drives and used a different letter, just use that letter for the SyncBack profile].

 

Run the SyncBack profile -- and it will only copy any files on the backup that aren't already on the server.  [in other words, those that are missing as a result of what you lost.]      Simple repeat this process for all of your backup disks, and when it's done, all of the files that were lost will have been restored.

 

Clearly this is a bit more complex if you have a lot of shares; or if the files on the backups aren't organized in folders that match your share names [unless every file on a disk belongs to a known share].

 

Booo-

No dice on the secondary Superblock after checking the /dev/md#

 

 

Let me preface this with: Ughhhhhh

 

This seem to come down to an EEOC error.

When reassigning drives, I accidentally swapped the Parity Drive with a Data disk.

This is why the one drive was unmountable and why no superblock could be found.

Now, you may take a moment to slap me upside the head for being stupid.

 

...Go ahead. I'll wait...

 

Now with that out of the way here is my plan.

Since my parity drive was not touched beyond the superblock scans, that should be ok.

The rest of the array was not really written to during the shrink EXCEPT for the drive that was accidentally made the 'new parity' - disk 1

This means I just have to address that one disk.

 

I was thinking of this:

1) re-initializing the array

2) set the the old assignments

3) tell Unraid to 'Trust my Parity'

4) Shut down and fail disk 1 by replacing the drive

5) Let a rebuild occur on disk 1

 

In theory this should get me back to square 1ish at which point I can run a crc check on the data against the backups

 

Does this sound reasonable? Is there something I'm not thinking of?

 

 

 

 

 

 

 

 

 

That will ALMOST work ... but there's a glitch in the process.

 

When you do a New Config with "Trust Parity" a parity check is automatically started when you first Start the array.

 

You can, of course, immediately CANCEL the check ... but it's very likely that at least SOME "corrections" will have already been written -- which will make the rebuild incorrect.    I've submitted a feature request to change this behavior, but as of now it's still the way it works.    [see here:  http://lime-technology.com/forum/index.php?topic=43765.msg417821#msg417821 ]

 

 

 

This might be one of the cases where you hire professional services.

http://lime-technology.com/services/

 

 

especially if the trust my parity option is going to start correcting things that it should not.

... As I was thinking about that issue, I thought of a possible way to force a rebuild without the intervening parity check.

 

I am NOT sure if this will work -- but you could TRY this (shouldn't hurt anything to try it) ...

 

(a)  Do a New Config, setting everything the way you want it, but do NOT Start the array.  Also be sure Auto-Start is NOT enabled.

 

(b)  Shut down and physically disconnect the drive you want to rebuild (so it will be missing when you start the array)

 

©  Boot; check the "Parity is already valid" box, and Start the array.    Since a drive will be missing, it shouldn't start a parity check.

 

(d)  Stop the array; assign the replacement drive to the slot; and Start the array.  Hopefully this will cause a rebuild to start.

 

I am NOT sure if this process will work -- but if it does please note that here so we can suggest it for similar cases until (hopefully) the "Trust Parity" option is modified to the automatic check does NOT start up.

 

This might be one of the cases where you hire professional services.

http://lime-technology.com/services/

 

 

especially if the trust my parity option is going to start correcting things that it should not.

 

I'd rather see the next release come out with the automatic parity check NOT enabled with the "Trust Parity" option.

 

Tom clearly KNOWS that shouldn't happen -- I've noticed since posting my feature request that this same issue was brought up way back in v5 days, and Tom indicated that was NOT his intent and that it would be fixed.    Not sure if it ever was ... but clearly it's back now and needs to be fixed again.

 

Yes I've noticed that a "trust parity" operation fires up a parity-check.  This is not the intent.  I intended that if you Start the array with "Parity is already valid" box checked, that no parity check is automatically started.  This has been fixed in 5.0.

This might be one of the cases where you hire professional services.

http://lime-technology.com/services/

 

 

especially if the trust my parity option is going to start correcting things that it should not.

 

I'd rather see the next release come out with the automatic parity check NOT enabled with the "Trust Parity" option.

 

Tom clearly KNOWS that shouldn't happen -- I've noticed since posting my feature request that this same issue was brought up way back in v5 days, and Tom indicated that was NOT his intent and that it would be fixed.    Not sure if it ever was ... but clearly it's back now and needs to be fixed again.

 

Yes I've noticed that a "trust parity" operation fires up a parity-check.  This is not the intent.  I intended that if you Start the array with "Parity is already valid" box checked, that no parity check is automatically started.  This has been fixed in 5.0.

 

 

Seeking guidance from the source may be prudent. Tom may know of a way to trust the parity without the parity check starting.

Seeking guidance from the source may be prudent. Tom may know of a way to trust the parity without the parity check starting.

 

I hope so ... in fact I asked him that in the thread I started asking this behavior to be changed ...

 

Tom => In lieu of this being fixed, is there a command line parameter that can be changed before starting the array that will keep this check from starting ??  [Or will perhaps at least change it to a non-correcting check so it won't cause any harm]

 

 

In any event, I certainly hope this behavior is changed in the next release.    This is the 3rd case in just the last couple months where this has happened ... and in both of the other 2 the changes made during the initial few seconds of the automated check destroyed the ability to do a successful rebuild.  "Trust parity" needs to MEAN that the system will trust parity !!