Drive won't mount, xfs_repair has a bug, how do I update xfs_repair?

[d4nk]

I have a healthy drive (according to SMART tests) but it is failing to mount. As far as I can tell, the next step is to run xfs_repair. However, when I run it against the drive it errors out with:

Quote

bogus .. inode number (549839724621) in directory inode 128, clearing inode number
xfs_repair: dir2.c:1419: process_dir2: Assertion `(ino != mp->m_sb.sb_rootino && ino != *parent) || (ino == mp->m_sb.sb_rootino && (ino == *parent || need_root_dotdot == 1))' failed.

This is apparently a known bug in xfs_repair version 4.16.1 (the version included in Unraid 6.6.6): https://access.redhat.com/solutions/3483841

This is fixed in the latest version of xfsprogs 4.20.0.

 ... So how can I update xfsprogs on unraid to the latest stable release? Or is that the right solution to this problem?

 

Thanks!!!

 

[trurl]

Unraid 6.7 release candidates have xfsprogs 4.19

 

See here for instructions on installing RCs:

 

[JonathanM]

1 hour ago, d4nk said:

when I run it against the drive

What command, precisely, are you running?

[trurl]

27 minutes ago, jonathanm said:

What command, precisely, are you running?

Didn't think to ask that.

 

@d4nk, if you run it from the webUI (click on the drive to get to its page) then you will be sure to run it on the correct device. If you run it from the command line, then you have to know exactly what you're doing or you will get it wrong. The right way to do it in Unraid may not be the same as you would expect from other uses or from googling.

[d4nk]

1 hour ago, trurl said:

Didn't think to ask that.

 

@d4nk, if you run it from the webUI (click on the drive to get to its page) then you will be sure to run it on the correct device. If you run it from the command line, then you have to know exactly what you're doing or you will get it wrong. The right way to do it in Unraid may not be the same as you would expect from other uses or from googling.

 

xfs_repair -v /dev/md1 (this is for Disk 1) The array is in Maintenance Mode.

 

So the upgrade to 6.7 RC5 did the trick as far as clearing the error, but now xfs_repair is freezing at Phase 7 - verify and correct link counts. It just sits at

"resetting inode 33093704 nlinks from 6 to 4" for hours. I also tried in the GUI, same thing, freezes at the same spot.

[JorgeB]

xfsprogrs 4.20 has been release since, you can try again after upgrading, see here how, if that doesn't help you'd need to ask for help in the xfs mailing list.

[d4nk]

20 hours ago, johnnie.black said:

xfsprogrs 4.20 has been release since, you can try again after upgrading, see here how, if that doesn't help you'd need to ask for help in the xfs mailing list.

Thanks! I followed that and installed xfsprogs 4.20 but I still ran into the hanging issue. I then found an article from 2009 that recommended using -P whenever xfs_repair hangs or becomes unresponsive while running and that did the trick!

 

After running xfs_repair -Pv /dev/md1, and then running again with just -v, xfs_repair reports that the drive is good!

 

However, the drive is disabled in the array in Unraid. How do I tell Unraid to re-enable it?

 

BTW, Unraid ran through a parity rebuild while it was unable to mount the drive due to the xfs errors, meaning parity does not have this drive's information. If I mount the drive using "Unassigned Devices" the drive mounts perfectly fine and I can access all of the files. So how do I tell Unraid that parity is false, drive is good, re-enable it and rebuild parity?

[phbigred]

I believe your next step is to go into tools and do a new config, of course noting which one is parity. Assign slots and assign parity drive. At that point when you start the array it will at that point rebuild parity.

 

 

[JorgeB]

2 hours ago, d4nk said:

However, the drive is disabled in the array in Unraid. How do I tell Unraid to re-enable it?

If the disk is disable then xfs_repair ran on the emulated disk, not the actual disk, best way forward depends on the status of the existing disk and why it got disabled, you can post the diagnostics, better if the server wasn't rebooted yet so you can give more informed advice, if you have one you can also rebuild to a spare disk, keeping the old disk intact, this is always the safest option.

[trurl]

8 hours ago, d4nk said:

BTW, Unraid ran through a parity rebuild while it was unable to mount the drive due to the xfs errors, meaning parity does not have this drive's information. If I mount the drive using "Unassigned Devices" the drive mounts perfectly fine and I can access all of the files. So how do I tell Unraid that parity is false, drive is good, re-enable it and rebuild parity?

This part is very confusing. Why did it do a parity rebuild? Are you sure that is what it did? A parity build (sync) is not the same thing as a parity check, and usually if you have a disabled disk it will really only do a read check that doesn't change parity at all. Do you have dual parity?

 

The only way it would have actually done a parity build is if you did a New Config, with or without that problem disk. Is that what you did?

 

 

[JorgeB]

1 minute ago, trurl said:

This part is very confusing. Why did it do a parity rebuild? Are you sure that is what it did?

I'm assuming he did a read check.

[trurl]

Also, I don't see how you could mount an assigned drive in Unassigned Devices. You have left out some critical details.

[d4nk]

On 3/6/2019 at 8:37 AM, trurl said:

This part is very confusing. Why did it do a parity rebuild? Are you sure that is what it did? A parity build (sync) is not the same thing as a parity check, and usually if you have a disabled disk it will really only do a read check that doesn't change parity at all. Do you have dual parity?

 

The only way it would have actually done a parity build is if you did a New Config, with or without that problem disk. Is that what you did?

 

 

I had a failed drive, it was 3TB, however I had 20TB of free space in the array so rather than go out and buy another drive which I don't need, I shrank the array by moving the files from the failed drive (emulated) to the other drives in the array using unBALANCE. I then created a new config without the failed drive and let parity rebuild following this procedure: https://wiki.unraid.net/Shrink_array

 

It was during this rebuild that I had a power failure, server rebooted, restarted parity sync but one of the drives didn't mount correctly, thus this problem.

 

On 3/6/2019 at 12:51 AM, phbigred said:

I believe your next step is to go into tools and do a new config, of course noting which one is parity. Assign slots and assign parity drive. At that point when you start the array it will at that point rebuild parity.

 

That did it! Everything is rebuilt, no data loss, array is healthy!

 

On 3/6/2019 at 8:39 AM, trurl said:

Also, I don't see how you could mount an assigned drive in Unassigned Devices. You have left out some critical details.

When the array is offline, you can remove a device from the assigned devices and then mount in unassigned devices to check it's contents.

[trurl]

On 3/6/2019 at 8:39 AM, trurl said:

You have left out some critical details.

In future, please consider that the advice you are given may only be as good as the information you have provided.