Was moving some files to my server and it seems I ran into an XFS error at some point. XFS seemed to shutdown the filesystem intentionally.

It seems it ran into an issue with a certain file, but I'm not sure entirely. I accidentally kicked the enclosure where my two parity drives are located at some point today, but I don't remember the exact time and if that lines up with the error. XFS was spitting out a lot of messages for the past 24 hours or more prior to that, so I'm uncertain if nudging the drives is related to the failure. My syslog is full of the messages about XFS and I didn't notice them before, so if there was a previous error, I can't seem to find it. For some context, I was able to browse the files from that disk earlier in the day prior to kicking it. None of the disks have failed out of the array either. Disk 15, the one with an invalid path, is in a different enclosure that wasn't affected by my blunder either.

I ran a short smart self-test on the disk and it doesn't appear there was a major issue with it.

I imagine an XFS repair will be needed, but I don't know exactly what steps to take first, so some guidance would be appreciated.

syslog 20250822 0642.zip HUH721010AL42C0_2TJPHM8D_35000cca26a97d5c4-20250822-0115.txt

Check filesystem for disk15

30 minutes ago, JorgeB said:

Check filesystem for disk15

Done. Shows that the File System is corrupted and gives the option to fix. Should I proceed?

1 hour ago, chris206 said:

Done. Shows that the File System is corrupted and gives the option to fix. Should I proceed?

yes.

Afterwards when you restart the array in normal mode the drive should mount fine.

You might want to check if there is now a lost+found folder on the drive which is where the repair process would put any file/folder it finds for which it could not find the directory entry. Not having this folder is a good sign that things went well during the repair.

Alright, so now it mentions the log is dirty and wants to zero it, but it says I should attempt to mount the filesystem first to replay it? Some files were put into a temp folder while the filesystem was down, not many though. Could those be what this error message is referring to? Should I back out of this and try to restart the array as normal first before doing this next step?

So, I stopped the array in maintenance mode and restarted the array normally. The disk mounted but said it was at almost 100% utilization. If I go back to the disk to look at check filesystem status, the repair hasn't reset at all and still has the same option before about zeroing the log. Any tips would be appreciated.

tower-syslog-20250823-0232.zip

Edited August 23, 2025Aug 23 by chris206

I also checked the filesystem on the drive and it seems to be back up without any lost+found files, so I'm not entirely sure what happened. Yesterday prior to the filesystem going down on the drive, I had roughly 1.8 TB of space left on it, so unless it stopped reporting the free space left accurately, I'm not sure how it managed to fill all the way up.

Edited August 23, 2025Aug 23 by chris206

If the disk is mounting, it should be resolved.

6 minutes ago, JorgeB said:

If the disk is mounting, it should be resolved.

Hmm alright,

I do get this message in the syslog, "XFS (md15p1): Per-AG reservation for AG 9 failed. Filesystem may run out of space." I'm guessing to resolve that I should move some files from that disk to another using krusader in order to free up space?

Stop the array, start in maintenance mode, and post the output from:

xfs_repair -v /dev/md15p1

53 minutes ago, JorgeB said:

Stop the array, start in maintenance mode, and post the output from:

xfs_repair -v /dev/md15p1

Before this reply, I had gone ahead and started the array with a new disk, which is in the process of being cleared. Prior to that, I had checked several of the files on the disk and everything seemed okay. In addition, I managed to free up ~280 GBs of space by cleaning up some folders. If still recommended I can run that command after the disk clear is done tomorrow. Question though, I can't seem to find documentation on what -v flag is for xfs_repair, what's it do?

I still recommend running it again.

-v adds verbosity, though with xfs_repair it's basically the same without it.

5 minutes ago, JorgeB said:

I still recommend running it again.

-v adds verbosity, though with xfs_repair it's basically the same without it.

Alright, should I run that through the terminal as well or is the GUI fine? I don't remember having the ability to change flags in the check filesystem section for the disk's page.

You can run using the GUI, CLI was just to confirm if it still needed to zero the log or not, in case the GUI was showing old info.

Finally reran xfs_repair through the CLI. The GUI for check filesystem doesn't seem like it'll refresh itself after taking the array out of maintenance mode.

Anyways, I uploaded the output from the CLI. Seems like it zeroed the log and mentioned moving files to the lost + found, but I didn't see that folder on the disk after starting the array. Everything still checks out okay and that previous warning I was worried about is no longer there. Not entirely sure what caused the issue with reporting the remaining freespace, but I've got a new drive in my server, so hopefully it won't happen again.

Thanks for all of the help.

xfs_repair_log_20250824.txt

filesystem looks healthy; the problem should be resolved.