biiiink Posted December 31, 2018 Share Posted December 31, 2018 (edited) I just built a new Unraid server. Moved all the drives from the old server to the new one. I used the unbalance plugin to move files around so I could convert the file system to XFS. When that was all complete, I had an error on disk 3, so I replaced it and rebuilt it. Now, my Media share shows as empty: However, if I check the individual disks, they all show the data from the Media share: I see an error in the log file: Dec 30 15:05:33 Tower kernel: XFS (md3): metadata I/O error in "xfs_trans_read_buf_map" at daddr 0x156c69f68 len 32 error 117 Dec 30 15:05:33 Tower kernel: XFS (md3): xfs_imap_to_bp: xfs_trans_read_buf() returned error -117. Dec 30 15:05:33 Tower kernel: XFS (md3): Metadata corruption detected at xfs_buf_ioend+0x4c/0x95 [xfs], xfs_inode block 0x156c69f68 xfs_inode_buf_verify Dec 30 15:05:33 Tower kernel: XFS (md3): Unmount and run xfs_repair Dec 30 15:05:33 Tower kernel: XFS (md3): First 128 bytes of corrupted metadata buffer: Dec 30 15:05:33 Tower kernel: 00000000b764e26b: be fa cc 07 c4 c8 71 c1 d6 3e 51 3e a0 cb af dc ......q..>Q>.... Dec 30 15:05:33 Tower kernel: 00000000ee47918b: d1 56 aa 91 0b c1 90 9f 95 2f af 7d 54 ee d1 a9 .V......./.}T... Dec 30 15:05:33 Tower kernel: 000000003f63270c: 1d 5d 78 9e 52 83 f6 b8 30 a5 de 46 6c 4d 1e f3 .]x.R...0..FlM.. Dec 30 15:05:33 Tower kernel: 00000000dcf7f7c5: 2a 1d 6a 11 a3 33 f4 1a 4f cf bd 60 8c 3b 09 e6 *.j..3..O..`.;.. Dec 30 15:05:33 Tower kernel: 000000009db33fd9: d4 38 5a e6 42 bf bf 8b 10 25 73 6a f1 58 5a f5 .8Z.B....%sj.XZ. Dec 30 15:05:33 Tower kernel: 000000002d170146: 93 98 cb 16 97 71 05 c2 a0 08 32 6b 5a 4e ab 26 .....q....2kZN.& Dec 30 15:05:33 Tower kernel: 00000000b756f334: 4e 32 80 f0 e3 72 0a 79 b8 0d 1a 64 68 c6 95 70 N2...r.y...dh..p Dec 30 15:05:33 Tower kernel: 00000000a507f273: 82 bb 13 a9 e6 a8 de 65 f7 b2 8f 14 6e 55 f4 ad .......e....nU.. I have attached my diagnostics files: tower-diagnostics-20181230-1506.zip Any help would be greatly appreciated Edited December 31, 2018 by biiiink Quote Link to comment
trurl Posted December 31, 2018 Share Posted December 31, 2018 Filesystem corruption on disk3. Here is the relevant link from the FAQ for Unraid V6 "sticky" pinned near the top of this same subforum: https://wiki.unraid.net/index.php/Check_Disk_Filesystems Quote Link to comment
trurl Posted December 31, 2018 Share Posted December 31, 2018 What was the error you had on disk3 that caused you to replace it anyway? Could be there was nothing wrong with the disk itself. Connection issues are much more common than bad disks. Possibly you already had filesystem corruption before the rebuild. And rebuilding from parity will just rebuild the corrupt filesystem. Quote Link to comment
biiiink Posted December 31, 2018 Author Share Posted December 31, 2018 im running xfs_repair right now. i dont quite remember what the error was, but googling it let me to information saying the drive was failing. Replacing the drive wasn't such a big deal as I have a few spares kicking around. Quote Link to comment
biiiink Posted January 1, 2019 Author Share Posted January 1, 2019 So the xfs_repair is still running. Been running for nearly 24 hrs. Is this normal? The disk is 3GB and would be probably 80% full. Most recent thing i see in the status window is: Phase 7 - verify and correct link counts... resetting inode 131 nlinks from 9 to 8 resetting inode 4427235023 nlinks from 5 to 4 resetting inode 222 nlinks from 5 to 4 resetting inode 2147483776 nlinks from 421 to 417 resetting inode 2147483866 nlinks from 5 to 4 Quote Link to comment
JorgeB Posted January 1, 2019 Share Posted January 1, 2019 1 hour ago, biiiink said: Been running for nearly 24 hrs. Is this normal? Not really, any disk activity? Quote Link to comment
biiiink Posted January 1, 2019 Author Share Posted January 1, 2019 1 hour ago, johnnie.black said: Not really, any disk activity? Doesn't look like it, judging by the number of reads Quote Link to comment
biiiink Posted January 2, 2019 Author Share Posted January 2, 2019 I cancelled the check, but I'm not sure where to go from here. I've tried googling the errors in the log but i'm getting nowhere. Any help would be greatly appreciated. Quote Link to comment
JorgeB Posted January 2, 2019 Share Posted January 2, 2019 Start xfs_repair again Quote Link to comment
biiiink Posted January 2, 2019 Author Share Posted January 2, 2019 5 hours ago, johnnie.black said: Start xfs_repair again I've done that through the Gui and the terminal, but it always gets stuck at the same spot Quote Link to comment
JorgeB Posted January 2, 2019 Share Posted January 2, 2019 Try mounting the disk once and then running xfs_repair again, if it doesn't help or the disk doesn't mount best bet is to ask for help on the xfs mailing list. Quote Link to comment
biiiink Posted January 2, 2019 Author Share Posted January 2, 2019 I've done that too. same result. To be clear, the disk mounts just fine, and shows all the files just fine. However, the share that the files belong to is completely empty Quote Link to comment
trurl Posted January 2, 2019 Share Posted January 2, 2019 35 minutes ago, biiiink said: I've done that too. same result. To be clear, the disk mounts just fine, and shows all the files just fine. However, the share that the files belong to is completely empty If that disk has the files but the user share doesn't then maybe that disk isn't the problem. Other things can break user shares, and other things can cause a user to report an empty share when user shares aren't broken. What specifically are you looking at that makes you think the share is empty? Go to Main - Array Devices, click on the folder icon at the far right on the same line as the disk, and post a screenshot. Go to User Shares, click on the Compute... link next to the share in question, and after it gives the result, post a screenshot. Quote Link to comment
biiiink Posted January 2, 2019 Author Share Posted January 2, 2019 (edited) 23 minutes ago, trurl said: If that disk has the files but the user share doesn't then maybe that disk isn't the problem. Other things can break user shares, and other things can cause a user to report an empty share when user shares aren't broken. What specifically are you looking at that makes you think the share is empty? Go to Main - Array Devices, click on the folder icon at the far right on the same line as the disk, and post a screenshot. Go to User Shares, click on the Compute... link next to the share in question, and after it gives the result, post a screenshot. Both of those screenshots are in my original post. Also, if i connect to the server and check the Media share, it shows as empty as well. I also notice that this error shows up twice when i go into the media share in the web Gui: (It also shows up, but only once, when i go into the Media folder on disk 3, but not on any other disk) Jan 2 06:49:25 Tower kernel: XFS (md3): Metadata corruption detected at xfs_buf_ioend+0x4c/0x95 [xfs], xfs_inode block 0x156c69f68 xfs_inode_buf_verify Jan 2 06:49:25 Tower kernel: XFS (md3): Unmount and run xfs_repair Jan 2 06:49:25 Tower kernel: XFS (md3): First 128 bytes of corrupted metadata buffer: Jan 2 06:49:25 Tower kernel: 000000005491da45: be fa cc 07 c4 c8 71 c1 d6 3e 51 3e a0 cb af dc ......q..>Q>.... Jan 2 06:49:25 Tower kernel: 0000000033c25846: d1 56 aa 91 0b c1 90 9f 95 2f af 7d 54 ee d1 a9 .V......./.}T... Jan 2 06:49:25 Tower kernel: 0000000040618539: 1d 5d 78 9e 52 83 f6 b8 30 a5 de 46 6c 4d 1e f3 .]x.R...0..FlM.. Jan 2 06:49:25 Tower kernel: 000000009e9334aa: 2a 1d 6a 11 a3 33 f4 1a 4f cf bd 60 8c 3b 09 e6 *.j..3..O..`.;.. Jan 2 06:49:25 Tower kernel: 000000009881fdfb: d4 38 5a e6 42 bf bf 8b 10 25 73 6a f1 58 5a f5 .8Z.B....%sj.XZ. Jan 2 06:49:25 Tower kernel: 000000004113fee1: 93 98 cb 16 97 71 05 c2 a0 08 32 6b 5a 4e ab 26 .....q....2kZN.& Jan 2 06:49:25 Tower kernel: 0000000049cb44db: 4e 32 80 f0 e3 72 0a 79 b8 0d 1a 64 68 c6 95 70 N2...r.y...dh..p Jan 2 06:49:25 Tower kernel: 00000000cc661843: 82 bb 13 a9 e6 a8 de 65 f7 b2 8f 14 6e 55 f4 ad .......e....nU.. Jan 2 06:49:25 Tower kernel: XFS (md3): metadata I/O error in "xfs_trans_read_buf_map" at daddr 0x156c69f68 len 32 error 117 Jan 2 06:49:25 Tower kernel: XFS (md3): xfs_imap_to_bp: xfs_trans_read_buf() returned error -117. Jan 2 06:49:25 Tower kernel: XFS (md3): Metadata corruption detected at xfs_buf_ioend+0x4c/0x95 [xfs], xfs_inode block 0x156c69f68 xfs_inode_buf_verify Jan 2 06:49:25 Tower kernel: XFS (md3): Unmount and run xfs_repair Jan 2 06:49:25 Tower kernel: XFS (md3): First 128 bytes of corrupted metadata buffer: Jan 2 06:49:25 Tower kernel: 00000000aa3c7d84: be fa cc 07 c4 c8 71 c1 d6 3e 51 3e a0 cb af dc ......q..>Q>.... Jan 2 06:49:25 Tower kernel: 00000000c13b8090: d1 56 aa 91 0b c1 90 9f 95 2f af 7d 54 ee d1 a9 .V......./.}T... Jan 2 06:49:25 Tower kernel: 00000000cb0ed316: 1d 5d 78 9e 52 83 f6 b8 30 a5 de 46 6c 4d 1e f3 .]x.R...0..FlM.. Jan 2 06:49:25 Tower kernel: 00000000de4d9b94: 2a 1d 6a 11 a3 33 f4 1a 4f cf bd 60 8c 3b 09 e6 *.j..3..O..`.;.. Jan 2 06:49:25 Tower kernel: 0000000021fff399: d4 38 5a e6 42 bf bf 8b 10 25 73 6a f1 58 5a f5 .8Z.B....%sj.XZ. Jan 2 06:49:25 Tower kernel: 000000008e981e48: 93 98 cb 16 97 71 05 c2 a0 08 32 6b 5a 4e ab 26 .....q....2kZN.& Jan 2 06:49:25 Tower kernel: 00000000808a9410: 4e 32 80 f0 e3 72 0a 79 b8 0d 1a 64 68 c6 95 70 N2...r.y...dh..p Jan 2 06:49:25 Tower kernel: 00000000b2bf6473: 82 bb 13 a9 e6 a8 de 65 f7 b2 8f 14 6e 55 f4 ad .......e....nU.. Jan 2 06:49:25 Tower kernel: XFS (md3): metadata I/O error in "xfs_trans_read_buf_map" at daddr 0x156c69f68 len 32 error 117 Jan 2 06:49:25 Tower kernel: XFS (md3): xfs_imap_to_bp: xfs_trans_read_buf() returned error -117. Edited January 2, 2019 by biiiink Quote Link to comment
trurl Posted January 2, 2019 Share Posted January 2, 2019 That certainly looks like corruption on disk3. And corruption on any disk has been known to break user shares. Maybe you could just try to copy whatever you can from that disk, have Unraid reformat it, and then see if the share comes back. Quote Link to comment
JorgeB Posted January 2, 2019 Share Posted January 2, 2019 12 minutes ago, trurl said: Maybe you could just try to copy whatever you can from that disk, have Unraid reformat it, If xfs_repair doesn't finish this is your best bet. Quote Link to comment
biiiink Posted January 2, 2019 Author Share Posted January 2, 2019 1 hour ago, trurl said: Maybe you could just try to copy whatever you can from that disk, have Unraid reformat it, and then see if the share comes back. I attempted this through unbalance, but it came back with this: So im going to do it manually. I will report back when its complete Quote Link to comment
biiiink Posted January 2, 2019 Author Share Posted January 2, 2019 So im trying to copy the files to an empty drive, but i get an error: cp: cannot stat 'Downloads/incomplete': Structure needs cleaning This error comes up for every folder im trying to copy. Googling it leads me back to running xfs_repair. So now im not sure what else to do Quote Link to comment
biiiink Posted January 2, 2019 Author Share Posted January 2, 2019 (edited) by the way, the command im running is: xfs_repair -L /dev/md3 I've attached the entire log in case there is something im missing xfs_repair.txt Edited January 2, 2019 by biiiink Quote Link to comment
JorgeB Posted January 3, 2019 Share Posted January 3, 2019 That looks like a very corrupt filesystem, still xfs_repair should fix would more or less data loss, so you could try asking for help on the xfs mailing list. Quote Link to comment
biiiink Posted January 9, 2019 Author Share Posted January 9, 2019 (edited) Just to update this thread: It was basically determined that nothing would fix the filesystem. It was just too corrupt. However, I could still mount the drive and see all the files. So I mounted the drive using the unassigned devices plugin, and manually copied all the folders to an empty drive. After checking the drive to ensure that there were no issues, i used the "new config" tool to create a new drive configuration. Reassigned all the drives, and I now have my Media share back. Thank you to everyone who offered their help/advice/opinion Edited January 9, 2019 by biiiink Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.