July 4, 20242 yr I'll number what I've done so far in case it helps. I have a server running 6.6.6 on Encrypted XFS that has been powered off for a few months and I want to move it into a new chassis. The old server seemed to work when I started it, but I decided to run a parity check before moving the drives to the new hardware. During the parity check, the machine suddenly shut down and would not restart, so I moved everything to the new case. After starting, both parity drives indicated "missing/no device" and did not appear in the dropdown. I moved them both to different SATA ports and they still didn't show up. I should have asked for advice at this point, but instead, I started the array to see if it would detect the parity drives as new so I could just re-add them and rebuild. When I started the array, the first data drive (Disk 1 • WDC_WD60EZRZ-00GZ5B1_WD-WX21DB6N4KN4 (sdh)) reported "Unmountable: Unsupported partition layout" I restarted the array in Maintenance mode to follow the XFS repair procedure. xfs_repair said "Device or resource busy" on Disk 1 (/dev/md1), so I also tried running it on /dev/sdh itself: root@unRAID:~# xfs_repair -v /dev/md1 xfs_repair: cannot open /dev/md1: Device or resource busy root@unRAID:~# xfs_repair -v /dev/sdh Phase 1 - find and verify superblock... bad primary superblock - bad magic number !!! attempting to find secondary superblock... ....found candidate secondary superblock... unable to verify superblock, continuing... .......................................... [12 hours later...] .........Sorry, could not find valid secondary superblock Exiting now. root@unRAID:/# One oddity I noticed was that during one of the restarts, the UI required me to enter the password twice (new and old) as if to change it in order to get the array to start. I used the same password as before, but doing that might have destroyed something. At this point, both parity drives appeared as "unassigned" rather than "missing/no device" as before. I finally realized it might be a power problem, so I found a spare power supply, put both parity drives on it and now they appear as "unassigned" but are available in their dropdowns. The start button indicates that "Start will start Parity-Sync and/or Data-Rebuild" so I haven't done that. It seems to me that if the data on both parity drives is the same, it's probably valid. Is there a way to check them, then tell Unraid to accept them as valid and use them to rebuild data Disk 1? What should my next move be?
July 4, 20242 yr Community Expert 3 minutes ago, neubert said: Maintenance mode to follow the XFS repair procedure. xfs_repair said "Device or resource busy" on Disk 1 (/dev/md1 Not sure what you really did there, but Maintenance mode doesn't mount any disks, so they can't be busy. You should try it again. 2 minutes ago, neubert said: running it on /dev/sdh And this was the incorrect command. You have to run it on the partitition, not the device. 8 minutes ago, neubert said: if the data on both parity drives is the same, it's probably valid Dual parity doesn't work like that. The data on the parity drives should be different. They wouldn't be independent and allow 2 failures if they were the same. 9 minutes ago, neubert said: What should my next move be? Attach diagnostics to your NEXT post in this thread.
July 4, 20242 yr Author Start the array in Maintenance Mode and run xfs_repair on /dev/md1: Yesterday partition /dev/sdh1 didn't appear: Diagnostics are attached. unraid-diagnostics-20240704-1317.zip
July 4, 20242 yr Community Expert You should always try to do the repair from the webUI instead of the command line. Easy to get the command wrong from the command line. /dev/md1 is incorrect for an encrypted disk. Why are you running an Unraid version that is over 5 years old? Can't tell anything about your filesystems from those diagnostics since the array is not started in normal mode. Try to do the repair from the webUI instead of the command line.
July 4, 20242 yr Community Expert To the title question, if the array was started without the parity drives then they are not valid anymore. Assuming all data drives are fine you can add them saying they are and then run a parity check to correct, but if you expect to rebuild a drive at this point you'll likely get corruption.
July 4, 20242 yr Author Okay, run from the UI on Disk 1 with -n gives this: Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... ALERT: The filesystem has valuable metadata changes in a log which is being ignored because the -n option was used. Expect spurious inconsistencies which may be resolved by first mounting the filesystem to replay the log. - scan filesystem freespace and inode maps... sb_fdblocks 45886556, counted 47498947 - found root inode chunk Phase 3 - for each AG... - scan (but don't clear) agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 No modify flag set, skipping phase 5 Phase 6 - check inode connectivity... - traversing filesystem ... - traversal finished ... - moving disconnected inodes to lost+found ... Phase 7 - verify link counts... No modify flag set, skipping filesystem flush and exiting. Run without -n gives this: Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... ERROR: The filesystem has valuable metadata changes in a log which needs to be replayed. Mount the filesystem to replay the log, and unmount it before re-running xfs_repair. If you are unable to mount the filesystem, then use the -L option to destroy the log and attempt a repair. Note that destroying the log may cause corruption -- please attempt a mount of the filesystem before doing this. Stopped the array, started the array in normal mode to see if it would mount and apply the log (it didn't), stopped the array, started in Maintenance Mode. Ran xfs-repair with -L: Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... ALERT: The filesystem has valuable metadata changes in a log which is being destroyed because the -L option was used. - scan filesystem freespace and inode maps... sb_fdblocks 45886556, counted 47498947 - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 2 - agno = 1 - agno = 3 - agno = 4 - agno = 5 Phase 5 - rebuild AG headers and trees... - reset superblock... Phase 6 - check inode connectivity... - resetting contents of realtime bitmap and summary inodes - traversing filesystem ... - traversal finished ... - moving disconnected inodes to lost+found ... Phase 7 - verify and correct link counts... Maximum metadata LSN (2:1770261) is ahead of log (1:2). Format log to cycle 5. done Stop the array, Start the array in normal mode. Disk 1 still says "Unmountable: Unsupported partition layout" Should I update Unraid to the newest version before going any further? Edited July 5, 20242 yr by neubert Forgot to request upgrade confirmation.
July 5, 20242 yr Author 3 hours ago, Kilrah said: To the title question, if the array was started without the parity drives then they are not valid anymore. Assuming all data drives are fine you can add them saying they are and then run a parity check to correct, but if you expect to rebuild a drive at this point you'll likely get corruption. If the array is started without parity and then data is written to the data drives, the parity would no longer be valid for the sections that were written. In theory, the rest of the parity would still be valid. I haven't written anything to the data drives without parity, so I would expect all of it to still be valid. However, I don't know how Unraid works, encryption might invalidate my assumptions, etc.
July 5, 20242 yr Community Expert 7 hours ago, neubert said: Stop the array, Start the array in normal mode. Disk 1 still says "Unmountable: Unsupported partition layout" Not seen that before The repair look like it ran successfully so I would expect the drive to have mounted when when you restarted in normal mode. Maybe you should post current diagnostics to see if we can spot a reason why it has still not mounted.
July 5, 20242 yr Community Expert 8 hours ago, neubert said: Disk 1 still says "Unmountable: Unsupported partition layout" This is a different problem, not filesystem corruption, since you don't have valid parity see if the disks mount with UD, but use the read-only option.
July 5, 20242 yr Community Expert 6 hours ago, neubert said: If the array is started without parity and then data is written to the data drives, the parity would no longer be valid for the sections that were written. In theory, the rest of the parity would still be valid. I haven't written anything to the data drives without parity, so I would expect all of it to still be valid. Even just mounting/unmounting a drive without explicitly writing files will change some metadata e.g. access times etc (unless mounted read-only). Sure most is still correct, but it's still likely to result in a hosed filesystem or corrupted files should a drive need to be emulated from that.
July 5, 20242 yr Author 4 hours ago, JorgeB said: This is a different problem, not filesystem corruption, since you don't have valid parity see if the disks mount with UD, but use the read-only option. Okay, I removed Disk 1 from the array to make it appear as an unassigned device, but I don't see any options in UD to mount it as read-only; it just says FORMAT where the other drives say MOUNT. Here's the drive log: Should I update Unraid from 6.6.6 to the newest 6.x.x and see if there are UD updates? My version is from 2019. Also, since xfs_repair didn't find any errors, I'm happy to pull that drive and try to mount it on a Linux system if that simplifies things. I have a laptop running Ubuntu 22.04 and a drive dock.
July 5, 20242 yr Author 3 hours ago, Kilrah said: Even just mounting/unmounting a drive without explicitly writing files will change some metadata e.g. access times etc (unless mounted read-only). Sure most is still correct, but it's still likely to result in a hosed filesystem or corrupted files should a drive need to be emulated from that. I see. I was hoping there might be a way to treat the good data drives as read-only, force the parity-drives back into service as read-only, and use that to rebuild the bad data drive. If that were to result in partial recovery of data on on the bad drive, with some file corruption, that would likely be a better outcome for me than losing everything on that bad drive. Since xfs-repair didn't find any problems, maybe the data can be recovered without getting complicated.
July 5, 20242 yr Community Expert Post the output of: fdisk -l /dev/sdX Replace X with correct letter.
July 5, 20242 yr Author 19 minutes ago, JorgeB said: Post the output of: fdisk -l /dev/sdX Replace X with correct letter. Disk 1 is the bad disk, using Encrypted XFS: Linux 4.18.20-unRAID. root@unRAID:~# fdisk -l /dev/sdj The primary GPT table is corrupt, but the backup appears OK, so that will be used. Disk /dev/sdj: 5.5 TiB, 6001175126016 bytes, 11721045168 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: gpt Disk identifier: ECE1170E-5D6F-49A0-940C-0729A1EF918F Device Start End Sectors Size Type /dev/sdj1 64 11721045134 11721045071 5.5T Linux filesystem
July 5, 20242 yr Community Expert 27 minutes ago, neubert said: The primary GPT table is corrupt, but the backup appears OK, so that will be used. You can try to fix this to see if it helps: Post the output of: gdisk /dev/sdj
July 5, 20242 yr Author 2 minutes ago, JorgeB said: You can try to fix this to see if it helps: Post the output of: gdisk /dev/sdj root@unRAID:~# gdisk /dev/sdj GPT fdisk (gdisk) version 1.0.3 Caution: invalid main GPT header, but valid backup; regenerating main header from backup! Partition table scan: MBR: protective BSD: not present APM: not present GPT: damaged **************************************************************************** Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk verification and recovery are STRONGLY recommended. **************************************************************************** Command (? for help):
July 5, 20242 yr Author Command (? for help): ? b back up GPT data to a file c change a partition's name d delete a partition i show detailed information on a partition l list known partition types n add a new partition o create a new empty GUID partition table (GPT) p print the partition table q quit without saving changes r recovery and transformation options (experts only) s sort partitions t change a partition's type code v verify disk w write table to disk and exit x extra functionality (experts only) ? print this menu Command (? for help):
July 5, 20242 yr Author Command (? for help): w Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING PARTITIONS!! Do you want to proceed? (Y/N): y OK; writing new GUID partition table (GPT) to /dev/sdj. The operation has completed successfully. root@unRAID:~# Start the array, but it still says Unmountable: Unsupported partition layout: Stop the array, back into the terminal: Linux 4.18.20-unRAID. Last login: Fri Jul 5 07:07:31 -0700 2024 on /dev/pts/1. root@unRAID:~# gdisk /dev/sdj GPT fdisk (gdisk) version 1.0.3 Partition table scan: MBR: protective BSD: not present APM: not present GPT: present Found valid GPT with protective MBR; using GPT. Command (? for help):
July 5, 20242 yr Community Expert 5 minutes ago, neubert said: Start the array, but it still says Unmountable: Unsupported partition layout: And with UD also the same?
July 5, 20242 yr Author Looking better. Remove Disk 1 from array and open UD. I still don't see a read-only option. Should I just press that MOUNT button?
July 5, 20242 yr Author Okay, it mounted and I can see the files under /mnt/disks: Linux 4.18.20-unRAID. Last login: Fri Jul 5 07:17:37 -0700 2024 on /dev/pts/1. root@unRAID:~# ls /mnt/ disks/ root@unRAID:~# ls /mnt/disks/ WDC_WD60EZRZ-00GZ5B1_WD-WX21DB6N4KN4/ root@unRAID:~# ls /mnt/disks/WDC_WD60EZRZ-00GZ5B1_WD-WX21DB6N4KN4/ [...] root@unRAID:~# du -hsc /mnt/disks/WDC_WD60EZRZ-00GZ5B1_WD-WX21DB6N4KN4/ 5.3T /mnt/disks/WDC_WD60EZRZ-00GZ5B1_WD-WX21DB6N4KN4/ 5.3T total Thank you! Unless you recommend something else, at this point, I'll think I'll connect another drive, copy everything off, reformat this drive, put it back into service, and rebuild parity.
July 5, 20242 yr Community Expert 10 minutes ago, neubert said: Unless you recommend something else, at this point, I'll think I'll connect another drive, copy everything off, reformat this drive, put it back into service, and rebuild parity. That's a good option.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.