Xandros Posted April 5 Share Posted April 5 Hi everyone, I'm hoping someone can help me out here.... I'm not the most knowledge on Unraid, but will do my best to provide additional information should it be required. I have 2 Unraid servers - 1 with several drives and 1 parity drive (we'll call it the Primary Server), the 2nd server is my backup server (we'll refer to that one as the Backup Server) that has no parity drive The backup server had a better MB and processor and I wanted that hardware to be used in my primary Server as Plex would run better on the upgraded hardware When I booted up the Primary server, I had 1 failed drive (Disk 6) that I couldn't get back online - I tried changing the SATA data cable and a different SATA port on the MB - I couldn't get it back online, so I removed it from the Server and tried to rely on parity to backup the main data that I had a concern about. I hadn't backed up the Primary Server data for a while because I was out of room on the Backup Server. I couldn't afford an additional hard drive for the longest time, but with this failure I decided I didn't have a choice but to make the purchase, so I bought a 22TB drive for the Backup Server. When pulling the data, it would process no problem until it started to hit files that were on Disk 7 - at that point it would slow right down and small sized jpegs would barely transfer at all... I manually checked some files from Disk 7 and found that they weren't accessible even though the drive appeared to be healthy in the array. I could browse and view the file structure on Disk 7 I took a few days to think about what was going on and came to the conclusion that I may be having a power issue. I went out and bought a fully modular 1000Watt Corsair power supply, installed that into the server and hooked up all the power/data cables. When I booted the server up, all the drives showed up in their respective places, so I started the Array. Once it started off, the parity rebuild started up right away. The strange thing I noticed right away, was that both disks 6 & 7 showed 'unmountable: Unsupported or no file system' instead of the drive size/used/free data. Even thought they appeared to be in that state, the parity rebuild was writing data to that drive with no problems... and it was also reading the data from disk 7 (at the same rate as the healthy drives), therefore contributing to the rebuild. I was concerned of course, but didn't want to interrupt the rebuild, thinking that the drives might come back online once the rebuild was done. Once the rebuild was done, Disks 6/7 stayed in the same state with the same error. I stopped the array, rebooted the server and then started the array back up. The drives are showing the same error. All drives show up as being healthy and all disks but 6 & 7 have mount points, 6 & 7 do not. One thing that I did notice, was that the drive descriptors had changed - the drives were showing up in the historical devices, but with a different identifier as shown. Here they are in the array: Disk 6 WDC_WD80EZAZ-11TDBA0_2SG1E64F - 8 TB (sdc) 26 C 1298 528 0 xfs Unmountable: Unsupported or no file system Disk 7 WDC_WD80EZAZ-11TDBA0_7HKS86DJ - 8 TB (sdh) 26 C 1299 528 0 xfs Unmountable: Unsupported or no file system Here are how the drives are shown in the historical devices: dev2 WDC_WD80EZAZ-11TDBA0_2SG1E64F sdm WDC_WD80EZAZ-11TDBA0_7HKS86DJ Could this change in the identifier be the cause of the inability to properly mount the drives? Any help would be greatly appreciated! I have attached the diagnostics from my server. I have uploaded a couple of screenshots that I took throughout this process - they will show the drive identifiers before and after The picture entitled 'Primary-Drives-04-02-2024' shows the drive info just after Disk 6 was removed - it also shows the Disk 7 errors that accumulated when I tried to pull data from it The picture entitled 'Capture1' shows the new drive identifiers and the 'Unmountable: Unsupported or no file system' errors The picture entitled 'HistoricalDevices' as they are currently shown. The 2 problems drives are circled in Red The picture entitled 'Capture3' shows the drive descriptors as seen by the unBALANCE plugin - taken before the rebuild Thanks! primarytower-diagnostics-20240405-1935.zip Quote Link to comment
Solution JorgeB Posted April 6 Solution Share Posted April 6 Check filesystem on both disks, run it without -n, and if it asks for -L use it. Quote Link to comment
Xandros Posted April 6 Author Share Posted April 6 Thanks for the quick reply JorgeB! Here is the output for Disk 6. I removed the -n and I was not prompted for the -L Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... - scan filesystem freespace and inode maps... sb_fdblocks 21167515, counted 9485879 - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - agno = 6 - agno = 7 - agno = 8 - agno = 9 - agno = 10 - agno = 11 - agno = 12 - agno = 13 - agno = 14 - agno = 15 - agno = 16 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 2 - agno = 5 - agno = 1 - agno = 4 - agno = 6 - agno = 7 - agno = 3 - agno = 8 - agno = 9 - agno = 10 - agno = 11 - agno = 12 - agno = 13 - agno = 14 - agno = 15 - agno = 16 Phase 5 - rebuild AG headers and trees... - reset superblock... Phase 6 - check inode connectivity... - resetting contents of realtime bitmap and summary inodes - traversing filesystem ... - traversal finished ... - moving disconnected inodes to lost+found ... Phase 7 - verify and correct link counts... Maximum metadata LSN (1:1218998) is ahead of log (1:1218678). Format log to cycle 4. done When I try to run it on Disk 7, I get the info as shown in the attached image - I did not proceed with running anything on Disk 7 - I would like to get some feedback on the following question first: Should I not bother with trying to get Disk 7 back online and instead try to get Disk 6 back online and remove Disk 7 - if I can get the array back into a functional yet degraded state (with hopefully data recovered from Disk 6) - I can try completing a full backup, then replace Disk 7 with another 8TB drive that I have on hand and rebuild parity at that point. I can take Disk 7, completely re-format it and add it back into the Array for additional capacity Any suggestions would be greatly appreciated! Thanks! Quote Link to comment
JorgeB Posted April 6 Share Posted April 6 18 minutes ago, Xandros said: Should I not bother with trying to get Disk 7 back online Not sure what you mean, the disk is already online, only unmountable, you should just use -L, basically is the only option, and usually it's a not a problem. Quote Link to comment
Xandros Posted April 6 Author Share Posted April 6 I see that was a bit confusing... I'll re-word it Should I not bother with trying to get Disk 7 to participate in the array, rather instead try to get Disk 6 back participating in the array and physically remove Disk 7 - if I can get the array back into a functional yet degraded state (with hopefully data recovered from Disk 6) - I can try completing a full backup, then after a successful backup, replace the slot that was previously occupied by Disk 7 with another 8TB drive that I have on hand and rebuild parity at that point. Once all that is done, then I can take the old corrupt Disk 7, completely re-format it and add it back into the Array for additional capacity if it is still viable Quote Link to comment
Xandros Posted April 6 Author Share Posted April 6 Or I can just go ahead with your recommendation... I think I may just be a little over-cautious as I'm not as familiar/comfortable with Unraid as you are Quote Link to comment
Xandros Posted April 6 Author Share Posted April 6 Ok, I ran it on Disk 7 - here is the output. I am about to try bringing the array up now. Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... ALERT: The filesystem has valuable metadata changes in a log which is being destroyed because the -L option was used. - scan filesystem freespace and inode maps... clearing needsrepair flag and regenerating metadata sb_fdblocks 3775398, counted 17402022 - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - agno = 6 - agno = 7 - agno = 8 - agno = 9 - agno = 10 - agno = 11 - agno = 12 - agno = 13 - agno = 14 - agno = 15 - agno = 16 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 1 - agno = 7 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - agno = 6 - agno = 8 - agno = 9 - agno = 10 - agno = 11 - agno = 12 - agno = 13 - agno = 14 - agno = 15 - agno = 16 clearing reflink flag on inodes when possible Phase 5 - rebuild AG headers and trees... - reset superblock... Phase 6 - check inode connectivity... - resetting contents of realtime bitmap and summary inodes - traversing filesystem ... - traversal finished ... - moving disconnected inodes to lost+found ... Phase 7 - verify and correct link counts... Maximum metadata LSN (1:456746) is ahead of log (1:2). Format log to cycle 4. done Quote Link to comment
itimpi Posted April 6 Share Posted April 6 Does in now mount and show expected contents when array started in normal mode? Quote Link to comment
Xandros Posted April 6 Author Share Posted April 6 JorgeB, I can't thank you enough! All my data is back as far as I can tell at this point... now I know a little bit more about Unraid and can be a bit more self-sufficient! Have a great day! 1 Quote Link to comment
Xandros Posted April 6 Author Share Posted April 6 4 minutes ago, itimpi said: Does in now mount and show expected contents when array started in normal mode? Thanks for checking in itimpi, the issue seems to be resolved now Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.