FearlessUser

Awesome, I’ll take it. Much better than losing almost 24tb of shows and movies. I have them on disc, but the time it would take to restore that route is not ideal. Looked at several folders and didn’t see any naming issues yet, hopefully that’s the case for what’s left too. And now I really want to build a backup… Thank you

The "Fix" button was super quick, only took a few seconds for each drive. Checked the filesystem on both again and they said they were fine. Restarted the array in regular mode, and its doing a parity check. I see a bunch of my content is in some "lost+found" folders, am I good just moving them back to restore those files? Diagnostics added again if anyone wants to see them omega-diagnostics-20250808-1449.zip

Went to bed at a normal time last night and I’m off to work now so I’ll do that this afternoon when I get home, thanks

Rebuild finished. Stopped array and started in maintenance mode. Drive 1 says this when I check the filesystem status Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... - scan filesystem freespace and inode maps... - found root inode chunk Phase 3 - for each AG... - scan (but don't clear) agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 bad CRC for inode 128 bad CRC for inode 128, would rewrite would clear root inode 128 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - agno = 6 - agno = 7 - agno = 8 - agno = 9 - agno = 10 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 1 - agno = 5 bad CRC for inode 128, would rewrite - agno = 4 would clear root inode 128 - agno = 2 - agno = 3 - agno = 6 - agno = 7 - agno = 8 - agno = 9 - agno = 10 No modify flag set, skipping phase 5 Phase 6 - check inode connectivity... would reinitialize root directory - traversing filesystem ... Metadata corruption detected at 0x488950, inode 0x80 dinode couldn't map inode 128, err = 117 - traversal finished ... - moving disconnected inodes to lost+found ... disconnected dir inode 131, would move to lost+found disconnected dir inode 132, would move to lost+found disconnected dir inode 12956713321, would move to lost+found Phase 7 - verify link counts... Metadata corruption detected at 0x488950, inode 0x80 dinode couldn't map inode 128, err = 117, can't compare link counts No modify flag set, skipping filesystem flush and exiting.Drive 2 says the same Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... - scan filesystem freespace and inode maps... - found root inode chunk Phase 3 - for each AG... - scan (but don't clear) agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 bad CRC for inode 128 bad CRC for inode 128, would rewrite Bad ctime nsec 1077710368 on inode 128, would reset to zero would clear root inode 128 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - agno = 6 - agno = 7 - agno = 8 - agno = 9 - agno = 10 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 1 bad CRC for inode 128, would rewrite Bad ctime nsec 1077710368 on inode 128, would reset to zero would clear root inode 128 - agno = 2 - agno = 4 - agno = 6 - agno = 5 - agno = 3 - agno = 7 - agno = 8 - agno = 9 - agno = 10 No modify flag set, skipping phase 5 Phase 6 - check inode connectivity... would reinitialize root directory - traversing filesystem ... Metadata corruption detected at 0x488950, inode 0x80 dinode couldn't map inode 128, err = 117 - traversal finished ... - moving disconnected inodes to lost+found ... disconnected dir inode 131, would move to lost+found disconnected dir inode 191, would move to lost+found Phase 7 - verify link counts... Metadata corruption detected at 0x488950, inode 0x80 dinode couldn't map inode 128, err = 117, can't compare link counts No modify flag set, skipping filesystem flush and exiting.I'm left with a button on both that says "Fix" after it did the file system check on both drives, followed by "file system corruption detected". The link provided doesn't say what will happen if I hit that. I really just want to know if that's safe to do and will fix the issue, or if there is another route I should take. New diagnostics attached as well omega-diagnostics-20250807-1838.zip

Gotcha. It's around 50% with the rebuild when I woke up so I'll wait for that to finish and do the check later today

Sorry if this is a dumb question or if I'm just tired, but I added the drives to the server and immediately went to rebuild them, no preclear done per the response earlier this week. The new drive wouldn't have a file system to check, correct? And a format wouldn't clear any data on the disk because they're new drives, right? Apart from the 15% or so from the rebuild anyway.

1 and 2 were the ones that failed and were replaced. They've had a history of issues. Do it on the replacement drives, or the old drives?

New diagnostics, thank you omega-diagnostics-20250807-0023.zip

So I got the new drives today and plugged them in. Added the drives to the array and started it. Now it says it's working on a data rebuild, but the drives both say "unmountable, wrong or no filesystem". Is this normal until the rebuild completes (because they're fresh drives), or do I need to format them before adding them to the array?

Awesome that will save time. Thank you

I've had on and off again issues with 2 drives. Every time parity check would run, the drives would go offline with errors. It would take a few tries to rebuild, but they would eventually rebuild. Parity check ran again yesterday, and the drives failed again. Removed them from the array and added them back, and the rebuild was running up until about 80% or so. Both drives failed again. Now the dashboard shows them as emulated, but the Main tab shows them as "unmountable, wrong or no file system". I already purchased new drives today, they should show up Wednesday. Here are the diagnosticsomega-diagnostics-20250803-1455.zip I want to plan ahead for Wednesday and the next few days. I'll need to preclear the drives when I get them, then rebuild them. Is that where I'm at? I only use this for Plex. Some shows and movies won't play saying the drives aren't mounted. I'm hoping it's just because of the file system error, even though the dashboard shows them as being emulated. Is it going to be as simple as adding the new drives and rebuilding, or is there more work to do?

Rebuild took a few hours longer than estimated and completed overnight. I’m back in business. Thanks @JorgeB for the help on this, super appreciated!

Finally got around to doing this. When I started a new config, disk 1 was fine, but then disk 2 gave me 1061 errors and went offline shortly after starting the array up. These drives can't be failing in sequence like this, then be fine after adding them back to the array. I replaced these SAS to SATA cables I got from Amazon 6 years ago when I built the server. The parity drives were in different spots in the server, but these 2 data drives that were failing were on the same cable. Once it was replaced, I booted up and tried another new config, and so far so good. At this point the rebuild has been going for about 20 minutes, much longer than when the drives would almost immediately throw errors and fail. The rebuild says it should take around 30 hours, and all data drives are still showing active with no errors.. On the old cable, 3 of the 4 cables that split out to the drives were bent to a crease. Fingers crossed that's what caused the issues.

Geez I can’t believe I missed that. Long week. Thanks

2 question on this new config tool. Do I want to preserve any assignments? And how much should I worry about this message in the tool: "DO NOT USE THIS UTILITY THINKING IT WILL REBUILD A FAILED DRIVE - it will have the opposite effect of making it impossible to rebuild an existing failed drive - you have been warned!"

Sweet, nice and easy. I’ll try it this afternoon when I get home, thanks

It’s still available. Still in the server all hooked up. That would be amazing if that could fix my screw up. Is there a guide I can follow to try this out when I get home tonight?

Disk 1 was never an issue until I was trying to rebuild parity a couple of days ago. As soon as the parity rebuild started, it immediately failed. So it sounds like there’s no chance of rebuilding that drive. Do I lose the entire array or just whatever is on that disk?

Memtest ran for over 24 hours, 6 passes completed, no issues. Not the RAM. Here are the diagnostics. In the smart folder, the drive that immediately fails is the one with the SN of ZTN0ASB1, and the new parity drive is the one with the SN of ZYD1PG7P. If I try to assign drive ZTN0ASB1 to data drive 1, the "start" button for the array gets greyed out and it says "Too many wrong and/or missing disks!" next to the start button. Unfortunately I didn't really have time to try anything else other than get the diagnostics tonight. I won't be able to try anything you might come up with until after work tomorrow, around 4 Phoenix time. omega-diagnostics-20250109-2106.zip

How's this for a comedy of errors. My server mainly runs Plex. I have 6 data drives and 2 parity drives. In the last couple of months, one of the parity drives would fail during the parity check. I'd reseat the cables and rebuild the drive, and it's been fine until another parity check when it fails again. The second parity drive just failed as well in the latest parity check, so I shut the server down. I finally got 2 new drives to replace the parity drives (just bought a house, so funds were tied up in that). One drive showed up, the other is on its way. Preclear was done on the first drive, so I added it to the array to rebuild parity. Almost immediately, one of the data drives was disabled because it had over 388k errors. Along with all the other data drives, this one never had any issues in the past when parity check would run. I shut the server down again to make sure I didn't have a loose cable or anything when I was in there replacing the parity drive. Everything was good, so I booted it back up. When I started the array again, the same data drive gave me the same exact number of errors. I let parity rebuild for a bit, but the server hung up when it got to around 17% of the rebuild. Then the dashboard of the GUI stopped responding. No errors were showing in any logs. I could see the logins that I had from my desktop and laptop being logged, but nothing else. Shut down and checked every connection in the case (reseated the RAM and LSI card, power and data cables for the drives were all good still). Booted up and it hung up on the rebuild at about 0.4%. I forget the exact error I got this time, but looking it up everyone is suggesting to run memtest, which is running now. It started about 12 hours ago and I was going to let it run for another 12 or so, unless I should let it go longer. So, I'm trying to rebuild parity, but one of the data drives keeps coming up with errors. If that data drive is really bad and parity can't be rebuilt from the other drives, is that drive lost? It's "just" TV shows and movies so it's not the worst thing in the world apart from my time to get it all back on there. If there is some other hardware issue, would that data drive come back up on its own once I resolve that? Or would it try to rebuild from the parity drive, the one that won't build properly because one of the data drives keeps giving me errors? If anyone wants to see logs, I don't have them now. I can get some after memtest is done, but that will be a bit. These were just my overnight thoughts on the topic, I wanted to see what people thought of my predicament.

I am not a smart man. This was one of those "user error" thing's I've heard so much about. I had 3 USB drives that were all the same, but I swore I only had 2. One was empty, and the other one that gave me error above is what I used to flash the HBA in my server to IT mode. The 3rd drive that I should have been using was in an anti-static bag. The drives are those Kingston Data Traveler drives that are silver, matching perfectly with an anti-static bag. The first 7 times I looked at the bag, I thought it was empty. My wife even went through that box and thought the bag was empty. 9th time is the charm I guess. Moral of the story: you're not using the right drive. Check all the anti-static bags, even the ones you think are empty. And make a backup before you move cross country. It's not a good idea to not have a backup.

Just found this old post that I'll try tomorrow. Something came up and I can't work on this anymore tonight. If anyone else has other ideas, feel free to add them. I'll get back on this after work tomorrow

It moves on to the next items in the boot order. It's a dual NIC motherboard with PXE. It looks for something on both NICs and fails because I don't have it connected to the network yet. That and I'm not set up for PXE. Then it tells me to insert the boot device and hit any key. If I put the USB drive back in, it tries to go through the PXE setup again, so it looks like it's stuck looking for that at this point until I shut it down

I moved cross country about a month ago. I'm not sure if it mattered, but I spent a bit of time making sure to label where every cable and drive was plugged in when I tore it down, and made sure everything was hooked back up the same way now that I'm finally getting around to setting the server back up at the new house. The picture below is what I get when I boot it up. The USB drive is recognized by Windows just fine, and it has no issues when I run a checkdisk on it. What am I missing here? It's still set to boot from the USB drive in the BIOS, that hasn't changed. The BIOS sees the USB drive by brand and name, so it seems like the drive is fine. I made the USB key back in May of 2019, and whatever version of unraid was available at the time. I've rebooted it several times over the years for OS updates and adding new drives. When I took it down, it said the uptime was something like 305 days, so it's been about a year since it was booted up last. The last OS update I did took the server to either 6.7 or 6.8, something like that. Could the mismatch between what's on the stick and something else cause this problem?

I have some 1tb Adata drives and have the issue. I figured it was because they are kind of cheaper drives. I bought some 860s to swap them out with. If I don't use them here, I have plenty of places to use them