Bartok27

Have a disabled drive with read errors. Should i troubleshoot or just replace drive. I have been away and came back to find my unraid Disk 2 dissabled and being emulated. In the logs there are a bunch of "Power-on or device reset occurred" errors followed by some read errors. From what i read online could be a problem with the cables or drive controller or the drive itself. The drive appears to have been disabled an emulated since May 31. Last time i had a drive fail i made some mistakes troubleshooting it and ended up not being able to rebuild it. So was looking for some advise before i screw anyting up. Since its been emulated since May 31. I assume i need to rebuild the drive either on itself or replace it? I do have a spare 14tb drive i could replace it with. Would you run any tests on the existing drive or just replace it at this point? Are there any tests that can be run to confirm the drive emulation is good before rebuilding the new drive. Since the array is up and running with the emulated drive is there any other info to collect, tests i should run or troubleshooting i should do now before stopping the system to add the new drive. Any help is greatly appreicated, Thanks tower-diagnostics-20250623-0736.zip

I was trying to do some cleanup in my media folder and right clicked on the disc1 beside one of the files thinking it would remove that file from disc1. It deleted the entire disc1 from the media folder. I stopped the array as i didnt want anything else written to the disk. Is there a way to undelete it or recover it. At this point if i pulled the drive as the little bit i could find talks about using recover software on another pc. If i can recover the folder i can then rebuild the parity as it will obviously mess up the parity if i can recover the drive. Any suggestions would be appreciate. I do not have a backup as its all just media i could download again but its 9 TB of media so if I can recover it instead that would be preffered.

Just wanted to close this off. Thanks the parity has been rebuilt and everything is working. Seem the one drive was bad and the rest is ok. I can still mount the drive on my laptop via a usb dock so will do some more research on if i can recovery any of the data. But my unraid server is back up and working thanks everyone.

I really dont want to make anything worse at this point. can i safely shutdown and restart the server to remove the old 12TB. Or should i do the new config and parity rebuild first. Dont know if its better to pull it first or rebuild then shutdown and pull it

Do i need to clear or format the new 14 TB before assigning it to the disk 3 spot. Not sure what state it is in after a partial ddrescue restore.

The remaining time is still bounces between 3000-4200 days remaining. Still not saying any read errors in ddrescue? However the smart info now states "Failing now" in the raw read error rate row. ID# ATTRIBUTE_NAME FLAGS VALUE WORST THRESH FAIL RAW_VALUE 1 Raw_Read_Error_Rate PO-R-- 012 012 016 NOW 4294942212 and ddrescue is showing GNU ddrescue 1.29.1 Press Ctrl-C to interrupt ipos: 174516 MB, non-trimmed: 0 B, current rate: 43690 B/s opos: 174516 MB, non-scraped: 0 B, average rate: 582 kB/s non-tried: 11825 GB, bad-sector: 0 B, error rate: 0 B/s rescued: 174516 MB, bad areas: 0, run time: 3d 11h 12m pct rescued: 1.45%, read errors: 0, remaining time: 3690d 10h time since last successful read: 0s Copying non-tried blocks... Pass 1 (forwards) So i assume the drive is done and probably time to give up on ddrescue? At this point i'm assume the drive is lost but thinking i will pull it and drop it in an external dock and see if i can pull anything off just to see what happens. If we assume the drive is dead and now there is no way to restore the emulated drive 5. Can we assume drive 5 (8TB) is good. It only contains media and some backups so i think i can just loose any changes that were made to the emulation. So can i add the remaining 4 drives to an array without loosing the data and the new 14TB drive to the array and rebuild the parity. If so could you point me to the correct steps to follow so i dont screw up anything else. tower-diagnostics-20250415-1313.zip Also Just for my information if i have any future issue. I wanted to confirm i took the correct steps for running ddrescue. I took the array offline (did not restart the server), set disc3 to no device so that the 12 TB drive showed up under "Unassigned Disk Device" and ran the command below (attached the current stats of the running command) root@Tower:~# ddrescue -f /dev/sdc /dev/sdb /boot/ddrescue.log On thing i did notice that was a bit weird is the drive is label dev2 and the other two devices are labelled Dev 1 and Dev 2. I assume that is just naming and has not impact but looks weird.

I've been running ddrescue hopping it would speed up but so far it should be done in just under 11 years. Really hoping this is just the bad sectors and it speeds up at some point. root@Tower:~# ddrescue -f /dev/sdc /dev/sdb /boot/ddrescue.log GNU ddrescue 1.29.1 Press Ctrl-C to interrupt ipos: 40878 MB, non-trimmed: 0 B, current rate: 32768 B/s opos: 40878 MB, non-scraped: 0 B, average rate: 571 kB/s non-tried: 11959 GB, bad-sector: 0 B, error rate: 0 B/s rescued: 40878 MB, bad areas: 0, run time: 19h 51m 18s pct rescued: 0.34%, read errors: 0, remaining time: 4010d 8h time since last successful read: 0s Copying non-tried blocks... Pass 1 (forwards)

I will try cloning it just curious why i need to upgrade to 71. before the rebuild.

Unfortunately I only have two new 14TB available for use.

I have tried several cables. But since i hadn't done it very systematically i ran through a bunch of boot tests on different cables. If i connect it to one of the onboard sata ports. I get a bunch of read errors and its listed as unmountable or no os. I tried the same cables and ports with other drives and they boot with no issues. If i connect it via the HBA card I get one I/O read error in one of 3 sectors depending on which cable i use and two device reset fails but it will mount the drive and is full acessable. So i have now tested 3 sata cable, 3 onboard sata ports, 3 HBA cables (4 sata each). So i am pretty confident the hardware seems to work. For the I/O read error i'm not entirely sure if its hitting a physically bad sector or if it just cant find the data that should be there. After doing some reading i also saw some people recommending putting a fan on you HBA card. Currently i don't have one on the HBA card but its in a node 304 case like your backup server so its pretty open with several case fans. But just to be sure i ordered an extra fan for it to try. So at this point i figure i will add the fan just in case just as an extra percaution. Then i'm open to suggestion on what you would try next. The options i can think of are : -check parity to see if it valid before trying a rebuild -run another extended smart test -try to rebuild disk5 onto itself (8TB drive) and if successful rebuild disk3 onto the 14tb drive -copy or clone the 12 to the 14, add the 14 and 8 as is to the array and rebuild parity (I'm not sure how this is done with data on all the drive drives or if it can be) Let me know if any of these options make sense or if you have a better suggestion on next steps. Thankstower-diagnostics-20250410-2318.zip

Yes i understand the problem I have got myself into and that with 2 failed drives its technically not recoverable and there could or likely will be data loss. It would also be great if i could test and confirm that disk3 was good but I don't think i have any way to confirm that 100%. Maybe if i had a spare PC or extra hardware to test with but unfortunately i don't have a great way to test that. I have replaced several cable throughout this troubleshooting and at this time i think Disc3 is as good as i can get it. Unless you have any suggestion on how i could test it further? I could try the extended smart test again but dont know if its best to try that first or just attempt to rebuild disc5. But I am definitely open to suggestions if you have any? Also not to add more complication but i am getting this error in the log now which appears to be it flushing the cache to the emulated drive5. Apr 9 17:00:01 Tower shfs: copy_file: /mnt/cache/Media/Media_Unraid/TV Shows/Land/Season 1/Land - S01E01 - Twist WEBDL-1080p.mkv /mnt/disk5/Media/Media_Unraid/TV Shows/Land/Season 1/Land - S01E01 - Twist WEBDL-1080p.mkv.partial (117) Structure needs cleaning Apr 9 17:00:01 Tower kernel: XFS (md5p1): Free Inode BTree record corruption in AG 0 detected at xfs_inobt_check_irec+0x8e/0xa7! Apr 9 17:00:01 Tower kernel: XFS (md5p1): start inode 0x7af66400, count 0x40, free 0x18 freemask 0x60000000bf8090, holemask 0x0 Does this need to be fixed before i try to rebuild drive5 if thats the route to take? If its only an issue with that folder i am fine just deleting the folder if that will fix it. So i backed up all the media that i really want off of the 8TB so its good to attempt a rebuild if you think is the best plan at this time. If i am understanding correctly to rebuild the 8tb drive it would only need to read the first 8tb worth of data off disc3 so depending what is wrong with the drive maybe it lasts that long. If it rebuilds and looks good then i would remove the 12 hope it emulates it and repeat the process on the 14. If it fails to rebuild the 8TB what options would i have: Format and mount the 14tb and copy as much as i can off the 12 onto it? Then would i be able to rebuild the array without erasing the drives? I have attached the latest diagnostics. Note the time reverted to April 1 at some point in the troubleshoot so that is actually yesterday and when i reset it to the 8th. Let me know what you suggest. Thanks for sticking with this the help and suggestions are appreciated. tower-diagnostics-20250409-1752.zip

Just want to update. I wasn't sure about the preclear and after some reading i started one last night. Looks like its still in the pre-read stage so i will cancel it if its not required. Also i was incorrect about there not being changes to Drive5. I was reading some more logs and the cache drive did get copied to the drive since it was emulated. So there is definitely a difference between the emulated version and whats on actual drive. So from my understanding adding it back to the array is out as the parity will be off. So i'm thinking i can backup the files i need on the 8TB to my external drive just in case. Then try to add the 8tb and see if it can rebuild it? I'm not sure if that will work or crash with the issues on the Disc3. If it does rebuild it then i could follow your steps to rebuild disc3 onto the new 14tb. Do you think there is a chance it can rebuild the 8TB successfully? Or would you suggest a different route? Thanks again for all the help.

Yes i have the a 14 TB thats connected as unassigned dev2 currrently and my parity is 14 so it will work. I haven't precleared it yet. So i assume i should still do that first or do you think i can skip that and just do a rebuild? Some people seem to be split on if its required

Thanks. I have to head out also. But will check the instructions when i get back.

Mounted no issues and last modify dates on all folders and the emulated drive are exactly the same. In that case should i add it back to the array? and if so what are the steps