July 22, 20223 yr Community Expert That looks better but On 7/19/2022 at 5:02 PM, trurl said: Can't tell whether your data is OK until you start the array
July 22, 20223 yr Author Ok, I thought it was started when I took those diagnostics. It's been running all night. Just took the attached ones, thanks! tower-diagnostics-20220722-1129.zip
July 22, 20223 yr Community Expert 19 hours ago, Ramshackleton said: assign my new drive as parity Go for it. What you should see on Main - Array Devices if it is going well is lots of writes to parity, lots of reads from all other disks, and zero in the Errors column for all disks.
July 22, 20223 yr Author 2 hours ago, trurl said: Go for it. What you should see on Main - Array Devices if it is going well is lots of writes to parity, lots of reads from all other disks, and zero in the Errors column for all disks. That is pretty much what I see, but the ETA on the parity rebuild is still really long, currently hovering around 100 days. Diagnostics attached. tower-diagnostics-20220722-1427.zip
July 22, 20223 yr Community Expert Lots of this Spoiler Jul 22 13:51:46 Tower kernel: ata2: link is slow to respond, please be patient (ready=0) Jul 22 13:51:51 Tower kernel: ata2: COMRESET failed (errno=-16) Jul 22 13:51:51 Tower kernel: ata2: hard resetting link Jul 22 13:51:51 Tower kernel: ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Jul 22 13:51:52 Tower kernel: ata2.00: configured for UDMA/133 Jul 22 13:51:52 Tower kernel: ata2: EH complete Jul 22 13:51:54 Tower kernel: ata2.00: limiting speed to UDMA/100:PIO4 Jul 22 13:51:54 Tower kernel: ata2.00: exception Emask 0x52 SAct 0xf814 SErr 0x4890c00 action 0xe frozen Jul 22 13:51:54 Tower kernel: ata2.00: irq_stat 0x0c400040, interface fatal error, connection status changed Jul 22 13:51:54 Tower kernel: ata2: SError: { Proto HostInt PHYRdyChg 10B8B LinkSeq DevExch } Jul 22 13:51:54 Tower kernel: ata2.00: failed command: READ FPDMA QUEUED Jul 22 13:51:54 Tower kernel: ata2.00: cmd 60/20:10:70:73:be/00:00:d7:02:00/40 tag 2 ncq dma 16384 in Jul 22 13:51:54 Tower kernel: res 40/00:00:b0:86:10/00:00:00:00:00/40 Emask 0x52 (ATA bus error) Jul 22 13:51:54 Tower kernel: ata2.00: status: { DRDY } Jul 22 13:51:54 Tower kernel: ata2.00: failed command: READ FPDMA QUEUED Jul 22 13:51:54 Tower kernel: ata2.00: cmd 60/20:20:68:cb:46/00:00:81:01:00/40 tag 4 ncq dma 16384 in Jul 22 13:51:54 Tower kernel: res 40/00:00:b0:86:10/00:00:00:00:00/40 Emask 0x52 (ATA bus error) Jul 22 13:51:54 Tower kernel: ata2.00: status: { DRDY } Jul 22 13:51:54 Tower kernel: ata2.00: failed command: READ FPDMA QUEUED Jul 22 13:51:54 Tower kernel: ata2.00: cmd 60/d0:58:e0:82:10/03:00:00:00:00/40 tag 11 ncq dma 499712 in Jul 22 13:51:54 Tower kernel: res 40/00:00:b0:86:10/00:00:00:00:00/40 Emask 0x52 (ATA bus error) Jul 22 13:51:54 Tower kernel: ata2.00: status: { DRDY } Jul 22 13:51:54 Tower kernel: ata2.00: failed command: READ FPDMA QUEUED Jul 22 13:51:54 Tower kernel: ata2.00: cmd 60/08:60:b0:86:10/04:00:00:00:00/40 tag 12 ncq dma 528384 in Jul 22 13:51:54 Tower kernel: res 40/00:00:b0:86:10/00:00:00:00:00/40 Emask 0x52 (ATA bus error) Jul 22 13:51:54 Tower kernel: ata2.00: status: { DRDY } Jul 22 13:51:54 Tower kernel: ata2.00: failed command: READ FPDMA QUEUED Jul 22 13:51:54 Tower kernel: ata2.00: cmd 60/50:68:b8:8a:10/01:00:00:00:00/40 tag 13 ncq dma 172032 in Jul 22 13:51:54 Tower kernel: res 40/00:00:b0:86:10/00:00:00:00:00/40 Emask 0x52 (ATA bus error) Jul 22 13:51:54 Tower kernel: ata2.00: status: { DRDY } Jul 22 13:51:54 Tower kernel: ata2.00: failed command: READ FPDMA QUEUED Jul 22 13:51:54 Tower kernel: ata2.00: cmd 60/08:70:08:8c:10/04:00:00:00:00/40 tag 14 ncq dma 528384 in Jul 22 13:51:54 Tower kernel: res 40/00:00:b0:86:10/00:00:00:00:00/40 Emask 0x52 (ATA bus error) Jul 22 13:51:54 Tower kernel: ata2.00: status: { DRDY } Jul 22 13:51:54 Tower kernel: ata2.00: failed command: READ FPDMA QUEUED Jul 22 13:51:54 Tower kernel: ata2.00: cmd 60/20:78:10:90:10/00:00:00:00:00/40 tag 15 ncq dma 16384 in Jul 22 13:51:54 Tower kernel: res 40/00:00:b0:86:10/00:00:00:00:00/40 Emask 0x52 (ATA bus error) Jul 22 13:51:54 Tower kernel: ata2.00: status: { DRDY } Jul 22 13:51:54 Tower kernel: ata2: hard resetting link Disk2 Jul 21 22:34:10 Tower kernel: ata2.00: ATA-9: WDC WD80EMAZ-00WJTA0, 2SGBLGTF, 83.H0A83, max UDMA/133 You should always double check all connections when mucking about inside.
July 22, 20223 yr Author 11 minutes ago, trurl said: Lots of this Reveal hidden contents Jul 22 13:51:46 Tower kernel: ata2: link is slow to respond, please be patient (ready=0) Jul 22 13:51:51 Tower kernel: ata2: COMRESET failed (errno=-16) Jul 22 13:51:51 Tower kernel: ata2: hard resetting link Jul 22 13:51:51 Tower kernel: ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Jul 22 13:51:52 Tower kernel: ata2.00: configured for UDMA/133 Jul 22 13:51:52 Tower kernel: ata2: EH complete Jul 22 13:51:54 Tower kernel: ata2.00: limiting speed to UDMA/100:PIO4 Jul 22 13:51:54 Tower kernel: ata2.00: exception Emask 0x52 SAct 0xf814 SErr 0x4890c00 action 0xe frozen Jul 22 13:51:54 Tower kernel: ata2.00: irq_stat 0x0c400040, interface fatal error, connection status changed Jul 22 13:51:54 Tower kernel: ata2: SError: { Proto HostInt PHYRdyChg 10B8B LinkSeq DevExch } Jul 22 13:51:54 Tower kernel: ata2.00: failed command: READ FPDMA QUEUED Jul 22 13:51:54 Tower kernel: ata2.00: cmd 60/20:10:70:73:be/00:00:d7:02:00/40 tag 2 ncq dma 16384 in Jul 22 13:51:54 Tower kernel: res 40/00:00:b0:86:10/00:00:00:00:00/40 Emask 0x52 (ATA bus error) Jul 22 13:51:54 Tower kernel: ata2.00: status: { DRDY } Jul 22 13:51:54 Tower kernel: ata2.00: failed command: READ FPDMA QUEUED Jul 22 13:51:54 Tower kernel: ata2.00: cmd 60/20:20:68:cb:46/00:00:81:01:00/40 tag 4 ncq dma 16384 in Jul 22 13:51:54 Tower kernel: res 40/00:00:b0:86:10/00:00:00:00:00/40 Emask 0x52 (ATA bus error) Jul 22 13:51:54 Tower kernel: ata2.00: status: { DRDY } Jul 22 13:51:54 Tower kernel: ata2.00: failed command: READ FPDMA QUEUED Jul 22 13:51:54 Tower kernel: ata2.00: cmd 60/d0:58:e0:82:10/03:00:00:00:00/40 tag 11 ncq dma 499712 in Jul 22 13:51:54 Tower kernel: res 40/00:00:b0:86:10/00:00:00:00:00/40 Emask 0x52 (ATA bus error) Jul 22 13:51:54 Tower kernel: ata2.00: status: { DRDY } Jul 22 13:51:54 Tower kernel: ata2.00: failed command: READ FPDMA QUEUED Jul 22 13:51:54 Tower kernel: ata2.00: cmd 60/08:60:b0:86:10/04:00:00:00:00/40 tag 12 ncq dma 528384 in Jul 22 13:51:54 Tower kernel: res 40/00:00:b0:86:10/00:00:00:00:00/40 Emask 0x52 (ATA bus error) Jul 22 13:51:54 Tower kernel: ata2.00: status: { DRDY } Jul 22 13:51:54 Tower kernel: ata2.00: failed command: READ FPDMA QUEUED Jul 22 13:51:54 Tower kernel: ata2.00: cmd 60/50:68:b8:8a:10/01:00:00:00:00/40 tag 13 ncq dma 172032 in Jul 22 13:51:54 Tower kernel: res 40/00:00:b0:86:10/00:00:00:00:00/40 Emask 0x52 (ATA bus error) Jul 22 13:51:54 Tower kernel: ata2.00: status: { DRDY } Jul 22 13:51:54 Tower kernel: ata2.00: failed command: READ FPDMA QUEUED Jul 22 13:51:54 Tower kernel: ata2.00: cmd 60/08:70:08:8c:10/04:00:00:00:00/40 tag 14 ncq dma 528384 in Jul 22 13:51:54 Tower kernel: res 40/00:00:b0:86:10/00:00:00:00:00/40 Emask 0x52 (ATA bus error) Jul 22 13:51:54 Tower kernel: ata2.00: status: { DRDY } Jul 22 13:51:54 Tower kernel: ata2.00: failed command: READ FPDMA QUEUED Jul 22 13:51:54 Tower kernel: ata2.00: cmd 60/20:78:10:90:10/00:00:00:00:00/40 tag 15 ncq dma 16384 in Jul 22 13:51:54 Tower kernel: res 40/00:00:b0:86:10/00:00:00:00:00/40 Emask 0x52 (ATA bus error) Jul 22 13:51:54 Tower kernel: ata2.00: status: { DRDY } Jul 22 13:51:54 Tower kernel: ata2: hard resetting link Disk2 Jul 21 22:34:10 Tower kernel: ata2.00: ATA-9: WDC WD80EMAZ-00WJTA0, 2SGBLGTF, 83.H0A83, max UDMA/133 You should always double check all connections when mucking about inside. Ok, so LGTF is the one I said was clicking (and I can hear it doing it now as well). Note this was NOT the original parity drive. This was a data drive. It is mounted but apparently unhealthy. Is he the one slowing things down? Should I try to salvage what's on it and replace?
July 22, 20223 yr Community Expert 6 minutes ago, Ramshackleton said: Note this was NOT the original parity drive. This was a data drive 18 minutes ago, trurl said: Disk2 It passed an extended test. Are you sure all connections, both ends, SATA and power, including splitters, are good? Clicking might also be caused by marginal power. Testing that one drive might be fine powerwise but parity rebuild is going to need good power to all drives simultaneously. What can you tell us about how that drive is connected (controller, power)?
July 22, 20223 yr Community Expert 11 minutes ago, Ramshackleton said: It is mounted but apparently unhealthy. Is he the one slowing things down? Should I try to salvage what's on it and replace? It is mounted, and not disabled. Since your single parity is invalid you can't rebuild that data disk. I guess you could remove it, New Config and rebuild parity without it. Then try to mount it Unassigned and copy its data to other array disks. It has nearly 7T data, a lot to copy off but disk13 has plenty of free space. But lets get more information about how it is connected, especially power.
July 22, 20223 yr Author 1 hour ago, trurl said: It is mounted, and not disabled. Since your single parity is invalid you can't rebuild that data disk. I guess you could remove it, New Config and rebuild parity without it. Then try to mount it Unassigned and copy its data to other array disks. It has nearly 7T data, a lot to copy off but disk13 has plenty of free space. But lets get more information about how it is connected, especially power. You know, I think you're onto something - all these problems started after I moved the guts of this server to a new case/power supply. I thought this would be better than my old one, which was full of molex -> SATA power adapters. This one though does have a fair # of splitters - the drive in question is running on the same line as at least 4 other drives and several fans, might be even more drives, it's a bit of a mess. I'm going to try to clean this up and will report back. But in the meantime, I assume I'm safe to abort the parity build so I can shut down?
July 22, 20223 yr Community Expert 1 minute ago, Ramshackleton said: abort the parity build so I can shut down yes
July 22, 20223 yr Community Expert 1 hour ago, trurl said: not disabled That fact just means a write to it hasn't failed. Unless you try to write to it no reason it should get disabled. There is the case where a read failure will cause Unraid to try to get the emulated data from the parity calculation and try to write it back to the disk, which might fail. But can't happen in your situation since you have no parity to emulate anything. 6 minutes ago, Ramshackleton said: the drive in question is running on the same line as at least 4 other drives and several fans, might be even more drives, it's a bit of a mess Sounds like a mess. What is the exact model of your power supply?
July 22, 20223 yr Author 57 minutes ago, trurl said: Sounds like a mess. What is the exact model of your power supply? Corsair RM750x. I had another Corsair model before, also 750W, it was just older and didn't have modular plugs. The problem here is that I have an untested and poorly-thought-out power scheme and doesn't ship with nearly enough connectors for this many drives. In any case I will clean this up ASAP. Edited July 22, 20223 yr by Ramshackleton
July 23, 20223 yr Community Expert 1 hour ago, Ramshackleton said: modular plugs Be careful with modular power supplies. There is no standard pinout for modular PSUs and many have fried disks trying to reuse cables from other modular PSUs.
July 23, 20223 yr Community Expert Found this manual: https://www.corsair.com/corsairmedia/sys_master/productcontent/RMx_2018_Manual.pdf You shouldn't try to split the SATA power cables, they already are carrying all they should. Split the peripheral cables if you need to and use molex - SATA cables with that.
July 23, 20223 yr Author 13 hours ago, trurl said: Found this manual: https://www.corsair.com/corsairmedia/sys_master/productcontent/RMx_2018_Manual.pdf You shouldn't try to split the SATA power cables, they already are carrying all they should. Split the peripheral cables if you need to and use molex - SATA cables with that. Thank you! I'm going to re-assess and revert back. Thank you so much for your help so far! Can't tell you how much I appreciate it.
July 25, 20223 yr Author Update: Tonight, after fiddling around a bit, I decided to rip out my new power supply and replace it with the old one. After all, it's power-related, right? Long story short: drives started not showing up upon boot, some of the same ones between power supply changes. Then I realized what the problem was: these were shucked drives that previously were probably connected via a Molex -> SATA converter, but now were straight SATA from the PSU. But they were never fixed to deal with the 3.3 V pin issue. This probably wasn't my original problem, as I'd think you'd either have power or not, so it wouldn't be the problem causing the clicking/power drops. Just goes to show what changing too many things at once can do to really complicate matters. I'm too tired to address it tonight, but I'll work on it in the morning/afternoon tomorrow to see if I can get it fixed. @trurl thanks again for all your help thus far.
July 25, 20223 yr Author Update #2: All affected drives now use Molex -> SATA adapters. All drives recognized, no clicking detected. Parity rebuild underway, looking MUCH better in terms of ETA, closer to the day-ish timeframe I was expecting. Hopefully this is the end of the saga. Thanks again for all the help!
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.