August 19, 20241 yr Hi guys, I got quite unlucky yesterday with my server, 2 drives showed up as disabled with no prior warning from unraid or read/write errors. I thought there might be something wrong didn't think the drives would die that suddenly so I rebooted the server to see if they popped up and it showed them as "Device is missing, content emulated". As I'm getting replacements for these disks another drive (Disk 13) started getting read errors and its got up to 10k now. After doing some research I decided to use unbalanced plugin to transfer data from the failed disks to the functional ones (excluding the one with errors) unfortunately this will take a few days to finish as the disks had 7tb of data each. The plan is to transfer the data, make a new config and rebuild parity then add the new replacement disks once they arrive. The content on the disks is not irreplaceable I've got a backup I can go back to, I just need some advice with regards to what is the best way of doing it. Should I use unbalance on the disk with errors as well, or should I get a new copy of what's in it? I made "ls -R" txt documents for each disk individually so I know what was in them in case another one dies on me, I know whats in it and I can get a back up. The problem is time... it took a while to transfer that data there in the first place I only have the diagnostics of the server after reboot, not sure if it can tell you much about what had happened. You guys are awesome thanks for helping out newbies like me! olympus-diagnostics-20240819-1219.zip
August 19, 20241 yr Community Expert Looks more like a power/connection issue, check cables for affected disks, especially if they share something in common, and post new diags after array start
August 19, 20241 yr Author 2 hours ago, JorgeB said: Looks more like a power/connection issue, check cables for affected disks, especially if they share something in common, and post new diags after array start Thank you for your prompt response! 2 days ago I installed an Intel ArcA310 GPU to my server, it didn't cross my mind that it would affect the power delivery to the drives because right after the installation everything seemed to be working fine. I moved the hard drives to the front bay and it seems to have fixed the reading errors! Im hoping it doesn't happen again. the 2 drives that failed are now recognizable by the server, I had deleted their configs when they were first disabled thinking Im gonna have to replace them so unraid did not automatically mount them. I know I screwed up as I should have waited, Ive got a lot to learn...They are popping up in UD as new drives (Dev 1 and Dev 2) but are unformattable, preclear doesn't work on them either. Is there any hope for them? New diagnostics attached after starting the array olympus-diagnostics-20240819-1443.zip Edited August 19, 20241 yr by lqpnote
August 19, 20241 yr Community Expert 13 minutes ago, lqpnote said: I had deleted their configs when they were first disabled What do mean by this?
August 19, 20241 yr Author Sorry shouldve been clearer. I was talking about the config files under "Historical Unassigned Devices" without them when I stop the array unraid does not know where they go
August 19, 20241 yr Community Expert 4 minutes ago, lqpnote said: I was talking about the config files under "Historical Unassigned Devices" without them when I stop the array unraid does not know where they go That's from the UD plugin and unrelated to array operations. Strange part is UD not offering to mount those disks, post the output of: smartctl -x /dev/sdk
August 19, 20241 yr Author 12 minutes ago, JorgeB said: That's from the UD plugin and unrelated to array operations. Strange part is UD not offering to mount those disks, post the output of: smartctl -x /dev/sdk smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.9.5-thor-Unraid+] (local build) Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Vendor: WDC Product: WUH721414AL4204 Revision: C240 Compliance: SPC-4 User Capacity: 14,000,519,643,136 bytes [14.0 TB] Logical block size: 4096 bytes LU is fully provisioned Rotation Rate: 7200 rpm Form Factor: 3.5 inches Logical Unit id: 0x5000cca264979aac Serial number: 9RJPBP8C Device type: disk Transport protocol: SAS (SPL-4) Local Time is: Mon Aug 19 08:11:03 2024 PDT device Test Unit Ready [medium or hardware error (serious)] A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options. I added -T permissive to the command this came out smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.9.5-thor-Unraid+] (local build) Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Vendor: WDC Product: WUH721414AL4204 Revision: C240 Compliance: SPC-4 User Capacity: 14,000,519,643,136 bytes [14.0 TB] Logical block size: 4096 bytes LU is fully provisioned Rotation Rate: 7200 rpm Form Factor: 3.5 inches Logical Unit id: 0x5000cca264979aac Serial number: 9RJPBP8C Device type: disk Transport protocol: SAS (SPL-4) Local Time is: Mon Aug 19 08:12:23 2024 PDT device Test Unit Ready [medium or hardware error (serious)] SMART support is: Available - device has SMART capability. SMART support is: Enabled Temperature Warning: Enabled Read Cache is: Enabled Writeback Cache is: Enabled === START OF READ SMART DATA SECTION === SMART Health Status: OK Current Drive Temperature: 0 C Drive Trip Temperature: 0 C Elements in grown defect list: 4
August 19, 20241 yr Community Expert 15 minutes ago, lqpnote said: device Test Unit Ready [medium or hardware error (serious)] This is not normal, does the other one show the same? Also post the output of: blkid /dev/sdk and fdisk -l /dev/sdk
August 19, 20241 yr Author 7 minutes ago, JorgeB said: blkid /dev/sdk This has no output at all Quote blkid /dev/sdaa (Other drive) No output Quote fdisk -l /dev/sdk fdisk: cannot open /dev/sdk: Input/output error Quote fdisk -l /dev/sdaa fdisk: cannot open /dev/sdaa: Input/output error Quote smartctl -x /dev/sdk smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.9.5-thor-Unraid+] (local build) Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Vendor: WDC Product: WUH721414AL4204 Revision: C240 Compliance: SPC-4 User Capacity: 14,000,519,643,136 bytes [14.0 TB] Logical block size: 4096 bytes LU is fully provisioned Rotation Rate: 7200 rpm Form Factor: 3.5 inches Logical Unit id: 0x5000cca264a46318 Serial number: 9RJXDM7L Device type: disk Transport protocol: SAS (SPL-4) Local Time is: Mon Aug 19 08:47:54 2024 PDT device Test Unit Ready [medium or hardware error (serious)] A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options. root@Olympus:~#
August 19, 20241 yr Community Expert Looks like there may really is a problem with those disks, try swapping cables/slots with other two disks and post new diags.
August 19, 20241 yr Author 1 hour ago, JorgeB said: Looks like there may really is a problem with those disks, try swapping cables/slots with other two disks and post new diags. I've tried swapping the cables again and got the same results, these must be truly dead. Unfortunate [medium or hardware error (serious)] This error keeps coming up Edited August 19, 20241 yr by lqpnote
August 19, 20241 yr Community Expert Are the emulated disks mounting correctly? If you start the array with those disks unassigned.
August 19, 20241 yr Author 4 minutes ago, JorgeB said: Are the emulated disks mounting correctly? If you start the array with those disks unassigned. Yes. It says contents are emulated and I can access them through /mnt/disk9 and /mnt/disk14/ Its just weird because they both went from being healthy to dead within a few minutes!
August 20, 20241 yr Community Expert It is kind of strange, just to make sure we are not seeing some weird SAS stuff, try rebuilding one of them on top.
August 21, 20241 yr Author Hi, Just updating this thread in case someone else encounters this issue I hope that helps you~ I'm fairly new to this, This issue popped up for me after doing some upgrades to my server and I think that's what triggered the errors. I did not think that the upgrades caused the issues because they didn't pop up instantly after booting the server. I understand the mistakes I did might be silly lol please bare with me.. Timeline: about a week ago I upgraded my server CPUs from E5-2630v3 to E5-2699v4 everything worked perfectly - no issues at all.. 5 days later I installed an Intel Arc310 things still worked fine - a day later 2 drives went from being healthy to disabled - no prior warnings. Rebooting the server got the HDDs to show up as "not installed contents emulated" The drives would not format, not pre-clear.. nothing I still need to figure out if they're completely dead.. @JorgeBgot me thinking about hardware issues and I thought maybe the new parts are causing issues.. went back to the motherboards manual to find out that the current ram set-up is not compatible with the new cpus or at least that's what I think fried the drives. My current ram set up is 4x8gb 2133mhz, manual states the v4 generation require 2400mhz. I understand Im getting a bit beyond unraid support, but I would really appreciate some help when it comes to Ram slotting https://prnt.sc/AfQlpeV9Dvio Here it says where the slots should be placed however, there is another chart saying how to populate the modules? https://prnt.sc/AGku2WcmRe2d I cant really figure out what they mean by this chart https://prnt.sc/b_4Mr8XaXNqX This is a picture of the motherboard appreciate any help. Thanks!
August 22, 20241 yr Community Expert 19 hours ago, lqpnote said: manual states the v4 generation require 2400mhz It won't be required, it's supported and it will perform better, but won't be the reason for the issues. How many RAM sticks do you have?
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.