razr

Ah, great! Thank you so much! So I assume there is nothing wrong with my data now. Is there a way to tell UNRAID, that it should re-add the disks to the array and just start up without doing any rebuilds? I would probably start a new parity sync afterwards, just to make sure the parity disks are up to date again.

Unraid Version 6.9.2 8x 8TB Seagate IronWolf in Array (2 Parity disks) 2x 480GB Marvell based SanDisk SSDs as Cache 1x 500GB Samsung NVMe mainly for Docker 1x 6TB external HDD with some arbitrary data Diagnostics zip attached! TL;DR Two of my array disks were disabled shortly after each other in the last week. One of those disks had the same issue two weeks back. Back then I removed and re-added it to the array and restarted the parity sync (as it was a parity disk). As it happened again now with multiple disks, I'd like to figure out if there is actually an issue here. How can I find out what the actual problem is (SATA cable, HBA, actually failing disk, ...)? The whole story: Three or four weeks back my OS flash drive failed. Before the server was running for more than two years without any major issues. I replaced the drive with a new one, did a re-setup of the OS, re-created the array and everything worked out for about a week. Then all of a sudden one of my parity drives failed. After some reading in the wiki and forums I removed from and re-added the disk to the array and restarted the parity sync. Which succeeded after about 12 hours. Beginning of this week I tried to access some files and saw, that now two of my disks are marked as _disabled_: One is the same parity disk, that was disabled two weeks back and the other is a data disk. Unfortunately I have no physical access to the machine until beginning of next week. And even more unfortunately my VPN app runs as a Docker container on the server. So powering it down or stopping the array would cut me off from accessing my network (yes, I need to fix that). So I kept it running. To prevent more damage being done (by writing data to the array) I stopped everything that does any writing on the array. My containers are running on an extra NVMe. Some read data from the array, but they are stopped now. My question now is: How do I proceed? How can I find out what exactly the problem is? I had a look at the diagnostics files myself and I saw a couple of read errors on disks 4, 5 and 6. If I remember correctly those three and the second parity disk (the one, that is disabled right now) are connected to my HBA (Dell Perc H200). I did not see any errors on the first parity disk or disks 1, 2 or 3. Which are all connected to the motherboard directly. Maybe that could be a hint? I tried to do a _SMART extended self-test_ on those disks, but it stopped at 10%. I don't see any errors. In the logs it just says _spinning down /dev/sdX_ shortly after. Would be great if someone could help me out here. Thanks in advance! coruscant-diagnostics-20220218-2153.zip

This is exactly what I did. I re-installed the OS on a new USB flash drive, booted with it and renewed my registration key. Afterwards I recreated the array in the same configuraion I had before and everything worked out fine!

A couple of days ago my server started acting weird. Last weekend I wanted to start a VM just to find out, that the VM menu is gone from the WebUI. Also no running Docker containers were shown. When accessing the Docker menu or the Tools, it just kept loading. There was the info, that a new version is available. The server was running for about a year. So I thought I restart and then install the new version. But the server never came back up. When I connected a display I was greeted with some message like "please plug in a disk with a working OS and restart". So I searched through the forums and found out that my flash drive might have an issue. So I plugged it into my notebook and checked dmesg and found a couple of these errors: [32236.970776] critical medium error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 [32236.970796] Buffer I/O error on dev sdb, logical block 0, async page read [32236.970851] ldm_validate_partition_table(): Disk read failed. [32236.975692] sd 1:0:0:0: [sdb] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s [32236.975701] sd 1:0:0:0: [sdb] tag#0 Sense Key : Medium Error [current] [32236.975706] sd 1:0:0:0: [sdb] tag#0 Add. Sense: Unrecovered read error [32236.975711] sd 1:0:0:0: [sdb] tag#0 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00 [32236.975714] critical medium error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 [32236.975730] Buffer I/O error on dev sdb, logical block 0, async page read [32236.979372] sd 1:0:0:0: [sdb] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s [32236.979384] sd 1:0:0:0: [sdb] tag#0 Sense Key : Medium Error [current] [32236.979392] sd 1:0:0:0: [sdb] tag#0 Add. Sense: Unrecovered read error [32236.979399] sd 1:0:0:0: [sdb] tag#0 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00 [32236.979404] critical medium error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 [32236.979419] Buffer I/O error on dev sdb, logical block 0, async page read They appear only on two sectors though (0 and 30031744). But I cannot mount the drive anymore. And I cannot run `fsck`. So I guess the drive is gone, right? Of course I have no working backup of it (because I'm an idiot). I checked the wiki and found this article, that says I should install it to a new flash device and put my registration key on it (I still have that one at least...). For the array and cache it says to ensure no data loss after the server is booted, you will need to make sure you assign each disk to the array / cache exactly as it was prior to the failure. Fortunately I did physically mark the disks with their position in the array (Parity, Disk 1, Disk 2, ...). With these markings and the serial numbers I hope I can get it back up. The two cache disks were mirrored. I have no idea which way around. But I hope this does not really matter. When I touched the flash drive it was pretty hot. Which might be (one of) the reason(s) the flash drive failed? It is a 16GB SanDisk Ultra Fit USB 3. I thought about getting a SanDisk Cruzer Fit, which is USB 2.0, as I read they are not getting as hot as the USB 3 ones. And since the OS is pulled into memory anyways, the speed should also not really make a difference. Is there anything else I need to consider? Besides creating regular backups next time... Thanks in advance! PS: The link in the bottom to the Wiki article "Replace Registration Key" is wrong. It leads me to https://wiki.unraid.net/UnRAID_6/Changing_The_Flash_Device when I think it should be https://wiki.unraid.net/Manual/Changing_The_Flash_Device.