November 4, 20241 yr Unraid version: Version 6.12.13 I have a cache pool consisting of two 4 TB WD Red SN700 NVMe SSD. It's formatted as a ZFS RAID0 (striped) if I remember correctly. I now get the following error: pool: cache state: SUSPENDED status: One or more devices are faulted in response to IO failures. action: Make sure the affected devices are connected, then run 'zpool clear'. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-HC config: NAME STATE READ WRITE CKSUM cache ONLINE 0 0 0 /dev/nvme4n1p1 ONLINE 0 0 0 /dev/nvme5n1p1 ONLINE 75 3.03G 0 errors: 21 data errors, use '-v' for a list I have not yet rebooted. I am trying to obtain the diagnostics file but it is stuck in the following state: I realize that I have probably lost whatever was on the drives but I am not sure if one drive is completely dead and what I should do. They have both been in use for a few months and I have probably written ~20 TB to each.
November 5, 20241 yr Community Expert A zfs pool will suspend if it loses more devices than redundancy can handle, in this case since there's no redundancy, it will suspend after a missing device. Probably the NVMe device dropped offline, see if you can get the syslog: cp /var/log/syslog /boot/syslog.txt Then power cycle the server, not reboot, but note that because of the suspended zfs pool, you will need to force the shutdown, then power back on and if the device comes back online, the pool should work again
November 5, 20241 yr Author I have the syslog now. Can I post it as it is or is there something I should edit/remove?
November 5, 20241 yr Community Expert Solution The NVMe device dropped, assuming it comes back after a power cycle, try this. Some NVMe devices have issues with power states on Linux, try this, on the main GUI page click on the flash drive, scroll down to "Syslinux Configuration", make sure it's set to "menu view" (top right) and add this to your default boot option, after "append initrd=/bzroot" nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off e.g.: append initrd=/bzroot nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off Reboot and see if it makes a difference.
November 7, 20241 yr Author I really appreciate your help! I postponed the power cycling because I wanted something else on the server to finish. In the meantime, I can no longer access the WebGUI. When I log into my modem/router, I can still see the server connected to the network with an unchanged IP address (I used the IP to connect to the WebGUI). I also seem to be able to ping it: What do you advise me to do? Edited November 7, 20241 yr by Di4rnu
November 7, 20241 yr Community Expert Do a quick press on the power button, if it doesn't power off after 5 minutes, you will need to force it.
November 7, 20241 yr Author 2 hours ago, JorgeB said: Do a quick press on the power button, if it doesn't power off after 5 minutes, you will need to force it. I had to do a forced shutdown On 11/5/2024 at 7:04 PM, JorgeB said: The NVMe device dropped, assuming it comes back after a power cycle, try this. Some NVMe devices have issues with power states on Linux, try this, on the main GUI page click on the flash drive, scroll down to "Syslinux Configuration", make sure it's set to "menu view" (top right) and add this to your default boot option, after "append initrd=/bzroot" nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off e.g.: append initrd=/bzroot nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off Reboot and see if it makes a difference. The line with "append initrd=/bzroot" currently reads initrd=/bzroot,/bzroot-gui Which of the following (if any) should I change it to: append initrd=/bzroot nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off append initrd=/bzroot,/bzroot-gui nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off append initrd=/bzroot nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off,/bzroot-gui Edited November 7, 20241 yr by Di4rnu
November 8, 20241 yr Community Expert 12 hours ago, Di4rnu said: initrd=/bzroot,/bzroot-gui That the GUI boot option, is that your default boot option?
November 8, 20241 yr Author The default boot option should be for network access via WebUI. That's also how I just booted. So I guess the answer is not. The GUI boot option is for a monitor attached directly to the server, right? Eventually I will switch to that option but not yet. Edited November 8, 20241 yr by Di4rnu
November 8, 20241 yr Author Ahh sorry! I completely overlooked the green box! 🤦♂️ On 11/5/2024 at 7:04 PM, JorgeB said: The NVMe device dropped, assuming it comes back after a power cycle, try this. Some NVMe devices have issues with power states on Linux, try this, on the main GUI page click on the flash drive, scroll down to "Syslinux Configuration", make sure it's set to "menu view" (top right) and add this to your default boot option, after "append initrd=/bzroot" nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off e.g.: append initrd=/bzroot nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off Reboot and see if it makes a difference. The drive was back up and running after the forced shutdown. I then implemented the above changes and rebooted. It is still running fine. Should I post diagnostics or just wait and see if the problem occurs again? Edited November 9, 20241 yr by Di4rnu
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.