Hi, I've been running unraid now for a few years with absolutely no issues, but recently upgraded all my hardware, including adding an NVME cache drive. This worked absolutely great for about a month, but after this, my NVME keeps dropping out with the "Input/Output" error. It only happens when I actually write data too it, it'll happily sit there mounted and attached all day if I don't write anything, but as soon as something uses it, it drops out.

Steps I've taken to troubleshoot

1. Booted the hardware to Ubuntu, ran multiple gig file transfers. Worked fine and stayed stable.

2. Tried a different m.2 slot

3. Tried another NVME, different brand, different model. In both slots

4. Checked and turned off all power management for NVMEs within the motherboard BIOS

5. Swapped from BTRFS to XFS for the FileSystem

6. Appended nvme_core. default_ps_max_latency_us=0 pcie_aspm=off and separately tried: nvme_core. default_ps_max_latency_us=0 pcie_aspm=off pci=nommconf to my Boot Config

At this stage I'm thinking it's not hardware, as I initially thought, as it used to work, I've tried it with multiple NVMEs and it works on Ubuntu.

This only leaves Unraid as the issue. I'm tearing my hair out with this one, so absolutely any advice/directions would be appreciated.

I've attached my diagnostics after one of these dropout events. tower-diagnostics-20210811-0946.zip

Thanks

1 hour ago, extinct_waffle said:

Appended nvme_core. default_ps_max_latency_us=0 pcie_aspm=off and separately tried: nvme_core. default_ps_max_latency_us=0 pcie_aspm=off pci=nommconf to my Boot Config

Post a screesnhof of how you added this, it's not using it on the current diags.

Interesting, I added it using the RAW view, but it doesn't seem to have taken correctly?

Anyway, here is the screenshots, thank you!

Apologies, the "Menu View" wasn't showing properly

This should show it better (sorry I'm on mobile)

Using the menu view, put those options in front of /bzroot, on the same line, then reboot.

Sorry it's hard to tell, they are actually on the same lines. I'll try and get a better screen capture later but essentially it's this

kernel /bzimage

append initrd=/bzroot nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off

The options are not being used according to the syslog.

Let me restart, wait for it to fail, and dump some more logs in here

New Diag. Logs,

Hopefully something in there sheds some light?

tower-diagnostics-20250715-1731.zip

Jul 15 17:14:00 Tower kernel: Command line: BOOT_IMAGE=/bzimage initrd=/bzroot nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off

Now it's using them; see if it makes any difference.

Glad it's using them, but no, it's made no difference. Any data written just crashes it out.

As I'm now using XFS, this is the error in the syslog I'm getting

Jul 15 17:32:08 Tower kernel: nvme0n1p1: writeback error on inode 136, offset 163577856, sector 80011328

Post the diags showing the device dropping; the first ones posted are from 2021.

Hi, I don't know what I'm doing wrong, the last diag that I uploaded was literally from the minute after the NVME dropped out.

I restarted, wrote some data, it dropped, then I created the Diag.

I looked in the syslog and it does show the dropout, albeit with a lot of noise.

10 hours ago, extinct_waffle said:

the last diag that I uploaded was literally from the minute after the NVME dropped out.

Yes, I missed it; was looking at the end of it, if the same thing happens with a different brand/model NVMe device, I don't really have any other suggestions than using a different board if possible.