I'm on Unraid 7.0.1, running on a AOOSTAR WTR PRO AMD NAS box. My cache pool is on two 1TB NVME SSD drives using BTRFS RAID1. Earlier today all my VMs and containers died and looking at the logs I see seems related to issues on one of the drives. Lots of these:

Oct 24 16:42:28 Tower kernel: BTRFS error (device nvme1n1p1: state EA): bdev /dev/nvme0n1p1 errs: wr 13726102, rd 291197, flush 80309, corrupt 802103, gen 0

It's a bit frustrating that I relatively recently added a second drive to my cache so I had redundancy, but this is now the second time that errors reported on just one drive seems to stop everything using the cache pool.

The Pool Devices section shows both drives as normal, active.

SMART shows no errors, though the pool device stats shows a massive amount of errors for the one drive.

Id Path           Write errors Read errors Flush errors Corruption errors Generation errors
-- -------------- ------------ ----------- ------------ ----------------- -----------------
 2 /dev/nvme1n1p1            0           0            0                 0                 0
 4 /dev/nvme0n1p1     13732034      291232        80309            802103                 0

This was the same situation I had a few months ago with another drive. This led me to believe it was a hardware issue, so I replaced the drive. This error is now for the replaced drive (a WD Blue 750), so I'm starting to think maybe these aren't hardware errors unless I've been incredibly unlucky. Could it be an issue with the M.2 slot maybe?

Here's the btrfs filesystem usage:

Overall:
    Device size:		   1.84TiB
    Device allocated:		 774.06GiB
    Device unallocated:		   1.08TiB
    Device missing:		     0.00B
    Device slack:		     0.00B
    Used:			 699.04GiB
    Free (estimated):		 591.96GiB	(min: 591.96GiB)
    Free (statfs, df):		 580.78GiB
    Data ratio:			      2.00
    Metadata ratio:		      2.00
    Global reserve:		 512.00MiB	(used: 0.00B)
    Multiple profiles:		        no

                  Data      Metadata System                              
Id Path           RAID1     RAID1    RAID1    Unallocated Total     Slack
-- -------------- --------- -------- -------- ----------- --------- -----
 2 /dev/nvme1n1p1 384.00GiB  3.00GiB 32.00MiB   566.84GiB 953.87GiB     -
 4 /dev/nvme0n1p1 384.00GiB  3.00GiB 32.00MiB   544.48GiB 931.51GiB     -
-- -------------- --------- -------- -------- ----------- --------- -----
   Total          384.00GiB  3.00GiB 32.00MiB     1.08TiB   1.84TiB 0.00B
   Used           347.70GiB  1.82GiB 80.00KiB                            

I can't help thinking there's something fundamentally wrong with my setup, as I would rather hope an error on just one of my RAID1 cache drives would flag an issue, but allow everything to continue on one drive.

Would be very grateful for any help to get my VMs back online as soon as possible, and then maybe understand what's going wrong here. At the moment I'd be happy to go back to my old cache arrangement of one drive that I backed up regularly, as that seemed much more fault tolerant than this RAID1 setup!

Have attached a diagnostics zip in case it's useful.

tower-diagnostics-20251024-1709.zip

Edited October 24, 2025Oct 24 by Richard Harnwell
added extra info.

Desperate to get my VMs (Blue Iris and Home Assistant) up and running again, so I've taken to seeking some help from CoPilot, while being very cautious with it's suggestions.

To try to get to a clean state I've ended up restarting, after having to manually kill off the two VM processes (the VMs tab was just blank so couldn't be done in the gui) to allow the array to stop. Now it's come back (array still stopped) I see the problem SSD shows as "missing". I remember this is exactly what happened last time. Back then I shutdown, reseated drive, powered up and it appeared, but I then ended up replacing the drive. This time round it seems unlikely the new drive has failed again. and I see there is a known error with NVME drive dropouts after boot in 7.0.1.

I'm thinking I should maybe stabilise the system before upgrading, and trying to get it running on just the one "good" cache drive. Unfortunately when I try to start the array with just one (or even none) cache drive it refuses.

Syslogs are already rotated, so can't see the beginning of the problem. Reboot and post new diags after array start.

Thanks. I have it up and running, on the "good" drive now. After the reboot I tried starting the array with just that one pool/cache drive, and Unraid refused as mentioned. I then tried it with no pool, and it refused, but interestingly when I then (for no good reason) tried again with just the one drive, it started up this time. All VMs and containers looked OK, so I took backups of everything and have left it in that state for now. I would love to understand why this has happened twice for me (on two different drives), especially why BTRFS RAID1 couldn't cope with a disk issue, but my main focus was to get Blue Iris and Home Assistant up and running again.

I see there is a known issue in 7.0.1 (that I'm on) with VVME drives disappearing. Looks like this is fixed in 7.2, so I'm thinking I should wait for the public release of that and upgrade. https://forums.unraid.net/bug-reports/prereleases/72-beta-3-request-kernel-bug-fix-for-nvme-ssd-reset-r4044/ That would seems like a good time to try to get the second drive re-introduced to the pool.

Would be very grateful for any thoughts on what might be happening or anything I can do to help with the robustness of my setup. I have taken another diagnostics dump - the log file appears to start straight after the reboot last night.

tower-diagnostics-20251025-1154.zip

Oct 24 04:40:04 Tower kernel: BTRFS warning (device nvme1n1p1): lost page write due to IO error on /dev/nvme0n1p1 (-5)

One of the devices dropped offline; the newer kernel on 7.2 may help with that. In some cases, adding nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off to syslinux.cfg also helps.

On 10/25/2025 at 12:41 PM, JorgeB said:

Oct 24 04:40:04 Tower kernel: BTRFS warning (device nvme1n1p1): lost page write due to IO error on /dev/nvme0n1p1 (-5)

One of the devices dropped offline; the newer kernel on 7.2 may help with that. In some cases, adding nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off to syslinux.cfg also helps.

Thank you (sorry, didn't see the notification for this at the time).

So, it looks like the above entry is disabling various low power states? Not sure whether to try that or wait a bit longer for 7.2 update when it comes....

It disables ASPM for PCIe devices.