This AM I noticed none of my docker were running.  Looking at the log file I found this:

 

Apr 23 00:00:02 tower Plugin Auto Update: Checking for available plugin updates
Apr 23 00:00:05 tower Plugin Auto Update: Update available for ca.mover.tuning.plg (Not set to Auto Update)
Apr 23 00:00:05 tower Plugin Auto Update: Update available for dynamix.wireguard.plg (Not set to Auto Update)
Apr 23 00:00:06 tower Plugin Auto Update: Update available for nvidia-driver.plg (Not set to Auto Update)
Apr 23 00:00:06 tower Plugin Auto Update: Update available for parity.check.tuning.plg (Not set to Auto Update)
Apr 23 00:00:06 tower Plugin Auto Update: Update available for preclear.disk.plg (Not set to Auto Update)
Apr 23 00:00:06 tower Plugin Auto Update: Update available for unassigned.devices.plg (Not set to Auto Update)
Apr 23 00:00:06 tower Plugin Auto Update: Community Applications Plugin Auto Update finished
Apr 23 02:15:15 tower kernel: nvme nvme0: I/O 370 QID 2 timeout, aborting
Apr 23 02:15:45 tower kernel: nvme nvme0: I/O 370 QID 2 timeout, reset controller
Apr 23 02:16:15 tower kernel: nvme nvme0: I/O 14 QID 0 timeout, reset controller
Apr 23 02:18:57 tower kernel: nvme nvme0: Device not ready; aborting reset, CSTS=0x1
Apr 23 02:18:57 tower kernel: blk_update_request: I/O error, dev nvme0n1, sector 1049736304 op 0x0:(READ) flags 0x80700 phys
_seg 1 prio class 0
Apr 23 02:18:57 tower kernel: blk_update_request: I/O error, dev nvme0n1, sector 1816114528 op 0x0:(READ) flags 0x80700 phys
_seg 1 prio class 0
Apr 23 02:18:57 tower kernel: nvme nvme0: Abort status: 0x371
Apr 23 02:21:05 tower kernel: nvme nvme0: Device not ready; aborting reset, CSTS=0x1
Apr 23 02:21:05 tower kernel: nvme nvme0: Removing after probe failure status: -19
Apr 23 02:23:13 tower kernel: nvme nvme0: Device not ready; aborting reset, CSTS=0x1
Apr 23 02:23:13 tower kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme0n1p1 errs: wr 0, rd 2, flush 0, corrupt 0, gen
0
Apr 23 02:23:13 tower kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme0n1p1 errs: wr 0, rd 2, flush 0, corrupt 0, gen

 

Prior to these log entries there is no messaging about my nvme0n1p1 except when it rebooted from updated a few weeks ago.

 

I've tried to stop the array, but just get a ton of errors about trying to unmount the cache drive:

Apr 23 10:19:41 tower root: umount: /mnt/cache: target is busy.
Apr 23 10:19:41 tower emhttpd: shcmd (42946): exit status: 32
Apr 23 10:19:41 tower emhttpd: Retry unmounting disk share(s)...
Apr 23 10:19:46 tower emhttpd: Unmounting disks...
Apr 23 10:19:46 tower emhttpd: shcmd (42947): umount /mnt/cache
Apr 23 10:19:46 tower root: umount: /mnt/cache: target is busy.
Apr 23 10:19:46 tower emhttpd: shcmd (42947): exit status: 32
Apr 23 10:19:46 tower emhttpd: Retry unmounting disk share(s)...

 

I'm pretty sure the cache drive isn't going to suddenly turn on, so how should I proceed?

 

thanks

david

NVMe device dropped offline, power cycling the server (not just rebooting) should bring it back, sometimes this helps with this issue:

 

Some NVMe devices have issues with power states on Linux, try this, on the main GUI page click on flash, scroll down to "Syslinux Configuration", make sure it's set to "menu view" (on the top right) and add this to your default boot option, after "append" and before "initrd=/bzroot"

 

nvme_core.default_ps_max_latency_us=0

Reboot and see if it makes a difference.

like so:

append vfio-pci.ids=8086:10d6 nvme_core.default_ps_max_latency_us=0 initrd=/bzroot

 

👍

wonderful the cache drive didn't come back.

 

It is only a few months old.

Ok rebooted in safe mode.  added the cache drive back, I guess unraid lost track of it.  Rebooted and its up and running.

 

thanks