September 19, 20196 yr Hi, This morning I tried to copy/create/move files on a share from windows and got this error : Error 0x8000FFFF: Catastrophic failure After a short "WTF / LOL" moment caused by how the failure is described as "catastrophic", I googled it, found out that some pleople talk about bad drive, and recommend scanning the drive for error. So I try to install binhex krusader to move files away from one drive to another prior to testing said drive but got this error: Error: open /var/lib/docker/tmp/GetImageBlob969712477: read-only file system Then I went to the system log and found this: Sep 19 08:53:04 Mozart kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 1, rd 67133, flush 1, corrupt 0, gen 0 Sep 19 08:53:04 Mozart kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme0n1p1 errs: wr 17, rd 79971, flush 0, corrupt 0, gen 0 Sep 19 08:53:04 Mozart kernel: print_req_error: I/O error, dev loop2, sector 1467944 Sep 19 08:53:04 Mozart kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 1, rd 67134, flush 1, corrupt 0, gen 0 Sep 19 08:53:05 Mozart kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme0n1p1 errs: wr 17, rd 79972, flush 0, corrupt 0, gen 0 Sep 19 08:53:05 Mozart kernel: print_req_error: I/O error, dev loop2, sector 1467944 Sep 19 08:53:05 Mozart kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 1, rd 67135, flush 1, corrupt 0, gen 0 Sep 19 08:53:05 Mozart kernel: print_req_error: I/O error, dev loop2, sector 1467944 Sep 19 08:53:06 Mozart kernel: print_req_error: I/O error, dev loop2, sector 1467944 Sep 19 08:53:06 Mozart kernel: print_req_error: I/O error, dev loop2, sector 1467944 Sep 19 08:53:07 Mozart kernel: print_req_error: I/O error, dev loop2, sector 250688 Sep 19 08:53:07 Mozart kernel: print_req_error: I/O error, dev loop2, sector 250688 Sep 19 08:53:07 Mozart kernel: vethde0bffa: renamed from eth0 Sep 19 08:53:07 Mozart kernel: docker0: port 4(vethd565f3d) entered disabled state Sep 19 08:53:08 Mozart kernel: btrfs_dev_stat_print_on_error: 26 callbacks suppressed Sep 19 08:53:08 Mozart kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme0n1p1 errs: wr 17, rd 79986, flush 0, corrupt 0, gen 0 Sep 19 08:53:08 Mozart kernel: print_req_error: 8 callbacks suppressed Sep 19 08:53:08 Mozart kernel: print_req_error: I/O error, dev loop2, sector 1467944 Sep 19 08:53:08 Mozart kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 1, rd 67149, flush 1, corrupt 0, gen 0 Sep 19 08:53:08 Mozart kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme0n1p1 errs: wr 17, rd 79987, flush 0, corrupt 0, gen 0 Sep 19 08:53:08 Mozart kernel: print_req_error: I/O error, dev loop2, sector 1467944 Sep 19 08:53:08 Mozart kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 1, rd 67150, flush 1, corrupt 0, gen 0 Sep 19 08:53:09 Mozart kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme0n1p1 errs: wr 17, rd 79988, flush 0, corrupt 0, gen 0 Sep 19 08:53:09 Mozart kernel: print_req_error: I/O error, dev loop2, sector 1467944 Sep 19 08:53:09 Mozart kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 1, rd 67151, flush 1, corrupt 0, gen 0 Sep 19 08:53:09 Mozart kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme0n1p1 errs: wr 17, rd 79989, flush 0, corrupt 0, gen 0 Sep 19 08:53:09 Mozart kernel: print_req_error: I/O error, dev loop2, sector 1467944 Sep 19 08:53:09 Mozart kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 1, rd 67152, flush 1, corrupt 0, gen 0 Sep 19 08:53:10 Mozart kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme0n1p1 errs: wr 17, rd 79990, flush 0, corrupt 0, gen 0 Sep 19 08:53:10 Mozart kernel: print_req_error: I/O error, dev loop2, sector 1467944 Sep 19 08:53:10 Mozart kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 1, rd 67153, flush 1, corrupt 0, gen 0 Sep 19 08:53:10 Mozart kernel: print_req_error: I/O error, dev loop2, sector 1467944 Sep 19 08:53:11 Mozart kernel: print_req_error: I/O error, dev loop2, sector 1467944 Sep 19 08:53:11 Mozart kernel: print_req_error: I/O error, dev loop2, sector 1467944 Sep 19 08:53:12 Mozart kernel: print_req_error: I/O error, dev loop2, sector 1467944 Sep 19 08:53:12 Mozart kernel: print_req_error: I/O error, dev loop2, sector 1467944 Sep 19 08:53:13 Mozart kernel: veth457c811: renamed from eth0 Sep 19 08:53:13 Mozart kernel: docker0: port 3(vethc7faf84) entered disabled state Sep 19 09:03:13 Mozart root: Fix Common Problems Version 2019.09.08 Sep 19 09:03:17 Mozart root: Fix Common Problems: Error: Unable to write to cache Sep 19 09:03:17 Mozart root: Fix Common Problems: Error: Unable to write to Docker Image Sep 19 09:04:03 Mozart emhttpd: req (22): cmd=/webGui/scripts/disk_log&arg1=nvme0n1&csrf_token=**************** Sep 19 09:04:03 Mozart emhttpd: cmd: /usr/local/emhttp/plugins/dynamix/scripts/disk_log nvme0n1 After some more googling, I tried to see what the Fix Common Problems plugin suggests and see that : Unable to write to cache Drive mounted read-only or completely full. Begin Investigation Here: [main] Unable to write to Docker Image Docker Image either full or corrupted. Investigate Here: [docker settings] Went to the "Main" tab and see plenty of free space : Cache Devices Device Identification Temp. Reads Writes Errors FS Size Used Free Cache [...] 500 GB (nvme0n1) * 232,978 18,446,744,073,709,486,080 0 btrfs 500 GB 111 GB 389 GB I guess the question is why does the cache drive would be mounted as "read only". It has obviously been written to a lot. Many thanks for your help. Other required info is below. G My unRaid version is: 6.7.2 2019-06-25 My installed plugins are: CA Auto Update Applications CA Backup / Restore Appdata CA Cleanup Appdata CA Mover Tuning Community Applications Dynamix Active Streams Dynamix Cache Directories Dynamix Date Time Dynamix S3 Sleep Dynamix SSD TRIM Dynamix System Information Dynamix System Statistics Dynamix System Temperature Fix Common Problems Nerd Tools Preclear Disks Tips and Tweaks Unassigned Devices unBALANCE My hardware is: Supermicro X11SSH-LN4F-O Intel Pentium G4560 Kingston UDIMM ECC RAM (32 GB) Crucial P1 SSD Bunch of WD, Seagate and Hitachi HDDs Diagnostic file: Please see attached Edited October 23, 20196 yr by Opawesome typo and marked as "solved"
September 19, 20196 yr Community Expert NVMe device dropped offline: Sep 18 23:51:01 Mozart kernel: nvme nvme0: I/O 136 QID 2 timeout, aborting Sep 18 23:51:01 Mozart kernel: nvme nvme0: I/O 137 QID 2 timeout, aborting Sep 18 23:51:01 Mozart kernel: nvme nvme0: I/O 138 QID 2 timeout, aborting Sep 18 23:51:01 Mozart kernel: nvme nvme0: I/O 139 QID 2 timeout, aborting Sep 18 23:51:31 Mozart kernel: nvme nvme0: I/O 136 QID 2 timeout, reset controller Sep 18 23:52:01 Mozart kernel: nvme nvme0: I/O 0 QID 0 timeout, reset controller Sep 18 23:53:33 Mozart kernel: nvme nvme0: Device not ready; aborting reset Sep 18 23:53:33 Mozart kernel: print_req_error: I/O error, dev nvme0n1, sector 14888064 Sep 18 23:53:33 Mozart kernel: print_req_error: I/O error, dev nvme0n1, sector 14888320 Sep 18 23:53:33 Mozart kernel: print_req_error: I/O error, dev nvme0n1, sector 14895648 Sep 18 23:53:33 Mozart kernel: print_req_error: I/O error, dev nvme0n1, sector 14895904 Sep 18 23:53:33 Mozart kernel: nvme nvme0: Abort status: 0x7
September 19, 20196 yr Author 2 hours ago, johnnie.black said: NVMe device dropped offline: Hi johnnie.black, Many thanks for your answer. It seems I should have investigate better before posting. Anyway, so it means it is a hardware issue ? Bad SSD or bad motherboard I guess. The machine is hooked up to a UPS so I can discard the power outage from the list.
September 19, 20196 yr Community Expert It's a hardware problem, it can be the device or the board, assuming it's not on a PCIe adapter, it can also be some firmware/compatibility problem, but once it drops Unraid can't do anything about it.
September 29, 20196 yr Author I replaced the Crucial P1 with the Intel 660p about 5 days ago and have not noticed any problem since. So it was either a compatibility issue with the Supermicro X11SSH-4LN-F motherboard, or a defective Crucial P1 SSD.
Archived
This topic is now archived and is closed to further replies.