benyaki Posted January 21, 2021 Share Posted January 21, 2021 (edited) Just noticed that log was at 100% so I had a quick look through it and the last hour was riddled with BTRFS errors. I currently have a mix of xfs drives and reiserfs drives (eliminating the later right now, just moving things around). Any insight as to why these errors are popping up? Of note - unbalance is currently running, moving data off reiserfs drives onto xfs drives before they are either pulled or reformatted. The NVME cache has been in use for a few weeks. Attached diagnostics. tower-diagnostics-20210121-1140.zip Edited January 21, 2021 by benyaki Quote Link to comment
JorgeB Posted January 21, 2021 Share Posted January 21, 2021 NVMe device dropped offline: Jan 21 10:23:41 Tower kernel: nvme nvme0: I/O 979 QID 6 timeout, aborting Jan 21 10:23:41 Tower kernel: nvme nvme0: I/O 980 QID 6 timeout, aborting Jan 21 10:23:41 Tower kernel: nvme nvme0: I/O 981 QID 6 timeout, aborting Jan 21 10:23:50 Tower kernel: nvme nvme0: I/O 422 QID 5 timeout, aborting Jan 21 10:23:50 Tower kernel: nvme nvme0: I/O 423 QID 5 timeout, aborting Jan 21 10:24:11 Tower kernel: nvme nvme0: I/O 979 QID 6 timeout, reset controller Jan 21 10:24:41 Tower kernel: nvme nvme0: I/O 7 QID 0 timeout, reset controller Jan 21 10:25:24 Tower kernel: nvme nvme0: Device not ready; aborting reset Jan 21 10:25:24 Tower kernel: nvme nvme0: Abort status: 0x7 ### [PREVIOUS LINE REPEATED 4 TIMES] ### Jan 21 10:25:46 Tower kernel: nvme nvme0: Device not ready; aborting reset Jan 21 10:25:46 Tower kernel: nvme nvme0: Removing after probe failure status: -19 Jan 21 10:26:09 Tower kernel: nvme nvme0: Device not ready; aborting reset Quote Link to comment
benyaki Posted January 21, 2021 Author Share Posted January 21, 2021 Any suggestions to fix or troubleshoot? Or is it just something that occurs randomly? Quote Link to comment
benyaki Posted January 22, 2021 Author Share Posted January 22, 2021 So I let it run (moving some data around) and came home, could not access any dockers. So I stopped and restarted the array. Cache drive is missing (not a visible device). Restarted - same thing. Looks like the drive has disappeared ... Attached diagnostics tower-diagnostics-20210121-2011.zip Quote Link to comment
JorgeB Posted January 22, 2021 Share Posted January 22, 2021 Try power cycling the server to see if it comes back. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.