Hi, 

My unraid server has suddenly got BTRFS errors on the cache drive that are causing docker to not run.

Please can someone offer some assistance as to how to fix this? Whilst the server has been up and running well for some time, I am definitely a newbie so layman's terms would be appreciated.

I have attached the diagnostics.

Thanks

diagnostics-20220901-1850.zip

Since btrfs was detecting some data corruption before this it's a good idea to run memtest first, then backup and re-format the pool.

Thanks for this. 

Please can you send a step by step guide to doing what you said? 

Thanks

 

Start with memtest, it's a boot option in the Unraid boot menu 

So I rad the mem test over the weekend for 3 days straight with no issues by the looks of it. 

Would the next stage be to backup and reformat the pool? If so, please can you outline how I do this? 

Thanks 

Assuming the pool is still mounting just copy anything important elsewhere, then re-format and restore the data, you can use for example the Dynamix File Manager plugin to easily copy the files.

I can't access krusader as the docker won't work with the current error. 

How do I go about copying all of the data to a USB hard drive that I have mounted using unasiged devices? Is it possible to do this through the unraid GUI? 

Thanks

9 minutes ago, ldog88 said:

I can't access krusader as the docker won't work with the current error. 

Since you are on 6.10, you could use the great File Manager plugin to copy data.

Just to double check, I need to go into Main then select Cache and then select all files and move to "remotes"? 

Sorry I'm very much a noob

If I am not mistaking 'remotes' would be a network path to another system.

 

An external USB drive attached to your server would be at /mnt/disks/xxx I think (or simply 'disks' in the plugin).

thanks for this. Managed to work it out but unfortunately the copy didn't work.

I have attached the diagnostics so if you could see any issues that would be much appreciated.

Thanks

diagnostics-20220906-2245.zip

Sep  6 21:44:17 Lawson-NAS kernel: nvme nvme1: I/O 908 QID 8 timeout, aborting
Sep  6 21:44:17 Lawson-NAS kernel: nvme nvme1: I/O 916 QID 8 timeout, aborting
Sep  6 21:44:17 Lawson-NAS kernel: nvme nvme1: I/O 917 QID 8 timeout, aborting
Sep  6 21:44:17 Lawson-NAS kernel: nvme nvme1: I/O 918 QID 8 timeout, aborting
Sep  6 21:44:17 Lawson-NAS kernel: nvme nvme1: I/O 919 QID 8 timeout, aborting
Sep  6 21:44:47 Lawson-NAS kernel: nvme nvme1: I/O 908 QID 8 timeout, reset controller
Sep  6 21:45:17 Lawson-NAS kernel: nvme nvme1: I/O 8 QID 0 timeout, reset controller
Sep  6 21:45:50 Lawson-NAS kernel: blk_update_request: I/O error, dev nvme1n1, sector 1068352 op 0x0:(READ) flags 0x80700 phys_seg 27 prio class 0
Sep  6 21:45:50 Lawson-NAS kernel: blk_update_request: I/O error, dev nvme1n1, sector 1836344 op 0x0:(READ) flags 0x80700 phys_seg 3 prio class 0
Sep  6 21:45:50 Lawson-NAS kernel: blk_update_request: I/O error, dev nvme1n1, sector 1837776 op 0x0:(READ) flags 0x80700 phys_seg 4 prio class 0
Sep  6 21:45:50 Lawson-NAS kernel: blk_update_request: I/O error, dev nvme1n1, sector 2003208 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
Sep  6 21:45:50 Lawson-NAS kernel: blk_update_request: I/O error, dev nvme1n1, sector 1838768 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
Sep  6 21:45:50 Lawson-NAS kernel: blk_update_request: I/O error, dev nvme1n1, sector 146702808 op 0x0:(READ) flags 0x80700 phys_seg 2 prio class 0
Sep  6 21:45:50 Lawson-NAS kernel: blk_update_request: I/O error, dev nvme1n1, sector 2263264 op 0x0:(READ) flags 0x80700 phys_seg 13 prio class 0
Sep  6 21:45:50 Lawson-NAS kernel: blk_update_request: I/O error, dev nvme1n1, sector 1068960 op 0x0:(READ) flags 0x80700 phys_seg 5 prio class 0
Sep  6 21:45:50 Lawson-NAS kernel: blk_update_request: I/O error, dev nvme1n1, sector 84863328 op 0x0:(READ) flags 0x80700 phys_seg 6 prio class 0
Sep  6 21:45:50 Lawson-NAS kernel: blk_update_request: I/O error, dev nvme1n1, sector 84862880 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0

 

Log shows issues with one of the NVMe devices, power cycle the server and try again.

So I think the nvme device is the issue here and that is what is set up as the cache drive. It appears that most if not all of the files have copied over into my USB hdd. 

How do I now go about formatting the cache drive? 

I tried removing it from the array but was unable to then format it. 

Thanks 

You can wipe it by stopping the array and typing

blkdiscard -f /dev/nvme#n1

then start the array to format

 

So I managed to clear it using preclear disks and then formatted. 

Having now added it back in and copied all of the data back to it, I am missing all of my docker containers. 

Is there a way to point docker to the cache so that it picks them up? 

Thanks

If you restored the appdata you can recreate the docker image.

So I followed the recreate docker image steps however it doenst appear to have worked. 

I have attached the diagnostics. 

Thanks

diagnostics-20220912-2013.zip

https://wiki.unraid.net/Manual/Docker_Management#Re-Create_the_Docker_image_file

 

https://wiki.unraid.net/Manual/Docker_Management#Re-Installing_Docker_Applications

Crisis averted. Looks like I managed to move some files to the wrong folders.
Thank you very much for your assistance!