I'm trying to move my domains from the cache to the array in order to change my cache setup.

I set the share to use cache: yes. I stopped the VM and docker service. When I invoke the mover, it will quit after a few seconds and all images remain where they are (on the cache).

1. Why is this happening?

2. (How) Can I move them "manually"?

Enable mover logging in settings, scheduler then try to move and then post your diagnostics

Sent via telekinesis

Sure, here you go

knowlage-diagnostics-20190430-1356.zip

I tried using CP, but I get these errors:

 

cp: error reading './domains/vm1/vdisk1.img': Input/output error
cp: error reading './domains/vm2/vdisk1.img': Input/output error
cp: error reading './domains/vm3/vdisk1.img': Input/output error
cp: error reading './domains/vm4/vdisk1.img': Input/output error

I also can see more restore messages/errors in the log:

 

Apr 30 15:26:32 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 1546661888 csum 0xe5c65c2b expected csum 0xa00f6081 mirror 1
Apr 30 15:26:32 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 1546661888 csum 0xe5c65c2b expected csum 0xa00f6081 mirror 2
Apr 30 15:26:32 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 1546661888 csum 0xe5c65c2b expected csum 0xa00f6081 mirror 1
Apr 30 15:26:32 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 1546661888 csum 0xe5c65c2b expected csum 0xa00f6081 mirror 2
Apr 30 15:26:32 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 1548656640 csum 0x0d4d842f expected csum 0xc923843e mirror 1
Apr 30 15:26:32 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 1548656640 csum 0x0d4d842f expected csum 0xc923843e mirror 2
Apr 30 15:26:32 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 1548656640 csum 0x0d4d842f expected csum 0xc923843e mirror 1
Apr 30 15:26:32 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 1548656640 csum 0x0d4d842f expected csum 0xc923843e mirror 2
Apr 30 15:26:32 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 1556733952 csum 0x98f94189 expected csum 0xd02899da mirror 1
Apr 30 15:26:35 Knowlage kernel: repair_io_failure: 8 callbacks suppressed
Apr 30 15:26:35 Knowlage kernel: BTRFS info (device sdp1): read error corrected: ino 1677672 off 1803350016 (dev /dev/sdd1 sector 51689624)
Apr 30 15:26:38 Knowlage kernel: btrfs_print_data_csum_error: 3 callbacks suppressed
Apr 30 15:26:38 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 2108080128 csum 0x506395a7 expected csum 0xf952d1b5 mirror 1
Apr 30 15:26:38 Knowlage kernel: BTRFS info (device sdp1): read error corrected: ino 1677672 off 2108080128 (dev /dev/sdd1 sector 50574296)
Apr 30 15:26:38 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 2116222976 csum 0x47ccaa93 expected csum 0x42d0672a mirror 1
Apr 30 15:26:38 Knowlage kernel: BTRFS info (device sdp1): read error corrected: ino 1677672 off 2116222976 (dev /dev/sdd1 sector 49899552)
Apr 30 15:26:39 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 2141528064 csum 0x332064af expected csum 0x7b624b14 mirror 1
Apr 30 15:26:39 Knowlage kernel: BTRFS info (device sdp1): read error corrected: ino 1677672 off 2141528064 (dev /dev/sdd1 sector 50575848)
Apr 30 15:26:39 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 2141564928 csum 0x21e67b16 expected csum 0xf7dc0737 mirror 1
Apr 30 15:26:39 Knowlage kernel: BTRFS info (device sdp1): read error corrected: ino 1677672 off 2141564928 (dev /dev/sdd1 sector 50576208)
Apr 30 15:26:39 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 2142257152 csum 0xda63fd1d expected csum 0xf5f764d1 mirror 1
Apr 30 15:26:39 Knowlage kernel: BTRFS info (device sdp1): read error corrected: ino 1677672 off 2142257152 (dev /dev/sdd1 sector 50576256)
Apr 30 15:26:42 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 2438111232 csum 0x4ba5fb49 expected csum 0x6114c5f6 mirror 1
Apr 30 15:26:42 Knowlage kernel: BTRFS info (device sdp1): read error corrected: ino 1677672 off 2438111232 (dev /dev/sdd1 sector 50802168)
Apr 30 15:26:43 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 2546335744 csum 0x35a76525 expected csum 0xd1525b2e mirror 1
Apr 30 15:26:43 Knowlage kernel: BTRFS info (device sdp1): read error corrected: ino 1677672 off 2546335744 (dev /dev/sdd1 sector 50453464)
Apr 30 15:26:43 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 2546434048 csum 0x984102f8 expected csum 0x56d0ab30 mirror 1
Apr 30 15:26:43 Knowlage kernel: BTRFS info (device sdp1): read error corrected: ino 1677672 off 2546434048 (dev /dev/sdd1 sector 50453768)
Apr 30 15:26:46 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 2782183424 csum 0x0184ad00 expected csum 0x46a10667 mirror 1
Apr 30 15:26:46 Knowlage kernel: BTRFS info (device sdp1): read error corrected: ino 1677672 off 2782183424 (dev /dev/sdd1 sector 51882232)
Apr 30 15:26:46 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 2791841792 csum 0x18043c69 expected csum 0x3809d3cc mirror 1
Apr 30 15:26:46 Knowlage kernel: BTRFS info (device sdp1): read error corrected: ino 1677672 off 2791841792 (dev /dev/sdd1 sector 51718560)
Apr 30 15:26:48 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 2953674752 csum 0x558f872e expected csum 0x78d2dc96 mirror 1
Apr 30 15:26:48 Knowlage kernel: BTRFS info (device sdp1): read error corrected: ino 1677672 off 2953674752 (dev /dev/sdd1 sector 51770576)
Apr 30 15:26:48 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 2957324288 csum 0xb68f6a0f expected csum 0xa0cd95fb mirror 1
Apr 30 15:26:48 Knowlage kernel: BTRFS info (device sdp1): read error corrected: ino 1677672 off 2957324288 (dev /dev/sdd1 sector 51722920)
Apr 30 15:26:51 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 3183108096 csum 0x31037cb8 expected csum 0xeb3565bb mirror 1
Apr 30 15:26:51 Knowlage kernel: BTRFS info (device sdp1): read error corrected: ino 1677672 off 3183108096 (dev /dev/sdd1 sector 51694936)
Apr 30 15:26:52 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 3270164480 csum 0xf5939b63 expected csum 0x3b4f24d1 mirror 1
Apr 30 15:26:52 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 3270164480 csum 0xf5939b63 expected csum 0x3b4f24d1 mirror 2
Apr 30 15:26:52 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 3270164480 csum 0xf5939b63 expected csum 0x3b4f24d1 mirror 1
Apr 30 15:26:52 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 3270164480 csum 0xf5939b63 expected csum 0x3b4f24d1 mirror 2
Apr 30 15:26:59 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 3976335360 csum 0x0a16375f expected csum 0x4d78f2b6 mirror 1
Apr 30 15:26:59 Knowlage kernel: BTRFS info (device sdp1): read error corrected: ino 1677672 off 3976335360 (dev /dev/sdd1 sector 51919064)

 

Edited April 30, 20197 yr by Jaster

Cache pool is running degraded, so it can't fix some errors:

 

Apr 30 15:26:32 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 1556733952 csum 0x98f94189 expected csum 0xd02899da mirror 1
Apr 30 15:26:35 Knowlage kernel: repair_io_failure: 8 callbacks suppressed

 

These are unfixable, and will give an i/o error to avoid copying a corrupt file.

 

Apr 30 15:26:59 Knowlage kernel: BTRFS warning (device sdp1): csum failed root 5 ino 1677672 off 3976335360 csum 0x0a16375f expected csum 0x4d78f2b6 mirror 1 
Apr 30 15:26:59 Knowlage kernel: BTRFS info (device sdp1): read error corrected: ino 1677672 off 3976335360 (dev /dev/sdd1 sector 51919064)

 

These are being fixed.

 

The corrupt files can be copied with btrfs restore, but they will still be corrupt.

 

 

 

 

 

 

 

 

 

 

Edited April 30, 20197 yr by johnnie.black

Thats kind of a destaster. Basicly all VM images are dead, even trought it is possible to start them, I can not do anything with the files?...

 

How can I try to restore the files using btrfs restore, while my pool is in raid5 mode?

Edited April 30, 20197 yr by Jaster

7 minutes ago, Jaster said:

How can I try to restore the files using btrfs restore

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?do=findComment&comment=543490