GraemeT

I have fritzbox and had issues with Unraid 6.11 with static IP addresses. The Fritzbox was very confused when identifying the Unraid server, even though the MAC matched and unraid had a static IP, the Fritxbox was trying to allocate another IP address... I ended up fixing it by resetting the Fritxbox and re-entering all my configuration! Not a fun task, but it seems to have resolved the issue I was seeing. Might not be related to your issue, but information in case it helps! (btw, Fritzbox is also the default router for Zen customers in the UK, so more countries to add to the list 😀 )

I also ditched the VLAN setup - although now it is working I may revisit that!

I managed to get it working - 3 things that I did (Perhaps not all of them needed...) 1) Set the docker networking to bridge mode 2) Set a port forward on the router to forward the wireguard port to the configured port on the host that the container is mapped to (shown on the docker information when running) 3) Set a static route on the router to point IP Addresses in the wireguard range to use the unraid server IP as the gateway (Probably not needed unless initiating connections from the LAN to the client - but I am doing that for one client)

I have been trying this docker after upgrading my switches to use VLANs seems to have broken the built in Unraid Wireguard that was previously working perfectly!

I am having exactly the same issue... did you ever resolve it?

I got is to work in the end, it was all to do with having the storage mapped through the user share rather that directly on the cache drive. The user share through the Unraid drive aggregation software does not support all the required filesystem linking operations (not sure exactly which ones) It might be worth updating the template to add that the data folder should not be on the user share.

I am having exactly the same issue. @xaositek did you ever get to the bottom of this?

I came across this performance comparison between SMB/NFS 4/SSHFS and it indicates from the testing performed that SSHFS should be on a par with NFS for most operations in their testing environment. Although I am seeing a larger difference between SMB and SSHFS.

I pulled the drive and performed a rebuild onto the same disk and it worked. Given the type of corruption and the exact same corruption in both LUKS Headers I would assume this issue was due to some LUKS activity that went wrong. The rebuild process seems to not be the cause. As everything is working fine now, and I am unable to recrate the issue, I am considering this closed.

It may be worth trying SSHFS as an alternative to SMB. I have done some simple testing with Windows -> Unraid and it appears to process small files 3x faster for my setup vs SMB. To set it up: On the Unraid side, you can use the openssh-server docker in the community application to "share" a folder - it allows you to setup a username and password (or ssh keys) to connect to Unraid and to restrict access to specific folders - e.g. /mnt/user/Media. For windows you can use sshfs-win (https://github.com/billziss-gh/sshfs-win) as a way to map the SSHFS to a drive letter. Performance thoughts: Doing this gives around 3x performance with small files, but peak throughput is around half.... probably due to the small file efficiencies in the protocol, but larger overheads of ssh encryption/decryption and docker filesystem layers that need to be traversed. Connecting directly to Unraid SSH as root is the same for peak throughput and small file processing is around 30% faster. But is not something I would want to do generally due to having to use root access. Might be an option for the specific use case of backing up millions of small files.

I have narrowed the issue down to a corrupted LUKS header: root@central:/mnt/cache/recover# cryptsetup luksDump /dev/sdb1 --debug # cryptsetup 2.3.4 processing "cryptsetup luksDump /dev/sdb1 --debug" # Running command luksDump. # Locking memory. # Installing SIGINT/SIGTERM handler. # Unblocking interruption on signal. # Allocating context for crypt device /dev/sdb1. # Trying to open and read device /dev/sdb1 with direct-io. # Initialising device-mapper backend library. # Trying to load any crypt type from device /dev/sdb1. # Crypto backend (OpenSSL 1.1.1j 16 Feb 2021) initialized in cryptsetup library version 2.3.4. # Detected kernel Linux 5.10.21-Unraid x86_64. # Loading LUKS2 header (repair disabled). # Acquiring read lock for device /dev/sdb1. # Opening lock resource file /run/cryptsetup/L_8:17 # Verifying lock handle for /dev/sdb1. # Device /dev/sdb1 READ lock taken. # Trying to read primary LUKS2 header at offset 0x0. # Opening locked device /dev/sdb1 # Veryfing locked device handle (bdev) # LUKS2 header version 2 of size 16384 bytes, checksum sha256. # Checksum:770a0e992143ca7be43ae402b12b9df9f6c0356b09937647d0151453371397f2 (on-disk) # Checksum:245e32401668452f3fc9e86faf5e388f49417c108350338501b069a04e2ec6f6 (in-memory) # LUKS2 header checksum error (offset 0). # Trying to read secondary LUKS2 header at offset 0x4000. # Reusing open ro fd on device /dev/sdb1 # LUKS2 header version 2 of size 16384 bytes, checksum sha256. # Checksum:6991844e5efb3ce3606c1dbfc629dfbb4149475b0d19b865e43cfe5a00d07f3f (on-disk) # Checksum:8679968f9c132a9687e9a5a116e23c108a38fa4521e338c135f5942b217dac42 (in-memory) # LUKS2 header checksum error (offset 16384). # Trying to read secondary LUKS2 header at offset 0x8000. # Reusing open ro fd on device /dev/sdb1 # Trying to read secondary LUKS2 header at offset 0x10000. # Reusing open ro fd on device /dev/sdb1 # Trying to read secondary LUKS2 header at offset 0x20000. # Reusing open ro fd on device /dev/sdb1 # Trying to read secondary LUKS2 header at offset 0x40000. # Reusing open ro fd on device /dev/sdb1 # Trying to read secondary LUKS2 header at offset 0x80000. # Reusing open ro fd on device /dev/sdb1 # Trying to read secondary LUKS2 header at offset 0x100000. # Reusing open ro fd on device /dev/sdb1 # Trying to read secondary LUKS2 header at offset 0x200000. # Reusing open ro fd on device /dev/sdb1 # Trying to read secondary LUKS2 header at offset 0x400000. # Reusing open ro fd on device /dev/sdb1 # LUKS2 header read failed (-22). # Device /dev/sdb1 READ lock released. Device /dev/sdb1 is not a valid LUKS device. # Releasing crypt device /dev/sdb1 context. # Releasing device-mapper backend. # Closing read only fd for /dev/sdb1. # Unlocking memory. Command failed with code -1 (wrong or missing parameters). It appears that somehow both copies of the header have been corrupted in the exact same way (they are identical in the parts that are defined to be identical) : 00000000 4c 55 4b 53 ba be 00 02 00 00 00 00 00 00 40 00 |LUKS..........@.| 00000010 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00 00 |................| 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 00000040 00 00 00 00 00 00 00 00 73 68 61 32 35 36 00 00 |........sha256..| 00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000060 00 00 00 00 00 00 00 00 60 63 7d 9c ab 52 b5 99 |........`c}..R..| 00000070 cf 5d a3 58 c0 06 3a 09 15 98 71 98 04 93 9c 5f |.].X..:...q...._| 00000080 be 70 9c ef b6 e1 af 4f 5b 7a 72 4b f0 38 d0 73 |.p.....O[zrK.8.s| 00000090 55 76 69 26 3f ce 8e 2f b1 9e 51 a2 47 79 8e 14 |Uvi&?../..Q.Gy..| 000000a0 3e 65 e0 83 8f d1 78 3a 34 38 39 64 62 65 34 37 |>e....x:489dbe47| 000000b0 2d 36 61 33 64 2d 34 38 61 33 2d 62 61 30 36 2d |-6a3d-48a3-ba06-| 000000c0 33 31 36 61 61 65 35 36 64 34 64 62 00 00 00 00 |316aae56d4db....| 000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 000001c0 77 0a 0e 99 21 43 ca 7b e4 3a e4 02 b1 2b 9d f9 |w...!C.{.:...+..| 000001d0 f6 c0 35 6b 09 93 76 47 d0 15 14 53 37 13 97 f2 |..5k..vG...S7...| 000001e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 00001000 7b 22 6b 65 79 73 6c 6f 74 73 22 3a 7b 22 30 22 |{"keyslots":{"0"| 00001010 3a 7b 22 74 79 70 65 22 3a 22 6c 75 6b 73 32 22 |:{"type":"luks2"| 00001020 2c 22 6b 65 79 5f 73 69 7a 65 22 3a 36 34 2c 22 |,"key_size":64,"| 00001030 61 66 22 3a 7b 22 74 79 70 65 22 3a 22 6c 75 6b |af":{"type":"luk| 00001040 73 31 22 2c 22 73 74 72 69 70 65 73 22 3a 34 30 |s1","stripes":40| 00001050 30 30 2c 22 68 61 73 68 22 3a 22 73 68 61 32 35 |00,"hash":"sha25| 00001060 36 22 7d 2c 22 61 72 65 61 22 3a 7b 22 74 79 70 |6"},"area":{"typ| 00001070 65 22 3a 22 72 61 77 22 2c 22 6f 66 66 73 65 74 |e":"raw","offset| 00001080 22 3a 22 33 32 37 36 38 22 2c 22 73 69 7a 65 22 |":"32768","size"| 00001090 3a 22 32 35 38 30 34 38 22 2c 22 65 6e 63 72 79 |:"258048","encry| 000010a0 70 74 69 6f 6e 22 3a 22 61 65 73 2d 78 74 73 2d |ption":"aes-xts-| 000010b0 70 6c 61 69 6e 36 34 22 2c 22 6b 65 79 5f 73 69 |plain64","key_si| 000010c0 7a 65 22 3a 36 34 7d 2c 22 6b 64 66 22 3a 7b 22 |ze":64},"kdf":{"| 000010d0 74 79 70 65 22 3a 22 61 72 67 6f 6e 32 69 22 2c |type":"argon2i",| 000010e0 22 74 69 6d 65 22 3a 34 2c 22 6d 65 6d 6f 72 79 |"time":4,"memory| 000010f0 22 3a 37 33 30 30 30 37 2c 22 63 70 75 73 22 3a |":730007,"cpus":| 00001100 34 2c 22 73 61 6c 74 22 3a 22 79 49 2b 56 77 44 |4,"salt":"yI+VwD| 00001110 7a 42 53 30 78 38 45 61 5a 41 4b 76 43 65 41 6a |zBS0x8EaZAKvCeAj| 00001120 63 54 44 4e 54 7a 56 51 36 46 63 35 53 6b 4f 75 |cTDNTzVQ6Fc5SkOu| 00001130 6c 47 57 7a 41 3d 22 7d 7d 7d 2c 22 74 6f 6b 65 |lGWzA="}}},"toke| 00001140 6e 73 22 3a 7b 7d 2c 22 73 65 67 6d 65 6e 74 73 |ns":{},"segments| 00001150 22 3a 7b 22 30 22 3a 7b 22 74 79 70 65 22 3a 22 |":{"0":{"type":"| 00001160 63 72 79 70 74 22 2c 22 6f 66 66 73 65 74 22 3a |crypt","offset":| 00001170 22 31 36 37 37 37 32 31 36 22 2c 22 73 69 7a 65 |"16777216","size| 00001180 22 3a 22 64 79 6e 61 6d 69 63 22 2c 22 69 76 5f |":"dynamic","iv_| 00001190 74 77 65 61 6b 22 3a 22 30 22 2c 22 65 6e 63 72 |tweak":"0","encr| 000011a0 79 70 74 69 6f 6e 22 3a 22 61 65 73 2d 78 74 73 |yption":"aes-xts| 000011b0 2d 70 6c 61 69 6e 36 34 22 2c 22 73 65 63 74 6f |-plain64","secto| 000011c0 72 5f 73 69 7a 65 22 3a 35 31 32 7d 7d 2c 22 64 |r_size":512}},"d| 000011d0 69 67 65 73 74 73 22 3a 7b 22 30 22 3a 7b 22 74 |igests":{"0":{"t| 000011e0 79 70 65 22 3a 22 70 62 6b 64 66 32 22 2c 22 6b |ype":"pbkdf2","k| 000011f0 65 79 73 6c 6f 74 73 22 3a 5b 22 30 22 5d 2c 22 |eyslots":["0"],"| 00001200 73 65 67 6d 65 6e 74 73 22 3a 5b 22 30 22 5d 2c |segments":["0"],| 00001210 22 68 61 73 68 23 3a 22 73 68 61 32 35 36 22 2c |"hash#:"sha256",| 00001220 22 69 74 6c 70 60 77 69 6f 6e 73 22 3a 34 30 32 |"itlp`wions":402| 00001230 30 36 2c 22 73 61 6c 74 22 0e 65 7b 47 39 2f 7b |06,"salt".e{G9/{| 00001240 78 54 02 40 34 5e 5d 5d 20 57 7b 56 5e 5d 69 5f |xT.@4^]] W{V^]i_| 00001250 30 5a 31 45 73 64 69 72 5d 19 50 5a 4c 29 02 20 |0Z1Esdir].PZL). | 00001260 4f 56 4c 7e 44 6b 3d 22 2c 22 64 69 67 65 73 74 |OVL~Dk=","digest| 00001270 22 3a 22 64 46 32 66 69 77 78 47 32 6d 44 6b 44 |":"dF2fiwxG2mDkD| 00001280 41 66 30 38 45 6b 71 6f 4e 6d 46 74 49 65 61 4a |Af08EkqoNmFtIeaJ| 00001290 56 52 73 4f 6b 70 78 4d 4e 49 49 4c 6d 6f 3d 22 |VRsOkpxMNIILmo="| 000012a0 7d 7d 2c 22 63 6f 6e 66 69 67 22 3a 7b 22 6a 73 |}},"config":{"js| 000012b0 6f 6e 5f 73 69 7a 65 22 3a 22 31 32 32 38 38 22 |on_size":"12288"| 000012c0 2c 22 6b 65 79 73 6c 6f 74 73 5f 73 69 7a 65 22 |,"keyslots_size"| 000012d0 3a 22 31 36 37 34 34 34 34 38 22 7d 7d 00 00 00 |:"16744448"}}...| 000012e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 00004000 53 4b 55 4c ba be 00 02 00 00 00 00 00 00 40 00 |SKUL..........@.| 00004010 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00 00 |................| 00004020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 00004040 00 00 00 00 00 00 00 00 73 68 61 32 35 36 00 00 |........sha256..| 00004050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00004060 00 00 00 00 00 00 00 00 fc a2 b5 4c c4 48 9e 04 |...........L.H..| 00004070 4c e6 a0 7b 4c c0 c1 f8 70 dc 81 67 07 06 4c 92 |L..{L...p..g..L.| 00004080 30 b8 eb ba 4c 3e b6 40 ae b7 bf d1 1e e6 04 2d |0...L>[email protected]| 00004090 38 0d 71 0f 6d 65 9c 66 69 e9 f6 c2 7d 68 58 fa |8.q.me.fi...}hX.| 000040a0 47 94 ba af a9 31 da 8d 34 38 39 64 62 65 34 37 |G....1..489dbe47| 000040b0 2d 36 61 33 64 2d 34 38 61 33 2d 62 61 30 36 2d |-6a3d-48a3-ba06-| 000040c0 33 31 36 61 61 65 35 36 64 34 64 62 00 00 00 00 |316aae56d4db....| 000040d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 00004100 00 00 00 00 00 00 40 00 00 00 00 00 00 00 00 00 |......@.........| 00004110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 000041c0 69 91 84 4e 5e fb 3c e3 60 6c 1d bf c6 29 df bb |i..N^.<.`l...)..| 000041d0 41 49 47 5b 0d 19 b8 65 e4 3c fe 5a 00 d0 7f 3f |AIG[...e.<.Z...?| 000041e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 00005000 7b 22 6b 65 79 73 6c 6f 74 73 22 3a 7b 22 30 22 |{"keyslots":{"0"| 00005010 3a 7b 22 74 79 70 65 22 3a 22 6c 75 6b 73 32 22 |:{"type":"luks2"| 00005020 2c 22 6b 65 79 5f 73 69 7a 65 22 3a 36 34 2c 22 |,"key_size":64,"| 00005030 61 66 22 3a 7b 22 74 79 70 65 22 3a 22 6c 75 6b |af":{"type":"luk| 00005040 73 31 22 2c 22 73 74 72 69 70 65 73 22 3a 34 30 |s1","stripes":40| 00005050 30 30 2c 22 68 61 73 68 22 3a 22 73 68 61 32 35 |00,"hash":"sha25| 00005060 36 22 7d 2c 22 61 72 65 61 22 3a 7b 22 74 79 70 |6"},"area":{"typ| 00005070 65 22 3a 22 72 61 77 22 2c 22 6f 66 66 73 65 74 |e":"raw","offset| 00005080 22 3a 22 33 32 37 36 38 22 2c 22 73 69 7a 65 22 |":"32768","size"| 00005090 3a 22 32 35 38 30 34 38 22 2c 22 65 6e 63 72 79 |:"258048","encry| 000050a0 70 74 69 6f 6e 22 3a 22 61 65 73 2d 78 74 73 2d |ption":"aes-xts-| 000050b0 70 6c 61 69 6e 36 34 22 2c 22 6b 65 79 5f 73 69 |plain64","key_si| 000050c0 7a 65 22 3a 36 34 7d 2c 22 6b 64 66 22 3a 7b 22 |ze":64},"kdf":{"| 000050d0 74 79 70 65 22 3a 22 61 72 67 6f 6e 32 69 22 2c |type":"argon2i",| 000050e0 22 74 69 6d 65 22 3a 34 2c 22 6d 65 6d 6f 72 79 |"time":4,"memory| 000050f0 22 3a 37 33 30 30 30 37 2c 22 63 70 75 73 22 3a |":730007,"cpus":| 00005100 34 2c 22 73 61 6c 74 22 3a 22 79 49 2b 56 77 44 |4,"salt":"yI+VwD| 00005110 7a 42 53 30 78 38 45 61 5a 41 4b 76 43 65 41 6a |zBS0x8EaZAKvCeAj| 00005120 63 54 44 4e 54 7a 56 51 36 46 63 35 53 6b 4f 75 |cTDNTzVQ6Fc5SkOu| 00005130 6c 47 57 7a 41 3d 22 7d 7d 7d 2c 22 74 6f 6b 65 |lGWzA="}}},"toke| 00005140 6e 73 22 3a 7b 7d 2c 22 73 65 67 6d 65 6e 74 73 |ns":{},"segments| 00005150 22 3a 7b 22 30 22 3a 7b 22 74 79 70 65 22 3a 22 |":{"0":{"type":"| 00005160 63 72 79 70 74 22 2c 22 6f 66 66 73 65 74 22 3a |crypt","offset":| 00005170 22 31 36 37 37 37 32 31 36 22 2c 22 73 69 7a 65 |"16777216","size| 00005180 22 3a 22 64 79 6e 61 6d 69 63 22 2c 22 69 76 5f |":"dynamic","iv_| 00005190 74 77 65 61 6b 22 3a 22 30 22 2c 22 65 6e 63 72 |tweak":"0","encr| 000051a0 79 70 74 69 6f 6e 22 3a 22 61 65 73 2d 78 74 73 |yption":"aes-xts| 000051b0 2d 70 6c 61 69 6e 36 34 22 2c 22 73 65 63 74 6f |-plain64","secto| 000051c0 72 5f 73 69 7a 65 22 3a 35 31 32 7d 7d 2c 22 64 |r_size":512}},"d| 000051d0 69 67 65 73 74 73 22 3a 7b 22 30 22 3a 7b 22 74 |igests":{"0":{"t| 000051e0 79 70 65 22 3a 22 70 62 6b 64 66 32 22 2c 22 6b |ype":"pbkdf2","k| 000051f0 65 79 73 6c 6f 74 73 22 3a 5b 22 30 22 5d 2c 22 |eyslots":["0"],"| 00005200 73 65 67 6d 65 6e 74 73 22 3a 5b 22 30 22 5d 2c |segments":["0"],| 00005210 22 68 61 73 68 23 3a 22 73 68 61 32 35 36 22 2c |"hash#:"sha256",| 00005220 22 69 74 6c 70 60 77 69 6f 6e 73 22 3a 34 30 32 |"itlp`wions":402| 00005230 30 36 2c 22 73 61 6c 74 22 0e 65 7b 47 39 2f 7b |06,"salt".e{G9/{| 00005240 78 54 02 40 34 5e 5d 5d 20 57 7b 56 5e 5d 69 5f |xT.@4^]] W{V^]i_| 00005250 30 5a 31 45 73 64 69 72 5d 19 50 5a 4c 29 02 20 |0Z1Esdir].PZL). | 00005260 4f 56 4c 7e 44 6b 3d 22 2c 22 64 69 67 65 73 74 |OVL~Dk=","digest| 00005270 22 3a 22 64 46 32 66 69 77 78 47 32 6d 44 6b 44 |":"dF2fiwxG2mDkD| 00005280 41 66 30 38 45 6b 71 6f 4e 6d 46 74 49 65 61 4a |Af08EkqoNmFtIeaJ| 00005290 56 52 73 4f 6b 70 78 4d 4e 49 49 4c 6d 6f 3d 22 |VRsOkpxMNIILmo="| 000052a0 7d 7d 2c 22 63 6f 6e 66 69 67 22 3a 7b 22 6a 73 |}},"config":{"js| 000052b0 6f 6e 5f 73 69 7a 65 22 3a 22 31 32 32 38 38 22 |on_size":"12288"| 000052c0 2c 22 6b 65 79 73 6c 6f 74 73 5f 73 69 7a 65 22 |,"keyslots_size"| 000052d0 3a 22 31 36 37 34 34 34 34 38 22 7d 7d 00 00 00 |:"16744448"}}...| 000052e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| The corruption appears to start at 1210 "hash#: should be "hash": and continues in the following few bytes. Most of it is fixable, but it appears that the salt was badly affected. It may be possible to write something to brute force that based on the sha256 for the header, but that assumes that non of the other data has corruption. I am not concerned about the data, but does anyone have any ideas how both the LUKS headers may have been corrupted in this subtle way? I do not think it was because I accidentally pulled the drive. If I don't get any other suggestions today, I will reformat and rebuild the disk. I will then pull the disk in the same way and see if I can replicate the issue.

I am having an issue with an encrypted array drive. I accidentally hot pulled the drive (stupid mistake as I was trying to pull an empty caddy and I managed to pull the wrong one!) What this has led to though is a walk through of what to do in the event of a failed disk.... and in my setup, a disk rebuild appears not to be working! I followed the unraid manual guidance for rebuilding back onto the same disk, which after 7 hours reported success.... however I am using encrypted XFS on the array and unraid reported that the disk was still unmountable after a rebuild. I ran cryptsetup to test if the encryption is OK... root@central:/mnt# cryptsetup -v isLuks /dev/sdb1 Command failed with code -1 (wrong or missing parameters). Apparently not! I am in the process of trying to rebuild for a second time: This time I have run a pre-clear on the disk for the first few hundred MB to wipe out any prior encryption or filesystem... the rebuild is currently on progress, but it is not looking promising as the luks headers are still not there according to the command. Does anyone have any ideas as to what I am doing wrong? This is all on Version: 6.9.1 Some screenshots to help show what I have done for the second attempt: State before bringing online (sdb is the disk that was in the array and I have run pre-clear on the first few hundred MB)

Just reading through this thread, one thing that might be related is the reported speed difference using ls on /mnt/user - https://forums.unraid.net/bug-reports/stable-releases/680-slow-shfs-file-listings-r818/ it seems the high CPU usage could be due to the find crawling that shfs aggregated folder.