nerv

I moved my docker folder back to my cache disk and that appears to have fixed the issue as everything is much faster. Not surprising but frustrating.

Hmm okay, it's on a 120TB pool though, but I was getting alerts before about docker space available that corresponded to the disk not the pool/share. Mostly I just want to make sure of the size before I move it back to my cache to see if it resolves my issues with updating taking so long they timeout. I guess I just need to wait for du -sh to finish to see how big it is?

I expect this issue predates 7.0. I noticed unraid is reporting my docker folder size as 3.89 TiB. Looking through the directories I traced it to overlay2, but it's so big with so many files du -sh just times out. Setting max depth 1 I can see there's some multi gig folders in here (just at 1 level). I've run docker system prune --all etc and nothing is getting cleaned up, and docker system df is showing ~15 GB of space. Any ideas why this folder is so big or advice on how to debug further? I'm running about 25 dockers, fairly standard plex, *arr, etc.

on 7.0, same issue

Did this ever get solved? I have the same problem. Is the timeout update the recommended solution?

FYI I did move the docker folder to another disk with higher capacity and the error went away. So that pretty much confirms the check is looking at the disk's space rather than the share it belongs to. Appreciate the help!

got it. I'm a little terrified to move the folder 😂, but agreed that's a good way to confirm the issue.

Sorry not fully following, are you saying the error message is correct? I do have split any directory enabled, so the disk capacity should not matter.

Looking at this more carefully, I think it's because the docker folder is on disk 4, which is at 13.1/14TB that rounds up to, 94%. That disk has been roughly at that usage since I changed to the docker directory a few months ago, because the array has been filling other lower disks the entire time. However, given the share is set to split at any directory, and can use all disks, perhaps the error is wrong? If the directory couldn't be split to another drive, or it was only allowed on that Drive it would make sense. But in practice won't unraid split this directory to another disk at some point?

Some of this just dates back to what was available at the time. E.g. I had some disks on reiser until a couple years ago until I took the time to migrate them all. I've been on Unraid for over a decade by now. Last I checked XFS was recommended for the array and BTRFS was what was available for the pool 🤷‍♂️ I may move the docker image back to the cache, but I haven't had performance issues as is. If there's nothing else to try I'll just silence the error. Fwiw I'm skeptical it has anything to do with free space on disks etc. The warning is complaining about the image (I don't have one) and 94% was the warning I got before migrating to docker directory to fix that error, and it has consistently been warning 94% for ~3 months despite the array changing in size a bunch since then (though admittedly most of the disks will have remained the same usage in that time). Seems like a stale error that won't go away but it doesn't really matter, I can ignore it. Appreciate the input.

No, the share is set to store on the Array.

What does the plugin base Docker image file is getting full on when using a docker directory? My docker image was getting full but I switched to a directory months ago. I deleted the old image files, and have rebooted the system, but I'm still getting this error. Afaik I have no docker image, so I don't know how to make this error go away. I can ignore it, but I'd rather not. thanks!

Closing this out, after 2 weeks no crashes. Seems very likely the UPS was just failing.

got it, thanks!

Thanks, that's what I saw as well. Are there specific types of hardware issues that don't show up in logs? I seem to recall memory issues and that type of thing typically showing up. Just trying to narrow where to start looking. For now I've moved it off the UPS to see if that is the cause.

I've attached diagnostics, but my server has started rebooting randomly every few days. I haven't upgraded the OS or really changed anything, so I'm wondering if there is a hardware issue. However, looking at sys logs, there's nothing around the time the reboot happens. Any ideas? My only theory atm is my battery backup is dieing and randomly cutting power. media-diagnostics-20240314-1046.zip syslog-192.168.86.42.log

Hmm, no that's probably it. The warning just showed up for the first time after a month which is odd, but that makes the most sense. Thanks!

Yea I did change the setting and the crashes stopped, but I got the warning again. Not sure why I'm getting the warning again.

No crashes still for a month+ so that definitely seems to have fixed it. Weirdly though I just got an alert from fix common errors that it found Macvlan call traces. I confirmed I still have docker set to ipvlan. Should I ignore this? I also have some VMs running.

So far so good, so I'm going to mark this solved and cross my fingers. Will reopen if I get a new crash + logs. Thanks @JorgeB!

Got it, I changed that. I'll follow up if that fixes the issue. That's actually the first time I've seen that log fwiw, or at least the first time fix common errors alerted me to it was the other day. I had assumed that was because Mullvad took down my VPN for maintenance and I had to switch. Maybe that's unrelated though. I'm not sure where that setting actually came from. Maybe an older legacy thing as I've had unraid for ~10 years? I do have a fairly unusual setup with dockers with individual dual IPs on a vlan + on a wireguard tunnel, but as far as I can tell I didn't set it that way due to one of those, and everything is still working with the new setting. Doesn't matter, just curious :).

Ever since upgrading to 6.11.5 my Unraid system has started crashing after having never previously crashed before. It was every 2 weeks or so, but recently it has increased to every 2-3 days. I've attached diagnostics and my syslog, any ideas on the cause? The most recent crash was sometime on the 25th, and I turned the server back on the morning of the 26th. There's a CPU warning in the logs right before then (see below), and I see something similar a few other places. Is this potentially related? Any help greatly appreciated. Thanks! Mar 25 06:55:24 Media kernel: ------------[ cut here ]------------ Mar 25 06:55:24 Media kernel: WARNING: CPU: 16 PID: 6805 at net/netfilter/nf_nat_core.c:594 nf_nat_setup_info+0x73/0x7b1 [nf_nat] Mar 25 06:55:24 Media kernel: Modules linked in: af_packet tcp_diag udp_diag inet_diag xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle ip6table_nat vhost_net tun vhost vhost_iotlb tap veth macvlan xt_nat xt_tcpudp xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user iptable_nat nf_nat br_netfilter xfs md_mod it87 hwmon_vid xt_connmark nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_mark iptable_mangle xt_comment xt_addrtype iptable_raw wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha ip6table_filter ip6_tables iptable_filter ip_tables x_tables bridge 8021q garp mrp stp llc bonding tls ixgbe xfrm_algo mdio x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel ipmi_ssif kvm crct10dif_pclmul crc32_pclmul crc32c_intel ast ghash_clmulni_intel drm_vram_helper i2c_algo_bit aesni_intel drm_ttm_helper ttm crypto_simd cryptd drm_kms_helper rapl intel_cstate drm intel_uncore mpt3sas backlight agpgart Mar 25 06:55:24 Media kernel: i2c_i801 i2c_smbus ahci syscopyarea i2c_core sysfillrect raid_class sysimgblt libahci fb_sys_fops scsi_transport_sas wmi acpi_ipmi ipmi_si button unix [last unloaded: xfrm_algo] Mar 25 06:55:24 Media kernel: CPU: 16 PID: 6805 Comm: kworker/16:2 Tainted: G W 5.19.17-Unraid #2 Mar 25 06:55:24 Media kernel: Hardware name: Cirrascale VB1416/GA-7PESH2, BIOS R17 06/26/2018 Mar 25 06:55:24 Media kernel: Workqueue: events macvlan_process_broadcast [macvlan] Mar 25 06:55:24 Media kernel: RIP: 0010:nf_nat_setup_info+0x73/0x7b1 [nf_nat] Mar 25 06:55:24 Media kernel: Code: 48 8b 87 80 00 00 00 48 89 fb 49 89 f4 76 04 0f 0b eb 0e 83 7c 24 1c 00 75 07 25 80 00 00 00 eb 05 25 00 01 00 00 85 c0 74 07 <0f> 0b e9 6a 06 00 00 48 8b 83 88 00 00 00 48 8d 73 58 48 8d 7c 24 Mar 25 06:55:24 Media kernel: RSP: 0018:ffffc90006784bc8 EFLAGS: 00010202 Mar 25 06:55:24 Media kernel: RAX: 0000000000000080 RBX: ffff8882a61a5700 RCX: ffff889099a38400 Mar 25 06:55:24 Media kernel: RDX: 0000000000000000 RSI: ffffc90006784cac RDI: ffff8882a61a5700 Mar 25 06:55:24 Media kernel: RBP: ffffc90006784c90 R08: 000000006756a8c0 R09: 0000000000000000 Mar 25 06:55:24 Media kernel: R10: 0000000000000158 R11: 0000000000000000 R12: ffffc90006784cac Mar 25 06:55:24 Media kernel: R13: 000000006756a800 R14: ffffc90006784d90 R15: 0000000000000000 Mar 25 06:55:24 Media kernel: FS: 0000000000000000(0000) GS:ffff88903fc00000(0000) knlGS:0000000000000000 Mar 25 06:55:24 Media kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Mar 25 06:55:24 Media kernel: CR2: 0000151c1dd49bf8 CR3: 000000000220a006 CR4: 00000000001726e0 Mar 25 06:55:24 Media kernel: Call Trace: Mar 25 06:55:24 Media kernel: <IRQ> Mar 25 06:55:24 Media kernel: ? xt_write_recseq_end+0xf/0x1c [ip_tables] Mar 25 06:55:24 Media kernel: ? __local_bh_enable_ip+0x56/0x6b Mar 25 06:55:24 Media kernel: ? ipt_do_table+0x57a/0x5bf [ip_tables] Mar 25 06:55:24 Media kernel: ? xt_write_recseq_end+0xf/0x1c [ip_tables] Mar 25 06:55:24 Media kernel: __nf_nat_alloc_null_binding+0x66/0x81 [nf_nat] Mar 25 06:55:24 Media kernel: nf_nat_inet_fn+0xc0/0x1a8 [nf_nat] Mar 25 06:55:24 Media kernel: nf_nat_ipv4_local_in+0x2a/0xaa [nf_nat] Mar 25 06:55:24 Media kernel: nf_hook_slow+0x3d/0x96 Mar 25 06:55:24 Media kernel: ? ip_protocol_deliver_rcu+0x164/0x164 Mar 25 06:55:24 Media kernel: NF_HOOK.constprop.0+0x79/0xd9 Mar 25 06:55:24 Media kernel: ? ip_protocol_deliver_rcu+0x164/0x164 Mar 25 06:55:24 Media kernel: ip_sabotage_in+0x4a/0x58 [br_netfilter] Mar 25 06:55:24 Media kernel: nf_hook_slow+0x3d/0x96 Mar 25 06:55:24 Media kernel: ? ip_rcv_finish_core.constprop.0+0x3b7/0x3b7 Mar 25 06:55:24 Media kernel: NF_HOOK.constprop.0+0x79/0xd9 Mar 25 06:55:24 Media kernel: ? ip_rcv_finish_core.constprop.0+0x3b7/0x3b7 Mar 25 06:55:24 Media kernel: __netif_receive_skb_one_core+0x77/0x9c Mar 25 06:55:24 Media kernel: process_backlog+0x8c/0x116 Mar 25 06:55:24 Media kernel: __napi_poll.constprop.0+0x2b/0x124 Mar 25 06:55:24 Media kernel: net_rx_action+0x159/0x24f Mar 25 06:55:24 Media kernel: __do_softirq+0x129/0x288 Mar 25 06:55:24 Media kernel: do_softirq+0x7f/0xab Mar 25 06:55:24 Media kernel: </IRQ> Mar 25 06:55:24 Media kernel: <TASK> Mar 25 06:55:24 Media kernel: __local_bh_enable_ip+0x4c/0x6b Mar 25 06:55:24 Media kernel: netif_rx+0x52/0x5a Mar 25 06:55:24 Media kernel: macvlan_broadcast+0x10a/0x150 [macvlan] Mar 25 06:55:24 Media kernel: macvlan_process_broadcast+0xbc/0x12f [macvlan] Mar 25 06:55:24 Media kernel: process_one_work+0x1ab/0x295 Mar 25 06:55:24 Media kernel: worker_thread+0x18b/0x244 Mar 25 06:55:24 Media kernel: ? rescuer_thread+0x281/0x281 Mar 25 06:55:24 Media kernel: kthread+0xe7/0xef Mar 25 06:55:24 Media kernel: ? kthread_complete_and_exit+0x1b/0x1b Mar 25 06:55:24 Media kernel: ret_from_fork+0x22/0x30 Mar 25 06:55:24 Media kernel: </TASK> Mar 25 06:55:24 Media kernel: ---[ end trace 0000000000000000 ]--- Mar 26 08:10:34 Media kernel: mdcmd (36): set md_write_method 1 peer-Media-wg0-1 (1).zip syslog-192.168.86.42 (1).log

Using the VPN tunneled access for docker, is it possible to not use the tunnel for some local IPs? My dockers on wg1 lose access to dockers I don't want on the tunnel running on another vlan. I tried modifying allowed IPS to exclude 192.168.0.0/16 with a calculator, but when I put that in allowed IPs the handshake fails. edit: This is on 6.11.5