So....docker on.....everything disabled except for two dockers....ten minutes later:

 

Dec  1 15:01:41 Svalbard kernel: ------------[ cut here ]------------
Dec  1 15:01:41 Svalbard kernel: kernel BUG at drivers/md/unraid.c:1617!
Dec  1 15:01:41 Svalbard kernel: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
Dec  1 15:01:41 Svalbard kernel: CPU: 12 PID: 12166 Comm: unraidd0 Tainted: P           O       6.1.118-Unraid #1
Dec  1 15:01:41 Svalbard kernel: Hardware name: Gigabyte Technology Co., Ltd. Z790M AORUS ELITE AX/Z790M AORUS ELITE AX, BIOS F10 09/27/2024
Dec  1 15:01:41 Svalbard kernel: RIP: 0010:unraidd+0x1051/0x1140 [md_mod]
Dec  1 15:01:41 Svalbard kernel: Code: 00 83 3d 83 50 00 00 03 7e 16 41 8b 56 98 89 e9 48 c7 c7 21 23 4d a0 48 8b 73 20 e8 23 96 39 e1 41 f6 86 69 ff ff ff 02 75 02 <0f> 0b 48 8b 43 20 49 03 47 10 41 c7 46 b0 00 10 00 00 49 8b 56 10
Dec  1 15:01:41 Svalbard kernel: RSP: 0018:ffffc9000b157df0 EFLAGS: 00010246
Dec  1 15:01:41 Svalbard kernel: RAX: 0000000000000000 RBX: ffff88815b424268 RCX: 0000000000000000
Dec  1 15:01:41 Svalbard kernel: RDX: 0000000000000000 RSI: ffffffff829f1720 RDI: ffff88810540aa38
Dec  1 15:01:41 Svalbard kernel: RBP: 0000000000000001 R08: 0000000000000000 R09: 0000000000000000
Dec  1 15:01:41 Svalbard kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff888148a5b110
Dec  1 15:01:41 Svalbard kernel: R13: ffff88815b424460 R14: ffff88815b4244d8 R15: ffff88815694d2d8
Dec  1 15:01:41 Svalbard kernel: FS:  0000000000000000(0000) GS:ffff88907f300000(0000) knlGS:0000000000000000
Dec  1 15:01:41 Svalbard kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Dec  1 15:01:41 Svalbard kernel: CR2: 000014ffe3390000 CR3: 000000000420a000 CR4: 0000000000750ee0
Dec  1 15:01:41 Svalbard kernel: PKRU: 55555554
Dec  1 15:01:41 Svalbard kernel: Call Trace:
Dec  1 15:01:41 Svalbard kernel: <TASK>
Dec  1 15:01:41 Svalbard kernel: ? __die_body+0x1a/0x5c
Dec  1 15:01:41 Svalbard kernel: ? die+0x30/0x49
Dec  1 15:01:41 Svalbard kernel: ? do_trap+0x7b/0xfe
Dec  1 15:01:41 Svalbard kernel: ? unraidd+0x1051/0x1140 [md_mod]
Dec  1 15:01:41 Svalbard kernel: ? unraidd+0x1051/0x1140 [md_mod]
Dec  1 15:01:41 Svalbard kernel: ? do_error_trap+0x6e/0x98
Dec  1 15:01:41 Svalbard kernel: ? unraidd+0x1051/0x1140 [md_mod]
Dec  1 15:01:41 Svalbard kernel: ? exc_invalid_op+0x4c/0x60
Dec  1 15:01:41 Svalbard kernel: ? unraidd+0x1051/0x1140 [md_mod]
Dec  1 15:01:41 Svalbard kernel: ? asm_exc_invalid_op+0x16/0x20
Dec  1 15:01:41 Svalbard kernel: ? unraidd+0x1051/0x1140 [md_mod]
Dec  1 15:01:41 Svalbard kernel: md_thread+0xf4/0x122 [md_mod]
Dec  1 15:01:41 Svalbard kernel: ? _raw_spin_rq_lock_irqsave+0x20/0x20
Dec  1 15:01:41 Svalbard kernel: ? signal_pending+0x1d/0x1d [md_mod]
Dec  1 15:01:41 Svalbard kernel: kthread+0xe4/0xef
Dec  1 15:01:41 Svalbard kernel: ? kthread_complete_and_exit+0x1b/0x1b
Dec  1 15:01:41 Svalbard kernel: ret_from_fork+0x1f/0x30
Dec  1 15:01:41 Svalbard kernel: </TASK>
Dec  1 15:01:41 Svalbard kernel: Modules linked in: ipvlan wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha tun nft_compat nf_tables xt_nat xt_tcpudp veth xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_addrtype br_netfilter macvtap macvlan tap cdc_ncm cdc_ether usbnet mii xfs md_mod tcp_diag inet_diag it87(O) hwmon_vid ip6table_filter ip6_tables iptable_filter ip_tables x_tables efivarfs bridge stp llc bonding tls r8169 realtek zfs(PO) intel_rapl_msr intel_rapl_common zunicode(PO) i915 zzstd(O) x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel zlua(O) iosf_mbi drm_buddy i2c_algo_bit ttm zavl(PO) icp(PO) kvm drm_display_helper crct10dif_pclmul crc32_pclmul crc32c_intel zcommon(PO) ghash_clmulni_intel btusb drm_kms_helper sha512_ssse3 btrtl btbcm sha256_ssse3 znvpair(PO) btintel sha1_ssse3 aesni_intel bluetooth spl(O)
Dec  1 15:01:41 Svalbard kernel: crypto_simd drm cryptd rapl ecdh_generic mei_hdcp mei_pxp gigabyte_wmi wmi_bmof intel_cstate ecc nvme intel_gtt mpt3sas agpgart intel_uncore i2c_i801 mei_me i2c_smbus ahci nvme_core i2c_core mei raid_class libahci scsi_transport_sas syscopyarea sysfillrect sysimgblt fb_sys_fops thermal fan video tpm_crb tpm_tis tpm_tis_core wmi tpm backlight intel_pmc_core acpi_tad acpi_pad button unix [last unloaded: realtek]
Dec  1 15:01:41 Svalbard kernel: ---[ end trace 0000000000000000 ]---
Dec  1 15:01:41 Svalbard kernel: RIP: 0010:unraidd+0x1051/0x1140 [md_mod]
Dec  1 15:01:41 Svalbard kernel: Code: 00 83 3d 83 50 00 00 03 7e 16 41 8b 56 98 89 e9 48 c7 c7 21 23 4d a0 48 8b 73 20 e8 23 96 39 e1 41 f6 86 69 ff ff ff 02 75 02 <0f> 0b 48 8b 43 20 49 03 47 10 41 c7 46 b0 00 10 00 00 49 8b 56 10
Dec  1 15:01:41 Svalbard kernel: RSP: 0018:ffffc9000b157df0 EFLAGS: 00010246
Dec  1 15:01:41 Svalbard kernel: RAX: 0000000000000000 RBX: ffff88815b424268 RCX: 0000000000000000
Dec  1 15:01:41 Svalbard kernel: RDX: 0000000000000000 RSI: ffffffff829f1720 RDI: ffff88810540aa38
Dec  1 15:01:41 Svalbard kernel: RBP: 0000000000000001 R08: 0000000000000000 R09: 0000000000000000
Dec  1 15:01:41 Svalbard kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff888148a5b110
Dec  1 15:01:41 Svalbard kernel: R13: ffff88815b424460 R14: ffff88815b4244d8 R15: ffff88815694d2d8
Dec  1 15:01:41 Svalbard kernel: FS:  0000000000000000(0000) GS:ffff88907f300000(0000) knlGS:0000000000000000
Dec  1 15:01:41 Svalbard kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Dec  1 15:01:41 Svalbard kernel: CR2: 000014ffe3390000 CR3: 00000001a19ca000 CR4: 0000000000750ee0
Dec  1 15:01:41 Svalbard kernel: PKRU: 55555554
Dec  1 15:01:41 Svalbard kernel: ------------[ cut here ]------------
Dec  1 15:01:41 Svalbard kernel: WARNING: CPU: 12 PID: 12166 at kernel/exit.c:816 do_exit+0x87/0x923
Dec  1 15:01:41 Svalbard kernel: Modules linked in: ipvlan wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha tun nft_compat nf_tables xt_nat xt_tcpudp veth xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_addrtype br_netfilter macvtap macvlan tap cdc_ncm cdc_ether usbnet mii xfs md_mod tcp_diag inet_diag it87(O) hwmon_vid ip6table_filter ip6_tables iptable_filter ip_tables x_tables efivarfs bridge stp llc bonding tls r8169 realtek zfs(PO) intel_rapl_msr intel_rapl_common zunicode(PO) i915 zzstd(O) x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel zlua(O) iosf_mbi drm_buddy i2c_algo_bit ttm zavl(PO) icp(PO) kvm drm_display_helper crct10dif_pclmul crc32_pclmul crc32c_intel zcommon(PO) ghash_clmulni_intel btusb drm_kms_helper sha512_ssse3 btrtl btbcm sha256_ssse3 znvpair(PO) btintel sha1_ssse3 aesni_intel bluetooth spl(O)
Dec  1 15:01:41 Svalbard kernel: crypto_simd drm cryptd rapl ecdh_generic mei_hdcp mei_pxp gigabyte_wmi wmi_bmof intel_cstate ecc nvme intel_gtt mpt3sas agpgart intel_uncore i2c_i801 mei_me i2c_smbus ahci nvme_core i2c_core mei raid_class libahci scsi_transport_sas syscopyarea sysfillrect sysimgblt fb_sys_fops thermal fan video tpm_crb tpm_tis tpm_tis_core wmi tpm backlight intel_pmc_core acpi_tad acpi_pad button unix [last unloaded: realtek]
Dec  1 15:01:41 Svalbard kernel: CPU: 12 PID: 12166 Comm: unraidd0 Tainted: P      D    O       6.1.118-Unraid #1
Dec  1 15:01:41 Svalbard kernel: Hardware name: Gigabyte Technology Co., Ltd. Z790M AORUS ELITE AX/Z790M AORUS ELITE AX, BIOS F10 09/27/2024
Dec  1 15:01:41 Svalbard kernel: RIP: 0010:do_exit+0x87/0x923
Dec  1 15:01:41 Svalbard kernel: Code: 24 74 04 75 13 b8 01 00 00 00 41 89 6c 24 60 48 c1 e0 22 49 89 44 24 70 4c 89 ef e8 1f 47 81 00 48 83 bb b0 07 00 00 00 74 02 <0f> 0b 48 8b bb d8 06 00 00 e8 21 46 81 00 48 8b 83 d0 06 00 00 83
Dec  1 15:01:41 Svalbard kernel: RSP: 0018:ffffc9000b157ee0 EFLAGS: 00010286
Dec  1 15:01:41 Svalbard kernel: RAX: 0000000000000000 RBX: ffff888140cb2000 RCX: 0000000000000000
Dec  1 15:01:41 Svalbard kernel: RDX: 0000000000000001 RSI: 0000000000002710 RDI: 00000000ffffffff
Dec  1 15:01:41 Svalbard kernel: RBP: 000000000000000b R08: 0000000000000000 R09: ffffffff829583f0
Dec  1 15:01:41 Svalbard kernel: R10: 00003fffffffffff R11: ffff8890bf7bb3be R12: ffff888108550c00
Dec  1 15:01:41 Svalbard kernel: R13: ffff888155350840 R14: 0000000000000002 R15: ffffffff820b440d
Dec  1 15:01:41 Svalbard kernel: FS:  0000000000000000(0000) GS:ffff88907f300000(0000) knlGS:0000000000000000
Dec  1 15:01:41 Svalbard kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Dec  1 15:01:41 Svalbard kernel: CR2: 000014ffe3390000 CR3: 00000001a19ca000 CR4: 0000000000750ee0
Dec  1 15:01:41 Svalbard kernel: PKRU: 55555554
Dec  1 15:01:41 Svalbard kernel: Call Trace:
Dec  1 15:01:41 Svalbard kernel: <TASK>
Dec  1 15:01:41 Svalbard kernel: ? __warn+0xab/0x122
Dec  1 15:01:41 Svalbard kernel: ? report_bug+0x109/0x17e
Dec  1 15:01:41 Svalbard kernel: ? do_exit+0x87/0x923
Dec  1 15:01:41 Svalbard kernel: ? handle_bug+0x41/0x6f
Dec  1 15:01:41 Svalbard kernel: ? exc_invalid_op+0x13/0x60
Dec  1 15:01:41 Svalbard kernel: ? asm_exc_invalid_op+0x16/0x20
Dec  1 15:01:41 Svalbard kernel: ? do_exit+0x87/0x923
Dec  1 15:01:41 Svalbard kernel: make_task_dead+0x11c/0x11c
Dec  1 15:01:41 Svalbard kernel: rewind_stack_and_make_dead+0x17/0x17
Dec  1 15:01:41 Svalbard kernel: RIP: 0000:0x0
Dec  1 15:01:41 Svalbard kernel: Code: Unable to access opcode bytes at 0xffffffffffffffd6.
Dec  1 15:01:41 Svalbard kernel: RSP: 0000:0000000000000000 EFLAGS: 00000000 ORIG_RAX: 0000000000000000
Dec  1 15:01:41 Svalbard kernel: RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
Dec  1 15:01:41 Svalbard kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
Dec  1 15:01:41 Svalbard kernel: RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
Dec  1 15:01:41 Svalbard kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
Dec  1 15:01:41 Svalbard kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
Dec  1 15:01:41 Svalbard kernel: </TASK>
Dec  1 15:01:41 Svalbard kernel: ---[ end trace 0000000000000000 ]---

 

So...what has changed? Still no extra network, all I did was enable docker and spin up two instances....admittedly two copies of the same docker image (2x Frigate). But nothing else....

@bmartino1 - I almost missed you there!

 

I saw something flash up just as it took me to the next page so I went back and looked and there you were!

 

I guess macVLAN might be worth a go then...if nothing else it is broken anyway so making it worse is unlikely.

 

Here is my current list:

NETWORK ID     NAME         DRIVER    SCOPE
5d933692e462   br0          ipvlan    local
d1d8b728a0af   bridge       bridge    local
4f8e4e9cff75   host         host      local
030024dbe919   none         null      local
1d649d9c9840   vpn_bridge   bridge    local

 

All default except for the vpn_bridge network that I need to bond my *arr's to GluTun. And the vpn_bridge looks like this:
 

[
    {
        "Name": "vpn_bridge",
        "Id": "1d649d9c98407eae0684f545004bb1811f4e3a695152301859c00c1828c6fce1",
        "Created": "2024-09-07T13:35:56.052832342+12:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.18.0.0/16",
                    "Gateway": "172.18.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {},
        "Options": {},
        "Labels": {}
    }
]

 

The individual dockers are then bonded to GlueTun with "--net=container:GluetunVPN" and GlueTun itself is connected to the vpn-bridge. Only vaguely remember the ins and outs....I followed a guide as it was my first time.

 

For starters I have now deleted the vpn_bridge network and put GlueTun on the 'normal' bridge. Everythign still seems to be working (so now I wonder what the custom network was for).

 

BUT

 

The really weird thing is that GlueTun starts by itself as soon as Docker is started eventhough it is *NOT* set to start automatically....that doesn't seem right!

 

 

so here is my recommend course of option for a reset.

Turn off any and all autostart items, dockers, VM, lxc...

lets recreate the docker libvirt system
turn off all docker autostart
in termianl run:
 

rm /var/lib/docker/network/files/local-kv.db

*this is the saved db file that has caused issues int eh past with resting docker networks.

then go to setting > docker

turn off docker and reboot.
*To reset the db file...
more info in this post:

at unraid boot when array is on, go to network settings and turn off bridging (unless you need it...) turn off bonding...

then go to settings > docker
delete the docker image.

review:

https://www.youtube.com/watch?v=WXBB4I0iLZM

setup macvlan:

so that your system is using and setup for
 

root@BMM-Unraid:~# docker network ls
NETWORK ID     NAME      DRIVER    SCOPE
9cbd343c82a0   bridge    bridge    local
0302aa0c1e94   eth0      macvlan   local
99ab079461ad   host      host      local
05b15108e8ce   none      null      local
root@BMM-Unraid:~# 

optional:
https://forums.unraid.net/topic/178033-bmartino1-user-scripts/#findComment-1492333
set vhost0 tap to a different ip...

go to docker add container and re-add your previous dockers. as configuration and other docker data should be on the pool/array disks...
*No data loss is expected in this process. I have done this in the past with no data loss. depends on docker configurations...

then re make the custom docker network. it can still be bridge/ipvlan.
 

docker network create \
    --driver=bridge \
    --subnet=172.18.0.0/16 \
    --gateway=172.18.0.1 \
    vpn_bridge

 

Edited December 1, 20241 yr by bmartino1
typo - data form new data

I still don't see the need for netwrok options bridging on or bonding on.

bridign on if you have more than 1 adapter that you want unraid to act like a switch and allow the two interface to talk to each other.
Bonding on if you have 2 adpaters dependent on bond (more for 2 connection for load balancing and fail over)

*if setup with 1 nic goes well add the second. to test stability.

Once your remove and reinstall eh docker img file and reset the unraid docker networks.

To set up Docker containers (like *arr apps) to use the GluTun container for networking and route traffic through NordVPN, you'll need to use network bridging and container bonding. Here's a guide to achieve this:

 

Setup Overview
GluTun Setup: The GluTun container acts as the VPN client, routing traffic through NordVPN.
Dependent Containers: *arr apps (like Sonarr, Radarr, etc.) are connected to the GluTun container network to route their traffic through the VPN.
Bonding Network: Containers bond to the GluTun network using --network=container:<GluTun container>.

 

*? - you may already have a docker tempate with dat...
 

Setup the GluTun Container

Use the official GluTun image and configure it for NordVPN:

docker run -d \
  --name=gluetun \
  --cap-add=NET_ADMIN \
  -e VPN_SERVICE_PROVIDER=nordvpn \
  -e VPN_TYPE=openvpn \
  -e NORDVPN_COUNTRY=<YourPreferredCountry> \
  -e NORDVPN_CATEGORY=legacy_standard \
  -e NORDVPN_PROTOCOL=udp \
  -e NORDVPN_USERNAME=<YourNordVPNUsername> \
  -e NORDVPN_PASSWORD=<YourNordVPNPassword> \
  -p 8888:8888/tcp \
  -p 8388:8388/tcp \
  -p 8388:8388/udp \
  qmcgaw/gluetun

 

Replace placeholders like <YourPreferredCountry>, <YourNordVPNUsername>, and <YourNordVPNPassword> with your NordVPN account details.

 

Connect arr Containers to GluTun

Use the --network=container:<gluetun> parameter to bond your *arr containers to the GluTun container network. Example for Sonarr:

docker run -d \
  --name=sonarr \
  --network=container:gluetun \
  -v /path/to/config:/config \
  -v /path/to/data:/data \
  linuxserver/sonarr

 

Repeat for Radarr, Lidarr, etc., using their respective images.

you may need other GluTun varaibles

Advanced Configuration

Allow Local LAN Access:

Add the following environment variable to the GluTun container to allow LAN access

-e EXTRA_SUBNETS=192.168.1.0/24

Configure PUID/PGID (Optional):

#run as root
-e PUID=0
-e PGID=0

#run defualt unraid:
-e PUID=99
-e PGID=100

DNS Resolution:

Add custom DNS settings to avoid leaks

-e DOT=on
-e DNS1=1.1.1.1
-e DNS2=1.0.0.1

 

Test Network Isolation:

Ensure *arr apps cannot access the internet without GluTun by disabling GluTun and testing connectivity.

Check NordVPN Connection:

Use tools like curl ifconfig.io or similar within the GluTun container to verify the VPN connection.

for testing id setup with custom eth0 on macvlan first then add bonding / brdign as we add teh seond nic back.

OK....those were some serious steps to work through!

1. Tried to swap out the USB key, no joy. Got stuck in a "you must license this box or else loop" so no option to run a trail license for a bit to see how it would go. I check the old key under windows, there were no problems, and the key is only a few months old...so I'm picking probably fine....after 45 minutes of mucking around I gave up. (Also the help page on the Unraid website is out of date - the terms and screenshots no longer match).
2. With docker on rand the rm command to purge the DB
3. Stopped docker, rebooted, turned off all the networking stuff (it then defaulted to macvlan all by itself which I didn't expect), deleted the image.
4. Added the containers again (and clean out the ones I no longer need). Set each container to Custom: Eth0 + an IP address. Rinse and repeat over and over. That actually worked like a charm, so very pleased with that as it's not something I've had to do before.
5. Add GlueTun back, *also* gave that a local IP, the same as all the other dockers. Started it, it connects to VPN like it always did.
6. Added the arr apps, set them all to use "no network" and use "--net=container:GluetunVPN" instead.

So interestingly, at that point everything just worked - that is with GluetunVPN up I could get SABNZBD to test server connections, with GluetunVPN stopped those tests failed. So I could get to SABNZBD and it could get out. If I snag a torrent the tracker reports the same address as that reported by GluetunVPN - so it is routing the qBitorrent traffic through the VPN....

 

This means that I did *NOT* need to do any of the other networking steps....note however that in the GlueTunVPN docker config I have mapped the ports to the connected dockers so that GluetunVPN will pass traffic to the slaved containers....

 

I can always go and mess with the network again - but I'm leaning towards leaning well enough alone, because more tinkering may make things worse....and I've already got problems. 

 

So everything is up, everything is working....and now we wait for a crash.....I'm going to kick off the parity check....that's bound to raise the stakes....

 

Here we go....

Hmmmm.....I remember why there needs to be a custom network.....it's because the arr dockers need to talk to each other....sigh.....

OK. Added the vpn_bridge using the command you provided. Set GluetunVPN to a fixed IP address and made sure everything else can hook into it....so now this is all going. I haven't added the command to allow GluetunVPN access to the local LAN (it doesn't need access) and it still reaches the Internet just fine.

 

Going to reboot now so we come up clean and the parity check will get kicked off again...

 

Fingers crossed. If nothing else it hasn't died yet, so that's positive in that if nothing else we're not worse off!  Have to think positive...that way you're less disappointed when it all catches fire again...  🙂

you can also have gluten docker conect to the ars docker with a side termainl camon user script.

 

Connect an existing container to a different network:

To connect a running Docker container to a different network, use the docker network connect command:

docker network connect my_custom_network container_name_or_id

Replace my_custom_network with the network's name or ID, and container_name_or_id with the name or ID of the container you want to connect.

 

Verify the connection:

To check if the container is connected to the network, you can use the following command:

docker network inspect my_custom_network

This will show the details of the network, including which containers are connected to it.

 

 

you can also create a ipvlan and run both macvlan and ipvlan on unraid...

 

docker network create \
  --driver ipvlan \
  --subnet 192.168.100.0/24 \
  --gateway 192.168.100.1 \
  --ip-range 192.168.100.128/25 \
  --aux-address 'my-address=192.168.100.10' \
  --opt parent=eth0 \
  ipvlan

 

Good to know!

 

So far 52% of the way through the parity check, no problems so far....logs are clean. 

 

Definitely heading in the right direction now I think!

Alas alas and much woe....at 57.7% it died again...

 

Dec  2 12:44:13 Svalbard kernel: ------------[ cut here ]------------
Dec  2 12:44:13 Svalbard kernel: kernel BUG at drivers/md/unraid.c:1617!
Dec  2 12:44:13 Svalbard kernel: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
Dec  2 12:44:13 Svalbard kernel: CPU: 14 PID: 11985 Comm: unraidd0 Tainted: P           O       6.1.118-Unraid #1
Dec  2 12:44:13 Svalbard kernel: Hardware name: Gigabyte Technology Co., Ltd. Z790M AORUS ELITE AX/Z790M AORUS ELITE AX, BIOS F10 09/27/2024
Dec  2 12:44:13 Svalbard kernel: RIP: 0010:unraidd+0x1051/0x1140 [md_mod]
Dec  2 12:44:13 Svalbard kernel: Code: 00 83 3d 83 50 00 00 03 7e 16 41 8b 56 98 89 e9 48 c7 c7 21 23 4e a0 48 8b 73 20 e8 23 96 38 e1 41 f6 86 69 ff ff ff 02 75 02 <0f> 0b 48 8b 43 20 49 03 47 10 41 c7 46 b0 00 10 00 00 49 8b 56 10
Dec  2 12:44:13 Svalbard kernel: RSP: 0018:ffffc90000bc7df0 EFLAGS: 00010246
Dec  2 12:44:13 Svalbard kernel: RAX: 0000000000000000 RBX: ffff888149586090 RCX: 0000000000000000
Dec  2 12:44:13 Svalbard kernel: RDX: 0000000000000000 RSI: ffffffff829f1720 RDI: ffff88810168ce38
Dec  2 12:44:13 Svalbard kernel: RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
Dec  2 12:44:13 Svalbard kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff88813980b108
Dec  2 12:44:13 Svalbard kernel: R13: ffff8881495861d8 R14: ffff888149586250 R15: ffff88814aebd218
Dec  2 12:44:13 Svalbard kernel: FS:  0000000000000000(0000) GS:ffff88907f380000(0000) knlGS:0000000000000000
Dec  2 12:44:13 Svalbard kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Dec  2 12:44:13 Svalbard kernel: CR2: 0000154ff45911e0 CR3: 000000022de42000 CR4: 0000000000750ee0
Dec  2 12:44:13 Svalbard kernel: PKRU: 55555554
Dec  2 12:44:13 Svalbard kernel: Call Trace:
Dec  2 12:44:13 Svalbard kernel: <TASK>
Dec  2 12:44:13 Svalbard kernel: ? __die_body+0x1a/0x5c
Dec  2 12:44:13 Svalbard kernel: ? die+0x30/0x49
Dec  2 12:44:13 Svalbard kernel: ? do_trap+0x7b/0xfe
Dec  2 12:44:13 Svalbard kernel: ? unraidd+0x1051/0x1140 [md_mod]
Dec  2 12:44:13 Svalbard kernel: ? unraidd+0x1051/0x1140 [md_mod]
Dec  2 12:44:13 Svalbard kernel: ? do_error_trap+0x6e/0x98
Dec  2 12:44:13 Svalbard kernel: ? unraidd+0x1051/0x1140 [md_mod]
Dec  2 12:44:13 Svalbard kernel: ? exc_invalid_op+0x4c/0x60
Dec  2 12:44:13 Svalbard kernel: ? unraidd+0x1051/0x1140 [md_mod]
Dec  2 12:44:13 Svalbard kernel: ? asm_exc_invalid_op+0x16/0x20
Dec  2 12:44:13 Svalbard kernel: ? unraidd+0x1051/0x1140 [md_mod]
Dec  2 12:44:13 Svalbard kernel: md_thread+0xf4/0x122 [md_mod]
Dec  2 12:44:13 Svalbard kernel: ? _raw_spin_rq_lock_irqsave+0x20/0x20
Dec  2 12:44:13 Svalbard kernel: ? signal_pending+0x1d/0x1d [md_mod]
Dec  2 12:44:13 Svalbard kernel: kthread+0xe4/0xef
Dec  2 12:44:13 Svalbard kernel: ? kthread_complete_and_exit+0x1b/0x1b
Dec  2 12:44:13 Svalbard kernel: ret_from_fork+0x1f/0x30
Dec  2 12:44:13 Svalbard kernel: </TASK>
Dec  2 12:44:13 Svalbard kernel: Modules linked in: xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle ip6table_nat iptable_mangle vhost_net vhost vhost_iotlb wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha tun nft_compat nf_tables veth xt_nat xt_tcpudp xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_addrtype br_netfilter bridge stp llc xfs md_mod tcp_diag inet_diag it87(O) hwmon_vid ip6table_filter ip6_tables iptable_filter ip_tables x_tables efivarfs macvtap macvlan tap zfs(PO) i915 intel_rapl_msr intel_rapl_common x86_pkg_temp_thermal intel_powerclamp zunicode(PO) zzstd(O) coretemp iosf_mbi drm_buddy i2c_algo_bit kvm_intel ttm zlua(O) drm_display_helper zavl(PO) drm_kms_helper icp(PO) btusb btrtl btbcm kvm btintel bluetooth drm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel sha512_ssse3 sha256_ssse3
Dec  2 12:44:13 Svalbard kernel: sha1_ssse3 zcommon(PO) aesni_intel znvpair(PO) crypto_simd spl(O) cryptd ecdh_generic mei_hdcp mei_pxp gigabyte_wmi wmi_bmof rapl ecc mpt3sas intel_cstate i2c_i801 intel_gtt nvme agpgart intel_uncore i2c_smbus mei_me r8169 ahci i2c_core nvme_core mei raid_class realtek libahci scsi_transport_sas syscopyarea sysfillrect sysimgblt fb_sys_fops thermal fan tpm_crb video tpm_tis tpm_tis_core wmi tpm backlight intel_pmc_core acpi_tad acpi_pad button unix
Dec  2 12:44:13 Svalbard kernel: ---[ end trace 0000000000000000 ]---
Dec  2 12:44:13 Svalbard kernel: RIP: 0010:unraidd+0x1051/0x1140 [md_mod]
Dec  2 12:44:13 Svalbard kernel: Code: 00 83 3d 83 50 00 00 03 7e 16 41 8b 56 98 89 e9 48 c7 c7 21 23 4e a0 48 8b 73 20 e8 23 96 38 e1 41 f6 86 69 ff ff ff 02 75 02 <0f> 0b 48 8b 43 20 49 03 47 10 41 c7 46 b0 00 10 00 00 49 8b 56 10
Dec  2 12:44:13 Svalbard kernel: RSP: 0018:ffffc90000bc7df0 EFLAGS: 00010246
Dec  2 12:44:13 Svalbard kernel: RAX: 0000000000000000 RBX: ffff888149586090 RCX: 0000000000000000
Dec  2 12:44:13 Svalbard kernel: RDX: 0000000000000000 RSI: ffffffff829f1720 RDI: ffff88810168ce38
Dec  2 12:44:13 Svalbard kernel: RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
Dec  2 12:44:13 Svalbard kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff88813980b108
Dec  2 12:44:13 Svalbard kernel: R13: ffff8881495861d8 R14: ffff888149586250 R15: ffff88814aebd218
Dec  2 12:44:13 Svalbard kernel: FS:  0000000000000000(0000) GS:ffff88907f380000(0000) knlGS:0000000000000000
Dec  2 12:44:13 Svalbard kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Dec  2 12:44:13 Svalbard kernel: CR2: 0000154ff45911e0 CR3: 000000022de42000 CR4: 0000000000750ee0
Dec  2 12:44:13 Svalbard kernel: PKRU: 55555554
Dec  2 12:44:13 Svalbard kernel: ------------[ cut here ]------------
Dec  2 12:44:13 Svalbard kernel: WARNING: CPU: 14 PID: 11985 at kernel/exit.c:816 do_exit+0x87/0x923
Dec  2 12:44:13 Svalbard kernel: Modules linked in: xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle ip6table_nat iptable_mangle vhost_net vhost vhost_iotlb wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha tun nft_compat nf_tables veth xt_nat xt_tcpudp xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_addrtype br_netfilter bridge stp llc xfs md_mod tcp_diag inet_diag it87(O) hwmon_vid ip6table_filter ip6_tables iptable_filter ip_tables x_tables efivarfs macvtap macvlan tap zfs(PO) i915 intel_rapl_msr intel_rapl_common x86_pkg_temp_thermal intel_powerclamp zunicode(PO) zzstd(O) coretemp iosf_mbi drm_buddy i2c_algo_bit kvm_intel ttm zlua(O) drm_display_helper zavl(PO) drm_kms_helper icp(PO) btusb btrtl btbcm kvm btintel bluetooth drm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel sha512_ssse3 sha256_ssse3
Dec  2 12:44:13 Svalbard kernel: sha1_ssse3 zcommon(PO) aesni_intel znvpair(PO) crypto_simd spl(O) cryptd ecdh_generic mei_hdcp mei_pxp gigabyte_wmi wmi_bmof rapl ecc mpt3sas intel_cstate i2c_i801 intel_gtt nvme agpgart intel_uncore i2c_smbus mei_me r8169 ahci i2c_core nvme_core mei raid_class realtek libahci scsi_transport_sas syscopyarea sysfillrect sysimgblt fb_sys_fops thermal fan tpm_crb video tpm_tis tpm_tis_core wmi tpm backlight intel_pmc_core acpi_tad acpi_pad button unix
Dec  2 12:44:13 Svalbard kernel: CPU: 14 PID: 11985 Comm: unraidd0 Tainted: P      D    O       6.1.118-Unraid #1
Dec  2 12:44:13 Svalbard kernel: Hardware name: Gigabyte Technology Co., Ltd. Z790M AORUS ELITE AX/Z790M AORUS ELITE AX, BIOS F10 09/27/2024
Dec  2 12:44:13 Svalbard kernel: RIP: 0010:do_exit+0x87/0x923
Dec  2 12:44:13 Svalbard kernel: Code: 24 74 04 75 13 b8 01 00 00 00 41 89 6c 24 60 48 c1 e0 22 49 89 44 24 70 4c 89 ef e8 1f 47 81 00 48 83 bb b0 07 00 00 00 74 02 <0f> 0b 48 8b bb d8 06 00 00 e8 21 46 81 00 48 8b 83 d0 06 00 00 83
Dec  2 12:44:13 Svalbard kernel: RSP: 0018:ffffc90000bc7ee0 EFLAGS: 00010286
Dec  2 12:44:13 Svalbard kernel: RAX: 0000000080000000 RBX: ffff8881048d3000 RCX: 0000000000000000
Dec  2 12:44:13 Svalbard kernel: RDX: 0000000000000001 RSI: 0000000000002710 RDI: 00000000ffffffff
Dec  2 12:44:13 Svalbard kernel: RBP: 000000000000000b R08: 0000000000000000 R09: ffffffff829583f0
Dec  2 12:44:13 Svalbard kernel: R10: 00003fffffffffff R11: ffff8890bf7bb176 R12: ffff8881058aa400
Dec  2 12:44:13 Svalbard kernel: R13: ffff8881048dca40 R14: 0000000000000002 R15: ffffffff820b440d
Dec  2 12:44:13 Svalbard kernel: FS:  0000000000000000(0000) GS:ffff88907f380000(0000) knlGS:0000000000000000
Dec  2 12:44:13 Svalbard kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Dec  2 12:44:13 Svalbard kernel: CR2: 0000154ff45911e0 CR3: 000000022de42000 CR4: 0000000000750ee0
Dec  2 12:44:13 Svalbard kernel: PKRU: 55555554
Dec  2 12:44:13 Svalbard kernel: Call Trace:
Dec  2 12:44:13 Svalbard kernel: <TASK>
Dec  2 12:44:13 Svalbard kernel: ? __warn+0xab/0x122
Dec  2 12:44:13 Svalbard kernel: ? report_bug+0x109/0x17e
Dec  2 12:44:13 Svalbard kernel: ? do_exit+0x87/0x923
Dec  2 12:44:13 Svalbard kernel: ? handle_bug+0x41/0x6f
Dec  2 12:44:13 Svalbard kernel: ? exc_invalid_op+0x13/0x60
Dec  2 12:44:13 Svalbard kernel: ? asm_exc_invalid_op+0x16/0x20
Dec  2 12:44:13 Svalbard kernel: ? do_exit+0x87/0x923
Dec  2 12:44:13 Svalbard kernel: make_task_dead+0x11c/0x11c
Dec  2 12:44:13 Svalbard kernel: rewind_stack_and_make_dead+0x17/0x17
Dec  2 12:44:13 Svalbard kernel: RIP: 0000:0x0
Dec  2 12:44:13 Svalbard kernel: Code: Unable to access opcode bytes at 0xffffffffffffffd6.
Dec  2 12:44:13 Svalbard kernel: RSP: 0000:0000000000000000 EFLAGS: 00000000 ORIG_RAX: 0000000000000000
Dec  2 12:44:13 Svalbard kernel: RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
Dec  2 12:44:13 Svalbard kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
Dec  2 12:44:13 Svalbard kernel: RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
Dec  2 12:44:13 Svalbard kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
Dec  2 12:44:13 Svalbard kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
Dec  2 12:44:13 Svalbard kernel: </TASK>
Dec  2 12:44:13 Svalbard kernel: ---[ end trace 0000000000000000 ]---

 

I'm out of ideas......how about you?

 

Fresh diagnostics attached for your enjoyment. 

 

svalbard-diagnostics-20241202-1452.zip

install the dynamx file intregirty plugin. Though we were close with this being a weird display of a network docker db corruptions

Some parts of that current call trace are core unraid components and how the array is handled. md_mdadm

I think the dynamx file intregrity plugin can assit here in checking integrity and enforcing some md/hashing.

will review the diag tomorrow.

usualy when this happens though, its hardware failure of some sort. which is weird since you ran and confirmed other possibilities. 1/2 more ideas outside that but requires USB test unraid sytem / move to beta

As this could be a kerneal driver issues with how things may have changed.
as in example the realke nic plugin can now be removed and 6.12.14 keral driver is working again. your hardware may have chips/other kernel 6 issues.

outside of that. Id have you temp run a 3rd party os like truenas? same instability?

?do you have the mover tuner plugin and/or parity check plugin?
what are you disk setting / plugin setting for this check?

for the time being id recommend dynamx file integrity plugin and hoepfully some more insight from the daig tomorrow

It be a pain alright! But every abject failure also reveals the nature of the beast....I'm now even more sure that it is docker problem of some description as with docker off we have no issues (after a reasonable period time anyway) while when we placing the docker network under load causes the crash.... Now I should be specific here - running the Frigate dockers which are on all the time causes no problem as they ran the whole time....but this afternoon when I got home I added some work to Sonarr and Radarr and the system got to work...within literal minutes we got a crash. So if not Gluetun and friends it might be a loading thing - too much in parallel and it dies.

 

Answer to your questions:

• No, I don't have mover tuning plugin
• Yes, I do have parity check tuning plugin (more on that below)
• The NIC thing is a pain, but I figured out a way to eliminate that as a cause as I decided to disable the RTL8125 onboard LAN chip in the BIOS, reinstall the Intel I210 card (just 1Gb/s sadly), and am now running on that - doing so *completely* removes any flakey 2.5Gb/s and RTL8*** issues from the equation. I have the workload back on, and the parity check running again....and now we wait.....but.....we crashed again...sigh. Plus side - it's not the NICs and not the drivers (unless everything is super buggy).
• Had a couple more random attempt to try different things - for example no IT87 drivers in case of conflict, but then to be fair we already know that in safe mode it can crash...which always points back at docker.

On the parity tuning plugin (and the Fix Common Problems plugin) I've noticed that they both show up frequently just before a crash....sometimes a few minutes, sometimes sooner...or in the case of FCP always immediately before the crash. This could be coincidence as they are there in the logs a lot anyway...but it is a tiny bit suspicious:

Nov 30 09:51:34 Svalbard Parity Check Tuning: Automatic Correcting Parity-Check detected
Nov 30 09:57:00 Svalbard root: Fix Common Problems Version 2024.11.12
Nov 30 10:15:32 Svalbard kernel: ------------[ cut here ]------------
Nov 30 10:15:32 Svalbard kernel: kernel BUG at drivers/md/unraid.c:1617!
Nov 30 10:15:32 Svalbard kernel: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
Nov 30 10:15:32 Svalbard kernel: CPU: 12 PID: 12150 Comm: unraidd0 Tainted: P 

Nov 30 17:51:30 Svalbard Parity Check Tuning: WARNING: marker file found for both automatic and manual check P
Nov 30 17:51:30 Svalbard Parity Check Tuning: Manual Correcting Parity-Check detected
Nov 30 17:52:06 Svalbard kernel: veth2c113da: renamed from eth0
Nov 30 17:52:07 Svalbard kernel: eth0: renamed from vethab1d7b1
Nov 30 17:52:07 Svalbard kernel: python3[3182]: segfault at 4fd5d018 ip 0000000000529470 sp 00007ffcb7d20120 error 4 in python3.9[41f000+288000] likely on CPU 12 (core 24, socket 0)
Nov 30 17:52:07 Svalbard kernel: Code: 24 f0 4c 89 47 08 48 81 fe 32 0d 4b 00 0f 85 4d ff ff ff e9 4d ff ff ff f6 82 ab 00 00 00 20 75 19 4c 8b 52 68 4d 85 d2 74 10 <49> 83 7a 18 00 74 09 58 5d 41 5c e9 80 c0 09 00 48 8b 52 18 48 8b
Nov 30 17:52:09 Svalbard kernel: veth50bbc37: renamed from eth0
Nov 30 17:52:09 Svalbard kernel: eth0: renamed from veth17f069d
Nov 30 17:52:12 Svalbard kernel: vethab1d7b1: renamed from eth0
Nov 30 17:52:12 Svalbard kernel: eth0: renamed from veth1ef0dd0
Nov 30 17:52:18 Svalbard kernel: veth1ef0dd0: renamed from eth0
Nov 30 17:52:18 Svalbard kernel: eth0: renamed from veth8d2e830
Nov 30 17:52:19 Svalbard kernel: traps: python3[6280] general protection fault ip:538bd0 sp:7fff126871f0 error:0 in python3.9[41f000+288000]
Nov 30 17:52:24 Svalbard kernel: veth8d2e830: renamed from eth0
Nov 30 17:52:25 Svalbard kernel: eth0: renamed from vethca43526
Nov 30 17:52:29 Svalbard kernel: vethca43526: renamed from eth0
Nov 30 17:52:29 Svalbard kernel: usb 2-9.1: reset SuperSpeed USB device number 8 using xhci_hcd
Nov 30 17:52:29 Svalbard kernel: usb 2-9.1: LPM exit latency is zeroed, disabling LPM.
Nov 30 17:52:31 Svalbard kernel: eth0: renamed from veth98feb48
Nov 30 17:52:49 Svalbard kernel: usb 2-9.2: reset SuperSpeed USB device number 7 using xhci_hcd
Nov 30 17:52:49 Svalbard kernel: usb 2-9.2: LPM exit latency is zeroed, disabling LPM.
Nov 30 18:00:00 Svalbard root: Fix Common Problems Version 2024.11.29
Nov 30 18:26:53 Svalbard kernel: ------------[ cut here ]------------
Nov 30 18:26:53 Svalbard kernel: kernel BUG at drivers/md/unraid.c:1617!
Nov 30 18:26:53 Svalbard kernel: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
Nov 30 18:26:53 Svalbard kernel: CPU: 14 PID: 12109 Comm: unraidd0 Tainted: P           O       6.1.118-Unraid #1
  
Dec  2 19:27:05 Svalbard sSMTP[16086]: Sent mail for [email protected] (221 2.0.0 Bye) uid=0 username=root outbytes=749
Dec  2 19:31:00 Svalbard root: Fix Common Problems Version 2024.11.29
Dec  2 19:34:40 Svalbard Parity Check Tuning: Manual Correcting Parity-Check detected
Dec  2 19:34:41 Svalbard Parity Check Tuning: Manual Correcting Parity-Check: Manually resumed
Dec  2 19:36:33 Svalbard kernel: ------------[ cut here ]------------
Dec  2 19:36:33 Svalbard kernel: kernel BUG at drivers/md/unraid.c:1617!
Dec  2 19:36:33 Svalbard kernel: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
Dec  2 19:36:33 Svalbard kernel: CPU: 14 PID: 12068 Comm: unraidd0 Tainted: P           O       6.1.118-Unraid #1  
  
Dec  2 20:06:30 Svalbard unassigned.devices: Using Gateway '192.168.6.1' for Remote Shares.
Dec  2 20:06:30 Svalbard unassigned.devices: Waiting 5 secs before mounting Remote Shares...
Dec  2 20:15:00 Svalbard root: Fix Common Problems Version 2024.11.29
Dec  2 20:17:44 Svalbard Parity Check Tuning: Automatic Correcting Parity-Check detected
Dec  2 20:17:56 Svalbard kernel: ------------[ cut here ]------------
Dec  2 20:17:56 Svalbard kernel: kernel BUG at drivers/md/unraid.c:1617!
Dec  2 20:17:56 Svalbard kernel: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
Dec  2 20:17:56 Svalbard kernel: CPU: 24 PID: 12078 Comm: unraidd0 Tainted: P           O       6.1.118-Unraid #1

 

For now I've also uninstalled the parity tuning plugin...just because it eliminates another potential source.

 

Next chance I get I will probably re-instate the onboard NIC (since this isn't the cause), and I may try to move the Gluetun stack to a different NIC to get it off the main interface. Not sure how to do that exactly but I'll probably figure it out. Failing that I may just delete the Gluetun container and leave the arr's on the normal network and see if I still get a crash or not.

 

I also need to read up on the file integrity plugin - looks like I need to set it up, but I'm nearly out of time (I've been reminded I have a family).

 

So tired of this....it's mad....but I really appreciate your help! 🤓

 

svalbard-diagnostics-20241202-1946.zip

i think it has to due with yor werid docker netwrk setup

you have a lot of internal docker network bridge drops and recreating.

this is consent with a docker server starting stopping starting.. Multiple times. check individual docker logs.

 

Quote

Nov 30 17:52:07 Svalbard kernel: python3[3182]: segfault at 4fd5d018 ip 0000000000529470 sp 00007ffcb7d20120 error 4 in python3.9[41f000+288000] likely on CPU 12 (core 24, socket 0) Nov 30 17:52:07 Svalbard kernel: Code: 24 f0 4c 89 47 08 48 81 fe 32 0d 4b 00 0f 85 4d ff ff ff e9 4d ff ff ff f6 82 ab 00 00 00 20 75 19 4c 8b 52 68 4d 85 d2 74 10 <49> 83 7a 18 00 74 09 58 5d 41 5c e9 80 c0 09 00 48 8b 52 18 48 8b Nov 30 17:52:09 Svalbard kernel: veth50bbc37: renamed from eth0 Nov 30 17:52:09 Svalbard kernel: eth0: renamed from veth17f069d Nov 30 17:52:12 Svalbard kernel: vethab1d7b1: renamed from eth0 Nov 30 17:52:12 Svalbard kernel: eth0: renamed from veth1ef0dd0 Nov 30 17:52:18 Svalbard kernel: veth1ef0dd0: renamed from eth0 Nov 30 17:52:18 Svalbard kernel: eth0: renamed from veth8d2e830 Nov 30 17:52:19 Svalbard kernel: traps: python3[6280] general protection fault ip:538bd0 sp:7fff126871f0 error:0 in python3.9[41f000+288000] Nov 30 17:52:24 Svalbard kernel: veth8d2e830: renamed from eth0 Nov 30 17:52:25 Svalbard kernel: eth0: renamed from vethca43526 Nov 30 17:52:29 Svalbard kernel: vethca43526: renamed from eth0

as you have a weird vpn docker network and may be breaking internal docker stuff.
Then as consequence as the docker stops and start breaks other things as it appears your in a party sync due to unclean shutdowns.
as the arrs docker should be assigned to vpn_network 

this is my curent theory...

Otherwise for 6.12.9 since the kerneal chages and driver change for hardware. I would recommend going to v7 beta 4 to confirm and check kernel and slackware curent applications.

So my curnet recomendation is turn off all and any autostart dockers.
go to setting docker off

run with out dockers. check if disk syncs. if yes then docker issue if trace in log again form a reboot, then try betat and lets start looing at other comands like lspci and other kernal level.

since a lot has hapened to your docker setup could we / I get a docker netwrok ls command to see what potenal docker netwrok you have made and what they are connecting to?

as my theory is more on the docker network side and how you are telling other dockers to muti network connect.
Review diag and logs for other issues.
 

I'm pretty sure it is all docker related...without docker running it will stay up for 24 hours with no errors, with it running a crash is bound to happen. 

 

So my docker -ls result is this:

NETWORK ID     NAME         DRIVER    SCOPE
990ae97238ca   bridge       bridge    local
a3edf9925e01   eth0         macvlan   local
dd759eb996bd   host         host      local
c1b3265776ec   none         null      local
926fdbc61565   vpn_bridge   bridge    local

 

My Gluetun is assigned the VPN bridge, but none of my other *arr dockers are (which you imply they should be). They are currently bonded to the Gluetun docker using "--net=container:GluetunVPN". The UI for docker allocations does should some weirdness as often the arr dockers will 'report' that they are connected/mapped to another docker's details:

 

They are **ALL** configured the same yet some (e.g. Sonarr) show blank mappings, while others (e.g. qBittorent) show mappings for 192.168.6.15:5055/TCP which actually belongs to another container (Overseerr on eth0). When the system starts, or Gluetun is changed and the containers rebuild, they all come up as blank, but over time many collect a random mapping - normally that of the last container restarted or rebuilt and they always get the same. This same behaviour is reported here, so I think a bug, but not related - though worth noting they too have a similar gluetun setup. 

 

As for the log, the 'random' network messages look to be correlated with dockers being turned on and off, or rebuilt....after that it doesn't seem to happen. For example you can see a bunch of this after 00:00 when AppDataBackup runs....I've attached the syslog for you so you can the full log for several days. I don't see it happening before a crash unless it happened to be doing legitimate start/stop activities. 

 

The full list of mappings is this:

 

 

Containers are either running against Gluetun, then Gluetun is on the vpn_bridge, and everything else is set to run like this against the eth0 interface so everything has it's own IP address:

 

I can re-map all the arr dockers to connect to the VPN_Bridge, but then how do I force their traffic out via Gluetun? If they are on the bridge they are free to access the internet without going through the VPN (just like Gluetun itself).

 

Hopefully that sheds a little more light....I don't think the network is complicated necessarily....but something is definitely breaking something somewhere. Otherwise feel free to suggest an alternate arrangements to trial...

 

I have a second hardware platform coming - as soon as the part arrive (over the next week) I will assemble it and then I will have a second machine to test on which might help as I can replicate (more or less) the set up and see if it breaks.... It's expensive but if I price my time trying to get to the bottom of this there's a huge cost there too...

 

Other random things left to try:

1. Move the HBA card to another slot (there are some reports of crashing if in a PCIeX16 slot (I've tried running as X16 and X8/X8 - both crash, but I have a PCIeX4 slot I can try)
2. Remove the HBA and run off SATA and a SATA expansion card (slow but old school)

Probably pointless, but again this eliminate other potential sources of error - if the crashing still happens then the slot and HBA are both off the hook as far as guilt. 

 

So far though I'm still looking at docker....the longest period of stable operation so far has been with docker not running - so one way or another it has to be there somewhere.

 

syslog-192.168.6.2.log

I have to think on that one a bit. unraid has some restraints with docker network due to how it bridges and taps the parent interface....
I'm nost saying your setup is wrong but i'm not sure what it would look like in the end and be stable...

*this may require extra ipv4 / ipv6 forwarding...
Systemctl stuff....
?maybe vpn_brigde... shouldn't be a docker bridge it may need to be custom eth0 so gluten has its own lan ip and ports open not causing conflict?

As I'm not seeing anything wrong with that setup. this is definitely a werid mystery...
How are you getting the network to declare its contaner: gluten ? I saw a net parmenter ealier in the forum...

I may need some extra info from within the vpn bridge...
docker network inspect vpn_bridge

You may have done part of it but this is what i would do:
###############

Configure the Target Containers

Ensure the containers you want to route through GluetunVPN (e.g., Sonarr, qBittorrent) are configured with the appropriate network settings:

*while its soulds right it done't look right in the docker web ui...

 

?stop and edit docker vai user script not template... use terminal commands...
docker stop <container_name>

I've shared the outside docker conect:
docker network connect vpn_bridge <container_name>

I would run the dockers to use gluten as network none. and assign them to gluten. (This may reguire a differnet network tryp for the VPN for bridge.)
docker network disconnect <old_network_name> <container_name>

docker start <container_name>
################

So let review. this is how I would aproache this:
*isolate and direct docerker to a vpn system:

 

Stop Docker Containers

First, stop all running containers that are using vpn_bridge or might be impacted during the setup process:

docker ps -q | xargs docker stop
 

Remove the Existing vpn_bridge Network

Remove the existing vpn_bridge Docker network

*I don't want docker bridge...Don't see the need for it...
docker network rm vpn_bridge

 

Assign GluetunVPN to eth0 with a Static IP

Reconfigure GluetunVPN to use the eth0 interface on Unraid's default macvlan network with a static IP (e.g., 192.168.2.253):
docker network connect \
    --ip=192.168.2.253 \
    eth0 \
    gluetunvpn

*web ui template change to eth0 and give its own static ip simalr to unraid having a static ip

Verify that GluetunVPN is working and has the static IP by running:

docker inspect gluetunvpn | grep "IPAddress"
console in and make sure it can ping google and is using the vpn...

*Let apraoch this at the netwrok level...
*Making our own br0 network...
Create a Virtual Bridge Interface (vpnbr0)

Use Unraid's networking tools (brctl) to create a new bridge interfac

brctl addbr vpnbr0
ifconfig vpnbr0 192.168.6.1 netmask 255.255.255.0 up

^- This may need to be a user script at frist array start Not sure is custum made bridges stay...
*This was we have a isoalted interface for static IP for the ars dockers...

Next Create a New Macvlan Network for the VPN (vpn_macvlan)

Create a macvlan network tied to the vpnbr0 bridge
 

docker network create \
  --driver=macvlan \
  --subnet=192.168.6.0/24 \
  --gateway=192.168.6.1 \
  --ip-range=192.168.6.100/25 \
  -o parent=vpnbr0 \
  vpn_macvlan

docker network ls
docker network inspect vpn_macvlan

all dockers that will use the gluten vpn will then use vpn_macvlan in the template and be assigned a static ip of 192.168.6.x

*Now the magic that needs testing: (untested but this is how i would aproach it...)
We add a second network to glutenvpn docker.
*docker may need to be stoped...

docker network connect \
    --ip=192.168.6.100 \
    vpn_macvlan \
    gluetunvpn

*now gluten has eth0 internet access and can talk to 192.168.6.x network...
--Other setting and thing done to this docker may be required...
*maybe make this a lxc and install in a debain instance for maroe network controls...
docker inspect gluetunvpn | grep "IPAddress"

 

GluetunVPN should now have two IPs: one on eth0 (192.168.2.253) and one on vpn_macvlan (192.168.6.100).
 

For testing lets use qbitorrent...
Assign Other Containers to vpn_macvlan

For containers that need to use GluetunVPN, assign them to vpn_macvlan and configure their static IPs in the 192.168.6.x subnet:
*web UI make the networtk change..

docker network connect \
    --ip=192.168.6.x \
    vpn_macvlan \
    <container_name>

 

console in qbittorrent.
 

Verify Routing and Connectivity

Check connectivity and routing for the containers:

Ping GluetunVPN from a connected container

docker exec <container_name> ping 192.168.6.100
docker exec <container_name> curl ifconfig.me
 

Side notes we may need systemctl edits..
 

sysctl -w net.ipv4.ip_forward=1

This setup ensures a clean separation between Unraid’s network, GluetunVPN, and the containers using the VPN.

 

This is how i would aproace it... Some work may need to be done option wise with gluten and to inside its container for the 2 nics...

HOWEVER.
going though this process has shed some light on your current setup.
To test. change gluten current network to custom eth0 with its own static IP.

you may be haiving port conflits due to docker network bridge driver.
 

Woah! Talk about a maximum effort reply!

 

I'm on my lunch break so can't do much, but I can start to respond. On the requested docker network inspect vpn_bridge command here is the output:

root@Svalbard:~# docker network inspect vpn_bridge
[
    {
        "Name": "vpn_bridge",
        "Id": "926fdbc61565a19b0b243daa011e73acd811882afd8029733b84c74b0959bb04",
        "Created": "2024-12-01T18:29:05.828317635+13:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.18.0.0/16",
                    "Gateway": "172.18.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "af99a9fafdb41e43f04e1d5d91de8b4e3296fd0d28f92afa9a524eec1ddf4f7c": {
                "Name": "GluetunVPN",
                "EndpointID": "cdfdd65fd756d96848f44d21db7aedee231211f65a81d04f3200635180c7493a",
                "MacAddress": "02:42:ac:12:00:02",
                "IPv4Address": "172.18.0.2/16",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {}
    }
]

 

The arr dockers are configured like this:

 

So Network Type = None, and the extra parameters to bind it to Gluetun. The WebUI url is the Unraid Host (which is on the vpn_bridge which in trun hangs off Eth0. For this particular docker the UI is on 8080/TCP and that is exposed in the Gluetun docker config like so:

 

Which drills down to this:

 

All the dockers behind Gluetun are set up this way. 

 

As for GlueTun itself, also a simple config (notwithstanding all the VPN bits):

 

I will have a play with the network reconfiguration when I get home. I have no doubt that Gluetun will work fine sitting on Eth0 with it's own address (which will be in the 192.168.6.0/24 space for me). I can create an internals virtual network on a different range (your proposed .6 I can't use as that's real for me, but I can make it something else like .60 or go for a 172.x.x.x subnet). I think most of what you wrote makes sense, except that I suspect I will need to make Gluetun the .1 address, or update the subnet gateway to be .100 (based on your addressing) so that the individual dockers know where to go to get off the subnet (with your numbering there's no real gateway at .1)

 

I have had the proposed Eth0 with no bridge network before (see back here from a couple of days ago). The problem there was that the arr stack couldn't talk amongst themselves (which they need) so adding a separate network for them should solve that problem.... This means there's possibly another design - if they are all slaved off Gluetun for their external connection, then their internal connection will only be just for them (i.e. it doesn't need to include Gluetun at all, doesn't include a gateway as there's nowhere to go). 

 

So much tinkering....  🙂

ok lets me try to make a revised command with a similar ip scheme:

lets use 60.2 for unraid network interface of vpnbr0
*this may need to be a user script to make at boot.
--some docker may need a autostart with a wait of 20 seconds for networking
Advance toggle auto and a wiat in seconds 

Create vpnbr0 Bridge Interface

brctl addbr vpnbr0
ifconfig vpnbr0 192.168.60.2 netmask 255.255.255.0 up

*I think your right gluten vpn should be at 192.168.60.1 in this regard.

 

Key Details:

Gateway: 192.168.60.1 (GluetunVPN).

Subnet: 192.168.60.0/24.

IP Range: 192.168.60.3-192.168.60.50.

 

Create the Macvlan Network for vpnbr0

Create a macvlan Docker network (vpn_macvlan) using the vpnbr0 bridge

docker network create \
  --driver=macvlan \
  --subnet=192.168.60.0/24 \
  --gateway=192.168.60.1 \
  --ip-range=192.168.60.3/27 \
  -o parent=vpnbr0 \
  vpn_macvlan

 

Configure Routing

Ensure GluetunVPN routes traffic for containers on vpn_macvlan

sysctl -w net.ipv4.ip_forward=1

*i think unraid has this option enabled.

use both the extra parameter of net and set other docker like qbittorrent to custom network vpn_macvlan for unraid local cross talk traffic such as hostnames.
so the Use with the Unraid Web UI to set --net=container:GluetunVPN for containers, may still be required.
 

This Ensure that containers using vpn_macvlan only use the static IPs in 192.168.60.x to avoid conflicts and can talk to 192.168.60.x with no internet access outside of 192.168.60.1 via gluten vpn as a iproute gateway...

we can then use unraids tempalte with gluten to add the second net over docker network add comand:
 

--net=vpn_macvlan --ip=192.168.60.1

*so gluten will use custom eth0 set to a static 192.168.6.x address and the above in extra permeates.

Console in and ping around.
ping another docker, ip and google...

docker exec <container_name> ping 192.168.60.1

docker exec <container_name> curl ifconfig.me


I think unraid web ui for gluten needs template network set to vpn_maclvan with a static ip of 192.168.60.1
then use the unraid termainl to docker netowrk add to gluten to allow it internet access:
 

docker network connect --ip=192.168.6.253 eth0 gluetunvpn

*setting a static ip of 6.253 for internet access to gluten only. but setting primary netowrk to vpn_maclvan for container corss talk.

This shold fix some docker network issues.
for user script and reboot purposes.

add a startup script:
 

#!/bin/bash

# Enable IP forwarding
sysctl -w net.ipv4.ip_forward=1

# Check if vpnbr0 exists
if ! brctl show | grep -q vpnbr0; then
    echo "Creating vpnbr0 bridge..."
    brctl addbr vpnbr0
    ifconfig vpnbr0 192.168.60.2 netmask 255.255.255.0 up
else
    echo "vpnbr0 bridge already exists."
fi

# Start the GluetunVPN container
if ! docker ps --format "{{.Names}}" | grep -q gluetunvpn; then
    echo "Starting GluetunVPN container..."
    docker start gluetunvpn
else
    echo "GluetunVPN container is already running."
fi

# Attach GluetunVPN to eth0 if not already connected
if ! docker network inspect eth0 | grep -q gluetunvpn; then
    echo "Connecting GluetunVPN to eth0..."
    docker network connect --ip=192.168.6.253 eth0 gluetunvpn
else
    echo "GluetunVPN is already connected to eth0."
fi

# Attach GluetunVPN to vpn_macvlan if not already connected
if ! docker network inspect vpn_macvlan | grep -q gluetunvpn; then
    echo "Connecting GluetunVPN to vpn_macvlan..."
    docker network connect --ip=192.168.60.1 vpn_macvlan gluetunvpn
else
    echo "GluetunVPN is already connected to vpn_macvlan."
fi

echo "Startup script completed."

 

Edited December 3, 20241 yr by bmartino1
typo - data

Just got home and starting on this now....

 

Is there a reason why the new isolated LAN needs a parent (i.e. vpnbr0)?

 

Can it not just exist in complete isolation - traffic should never leave it except via Gluetun on .1...and it in turn is just sitting on the external eth0 on 192.168.6.50....

 

 

more to do with staying and using macvlan and how macvlan need a parent interface. also helps define this contain at this network at this ip.

Answering: what where and how. in terms of the network.
?in theroy a docker bridge but due to ports with a contain on macvlan we don't have to worry about port as they are assigned the static ip and all port are open to the continer for its use..

 

in this setup traffic never leave vpnbr0 as no iprouter exist off unraid. its more to send and vpn hostname traffic through unraid local net and using it as a temp bridge for macvlan

in this setup vpnbr0 done't have internet access but allows unraid docker container using it to communicate it over a virtual switch with in the docker network.

Edited December 3, 20241 yr by bmartino1
typo - data

if needed later we can then set a ip route with in unraid to send vpnbr0 traffic to 192.168.6.1
 

ip route add 192.168.60.0/24 dev vpnbr0 via 192.168.60.1

*May also need to be in the user startup scirpt

In Essen i would use unraid network and networking commands to make my own brdige to guarantee this ip this route this dns server etc.

this way future docker that need to be in that netwrok and vpn only internet i just add vpn_macvlan and set a static ip
 

I've created the new maclan:

root@Svalbard:~# docker network ls
NETWORK ID     NAME          DRIVER    SCOPE
990ae97238ca   bridge        bridge    local
a3edf9925e01   eth0          macvlan   local
dd759eb996bd   host          host      local
c1b3265776ec   none          null      local
db038511e832   vpn_macvlan   macvlan   local


root@Svalbard:~# docker network inspect vpn_macvlan 
[
    {
        "Name": "vpn_macvlan",
        "Id": "db038511e8325595f363325c550c376a1cd8fb453b4faeb9ff661d5fdb6dbdc8",
        "Created": "2024-12-03T17:01:16.423589416+13:00",
        "Scope": "local",
        "Driver": "macvlan",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "192.168.60.0/24",
                    "IPRange": "192.168.60.3/27",
                    "Gateway": "192.168.60.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {},
        "Options": {
            "parent": "vpnbr0"
        },
        "Labels": {}
    }
]

 

I cannot assign a container (i.e. Gluetun) to 192.168.60.1 as this is reserved for the "gateway". Even if I leave the gateway off the macvlan creation command the .1 is automatically reserved.

 

I can do this:

Extra Parameters:  --network=vpn_macvlan --ip=192.168.60.5

 

But this means that there is now a conflict been the subnet's default gateway (.1) and the gateway we actually need the containers to use (being the .5 now assigned to Gluetun). It is (I believe) possible to get containers to do this but for that it's a case of running form the command line or hacking a docker-compose file...

 

It also didn't seem to work having both the extra parameters for the vpn lan and the default "use eth0" settings....and somewhere in the middle there I got a truly hard crash and Unraid dropped off the network completely and needed a hard reboot to bring it back. Which in turn proved that the vpnbr0 Bridge Interface doesn't survive reboot as you suspected, which stopped Gluetun and friends from loading which probably saved me from a crash loop.

 

Anyway, after much playing around I'm finding myself stuck as I'm unable to attach the Gluetun container to both Eth0 and vpn_maclan. So....where did I land?

 

For now it seems the thing I tried a few days ago is actually working (now that I've done it properly) and it is about as simple as it is possible to be:

1. There is no bridge (at all)
2. GluetunVPN is on eth0 @ 192.168.6.50
3. Arr's are bonded with --net=container:GluetunVPN
4. Arr's are configured to talk to each other via 192.168.6.50:abcd when abcd is the port for each service.
5. Arr's *can* talk to each other, and can ping *anything* on the 192.168.6.0/24 network (which makes sense as that's their segment via GluetunVPN)
6. Arr's *cannot* ping outside the local subnet (e.g. 192.168.1.10) but they *can* ping on the Internet (e.g. 1.1.1.1) - so they have www access while still being "semi" isolated.
7. If I check my tracking on a torrent site the reported IP address matches the VPN address provided by Gluetun / NordVPN.

So, peanut-brain-me says - it is working, it is all talking to each other, there's no bridge stuff and custom scripts. So it will not get any less complex than it is now....

 

Did it crash? Hell yes. Of course it crashed. 😭

 

Rebooting now so that it all comes up completely clean as obviously there was still some bridge building and breaking in this run.

 

Do I expect it to break again? Yes. But we need to be sure. And it is not likely to take long to see if it burns.....

 

After that - start looking at the HBA slot and possibly replacing it with a SATA card. But that will become a job for tomorrow I expect...

 

Edited December 3, 20241 yr by ChirpyTurnip

It died again (like there was any doubt). Moving the HBA from the PCIeX16 to a PCIeX4 slowed it down, but it still crashes. So the card is still vaguely suspect, but the PCI slots are in the clear.

 

Using Tips and Tweaks I've disabled Flow Control / Offload on the NIC....just in case it creates a problem - not likely but another thing ticked off. Currently running a parity check with docker running but all dockers down. After so many crashes I might need to let this one run all the way through.

 

We will see if it crashes again...

 

I just wish I could figure out the common point causing Unraid to crash....docker doesn't seem to skip a beat as everythign just keeps running...but Unraid is crippled such that it cannot even reboot or halt the hardware. We've been looking hard at the network but we've changed that all around (twice) and we played musical chairs with the interfaces and nothing is making a difference. It could simply be that we are digging in the wrong place...maybe it's something else?

1. coral.ai USB adapters (I have two) interfering with Unraid USB? Not likely as Unraid shoul b in RAM I think?
2. a storage-level thing maybe? 3x M.2 NVME ZFS pool (for cache) somehow freaking out during an array parity check?
3. a weird driver / config thing?

My computer parts have started arriving, but the power supply and the HDD haven't shipped yet....so that's sub optimal.

 

Another random question, since I'm staring at the screen so long....should appdata be on the array on in the ZFS pool? I'm detecting a lot of writes to the cache as containers are still logging all the time....sometime the most stupid stuff and it's not always possible to disable that. My ZFS pool will not wear out nearly as quickly if I reserve it for file transfers and not docker files.....they're 2 months old (give or take) and have written 20TB on the 2TB drives (so x10 of a x600 duty cycle --> 10 years at this rate assuming that's constant and not getting worse as I pile on load).

 

In other news: Unraid 7 RC1 is out.....so that's nice....

Given the network reroute and issues and the HBA moving clearly affecting it could be hardware failure in the HBA. It was worth a shot.

In terms of docker cross talk.
you could try and run a dns server within it like pihole in that configuration over gluten. then use a 1 to 1 to replace the docker resolv.conf to use pihole for dns this way you have dns control within the network...

resolv.conf for dns as seen here:

Otherwises I think it would be easier to make a folder in appdata idk arrs1to1... and make a host file replacement and 1 to 1 read only file replacment in each docker to be able to cross talk then at that point....

so each docker has a /etc/host
and edit 1 file for them all.

Example hosts
192.168.6.x qtbittorrent
192.168.6.x pihole

like the resolve config 1 to 1 then test within the docker.

But this doesn't fix the constant crashing you are experiencing. Originally thought the " --net=container:GluetunVPN" may be causing these issues, but on my other test systems i'm not seeing instability.

so weird docker disk access and how unraid driver for your HBA is my  last sus component and thought.
Sorry man. I wish you luck

your correct, it is hard messing with subnets within a docker and multiple defined networks. I went with a different approach LXC in my test as I want a bit move control I don't use nord vpn I use other things like twingate docker in the test. Info above worked for me. Not sure where in the configuration process, as it is long and complicated. (lxc / vm was able to easily add two seperate interface one to the docker network vpn_macvlan and eth0...

Outside of sus HBA, you clearly state that docker is misbehaving and the system is stable if docker is not running.

we removed any possible networking thing at this point form no br0 no bond0, straight single eth0 interface. and even tried making or own br0 like interface and macvlan.
so if its not in the docker network its wiht a individual docker or docker component as you may have had network db corruption fixed earlier. a corrupt docker.img due to parity disk failure and writes?

so my last troubleshooting where i'm out of ideas is the hard drive controller, which could be motherboard / pcie card incompatibly or other hardware error with in.

with all your docker runnign you could try this script the prune and help clear/cleanup docker: