lousek

I'm finally making progress! It works immediately after running this command on the Unraid host: echo 0 > /proc/sys/net/bridge/bridge-nf-call-iptables And it immediately stops working after setting it back to the standard value again: echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables I've also tried to add this only for the docker container, but this did not work (I did not expect it to): --sysctl net.bridge.bridge-nf-call-iptables=0 Issue is: no idea yet what else I might break when setting this to 0 on the Unraid host, and why this being 1 is an issue at all ...

That's what I've feared, but after spending hours trying to figuring out what the issue is, I thought at least it does not hurt Btw: I did some further tests: 1) On the same Unraid host, I've installed a Debian 10 VM, and inside the Debian VM inside docker, created the br0 network with the same config as on Unraid (except that the parent was enp1s0 instead of br0) and run the network-multitool docker container the same way as described above - works as it should, it always matches inbound and output rule 1! As it is the same container but a different host, I suspect Unraid (not the docker container): root@docker-test:~# tail -n0 -f /var/log/syslog | grep filter | grep TCP Apr 11 22:08:59 docker-test kernel: [ 2176.338229] TRACE: filter:OUTPUT:policy:2 IN= OUT=eth0 SRC=192.168.128.91 DST=188.114.96.22 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=46724 DF PROTO=TCP SPT=38464 DPT=80 SEQ=4059499213 ACK=0 WINDOW=64240 RES=0x00 SYN URGP=0 OPT (020405B40402080A2CBC36B60000000001030307) UID=0 GID=0 Apr 11 22:08:59 docker-test kernel: [ 2176.368612] TRACE: filter:INPUT:rule:1 IN=eth0 OUT= MAC=02:42:c0:a8:80:5b:00:0d:b9:55:17:96:08:00 SRC=188.114.96.22 DST=192.168.128.91 LEN=52 TOS=0x00 PREC=0x00 TTL=55 ID=0 DF PROTO=TCP SPT=80 DPT=38464 SEQ=1391846030 ACK=4059499214 WINDOW=65535 RES=0x00 ACK SYN URGP=0 OPT (02040578010104020103030A) UID=0 GID=0 Apr 11 22:08:59 docker-test kernel: [ 2176.368685] TRACE: filter:OUTPUT:rule:1 IN= OUT=eth0 SRC=192.168.128.91 DST=188.114.96.22 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=46725 DF PROTO=TCP SPT=38464 DPT=80 SEQ=4059499214 ACK=1391846031 WINDOW=502 RES=0x00 ACK URGP=0 UID=0 GID=0 Apr 11 22:08:59 docker-test kernel: [ 2176.368746] TRACE: filter:OUTPUT:rule:1 IN= OUT=eth0 SRC=192.168.128.91 DST=188.114.96.22 LEN=115 TOS=0x00 PREC=0x00 TTL=64 ID=46726 DF PROTO=TCP SPT=38464 DPT=80 SEQ=4059499214 ACK=1391846031 WINDOW=502 RES=0x00 ACK PSH URGP=0 UID=0 GID=0 Apr 11 22:08:59 docker-test kernel: [ 2176.404062] TRACE: filter:INPUT:rule:1 IN=eth0 OUT= MAC=02:42:c0:a8:80:5b:00:0d:b9:55:17:96:08:00 SRC=188.114.96.22 DST=192.168.128.91 LEN=40 TOS=0x00 PREC=0x00 TTL=55 ID=45058 DF PROTO=TCP SPT=80 DPT=38464 SEQ=1391846031 ACK=4059499289 WINDOW=64 RES=0x00 ACK URGP=0 UID=0 GID=0 Apr 11 22:08:59 docker-test kernel: [ 2176.568538] TRACE: filter:INPUT:rule:1 IN=eth0 OUT= MAC=02:42:c0:a8:80:5b:00:0d:b9:55:17:96:08:00 SRC=188.114.96.22 DST=192.168.128.91 LEN=652 TOS=0x00 PREC=0x00 TTL=55 ID=45059 DF PROTO=TCP SPT=80 DPT=38464 SEQ=1391846031 ACK=4059499289 WINDOW=64 RES=0x00 ACK PSH URGP=0 UID=0 GID=0 Apr 11 22:08:59 docker-test kernel: [ 2176.568587] TRACE: filter:OUTPUT:rule:1 IN= OUT=eth0 SRC=192.168.128.91 DST=188.114.96.22 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=46727 DF PROTO=TCP SPT=38464 DPT=80 SEQ=4059499289 ACK=1391846643 WINDOW=501 RES=0x00 ACK URGP=0 UID=0 GID=0 Apr 11 22:08:59 docker-test kernel: [ 2176.568748] TRACE: filter:OUTPUT:rule:1 IN= OUT=eth0 SRC=192.168.128.91 DST=188.114.96.22 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=46728 DF PROTO=TCP SPT=38464 DPT=80 SEQ=4059499289 ACK=1391846643 WINDOW=501 RES=0x00 ACK FIN URGP=0 UID=0 GID=0 Apr 11 22:08:59 docker-test kernel: [ 2176.597637] TRACE: filter:INPUT:rule:1 IN=eth0 OUT= MAC=02:42:c0:a8:80:5b:00:0d:b9:55:17:96:08:00 SRC=188.114.96.22 DST=192.168.128.91 LEN=40 TOS=0x00 PREC=0x00 TTL=55 ID=45060 DF PROTO=TCP SPT=80 DPT=38464 SEQ=1391846644 ACK=4059499290 WINDOW=64 RES=0x00 ACK URGP=0 Apr 11 22:08:59 docker-test kernel: [ 2176.804972] TRACE: filter:INPUT:rule:1 IN=eth0 OUT= MAC=02:42:c0:a8:80:5b:00:0d:b9:55:17:96:08:00 SRC=188.114.96.22 DST=192.168.128.91 LEN=40 TOS=0x00 PREC=0x00 TTL=55 ID=45061 DF PROTO=TCP SPT=80 DPT=38464 SEQ=1391846643 ACK=4059499290 WINDOW=64 RES=0x00 ACK PSH FIN URGP=0 Apr 11 22:08:59 docker-test kernel: [ 2176.805008] TRACE: filter:OUTPUT:rule:1 IN= OUT=eth0 SRC=192.168.128.91 DST=188.114.96.22 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=38464 DPT=80 SEQ=4059499290 ACK=1391846644 WINDOW=501 RES=0x00 ACK URGP=0 The only difference I see for e.g. packet 5 (the first INPUT packet after the TCP handshake) is that on the docker container running on Unraid, there is also the information about the physical interface (PHYSIN=bond0) ... otherwise it looks exactly the same. 2) I've also tried to remove all iptables rules (iptables -F) on the Unraid host - did not make a difference ...

Dear Unraid masters, I've noted a weird issue: when using iptables with conntrack inside a docker container on a macvlan network (custom bridge), the iptables rules for ESTABLISHED and RELATED do no longer match any traffic for TCP return traffic (after handshake) or for UDP return traffic in general. I've first noticed this when trying to use bubuntux/nordlynx container on a custom bridge interface (br0 or br0.1161), but it is also reproducible with the praqma/network-multitool container which is probably easier as there are no custom ip rules / routes involved etc. But both are based on alpine-linux. It seem to work fine when using normal Bridge mode (docker0) btw. I've noticed it first on Unraid 6.8.3 and then upgraded to 6.9.2 - no difference in behaviour. So my questions are basically: Why are the normal TCP packets between TCP handshake and TCP teardown not matching the INPUT rule (#1), although there is an established session? How to get it working? Thanks for all your help in advance! lousek Steps to reproduce: Run praqma/network-multitool docker container As "Network Type" select "Custom: br0" Optionally, as "Fixed IP address (optional)", define a specific IP address (does not seem to make a difference) At "Extra Parameters" fill in: --cap-add=NET_ADMIN --sysctl net.ipv4.conf.all.src_valid_mark=1 --sysctl net.ipv6.conf.all.disable_ipv6=1 --dns=1.1.1.1 (copied from the nordlynx container, except dns servers) Run bash inside docker container: docker exec -it network-multitool bash Add iptables rule to permit return traffic for established sessions: iptables -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT Add a rule to allow docker stuff and DNS: iptables -A INPUT -i lo -j ACCEPT && iptables -A INPUT -s 1.1.1.1/32 -j ACCEPT Change policy / default action of INPUT chain to DROP: iptables -P INPUT DROP Run HTTP query (should return your public IP): curl ifconfig.io Optionally: Install conntrack-tools inside docker container before step 3: apk add conntrack-tools On the Unraid host: enable logging of all nf namespaces into the host syslog: echo 1 > /proc/sys/net/netfilter/nf_log_all_netns Inside the docker container, add trace rules: iptables -t raw -I PREROUTING -j TRACE && iptables -t raw -I OUTPUT -j TRACE Inside the docker container, add a rule to permit outgoing traffic for established sessions (helpful for seing that conntrack works for outbound packets): iptables -A OUTPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT So the iptables configuration inside the network-multitool containers looks like this: bash-5.1# iptables -vnL --line-numbers Chain INPUT (policy DROP 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination 1 76 7753 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED 2 30 3030 ACCEPT all -- * * 1.1.1.1 0.0.0.0/0 3 28 1596 ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0 Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) num pkts bytes target prot opt in out source destination 1 0 0 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED bash-5.1# Or as full output of iptables-save: bash-5.1# iptables-save # Generated by iptables-save v1.8.6 on Sat Apr 9 16:13:51 2022 *raw :PREROUTING ACCEPT [88:12316] :OUTPUT ACCEPT [56:3578] -A PREROUTING -j TRACE -A OUTPUT -j TRACE COMMIT # Completed on Sat Apr 9 16:13:51 2022 # Generated by iptables-save v1.8.6 on Sat Apr 9 16:13:51 2022 *filter :INPUT DROP [1:72] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [0:0] -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A INPUT -s 1.1.1.1/32 -j ACCEPT -A INPUT -i lo -j ACCEPT -A OUTPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT COMMIT # Completed on Sat Apr 9 16:13:51 2022 # Generated by iptables-save v1.8.6 on Sat Apr 9 16:13:51 2022 *nat :PREROUTING ACCEPT [0:0] :INPUT ACCEPT [37:3770] :OUTPUT ACCEPT [101:5913] :POSTROUTING ACCEPT [140:8339] :DOCKER_OUTPUT - [0:0] :DOCKER_POSTROUTING - [0:0] -A OUTPUT -d 127.0.0.11/32 -j DOCKER_OUTPUT -A POSTROUTING -d 127.0.0.11/32 -j DOCKER_POSTROUTING -A DOCKER_OUTPUT -d 127.0.0.11/32 -p tcp -m tcp --dport 53 -j DNAT --to-destination 127.0.0.11:42583 -A DOCKER_OUTPUT -d 127.0.0.11/32 -p udp -m udp --dport 53 -j DNAT --to-destination 127.0.0.11:42566 -A DOCKER_POSTROUTING -s 127.0.0.11/32 -p tcp -m tcp --sport 42583 -j SNAT --to-source :53 -A DOCKER_POSTROUTING -s 127.0.0.11/32 -p udp -m udp --sport 42566 -j SNAT --to-source :53 COMMIT # Completed on Sat Apr 9 16:13:51 2022 bash-5.1# Now the weird thing, sometimes curl ifconfig.io works, but most of the time it fails with "Connection reset by peer": bash-5.1# curl ifconfig.io xx.xx.108.95 bash-5.1# curl ifconfig.io curl: (56) Recv failure: Connection reset by peer bash-5.1# tcpdump itself looks fine except TCP retransmissions (I've numbered the packets for easier correlation): bash-5.1# tcpdump -ni any tcpdump: data link type LINUX_SLL2 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes 1 16:14:20.446663 eth0 Out IP 192.168.128.99.60992 > 188.114.97.22.80: Flags [S], seq 1132975168, win 42340, options [mss 1460,sackOK,TS val 2787308685 ecr 0,nop,wscale 9], length 0 2 16:14:20.462695 eth0 In IP 188.114.97.22.80 > 192.168.128.99.60992: Flags [S.], seq 1825159899, ack 1132975169, win 65535, options [mss 1400,nop,nop,sackOK,nop,wscale 10], length 0 3 16:14:20.462775 eth0 Out IP 192.168.128.99.60992 > 188.114.97.22.80: Flags [.], ack 1, win 83, length 0 4 16:14:20.462884 eth0 Out IP 192.168.128.99.60992 > 188.114.97.22.80: Flags [P.], seq 1:76, ack 1, win 83, length 75: HTTP: GET / HTTP/1.1 5 16:14:20.477102 eth0 In IP 188.114.97.22.80 > 192.168.128.99.60992: Flags [.], ack 76, win 64, length 0 6 16:14:20.601025 eth0 In IP 188.114.97.22.80 > 192.168.128.99.60992: Flags [P.], seq 1:597, ack 76, win 64, length 596: HTTP: HTTP/1.1 200 OK 7 16:14:20.602559 eth0 In IP 188.114.97.22.80 > 192.168.128.99.60992: Flags [F.], seq 597, ack 76, win 64, length 0 8 16:14:20.602609 eth0 Out IP 192.168.128.99.60992 > 188.114.97.22.80: Flags [.], ack 1, win 83, options [nop,nop,sack 1 {597:598}], length 0 9 16:14:20.627440 eth0 In IP 188.114.97.22.80 > 192.168.128.99.60992: Flags [P.], seq 1:597, ack 76, win 64, length 596: HTTP: HTTP/1.1 200 OK 10 16:14:20.849498 eth0 In IP 188.114.97.22.80 > 192.168.128.99.60992: Flags [P.], seq 1:597, ack 76, win 64, length 596: HTTP: HTTP/1.1 200 OK 11 16:15:21.128343 eth0 Out IP 192.168.128.99.60992 > 188.114.97.22.80: Flags [.], ack 1, win 83, options [nop,nop,sack 1 {597:598}], length 0 12 16:15:21.158614 eth0 In IP 188.114.97.22.80 > 192.168.128.99.60992: Flags [R], seq 1825159900, win 0, length 0 But when looking at the iptables TRACE log, it seems that for some reason the normal packets (#5-6, 9-11) are not matching the INPUT rule 1 (which would ACCEPT the packet) but only the default action (DROP). The TCP control packets (SYN, SYN+ACK, ACK, FIN+ACK, RST) are handled correctly. root@nas01:~# tail -n0 -f /var/log/syslog | grep filter # -> Handshake SYN packet matching the OUTPUT chain policy / default action (ACCEPT) - OK 1 Apr 9 18:14:20 nas01 kernel: TRACE: filter:OUTPUT:policy:2 IN= OUT=eth0 SRC=192.168.128.99 DST=188.114.97.22 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=17584 DF PROTO=TCP SPT=60992 DPT=80 SEQ=1132975168 ACK=0 WINDOW=42340 RES=0x00 SYN URGP=0 OPT (020405B40402080AA622F48D0000000001030309) UID=0 GID=0 # <- Handshake SYN+ACK packet matching rule 1 in the INPUT chain (for RELATED & ESTABLISHED sessions) - OK 2 Apr 9 18:14:20 nas01 kernel: TRACE: filter:INPUT:rule:1 IN=eth0 OUT= PHYSIN=bond0 MAC=02:42:c0:a8:80:63:00:0d:b9:55:17:96:08:00 SRC=188.114.97.22 DST=192.168.128.99 LEN=52 TOS=0x00 PREC=0x00 TTL=54 ID=0 DF PROTO=TCP SPT=80 DPT=60992 SEQ=1825159899 ACK=1132975169 WINDOW=65535 RES=0x00 ACK SYN URGP=0 OPT (02040578010104020103030A) UID=0 GID=0 # -> Handshake ACK packet matching rule 1 in the OUTPUT chain (for RELATED & ESTABLISHED sessions) - OK 3 Apr 9 18:14:20 nas01 kernel: TRACE: filter:OUTPUT:rule:1 IN= OUT=eth0 SRC=192.168.128.99 DST=188.114.97.22 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=17585 DF PROTO=TCP SPT=60992 DPT=80 SEQ=1132975169 ACK=1825159900 WINDOW=83 RES=0x00 ACK URGP=0 UID=0 GID=0 # -> HTTP request packet matching rule 1 in the OUTPUT chain (for RELATED & ESTABLISHED sessions) - OK 4 Apr 9 18:14:20 nas01 kernel: TRACE: filter:OUTPUT:rule:1 IN= OUT=eth0 SRC=192.168.128.99 DST=188.114.97.22 LEN=115 TOS=0x00 PREC=0x00 TTL=64 ID=17586 DF PROTO=TCP SPT=60992 DPT=80 SEQ=1132975169 ACK=1825159900 WINDOW=83 RES=0x00 ACK PSH URGP=0 UID=0 GID=0 # <- ACK for handshake matching the INPUT chain policy / default action (DROP) instead of the INPUT rule 1 (for RELATED & ESTABLISHED sessions) - NOT OK 5 Apr 9 18:14:20 nas01 kernel: TRACE: filter:INPUT:policy:4 IN=eth0 OUT= PHYSIN=bond0 MAC=02:42:c0:a8:80:63:00:0d:b9:55:17:96:08:00 SRC=188.114.97.22 DST=192.168.128.99 LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=45466 DF PROTO=TCP SPT=80 DPT=60992 SEQ=1825159900 ACK=1132975244 WINDOW=64 RES=0x00 ACK URGP=0 UID=0 GID=0 # <- HTTP reply matching the INPUT chain policy / default action (DROP) instead of the INPUT rule 1 (for RELATED & ESTABLISHED sessions) - NOT OK 6 Apr 9 18:14:20 nas01 kernel: TRACE: filter:INPUT:policy:4 IN=eth0 OUT= PHYSIN=bond0 MAC=02:42:c0:a8:80:63:00:0d:b9:55:17:96:08:00 SRC=188.114.97.22 DST=192.168.128.99 LEN=636 TOS=0x00 PREC=0x00 TTL=54 ID=45467 DF PROTO=TCP SPT=80 DPT=60992 SEQ=1825159900 ACK=1132975244 WINDOW=64 RES=0x00 ACK PSH URGP=0 UID=0 GID=0 # <- FIN+ACK matching the rule 1 in the INPUT chain (for RELATED & ESTABLISHED sessions) - OK 7 Apr 9 18:14:20 nas01 kernel: TRACE: filter:INPUT:rule:1 IN=eth0 OUT= PHYSIN=bond0 MAC=02:42:c0:a8:80:63:00:0d:b9:55:17:96:08:00 SRC=188.114.97.22 DST=192.168.128.99 LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=45468 DF PROTO=TCP SPT=80 DPT=60992 SEQ=1825160496 ACK=1132975244 WINDOW=64 RES=0x00 ACK FIN URGP=0 UID=0 GID=0 # -> Retransmission of the handshake ACK (packet 3) with selective ACK for the FIN+ACK (packet 7) - matching rule 1 in the OUTPUT chain (for RELATED & ESTABLISHED sessions) - OK 8 Apr 9 18:14:20 nas01 kernel: TRACE: filter:OUTPUT:rule:1 IN= OUT=eth0 SRC=192.168.128.99 DST=188.114.97.22 LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=17587 DF PROTO=TCP SPT=60992 DPT=80 SEQ=1132975244 ACK=1825159900 WINDOW=83 RES=0x00 ACK URGP=0 OPT (0101050A6CC9BD306CC9BD31) UID=0 GID=0 # <- Retransmission of the HTTP reply (packet 6) - matching the INPUT chain policy / default action (DROP) instead of the INPUT rule 1 (for RELATED & ESTABLISHED sessions) - NOT OK 9 Apr 9 18:14:20 nas01 kernel: TRACE: filter:INPUT:policy:4 IN=eth0 OUT= PHYSIN=bond0 MAC=02:42:c0:a8:80:63:00:0d:b9:55:17:96:08:00 SRC=188.114.97.22 DST=192.168.128.99 LEN=636 TOS=0x00 PREC=0x00 TTL=54 ID=45469 DF PROTO=TCP SPT=80 DPT=60992 SEQ=1825159900 ACK=1132975244 WINDOW=64 RES=0x00 ACK PSH URGP=0 UID=0 GID=0 # <- Retransmission of the HTTP reply (packet 6) - matching the INPUT chain policy / default action (DROP) instead of the INPUT rule 1 (for RELATED & ESTABLISHED sessions) - NOT OK 10 Apr 9 18:14:20 nas01 kernel: TRACE: filter:INPUT:policy:4 IN=eth0 OUT= PHYSIN=bond0 MAC=02:42:c0:a8:80:63:00:0d:b9:55:17:96:08:00 SRC=188.114.97.22 DST=192.168.128.99 LEN=636 TOS=0x00 PREC=0x00 TTL=54 ID=45470 DF PROTO=TCP SPT=80 DPT=60992 SEQ=1825159900 ACK=1132975244 WINDOW=64 RES=0x00 ACK PSH URGP=0 UID=0 GID=0 # -> Retransmission of the selective ACK (packet 8) - matching the policy / default action (ACCEPT) instead of the OUTPUT rule 1 (for RELATED & ESTABLISHED sessions) - NOT OK I think 11 Apr 9 18:15:21 nas01 kernel: TRACE: filter:OUTPUT:policy:2 IN= OUT=eth0 SRC=192.168.128.99 DST=188.114.97.22 LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=17588 DF PROTO=TCP SPT=60992 DPT=80 SEQ=1132975243 ACK=1825159900 WINDOW=83 RES=0x00 ACK URGP=0 OPT (0101050A6CC9BD306CC9BD31) UID=0 GID=0 # <- RST from the server matching the rule 1 in the INPUT chain (for RELATED & ESTABLISHED sessions) - OK 12 Apr 9 18:15:21 nas01 kernel: TRACE: filter:INPUT:rule:1 IN=eth0 OUT= PHYSIN=bond0 MAC=02:42:c0:a8:80:63:00:0d:b9:55:17:96:08:00 SRC=188.114.97.22 DST=192.168.128.99 LEN=40 TOS=0x00 PREC=0x00 TTL=55 ID=0 DF PROTO=TCP SPT=80 DPT=60992 SEQ=1825159900 ACK=0 WINDOW=0 RES=0x00 RST URGP=0 UID=0 GID=0 Conntrack inside the Docker container shows the session (but already as CLOSE_WAIT which seems to be fine due to the FIN+ACK packet (#7): bash-5.1# conntrack -L tcp 6 56 CLOSE_WAIT src=192.168.128.99 dst=188.114.97.22 sport=60992 dport=80 src=188.114.97.22 dst=192.168.128.99 sport=80 dport=60992 [ASSURED] mark=0 use=1 udp 17 26 src=192.168.128.99 dst=1.1.1.1 sport=42906 dport=53 [UNREPLIED] src=1.1.1.1 dst=192.168.128.99 sport=53 dport=42906 mark=0 use=1 udp 17 26 src=192.168.128.99 dst=1.1.1.1 sport=47285 dport=53 [UNREPLIED] src=1.1.1.1 dst=192.168.128.99 sport=53 dport=47285 mark=0 use=1 udp 17 26 src=127.0.0.1 dst=127.0.0.11 sport=36611 dport=53 src=127.0.0.11 dst=127.0.0.1 sport=42566 dport=36611 [ASSURED] mark=0 use=1 conntrack v1.4.6 (conntrack-tools): 4 flow entries have been shown. For the seldom cases where it actualyl works, it seems that the web server actually sends a FIN+PSH+ACK (so basically an optimization to provide the result and initiating the TCP teardown in one go), which is then matching correctly the INPUT rule 1 (I guess because it's a control packet). I guess this is more a coincidence than really a hint; afterwards there are still some TCP retransmits because e.g. the incoming ACKs are not matching the established session: tcpdump: 00:30:08.007016 eth0 In IP 188.114.96.22.80 > 192.168.128.99.48872: Flags [FP.], seq 1:617, ack 76, win 64, length 616: HTTP: HTTP/1.1 200 OK iptables trace: Apr 10 02:30:08 nas01 kernel: TRACE: filter:INPUT:rule:1 IN=eth0 OUT= PHYSIN=bond0 MAC=02:42:c0:a8:80:63:00:0d:b9:55:17:96:08:00 SRC=188.114.96.22 DST=192.168.128.99 LEN=656 TOS=0x00 PREC=0x00 TTL=54 ID=18624 DF PROTO=TCP SPT=80 DPT=48872 SEQ=2154576658 ACK=659325680 WINDOW=64 RES=0x00 ACK PSH FIN URGP=0 UID=0 GID=0 connrack: tcp 6 27 LAST_ACK src=192.168.128.99 dst=188.114.96.22 sport=48872 dport=80 src=188.114.96.22 dst=192.168.128.99 sport=80 dport=48872 [ASSURED] mark=0 use=1

The correct command inside the docker container would be /usr/share/elasticsearch/bin/elasticsearch-plugin install --batch ingest-attachment ... is there an easy possibility in unRAID to run this command after the Docker container has started?