Unraid server not reachable via WebGUI or SSH from time to time - Unraid v6.8.3

[JonasGovaerts]

Hi Forum

 

Strange issue happening over here. From time to time when I want to access the Unraid WebGUI or the terminal I'm getting connection refused.

When checking connectivity, I can see it's giving connection refused for about a minute until it will accept traffic again.

 

Viewing the logs, I can see something about a veth (docker network interface) going into a blocked state, but this shouldn't impact the connectivity in my opinion.

Connection towards VMs running on Unraid is working perfectly fine as well.

 

Any idea what could be going on?

 

Attached diagnostics as well.

 

FYI, veth2274d2b is a sonarr container.

Sep 15 21:21:27 NAS-Jonas kernel: docker0: port 11(veth2274d2b) entered blocking state
Sep 15 21:21:27 NAS-Jonas kernel: docker0: port 11(veth2274d2b) entered forwarding state
Sep 15 21:21:27 NAS-Jonas kernel: docker0: port 11(veth2274d2b) entered disabled state
Sep 15 21:21:27 NAS-Jonas kernel: usb 5-2: reset full-speed USB device number 2 using xhci_hcd
Sep 15 21:21:27 NAS-Jonas kernel: usb 3-4: reset low-speed USB device number 3 using xhci_hcd
Sep 15 21:21:28 NAS-Jonas webGUI: Successful login user root from 192.168.0.137
Sep 15 21:21:28 NAS-Jonas kernel: usb 3-3: reset high-speed USB device number 2 using xhci_hcd
Sep 15 21:21:28 NAS-Jonas kernel: eth0: renamed from vethc43b009
Sep 15 21:21:28 NAS-Jonas kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth2274d2b: link becomes ready
Sep 15 21:21:28 NAS-Jonas kernel: docker0: port 11(veth2274d2b) entered blocking state
Sep 15 21:21:28 NAS-Jonas kernel: docker0: port 11(veth2274d2b) entered forwarding state
Sep 15 21:21:28 NAS-Jonas rc.docker: sonarr: started succesfully!
Sep 15 21:21:29 NAS-Jonas avahi-daemon[11712]: Joining mDNS multicast group on interface vethd86e111.IPv6 with address fe80::f4af:d9ff:fe82:e62c.
Sep 15 21:21:29 NAS-Jonas avahi-daemon[11712]: New relevant interface vethd86e111.IPv6 for mDNS.
Sep 15 21:21:29 NAS-Jonas avahi-daemon[11712]: Registering new address record for fe80::f4af:d9ff:fe82:e62c on vethd86e111.*.
Sep 15 21:21:30 NAS-Jonas rc.docker: vlmcsd: started succesfully!
Sep 15 21:21:30 NAS-Jonas avahi-daemon[11712]: Joining mDNS multicast group on interface veth2274d2b.IPv6 with address fe80::28c9:53ff:fec2:8d7c.
Sep 15 21:21:30 NAS-Jonas avahi-daemon[11712]: New relevant interface veth2274d2b.IPv6 for mDNS.
Sep 15 21:21:30 NAS-Jonas avahi-daemon[11712]: Registering new address record for fe80::28c9:53ff:fec2:8d7c on veth2274d2b.*.
Sep 15 21:21:36 NAS-Jonas kernel: BTRFS critical (device sdb1): corrupt leaf: root=5 block=481189888000 slot=7 ino=11841754, dir item with invalid data len, have 32 expect 0
Sep 15 21:21:36 NAS-Jonas kernel: BTRFS critical (device sdb1): corrupt leaf: root=5 block=481189888000 slot=7 ino=11841754, dir item with invalid data len, have 32 expect 0
Sep 15 21:21:36 NAS-Jonas kernel: BTRFS critical (device sdb1): corrupt leaf: root=5 block=481189888000 slot=7 ino=11841754, dir item with invalid data len, have 32 expect 0
Sep 15 21:21:36 NAS-Jonas kernel: BTRFS critical (device sdb1): corrupt leaf: root=5 block=481189888000 slot=7 ino=11841754, dir item with invalid data len, have 32 expect 0
Sep 15 21:21:36 NAS-Jonas kernel: BTRFS critical (device sdb1): corrupt leaf: root=5 block=481189888000 slot=7 ino=11841754, dir item with invalid data len, have 32 expect 0
Sep 15 21:21:54 NAS-Jonas sshd[19841]: Accepted password for root from 192.168.0.101 port 64155 ssh2
Sep 15 21:22:34 NAS-Jonas webGUI: Successful login user root from 192.168.0.1
Sep 15 21:26:01 NAS-Jonas sshd[21566]: error: kex_exchange_identification: Connection closed by remote host
Sep 15 21:26:07 NAS-Jonas ntpd[2028]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized
Sep 15 21:31:00 NAS-Jonas root: Fix Common Problems Version 2020.08.02
Sep 15 21:31:01 NAS-Jonas root: Fix Common Problems: Warning: Share appdata set to cache-only, but files / folders exist on the array
Sep 15 21:31:01 NAS-Jonas root: Fix Common Problems: Warning: Share Dockerconfig set to cache-only, but files / folders exist on the array
Sep 15 21:31:01 NAS-Jonas root: Fix Common Problems: Warning: Share domains set to not use the cache, but files / folders exist on the cache drive
Sep 15 21:31:08 NAS-Jonas root: Fix Common Problems: Warning: Template URL for docker application ApacheGuacamole is missing.
Sep 15 21:31:08 NAS-Jonas root: Fix Common Problems: Warning: Template URL for docker application EmbyServer is missing.
Sep 15 21:31:08 NAS-Jonas root: Fix Common Problems: Warning: Template URL for docker application pihole is missing.
Sep 15 21:31:08 NAS-Jonas root: Fix Common Problems: Warning: Template URL for docker application swag is not the as what the template author specified.
Sep 15 21:31:09 NAS-Jonas root: Fix Common Problems: Other Warning: Unassigned Devices Plus not installed
Sep 15 21:31:09 NAS-Jonas root: Fix Common Problems: Other Warning: Background notifications not enabled

{"timestamp":"15-09-2020T15:34:05","message":"Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 192.168.0.100:8085. Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds."}
{"timestamp":"15-09-2020T15:34:20","message":"Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 192.168.0.100:8085. Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds."}
{"timestamp":"15-09-2020T15:34:35","message":"Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 192.168.0.100:8085. Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds."}
{"timestamp":"15-09-2020T15:34:50","message":"Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 192.168.0.100:8085. Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds."}
{"timestamp":"15-09-2020T15:35:05","message":"Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 192.168.0.100:8085. Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds."}
{"timestamp":"15-09-2020T15:35:20","message":"Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 192.168.0.100:8085. Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds."}
{"timestamp":"15-09-2020T15:35:35","message":"Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 192.168.0.100:8085. Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds."}
{"timestamp":"15-09-2020T15:35:50","message":"Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connection refused."}
{"timestamp":"15-09-2020T15:36:05","message":"Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connection refused."}
{"timestamp":"15-09-2020T15:36:20","message":"Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connection refused."}
{"timestamp":"15-09-2020T15:36:35","message":"Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connection refused."}
{"timestamp":"15-09-2020T15:36:50","message":"Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connection refused."}
{"timestamp":"15-09-2020T15:37:05","message":"Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 192.168.0.100:8085. Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds."}
{"timestamp":"15-09-2020T15:37:20","message":"Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 192.168.0.100:8085. Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds."}
{"timestamp":"15-09-2020T15:37:35","message":"Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 192.168.0.100:8085. Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds."}
{"timestamp":"15-09-2020T15:37:50","message":"Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 192.168.0.100:8085. Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds."}
{"timestamp":"15-09-2020T15:38:05","message":"Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 192.168.0.100:8085. Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds."}
{"timestamp":"15-09-2020T15:38:20","message":"Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 192.168.0.100:8085. Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds."}
{"timestamp":"15-09-2020T15:38:35","message":"Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 192.168.0.100:8085. Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds."}

Kind regards

Jonas

nas-jonas-diagnostics-20200915-2139.zip

[Squid]

Those messages are normal for when the apps are starting up.

 

This however

Sep 15 21:21:22 NAS-Jonas kernel: BTRFS critical (device sdb1): corrupt leaf: root=5 block=481189888000 slot=7 ino=11841754, dir item with invalid data len, have 32 expect 0

is more worrisome, and should have @JorgeB advise

 

Sep 15 21:26:01 NAS-Jonas sshd[21566]: error: kex_exchange_identification: Connection closed by remote host

This I would guess is the probably cause of what you're actually seeing.  Best I can offer is to delete the contents of /config/ssh and /config/ssl on the flash drive and reboot to regenerate the keys

[JorgeB]

9 hours ago, Squid said:

This however

That is a corrupt cache filesystem, best bet is to backup and re-format.

 

RAM is overclocked for current config, see here, it's a known issue with Ryzen and in some cases a reason for data corruption.