Unraid API Error - CLOUD: Client Closed Connection

[Titan84]

On my backup server im running 6.12-RC3 and i have been having this issue ever since upgrading. Something that i have noticed is that if i disable docker, this error never comes back but as soon as Docker is enabled the error returns.

 

For now im leaving Docker disabled as i dont need it on but this fixes the issue for me.

[IGHOR]

unraid-api restart and unregister/register didn't help

[Verme123]

Can confirm, something is wrong. I'm online in the forums status, however the unraid server has an API error. Restarting the unraid-api is a no-go.

[ConnerVT]

Last reply in this thread from someone associated with Limetech was April 3.

[ljm42]

Hey folks, please go to the Plugins tab and check for updates, this will upgrade you to Unraid Connect:

https://forums.unraid.net/topic/138392-unraid-connect-plugin-announcements/#comment-1255739

 

 

There have been many changes and improvements which should resolve many problems. Of course, if you continue to have issues please let us know.

[hawkcfa]

Didn't help me.  Running 6.11.5 and Unraid Connect 2023.04.25.0813 and still have the Unraid API Error Network:Unable to connect.  Server shows offline on connect.myunraid.net as well.

[ljm42]

6 hours ago, hawkcfa said:

Didn't help me.  Running 6.11.5 and Unraid Connect 2023.04.25.0813 and still have the Unraid API Error Network:Unable to connect.  Server shows offline on connect.myunraid.net as well.

 

Tell me about your network, do you have any kind of firewall that could be blocking outgoing communication from the server?

 

On the Settings -> Network Settings page, what are your DNS servers set to?

 

Also, please open a web terminal and provide the output from `unraid-api report -v`

[hawkcfa]

23 minutes ago, ljm42 said:

 

Tell me about your network, do you have any kind of firewall that could be blocking outgoing communication from the server?

 

On the Settings -> Network Settings page, what are your DNS servers set to?

 

Also, please open a web terminal and provide the output from `unraid-api report -v`

 

I do have a firewall, but have not made any changes to the firewall or any rules in several months.  Long before the API stopped working.  My DNS servers on my internal network are Pihole's but I have turned off the blocking of them to allow queues to go directly to CloudFlare or OpenDNS's DNSSEC.

 

Got another update to Unraid Connect.  Now version 2023.04.25.1448.  Restarted the API again after the new update and still no dice.

 

<-----UNRAID-API-REPORT----->
SERVER_NAME: UNRAID
ENVIRONMENT: production
UNRAID_VERSION: 6.11.5
UNRAID_API_VERSION: 3.0.1
UNRAID_API_STATUS: running
API_KEY: invalid
MY_SERVERS: authenticated
MY_SERVERS_USERNAME: hawkcfa
CLOUD: 
        STATUS: [error]  
MINI-GRAPH: 
        STATUS: [PRE_INIT]
        ERROR: [API Disconnected]
SERVERS:
        ONLINE: 
        OFFLINE: UNRAID[owner="hawkcfa"]
ALLOWED_ORIGINS: 
</----UNRAID-API-REPORT----->

 

[hawkcfa]

The Unraid server IPv6 broke.  I couldn't ping mothership.unraid.net from the server when it was using IPv6.  But I could ping mothership.unraid.net's IPv6 IP from other workstations on my network.  

 

root@UNRAID:~# ping mothership.unraid.net
PING mothership.unraid.net(2606:4700:20::681a:275 (2606:4700:20::681a:275)) 56 data bytes
From 2604:2d80:500a:6200::949 icmp_seq=1 Destination unreachable: Address unreachable
From 2604:2d80:500a:6200::949 icmp_seq=2 Destination unreachable: Address unreachable

 

Flushed the IPv6 on the interface and restarted the API

root@UNRAID:~# ip -6 addr flush dev br0
root@UNRAID:~# unraid-api restart

 

Via v4 I can get to mothership.unraid.net.  And it is showing connected and online on connect.unraid.net

 

root@UNRAID:~# ping mothership.unraid.net
PING mothership.unraid.net (172.67.69.176) 56(84) bytes of data.
64 bytes from 172.67.69.176 (172.67.69.176): icmp_seq=1 ttl=54 time=17.6 ms
64 bytes from 172.67.69.176 (172.67.69.176): icmp_seq=2 ttl=54 time=15.1 ms
64 bytes from 172.67.69.176 (172.67.69.176): icmp_seq=3 ttl=54 time=14.6 ms
64 bytes from 172.67.69.176 (172.67.69.176): icmp_seq=4 ttl=54 time=15.9 ms

--- mothership.unraid.net ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 15069ms
rtt min/avg/max/mdev = 14.644/15.812/17.627/1.133 ms

 

<-----UNRAID-API-REPORT----->
SERVER_NAME: UNRAID
ENVIRONMENT: production
UNRAID_VERSION: 6.11.5
UNRAID_API_VERSION: 3.0.1
UNRAID_API_STATUS: running
API_KEY: valid
MY_SERVERS: authenticated
MY_SERVERS_USERNAME: hawkcfa
CLOUD: 
        STATUS: [ok] 
        IP: [104.26.2.117] 
MINI-GRAPH: 
        STATUS: [CONNECTED]
SERVERS:
        ONLINE: UNRAID[owner="hawkcfa"]
        OFFLINE: 
ALLOWED_ORIGINS: http://localhost, https://localhost, http://unraid.local, http://IPV4ADDRESS, http://unraid, https://connect.myunraid.net, https://staging.connect.myunraid.net, https://dev-my.myunraid.net:4000
</----UNRAID-API-REPORT----->

 

I still can't ping outside my internal network via IPv6 from Unraid.  So it is only band-aided to connect via v4.  It still can't/won't connect via v6 because the Unraid server itself IPv6 is broken.  I have VMs on the Unraid that get IPv6 from my DHCPv6 server and they are working just fine.  So the problem is definitely with IPv6 on Unraid itself.  It really is acting like the Unraid server has lost or unable to implement the default route for v6.

[hawkcfa]

After shutting down Dockers and VMs disabling and re-enabling IPv6 then restarting the unraid-api, it came up on IPv6.

 

<-----UNRAID-API-REPORT----->
SERVER_NAME: UNRAID
ENVIRONMENT: production
UNRAID_VERSION: 6.11.5
UNRAID_API_VERSION: 3.0.1
UNRAID_API_STATUS: running
API_KEY: valid
MY_SERVERS: authenticated
MY_SERVERS_USERNAME: hawkcfa
CLOUD: 
        STATUS: [ok] 
        IP: [2606:4700:20::681a:375] 
MINI-GRAPH: 
        STATUS: [CONNECTED]
SERVERS:
        ONLINE: UNRAID[owner="hawkcfa"]
        OFFLINE: 
ALLOWED_ORIGINS: http://localhost, https://localhost, http://unraid.local, http://IPV4ADDRESS, http://unraid, https://connect.myunraid.net, https://staging.connect.myunraid.net, https://dev-my.myunraid.net:4000
</----UNRAID-API-REPORT----->

[ljm42]

30 minutes ago, hawkcfa said:

After shutting down Dockers and VMs disabling and re-enabling IPv6 then restarting the unraid-api, it came up on IPv6.

 

Glad you got it working! If you run into ipv6 issues again you might grab diagnostics and start a new thread under general support 

[ljm42]

1 hour ago, hawkcfa said:

I do have a firewall, but have not made any changes to the firewall or any rules in several months.  Long before the API stopped working.  My DNS servers on my internal network are Pihole's but I have turned off the blocking of them to allow queues to go directly to CloudFlare or OpenDNS's DNSSEC.

 

Would you do me a favor and enable logging on your firewall so you can confirm whether it is blocking traffic?

[CorruptComputer]

I've been dealing with this problem for a couple months now and I didn't realize that it was a more widespread issue. I have had this Unraid server deployed for a long time now without having any network related issues, but in March (with no changes on my end) it started acting up. However, when I enable IPv6 everything works without issue.

So what I have been doing for months now, is when I need to update my Plugins and Docker containers I will enable IPv6, update those, then disable IPv6 again. Currently on Unraid 6.11.5 and Unraid Connect version 2023.05.01.1138. Using NextDNS for my DNS with a fallback on Cloudflare as my third DNS server, and looking through my NextDNS logs nothing is being blocked that would be needed for Unraid to work properly.

 

The weirdest part of it all is that when I go to the Unraid Connect website I can see that it shows my server as online, however when I go to my server these just show as 'not available':

 

 

I've attached my servers anonymized diagnostics, if needed I can privately send an employee of Limetech the full version of it.

neptune-diagnostics-20230501-1508.zip

Edited May 1, 2023 by CorruptComputer
fix wording

[ljm42]

3 hours ago, CorruptComputer said:

So what I have been doing for months now, is when I need to update my Plugins and Docker containers I will enable IPv6, update those, then disable IPv6 again.

 

This area of the forum is for support with Unraid Connect specifically. As you are having general networking issues, please start a new thread under General Support where a lot more people will see it.

[user00265]

On 5/1/2023 at 6:21 PM, ljm42 said:

 

This area of the forum is for support with Unraid Connect specifically. As you are having general networking issues, please start a new thread under General Support where a lot more people will see it.

I would love to challenge that -- fully working IPv6 setup here through Hurricane Electric. All device that are dual-stack capable on the network work... except for my Unraid NAS.

 

I've tried everything else on this thread and nothing got the connection to work until IPv6 was disabled. As mentioned by @CorruptComputer, turning off IPv6 on the NIC gets the connection re-established. Turning it back on causes the connection to not work. This is the first device I have found with any issues in my network, so I don't believe this to be "a general networking issue" and perhaps more of the underlying cause in the specific setup.

 

Is the cloud service endpoint IPv6 capable? Is it advertising an IPv6 address? Is it responding normally like the IPv4 counterpart? Is the code able to handle either address family? Anything hard-coded that may expect IPv4 but will not handle an IPv6 address?

 

A simplistic example is the Transmission portcheck service. For a while it was working fine with IPv4 queries, but any dual-stack hosts that used the (published in DNS) IPv6 address would get a "400 Bad Request" error. I believe this was fixed this year, even.