njdowdy Posted April 8, 2021 Posted April 8, 2021 (edited) SOLVED: Turned out to be address via a later firmware update to the router. Hello! Thanks in advance for checking out my question. TLDR; I installed a new router and now my plugins and community applications pages are very slow to load and pulling new docker containers does not work / takes a very long time. I tried many steps to resolve, but the issue persists. Before yesterday I was running my unraid server (version 6.9.0-rc2) on my home network. I purchased a new router (QNAP QHora-301W w/ up-to-date firmware) and plugged my unraid server into a 10Gbit LAN port. Nothing else was changed. I updated the LAN IP range in the router (192.168.100.X > 10.20.30.X), set my unraid server to the same static IP it was on prior to the new router (10.20.30.222; managed by the router), and set DNS to 1.1.1.1 (was 8.8.8.8 before). I did NOT bring over my old port forwarding mappings, because I felt I could use a fresh start on that (I probably had more open ports than I really needed). Everything ISP-related stayed exactly the same. I access some docker containers via the internet using Nginx Proxy Manager (NPM), and so I did forward port 80 > 1880 and 443 > 18443. At this point, everything on the server is seemingly working great. However, I tried to log into the NPM web GUI and discovered that I could not, with the error of "Bad Gateway". I tried to Google that error and didn't find any good leads. I figured I'd just backup my appdata folder for NPM, delete the docker container, and start fresh to see if that fixed anything. I then went to Community Applications (CA) and tried to re-download the NPM container. CA (and also the "Plugins" tab) was much slower than usual to load. I also could not pull down any containers (NPM or anything else). These were the errors from Tools>Diagnostics: time="2021-04-07T08:39:55.878436185-07:00" level=error msg="Download failed, retrying: read tcp 10.20.30.222:33386->104.18.123.25:443: read: connection timed out" time="2021-04-07T08:39:55.878487095-07:00" level=error msg="Download failed, retrying: read tcp 10.20.30.222:33382->104.18.123.25:443: read: connection timed out" time="2021-04-07T08:39:55.878441744-07:00" level=error msg="Not continuing with pull after error: error pulling image configuration: read tcp 10.20.30.222:33384->104.18.123.25:443: read: connection timed out" time="2021-04-07T08:39:56.390430850-07:00" level=error msg="Download failed, retrying: read tcp 10.20.30.222:33392->104.18.123.25:443: read: connection timed out" time="2021-04-07T08:40:49.127467179-07:00" level=error msg="Not continuing with pull after error: error pulling image configuration: read tcp 10.20.30.222:56906->104.18.125.25:443: read: connection timed out" time="2021-04-07T08:40:57.830529626-07:00" level=error msg="Not continuing with pull after error: error pulling image configuration: read tcp 10.20.30.222:56916->104.18.125.25:443: read: connection timed out" time="2021-04-07T08:41:50.054561692-07:00" level=error msg="Not continuing with pull after error: error pulling image configuration: read tcp 10.20.30.222:46292->104.18.124.25:443: read: connection timed out" And on the docker pull page from community applications: IMAGE ID [1610037955]: Pulling from jlesage/nginx-proxy-manager. IMAGE ID [df20fa9351a1]: Already exists. IMAGE ID [c29f2a9687c5]: Already exists. IMAGE ID [f2b10fbfc380]: Already exists. IMAGE ID [529722c2e3cf]: Already exists. IMAGE ID [f0cf5f38d987]: Already exists. IMAGE ID [21fb739242f4]: Already exists. IMAGE ID [b17e90563eea]: Pulling fs layer. Downloading 92% of 532 KB. IMAGE ID [bff4d859ae50]: Pulling fs layer. Downloading 67% of 4 MB. IMAGE ID [fd1567abff3c]: Pulling fs layer. Downloading 92% of 34 MB. IMAGE ID [90a3e8820aa7]: Pulling fs layer. IMAGE ID [4704d454c63e]: Pulling fs layer. IMAGE ID [138ba29e4057]: Pulling fs layer. It is strange to me that some data is coming through (92% of 34MB), but not enough! Those downloads stall at those percentages after about 5-10 seconds. After about 1 minute elapses, this appears: root@localhost:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name='NginxProxyManager' --net='bridge' -e TZ="America/Los_Angeles" -e HOST_OS="Unraid" -e 'USER_ID'='99' -e 'GROUP_ID'='100' -e 'UMASK'='000' -e 'APP_NICENESS'='' -e 'DISABLE_IPV6'='0' -p '7818:8181/tcp' -p '1880:8080/tcp' -p '18443:4443/tcp' -v '/mnt/user/appdata/NginxProxyManager':'/config':'rw' 'jlesage/nginx-proxy-manager' Please wait ... Unable to find image 'jlesage/nginx-proxy-manager:latest' locally latest: Pulling from jlesage/nginx-proxy-manager df20fa9351a1: Already exists c29f2a9687c5: Already exists f2b10fbfc380: Already exists 529722c2e3cf: Already exists f0cf5f38d987: Already exists 21fb739242f4: Already exists b17e90563eea: Pulling fs layer bff4d859ae50: Pulling fs layer fd1567abff3c: Pulling fs layer 90a3e8820aa7: Pulling fs layer 4704d454c63e: Pulling fs layer 138ba29e4057: Pulling fs layer 4704d454c63e: Waiting 90a3e8820aa7: Waiting 138ba29e4057: Waiting And then we wait for a long, long time. These are the abbreviated steps I took next, more or less in this order: Unraid is able to ping Google just fine All docker containers (except NPM's Bad Gateway) are working perfectly fine via local ports (but not outside network, because NPM is gone now) Checked "Fix Common Problems" and removed preclear plugin (probably unrelated); no other problems show up; Problem persists Googled around and determined it might be a DNS issue; tried openDNS (208.67.222.222), Google DNS server (8.8.8.8), as well as a different CloudFlare DNS (1.0.0.1) all set via the router (unraid set to get automatically); Problem persists Checked that Docker Hub was online and functional Checked that GitHub was online and functional Shutdown server, shutdown router, started router, started server; Problem persists Turned off port forwarding of 80 and 443 to nginx ports (since the docker container is gone) Shutdown server, shutdown router, started router, started server; Problem persists Opened up some high-valued ports near those that I saw referenced in the docker diagnostics (~30000-50000); Problem persists Switch to a 1Gbit port on router (was 10Gbit port); Problem persists Settings > Docker > Scrub docker btrfs with correct errors checked. No errors found.; Problem persists Reboot server in GUI Safe Mode, installed CA, try again; Problem persists In GUI Safe Mode, I decided to just leave the docker container download going for a long time (~1 hour). I came back to this: root@localhost:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name='NginxProxyManager' --net='bridge' -e TZ="America/Los_Angeles" -e HOST_OS="Unraid" -e 'USER_ID'='99' -e 'GROUP_ID'='100' -e 'UMASK'='000' -e 'APP_NICENESS'='' -e 'DISABLE_IPV6'='0' -p '7818:8181/tcp' -p '1880:8080/tcp' -p '18443:4443/tcp' -v '/mnt/user/appdata/NginxProxyManager':'/config':'rw' 'jlesage/nginx-proxy-manager' Please wait . Unable to find image 'jlesage/nginx-proxy-manager:latest' locally latest: Pulling from jlesage/nginx-proxy-manager df20fa9351a1: Already exists c29f2a9687c5: Already exists f2b10fbfc380: Already exists 529722c2e3cf: Already exists f0cf5f38d987: Already exists 21fb739242f4: Already exists b17e90563eea: Pulling fs layer bff4d859ae50: Pulling fs layer fd1567abff3c: Pulling fs layer 90a3e8820aa7: Pulling fs layer 4704d454c63e: Pulling fs layer 138ba29e4057: Pulling fs layer 138ba29e4057: Waiting 90a3e8820aa7: Waiting 4704d454c63e: Waiting b17e90563eea: Retrying in 5 seconds bff4d859ae50: Retrying in 5 seconds b17e90563eea: Retrying in 4 seconds bff4d859ae50: Retrying in 4 seconds b17e90563eea: Retrying in 3 seconds bff4d859ae50: Retrying in 3 seconds fd1567abff3c: Retrying in 5 seconds b17e90563eea: Retrying in 2 seconds bff4d859ae50: Retrying in 2 seconds fd1567abff3c: Retrying in 4 seconds b17e90563eea: Retrying in 1 second bff4d859ae50: Retrying in 1 second fd1567abff3c: Retrying in 3 seconds fd1567abff3c: Retrying in 2 seconds bff4d859ae50: Download complete b17e90563eea: Verifying Checksum b17e90563eea: Download complete b17e90563eea: Pull complete bff4d859ae50: Pull complete fd1567abff3c: Retrying in 1 second fd1567abff3c: Download complete 138ba29e4057: Verifying Checksum 138ba29e4057: Download complete fd1567abff3c: Pull complete and in docker logs via diagnostics: time="2021-04-07T18:43:31.729223056-07:00" level=error msg="Download failed, retrying: read tcp 10.20.30.222:43780->104.18.124.25:443: read: connection timed out" time="2021-04-07T18:43:31.729239234-07:00" level=error msg="Download failed, retrying: read tcp 10.20.30.222:43782->104.18.124.25:443: read: connection timed out" time="2021-04-07T18:43:34.289240096-07:00" level=error msg="Download failed, retrying: read tcp 10.20.30.222:43784->104.18.124.25:443: read: connection timed out" time="2021-04-07T18:48:39.953230819-07:00" level=error msg="Download failed, retrying: read tcp 10.20.30.222:43930->104.18.124.25:443: read: connection timed out" time="2021-04-07T18:48:40.978206276-07:00" level=error msg="Download failed, retrying: read tcp 10.20.30.222:43928->104.18.124.25:443: read: connection timed out" Amazingly, the docker container is installed at this point. I guess it constantly tried different ports until it could make a connection to 104.18.124.25:443? That IP address (104.18.124.25:443) appears to be a cloudflare server, not sure if there's significance to that, or that's just what docker hub uses. However, NPM still displays the "Bad Gateway" error from before (even with new appdata path). So, I got my short-term problem of getting access to my containers via the internet fixed after re-forwarding ports for NPM. However, I am unable to install new docker containers (at least in a reasonable amount of time), I probably can't update them (not tested yet), nor add new subdomains to NPM (due to "Bad Gateway" problem). I don't see much in the NPM logs except: [4/7/2021] [8:03:10 PM] [Migrate ] › ℹ info Current database version: none Nothing else on my network is having any issues with this new router. What have I missed on the Unraid side? Thanks so much!! Edited April 14, 2021 by njdowdy wrote the NPM port wrong, SOLVED Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.