Nginx authlimit errors filling up the log

[esaru]

The last week my log keeps filling to 100% with these errors:

 

Apr 18 04:30:10 Tower nginx: 2021/04/18 04:30:10 [error] 6058#6058: *800608 limiting requests, excess: 20.366 by zone "authlimit", client: 192.168.3.16, server: , request: "PROPFIND /login HTTP/2.0",

 

I have no idea how to troubleshoot the issue and would love some guidance. I tried searching the forums and found a few others mentioning the same issue but no solution has been posted. 

[ljm42]

The computer with IP 192.168.3.16 is repeatedly trying to connect to your server. Possibly a stale version of the webgui open, find that computer and close the browser.

[esaru]

12 hours ago, ljm42 said:

The computer with IP 192.168.3.16 is repeatedly trying to connect to your server. Possibly a stale version of the webgui open, find that computer and close the browser.

Right, I forgot to mention this IP belongs to my workstation which is a VM on the server. I made sure to close all tabs related to unraid, but it seems the errors continue anyway. I use the VM pretty much all day and I like to keep my browser open. I guess closing the browser completely or powering down the VM could be a step for troubleshooting but it won't really fix the issue, no? I am positive that the error occurs regardless of webgui tab being closed or open and logged in to. 

 

Further assistance would be very appreciated.

 

 

[ljm42]

In order to keep the Unraid webgui current, the webgui polls the server on a regular basis. If an open page becomes unauthenticated then the individual poll requests will redirect to the login page, which is what we are seeing in the log. Later versions of Unraid will redirect the whole tab to the log script (and thus stop polling) if they detect this situation, but it is possible that there are scenarios that are not detected.

 

That is the most likely explanation for the log entries you are seeing. The quick fix is to close any open tabs that are pointing to the webgui. If you can't find them, reboot the system at 192.168.3.16

 

If you want to actually troubleshoot the problem, first confirm that you are running Unraid 6.9.2 and that all plugins are current (there isn't much value in troubleshooting older versions as work has been done in this area.) Then find the tab that is pointed at the webgui, open the browser's developer tools and switch to the network tab. Do not refresh the tab. You should see a new entry being added to the network tab on a regular basis, let me know what it is. Also let me know the url the tab is showing (the part after the hostname at least). Diagnostics would be good too.

[esaru]

On 4/20/2021 at 7:52 PM, ljm42 said:

In order to keep the Unraid webgui current, the webgui polls the server on a regular basis. If an open page becomes unauthenticated then the individual poll requests will redirect to the login page, which is what we are seeing in the log. Later versions of Unraid will redirect the whole tab to the log script (and thus stop polling) if they detect this situation, but it is possible that there are scenarios that are not detected.

 

That is the most likely explanation for the log entries you are seeing. The quick fix is to close any open tabs that are pointing to the webgui. If you can't find them, reboot the system at 192.168.3.16

 

If you want to actually troubleshoot the problem, first confirm that you are running Unraid 6.9.2 and that all plugins are current (there isn't much value in troubleshooting older versions as work has been done in this area.) Then find the tab that is pointed at the webgui, open the browser's developer tools and switch to the network tab. Do not refresh the tab. You should see a new entry being added to the network tab on a regular basis, let me know what it is. Also let me know the url the tab is showing (the part after the hostname at least). Diagnostics would be good too.

I am running 6.9.2, all plugins updated. The issue started after upgrading to 6.9.2 from the preceding beta version.

 

Surprisingly the error seems to return even when the browser is closed. I also tried switching from Chrome to Firefox to no avail. I've rebooted many times, both the VM and the tower.

 

The url is https://e796dc2d0c489161d612bf0fd71beb13294217db.unraid.net/Dashboard

 

Full error message: (changed to a static ip. 222 instead of 16).

Quote

Apr 27 14:46:42 Tower nginx: 2021/04/27 14:46:42 [error] 6002#6002: *106834 limiting requests, excess: 20.041 by zone "authlimit", client: 192.168.3.222, server: , request: "PROPFIND /login HTTP/2.0", host: "e796dc2d0c489161d612bf0fd71beb13294217db.unraid.net"

 

The more recent errors occured with no webgui tab open.

 

Thank you!

tower-diagnostics-20210427-1452.zip

Edited April 27, 2021 by esaru

[esaru]

On 4/20/2021 at 7:52 PM, ljm42 said:

In order to keep the Unraid webgui current, the webgui polls the server on a regular basis. If an open page becomes unauthenticated then the individual poll requests will redirect to the login page, which is what we are seeing in the log. Later versions of Unraid will redirect the whole tab to the log script (and thus stop polling) if they detect this situation, but it is possible that there are scenarios that are not detected.

 

That is the most likely explanation for the log entries you are seeing. The quick fix is to close any open tabs that are pointing to the webgui. If you can't find them, reboot the system at 192.168.3.16

 

If you want to actually troubleshoot the problem, first confirm that you are running Unraid 6.9.2 and that all plugins are current (there isn't much value in troubleshooting older versions as work has been done in this area.) Then find the tab that is pointed at the webgui, open the browser's developer tools and switch to the network tab. Do not refresh the tab. You should see a new entry being added to the network tab on a regular basis, let me know what it is. Also let me know the url the tab is showing (the part after the hostname at least). Diagnostics would be good too.

It's still recurring after several restarts of both my VM and the tower itself.

 

The log gets to 100% in like 24 hours and as soon as it's full I get this nagging feeling that I have to restart. What are the actual implications of having a 100% full log, in what way (if any) does it "harm" my servers wellbeing?

[ljm42]

*something* on 192.168.3.222 is contacting the server on a regular basis and is being redirected to /login . It is sending a PROPFIND command, looks like that might be related to WebDAV. Does that give you any clues?

[esaru]

51 minutes ago, ljm42 said:

*something* on 192.168.3.222 is contacting the server on a regular basis and is being redirected to /login . It is sending a PROPFIND command, looks like that might be related to WebDAV. Does that give you any clues?

I run Cryptomator, I think it utilizes WebDAV. I’ll check it out! 

[esaru]

On 5/2/2021 at 11:06 PM, ljm42 said:

*something* on 192.168.3.222 is contacting the server on a regular basis and is being redirected to /login . It is sending a PROPFIND command, looks like that might be related to WebDAV. Does that give you any clues?

Yesterday i exited Cryptomator and a bunch of other services for good measure (Veeam, google drive sync). I also uninstalled a bunch of unused apps and updated Windows. No dice, the log is 100% again this morning.

 

Any other advice/pointers? Is there some additional information I could provide for further troubleshooting? Thanks.

[ChatNoir]

37 minutes ago, esaru said:

Yesterday i exited Cryptomator and a bunch of other services for good measure (Veeam, google drive sync). I also uninstalled a bunch of unused apps and updated Windows. No dice, the log is 100% again this morning.

 

Any other advice/pointers? Is there some additional information I could provide for further troubleshooting? Thanks.

Did you reboot after that ?

There is not way to clear the log, so whether your actions improve things or not, the logs would stay at 100%.

 

If you did reboot, new diags could have information about what is happening now.

[esaru]

2 minutes ago, ChatNoir said:

Did you reboot after that ?

There is not way to clear the log, so whether your actions improve things or not, the logs would stay at 100%.

 

If you did reboot, new diags could have information about what is happening now.

I did reboot the tower around 10 yesterday, however only the VM was rebooted after exiting Cryptomator etc in the afternoon. When I took those measures the log was not full at all, it filled during the late evening/night.

 

tower-diagnostics-20210504-1040.zip

[ChatNoir]

Looks like it is still the same local IP trying authenticate 20 time a second. That's out of my wheelhouse.  

Hopefully, someone can help you on this.

[esaru]

8 minutes ago, ChatNoir said:

Looks like it is still the same local IP trying authenticate 20 time a second. That's out of my wheelhouse.  

Hopefully, someone can help you on this.

Thank you for trying! 🙂

[esaru]

Rebooted tower again, it's still ongoing. Not sure what to do next, I've been shutting down pretty much every app/service I could find in task manager. Let's see if that has an impact, otherwise I'm just clueless.

 

Edit: it's still happening. Seems like nuking my VM and leaving windows forever is like the only viable option at this point..

Edited May 4, 2021 by esaru
Update

[esaru]

Tried putting my VM to sleep, just to try something. When the VM is sleeping, this error shows instead:

Quote

May 5 12:09:16 Tower nginx: 2021/05/05 12:09:16 [alert] 5228#5228: worker process 2208 exited on signal 6

 

Does this possibly lead to some conclusion?

Edited May 5, 2021 by esaru

[esaru]

On 5/2/2021 at 11:06 PM, ljm42 said:

*something* on 192.168.3.222 is contacting the server on a regular basis and is being redirected to /login . It is sending a PROPFIND command, looks like that might be related to WebDAV. Does that give you any clues?

I deleted a whole bunch of apps, including Cryptomator which is the only one that I know uses WebDAV. It's still happening. Am I correct to assume that it must be an app/program on my VM that is contacting my servers webui? Could it be some aspect of Windows itself doing this stuff?

Some more instructions for troubleshooting would be very appreciated.

[ProphetSe7en]

Did you figure this out? I have the same issue. The ip in the log belongs to my laptop. Restarting my laptop stops it, then after a day or so it fils the log again with the same error.

[ljm42]

On 5/8/2021 at 6:42 AM, esaru said:

I deleted a whole bunch of apps, including Cryptomator which is the only one that I know uses WebDAV. It's still happening. Am I correct to assume that it must be an app/program on my VM that is contacting my servers webui? Could it be some aspect of Windows itself doing this stuff?

Some more instructions for troubleshooting would be very appreciated.

 

I don't really have a lot more to add. What we know is that that IP address is making repeated unauthenticated connections to the Unraid webgui. The request is "PROPFIND /login HTTP/2.0", when I look up PROPFIND that is related to Webdav.

 

This is not a standard thing that Windows does

[esaru]

5 hours ago, ljm42 said:

 

I don't really have a lot more to add. What we know is that that IP address is making repeated unauthenticated connections to the Unraid webgui. The request is "PROPFIND /login HTTP/2.0", when I look up PROPFIND that is related to Webdav.

 

This is not a standard thing that Windows does

I see. Is there any duct tape solution available? Perhaps changing the nginx conf to just refuse this type of connection or something?

 

Should I reboot the server when log hits 100%? Are there any risks involved with leaving it at 100%? I can live with not being able to get fresh log messages, but if there are harsher potential consequences I'd be more motivated to adress the issue by reinstalling windows or something.

[Kodey]

I just had this occur on my server. Did you ever figure this out?

[esaru]

11 hours ago, Kodey said:

I just had this occur on my server. Did you ever figure this out?

Nope, sorry. I had to create a new VM to get rid of this, unfortunately.. I still have no idea what caused the log spam. Please post here if you find a solution! 

[mchristo]

Having the same issue here as well - sorry for opening this thread up again.

My alteration of this is coming from my main desktop, where I access the Unraid UI. 

 

I'm on version 6.10.0-rc1 - everything is up-to-date, including my PC. 

 

[matty2k]

Hi folks,

same here. Requests come from one client in the network which is using only SMB access to the server.

Restarting the client did not solve the problem.

regards

[JohnnyCache]

found this thread after seeing the same thing. IP is my main desktop. 

[JohnnyCache]

Nov 9 09:24:41 nas nginx: 2021/11/09 09:24:41 [error] 27188#27188: *974740 limiting requests, excess: 20.172 by zone "authlimit", client: 192.168.1.159, server: , request: "GET /login HTTP/2.0", host: "nas", referrer: "https://nas/Docker"
Nov 9 09:24:41 nas nginx: 2021/11/09 09:24:41 [error] 27188#27188: *974740 limiting requests, excess: 20.169 by zone "authlimit", client: 192.168.1.159, server: , request: "GET /login HTTP/2.0", host: "nas", referrer: "https://nas/Docker"
Nov 9 09:24:41 nas nginx: 2021/11/09 09:24:41 [error] 27188#27188: *974740 limiting requests, excess: 20.166 by zone "authlimit", client: 192.168.1.159, server: , request: "GET /login HTTP/2.0", host: "nas", referrer: "https://nas/Docker"
Nov 9 09:24:41 nas nginx: 2021/11/09 09:24:41 [error] 27188#27188: *974740 limiting requests, excess: 20.164 by zone "authlimit", client: 192.168.1.159, server: , request: "GET /login HTTP/2.0", host: "nas", referrer: "https://nas/Docker"
Nov 9 09:24:41 nas nginx: 2021/11/09 09:24:41 [error] 27188#27188: *974740 limiting requests, excess: 20.161 by zone "authlimit", client: 192.168.1.159, server: , request: "GET /login HTTP/2.0", host: "nas", referrer: "https://nas/Docker"
Nov 9 09:24:41 nas nginx: 2021/11/09 09:24:41 [error] 27188#27188: *974740 limiting requests, excess: 20.160 by zone "authlimit", client: 192.168.1.159, server: , request: "GET /login HTTP/2.0", host: "nas", referrer: "https://nas/Docker"
Nov 9 09:24:41 nas nginx: 2021/11/09 09:24:41 [error] 27188#27188: *974740 limiting requests, excess: 20.158 by zone "authlimit", client: 192.168.1.159, server: , request: "GET /login HTTP/2.0", host: "nas", referrer: "https://nas/Docker"

 

 

I'm still seeing these too. The weird thing is that the IP changes; this issue is not isolated to my desktop. The IP I'm seeing in my logs is from a brand new laptop with almost nothing on it other than Windows. All I did to interact with Unraid is login to the webGUI; have no shares or mapped or anything.