Greetings fellow Unraid-ers!

 

I've encountered an issue with my Unraid server and after trying to figure it out myself for a few days with no success am asking for help.  The last few days my Unraid server, running on ESXI 6.7, has started randomly freezing and I have to kill the VM, disconnect and reconnect the USB Drive, then restart the VM to get it going again.  The amount of time it runs before freezing seems to be random, its lasted anywhere from 1 hour to 1 +days and I can't determine a trigger time or process. I feel its worth noting that when the VM locks up the USB drive, attached to ESXI, becomes unresponsive until I disconnect and reconnect it, its still recognized by the OS but Unraid won't boot from it and I can't assign it to another VM for diagnostics.  There are several VM's running on ESXI and Unraid is the only one having trouble which of course doesn't completely rule this out as an ESXI issue but has me thinking its more an issue with my Unraid install than something else.

 

In the days leading up to this starting I cannot think of any changes that could have caused this.  No Unraid or Docker container updates, no physical hardware changes, no esxi changes.  My setup has been pretty much in maintenance mode for a couple of years and has just worked with no issue so this is very perplexing!

 

Here's what I've done so far:

 

A lot of Googling!

Swapped USB ports on the ESXI host

Replaced the USB Drive (because it was probably 6+ years old) as per LimeTech's instructions using a backup

Verified docker is set to use ipvlan

Set the syslog to USB option and reviewed the logs looking for clues

In the syslog the last entries appear to be for the mover trying to copy some files with long file files, I've now removed those files but they have been there since May 2024 so I don't think that's it.

 

Hopefully I'm not leaving out any useful information.  Does anyone have suggestions of where to look next?

 

Thanks in advance!

 

Edit:  I know I'm a version behind on UnraidOS but have been hesitant to upgrade until I figure out the freezing problem.

 

syslog_0124250902 unraid-diagnostics-20250124-1107.zip

Edited January 24, 20251 yr by andrewh91

I'm not seeing anything relevant logged, and since you are virtualizing Unraid, and that introduces other variables, I'm going to move this to the appropriate subforum to see if anyone else can help.

Ah, maybe, finally, I have a clue as to what's going on.  I was home Friday (when I made this thread), Saturday, and Sunday and of course my server ran flawlessly all weekend.  Today I'm at work and at approximately 3:30pm the Nextcloud client on my PC started complaining about "connection lost" so I'm like here we go again.  I try to access the WebUI and it doesn't respond, I check my syslog server and there's no entries since 6:54am when I ran a fix common problems scan, then I check ESXi and the VM is running but unresponsive.  Just like it was doing last week.

 

Just by chance I left the syslog up on my screen in a small window and at 4:12pm I noticed a rush of logs flash across the screen.  I've uploaded a new diagnostics file and looking at the syslog entries it looks to me like docker ran out of memory and killed off processes.  After that happened the server was responsive again.

 

I haven't been able to really analyze it yet but can I tell from the logs what cause docker to run out of memory?

unraid-diagnostics-20250127-1639.zip

Jellyfin appears to have been the problem