6.12.6 - Random crashes/unresponsiveness overnight

[MassimoMx]

I recently bought a basic license, but my unraid has become unresponsive overnight every 2-3 days, and I have no idea how to fix that.

The only fix was to shutdown the server using the case button, and turning it on again.

It's worth mention that I noticed this behaviour after I set my SSD as cache drive, before I was using the SSD as unsassigned device and everything seemed to work just fine.

My Unraid is running on a optiplex 5050 SFF, with the latest bios 1.28.0
- CPU: i7 7700
- RAM: Ballistix Sport LT 16GB Kit (8GBx2) DDR4 2400 MT/s
- Cache SSD: SanDisk X400 M.2 2280 128GB
- Array HDD: Seagate Barracuda Sata III 7200rpm 2TB
- Everything else is stock Optiplex 5050 SFF.

Memtest 86+ passed: https://imgur.com/YPYZ0ok

I'm also using Docker:
- 1 24/7 container using `assaro/ddbot:latest`
- 1 container with `scrutiny` running only when needed
- 1 container with `Palworld` (ich777/steamcmd) running only when needed
- Docker set to use IPVLAN My settings here (imgur)

Network Protocol is set to IPv4 Only.

I did enable syslogs, and I run all diagnostics. I will attach the logs and diagnostics to this post.

minimax-diagnostics-20240303-1221.zip syslog-192.168.0.55.logminimax-diagnostics-20240303-1221.zipsyslog-192.168.0.55.log

Edited March 3 by MassimoMx
more info, even more info

[MassimoMx]

@Moderators I got told that apparently the attachments I shared are unavailable

Added a backup link for my syslog and diagnostics: https://mega.nz/folder/MUEX2ZRT#HAFjmTNMFTwZ0VxYA8lizQ

Edited March 3 by MassimoMx
Added a backup link for my syslog and diagnostics

[MassimoMx]

Bumping in case someone has any suggestions.

[JorgeB]

Unfortunately there's nothing relevant logged, this usually points to a hardware issue, one thing you can try is to boot the server in safe mode with all docker containers/VMs disabled, let it run as a basic NAS for a few days, if it still crashes it's likely a hardware problem, if it doesn't start turning on the other services one by one.

[MassimoMx]

Okay, I guess I will give it a try, thank you.

 

[MassimoMx]

Sorry for the delay, but I have good news! My minimax looks like it's working just fine and I'm on 5 days and 20 hours of it running just fine with a docker container running 24/7

For transparency sake I have do say that I don't know exactly what fixed it.

But I can share a list of things I did that might have helped fixing the problem, in order of when I did it.

1. Reseated my RAM as I did get a memtest fail on a single address one night and reseating them didnt cause any fail over 3 different overnight tests.
   - I had to make sure the ram was matching the slot 1 and 2 of my MOBO to ensure it worked in dual channel.
2. Made a Backup of my cache and 2TB HHD (to use in later step)
3. Unraid 6.12.8 Fresh installed on my USB (Previously it was 6.12.6)
4. Formatted my Cache and HDD to use zfs (Previously xfs)
5. Set my `appdata` `domains` `isos` and `system` shares Primary storage to Cache, and Secondary Storage to None
6. I disabled the VM Manager (I don't need VMs right now, so, I turned it off before changing my docker setting)
7. Set my Docker vDisk location to `/mnt/cache/system/docker/docker.img`
8. Set my Docker Default appdata storage location to `/mnt/cache/appdata/`
9. I left the Docker custom network type to it's default (ipvlan)
10. This time I did NOT run the New Permissions tool to edit my Docker Container files from my Windows machine, instead I followed this guide to set the permissions of my appdata share: 
11. I reinstalled the container `assaro/ddbot:latest`
12. I did NOT install the container `scrutiny` and `Palworld`
13. Restored my HDD and cache data from backup and docker config/data files from backup
14. No more steps, I turned on the docker container I needed to run 24/7 and it looks like it's working just fine for now, 5 days and 20 hours later it's still working fine.

 

 

Notes:

1. I copied the docker location settings from this video 
2. I forgot to set up the syslog server, but I will after sending this message.

 

Edited March 16 by MassimoMx

[MassimoMx]

This thread could be related to the this one too, as we both had (almost) the same issue, and similiar solutions. 

 

[ich777]

On 3/3/2024 at 1:27 PM, MassimoMx said:

RAM: Ballistix Sport LT 16GB Kit (8GBx2) DDR4 2400 MT/s

This is way too less memory for that container.

 

See my explanation here:

 

[MassimoMx]

Thank you for the reply, Interestingly I didn't have many issues running the Container itself as we only had 3 people at best online  at the same time.
 

Regardless I took the opportunity to finally get the 4th RAM stick to go with the 3rd I kept spare. 

So, right now my miniserver has 33GB of Ram and i'm running a memtest86 to ensure everything is fine.