Hello,

 

 

since a while unraid always crashes for me without any log entry.

I thought in the beginning, that it may helps to restart my server once a day with a cron job, but that also did not helped, as even the cron job is not being executed. 

The server is still reachable via MeshCOmmander, so that atleast I can reboot it via iKVM. So the error must be on the unraid side somewhere.

 

One interesting point: When I try to reach subdomain.domain.com (which I did setup in Nginx Proxy Manager) it shows, that 502 Bad Gateway Info. WHen I try to reach directly my Server via its internal IP, the site will not load. 

Also it is still able to receive PING signals (maybe because of the iKVM, but I am not sure), so atleast some basic functionalty works.

 

There are also no suspcious entries int he log file. 

 

I would very much appricate any help.

 

syslog-127.0.0.1_unraid.log

setup a syslog server..

Generic trouble shooting:

The issue of Unraid crashing without logs can be challenging to troubleshoot, especially if no obvious errors appear. Based on the problem and the provided logs, here's a step-by-step approach to diagnose and resolve the issue:

 

1. Check for Hardware Issues

Since your server is still responsive to pings and accessible via iKVM, the crash might be related to Unraid's OS or services rather than a complete hardware failure.

RAM Test:

Run a Memtest (available in the Unraid boot menu) to check for memory errors.

Storage Check:

Verify the USB flash drive containing Unraid is healthy.

Use a different USB port (preferably USB 2.0) to rule out port-specific issues.

Cooling and Power:

Monitor CPU and motherboard temperatures in Dashboard > System Stats before the crash.

Check PSU health and ensure it provides stable power to all components.

 

2. Subdomain and Network Configuration

The 502 Bad Gateway error and inability to access the server via its internal IP might indicate an issue with Nginx Proxy Manager or the Unraid network configuration.

Nginx Logs:

Check the Nginx Proxy Manager logs for errors. These are usually in /mnt/user/appdata/nginx-proxy-manager/log/.

Direct Access Test:

Access the server's IP directly without using Nginx Proxy Manager to verify Unraid's WebUI availability.

Static IP Configuration:

Ensure Unraid has a static IP address or proper DHCP reservation in your router.

Check Settings > Network Settings for any misconfigurations.

*could be a iproute netowrk issue / docker host replacing ports issue...

 

3. Crash Diagnosis

If there are no entries in the system logs before the crash, try enabling persistent logs and use an external logging mechanism.

Persistent Logs:

Go to Settings > Syslog Server.

Enable the syslog server to write logs to an external machine or file share.

Kernel Panic Capture:

If Unraid crashes silently, it might be experiencing a kernel panic.

Connect to the server via iKVM or a monitor and enable VGA Console in Unraid.

Monitor the screen for kernel panic messages during the crash.

 

4. Docker and Plugin-Related Issues

Certain Docker containers or plugins might be causing instability.

Disable Containers and Plugins:

Temporarily disable all Docker containers and plugins.

Gradually re-enable them one by one to isolate the culprit.

Update Containers:

Ensure all Docker containers are updated to their latest versions.

Review logs for containers like Nginx Proxy Manager or other high-resource usage containers.

*Boot unraid in safe mode (no plugins...)

 

5. Advanced Debugging

If the above steps don’t resolve the issue, here are additional diagnostics to try:

Safe Mode:

Boot Unraid in Safe Mode (no plugins or Docker containers).

Run the server to see if the crash still occurs.

BIOS Update:

Check your motherboard manufacturer’s site for BIOS updates and apply them if necessary.

Network Isolation:

If the crash seems network-related, disconnect the server from the internet and monitor for stability.

 

Analyze Existing Logs

Let me analyze the uploaded log (syslog-127.0.0.1_unraid.log) to spot potential issues or patterns. One moment!

The log indicates an issue with a Docker network (br0) already existing, which might be causing conflicts in your setup. Here's how to resolve this:

(Better if you bots diag file...)
 

Fix Docker Network Conflict

Stop the Docker Service:

Navigate to Settings > Docker and stop the Docker service.

Reset the Docker Network:

SSH into your Unraid server or use the console.
 

docker network prune

 

This will remove unused Docker networks, including conflicting ones.

Reconfigure Docker:

Check the Docker network settings in Settings > Docker.

Ensure Host access to custom networks is enabled if you use bridge networks with Nginx Proxy Manager.

Start the Docker Service:

Restart the Docker service in the Unraid settings.

 

Additional Steps

If the crash persists, try running the server in Safe Mode or disabling Docker entirely to isolate whether this conflict is related to the crashes.

 

Thanks @bmartino1 or shall I say ChatGPT?  

 

Ineed I check the docker settings, which might cause a issue/conflict and found out, that IPv4 custom network on interface br0 was set to 192.168.1.0/24, which might could have caused the issue. 

I disabled the option, and I will see if it helps.

 

FOr sure I did setup a syslog server, to localhost, but also this did not wrote anymore any settings after the unknown crash.

 

I will update the thread, after a days, as those crashs are random mostly within a few days, if it solved the problem.

 

If there's nothing relevant logged in the permanent syslog, it can be a hardware issue, one thing you can try is to boot the server in safe mode with all docker containers/VMs disabled, let it run as a basic NAS for a few days, if it still crashes it's likely a hardware problem, if it doesn't start turning on the other services one by one, including the individual docker containers. 

AN update  regards this: It was indeed the br0 Interface, which caused problems. Since I "solved" this issue I had no crashes anymore and the serve ris running stable- 

Thanks again for the input!

On 11/21/2024 at 12:18 AM, bmartino1 said:

If Unraid crashes silently, it might be experiencing a kernel panic.

Connect to the server via iKVM or a monitor and enable VGA Console in Unraid.

Monitor the screen for kernel panic messages during the crash.

I'm experiencing frequent crashes with nothing indicated in the local syslogs. How do I enable VGA Console in Unraid? Thanks.