Hello,

Since a couple of weeks, it happens that my Unraid server is unreachable through UI or SSH.

The docker which are running are reachable (for example HA, Plex, ... ) - VMs aren't, since the whole server seems to be unreachable (dunno how dockers are..)

I don't know why it's happening nor where to start searching.
To be able to get access again I have to reboot the whole server.

Sometimes I'm able to get access through ILO and I can login at least by console, then I've tried with:

/etc/rc.d/rc.nginx restart

/etc/rc.d/rc.nginx reload

But it seems NGIX isn't running at all, and with START command, it fails:

I saw that when i tried to do ls on /boot it was empty - like if the server was not able to get access to the Unraid USB anymore (always hated to rely on an USB Stick for OS - but that's me)

But now it also happen that, when I try to access per ILO, as soon as I try to login with root (so I write root and press enter), I get:

And then I have to reboot it "physically".
I don't have a clue what's going on.
any help to find out what's happening is really appreciated.

Thanks.

(in the attachement the diagnostic after rebooting the server & stopped the check of the array)sekiro-diagnostics-20250710-0150.zip

Edited July 9, 2025Jul 9 by TDA

Enable the syslog server and post that after it happens again

On 7/10/2025 at 8:50 AM, JorgeB said:

Enable the syslog server and post that after it happens again

How to configure at best to give you the logs of the syslog?

Should I mirror it to the flash, or simply to a share?

Either option should work.

On 7/14/2025 at 9:05 AM, JorgeB said:

Either option should work.

Just enabled it - since it went down again after 4 days (god knows why)

Set it up like this, let's see what's happen in 4-5 days:

That is just ‘listening’ for syslog messages from elsewhere and will not capture anything. As mentioned in the syslog server link you either need to enable the “Mirror syslog to flash” option, or put your server’s IP address into the Remote Syslog server field.

On 7/18/2025 at 8:51 PM, itimpi said:

That is just ‘listening’ for syslog messages from elsewhere and will not capture anything. As mentioned in the syslog server link you either need to enable the “Mirror syslog to flash” option, or put your server’s IP address into the Remote Syslog server field.

I've just setup as you suggested, after another carsh.
As soon as it happens again, I'll post the file.

On 7/18/2025 at 8:51 PM, itimpi said:

That is just ‘listening’ for syslog messages from elsewhere and will not capture anything. As mentioned in the syslog server link you either need to enable the “Mirror syslog to flash” option, or put your server’s IP address into the Remote Syslog server field.

On 7/10/2025 at 8:50 AM, JorgeB said:

Enable the syslog server and post that after it happens again

Hello together,
I don't know If I've enabled it correctly but here the file I got:

syslog-previous

There appears to be a container constantly restarting; see if you can find out which one by looking at their uptimes.

Also, if you are not using Bluetooth try disabling it in the BIOS, since it's spamming the logs.

5 minutes ago, JorgeB said:

There appears to be a container constantly restarting; see if you can find out which one by looking at their uptimes.

Also, if you are not using Bluetooth try disabling it in the BIOS, since it's spamming the logs.

Hello,
Thanks for the analysis :-)

I will have to check what's restarting (is there a way to check it or to keep track of it via cli? )

Bluetooth is needed because I have to use it for my fan - and I had to create one container with docker compose:

services:

tunnel_socket:

container_name: tunnel_socket

image: python:3

environment:

- TUNNEL_SOCKET_FILE=/tunnel_socket/hci.sock

volumes:

- /mnt/user/homeassistant-bt-proxy:/tunnel_socket

entrypoint: ["python", "-u", "/tunnel_socket/async_server.py"]

cap_add:

- NET_ADMIN

network_mode: host

restart: unless-stopped

Source: https://github.com/NicoIIT/ha-ble-adv/wiki/Workaround-for-HA-non-'network_mode:-host'-or-non-root-installations

45 minutes ago, TDA said:

I will have to check what's restarting (is there a way to check it or to keep track of it via cli? )

You should be able to do that by looking at their uptimes, if a container is showing a few seconds, that's the one.

18 minutes ago, JorgeB said:

You should be able to do that by looking at their uptimes, if a container is showing a few seconds, that's the one.

At the moment I don't see anything which is restarting.
Although I didn't started the "bt-proxy" - cause tomorrow I have to go away for a few days and I don't want to connect remotely to restart the server if it hangs.
Will keep an eye on the docker-compose docker, and see if it's the culprit.
Anyway - I see some docker show the uptime and (healthy), and other just the uptime - why is that?

Not sure, probably some container option that some don't have.

hello agian,

I've now runned the server without problems for about 13days (with the bt-proxy docker disabled), but all of the sudden it was blocked again.

Now after just 2-3 days of uptime, it blocked itself again.

Attached you find the new syslog :-)
syslog-previoussekiro-diagnostics-20250910-1833.zip

Sep 10 16:53:12 Sekiro kernel: igc 0000:08:00.0 eth0: PCIe link lost, device now detached

Problem with the NIC, it got detached. Look for a BIOS update or try using an add-on NIC.

On 9/10/2025 at 7:00 PM, JorgeB said:

Sep 10 16:53:12 Sekiro kernel: igc 0000:08:00.0 eth0: PCIe link lost, device now detached

Problem with the NIC, it got detached. Look for a BIOS update or try using an add-on NIC.

Hi Jorge,

I'll try to update the BIOS.
What's "odd" if you want - is that the time it was good for almost 2weeks, no VM were running.

When it started again to do "stupid things", VM were running.

But I'll update the BIOS as first thing now :-)
Thanks again for your support!

Are you passing through any PCIe devices to the VM and/or do you have PCIe override enabled?

7 minutes ago, JorgeB said:

Are you passing through any PCIe devices to the VM and/or do you have PCIe override enabled?

I have a VM which has a passthrough on an NVME SSD.
But this particular VM was turned off - I used one which was hosted on the cache disk

If ACS PCIe override in VM Settings is disabled, it should not be related to that, but if it is, it may be worth disabling it and retesting.

1 minute ago, JorgeB said:

If ACS PCIe override in VM Settings is disabled, it should not be related to that, but if it is, it may be worth disabling it and retesting.

ACS PCIe override? Sorry for my newbiness but I don't get what it should be :-)

In any case - the VM which was running - was without any passhtroughs.

It's probably disabled, but go to Settings - VM Manager - PCIe ACS override

39 minutes ago, JorgeB said:

It's probably disabled, but go to Settings - VM Manager - PCIe ACS override

Yes it's disabled indeed.
I've upgraded the BIOS and will fire up the VM again - let's see if it will run without issues or not :-)