My unraid server rebooted twice within 24h (first in the morning, did auto parity check which took 12h, rebooted 3h after finishing or so).
I did not do anything new really, I checked the logs but couldn’t find anything. I know about disks HDD UDMA errors, they are there since always, maybe controller is not fully supported. These error never caused a reboot so it’s not that and they are on all 6 disks if they are busy (there are no sata cables, it’s NAS).
It’s a problem because server is my home automation center so if it’s off nothing will work in my smart home.

Attacking diagnostics, maybe youll find something I can’t see. No idea why it started now, my last uptime was 2 months till today.

Would appreciate if you find any clues.

tower-diagnostics-20250907-0231.zip

Setup Syslog Server

I set it to mirror to flash. Is it enough or shall I set one on a network? I actually did it immediately after second reboot and go ogling where are old logs...

EDIT: never mind, I set a remote syslog server as well. Now we wait I suppose?

4 minutes ago, trurl said:

Setup Syslog Server

Edited September 7, 2025Sep 7 by Namecaps

9 hours ago, Namecaps said:

Now we wait I suppose?

Yes, post that after the next event, but a server rebooting on its own is almost always a hardware problem.

17 minutes ago, JorgeB said:

Yes, post that after the next event, but a server rebooting on its own is almost always a hardware problem.

Yeah I suppose but it's less than a year and was working great till yesterday. Anyways let's wait and see

1 hour ago, Namecaps said:

and was working great till yesterday

Keep in mind that every piece of hardware that goes bad was working fine before.

The commonest causes of unexpected reboots is the RAM and PSU, both of which can go bad even though they were working OK earlier.

Unraid died again with automatic reboot. Unfortunately there is nothing interesting in the remote log :/

Last ones are these

Sep 8 10:29:41 Tower sshd-session[3167839]: Accepted keyboard-interactive/pam for root from 192.168.1.189 port 55222 ssh2

Sep 8 10:29:41 Tower sshd-session[3167839]: pam_unix(sshd:session): session opened for user root(uid=0) by (uid=0)

Sep 8 10:29:41 Tower sshd-session[3167839]: User child is on pid 3167844

Sep 8 10:29:41 Tower sshd-session[3167844]: Starting session: shell on pts/1 for root from 192.168.1.189 port 55222 id 0

Sep 8 10:29:41 Tower sshd-session[3167844]: Starting session: command for root from 192.168.1.189 port 55222 id 1

Sep 8 10:29:41 Tower sshd-session[3167844]: Close session: user root from 192.168.1.189 port 55222 id 1

Sep 8 10:29:41 Tower sshd-session[3167844]: Starting session: command for root from 192.168.1.189 port 55222 id 2

Sep 8 10:29:41 Tower sshd-session[3167844]: Close session: user root from 192.168.1.189 port 55222 id 2

Sep 8 10:50:22 Tower flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup update

Sep 8 11:17:26 Tower emhttpd: read SMART /dev/sdf

Sep 8 11:28:57 Tower autofan: Highest disk temp is 36C, adjusting fan speed from: OFF (0% @ 0rpm) to: 181 (70% @ 1377rpm)

Sep 8 11:29:04 Tower autofan: Highest disk temp is 36C, adjusting fan speed from: OFF (0% @ 0rpm) to: 181 (70% @ 1386rpm)

EDIT:

Wait, this time is different. After reboot one of my disk dissapeared from Array. I rebooted manually and it showed again… This wasn’t the case in last 2 reboots.

I’ll investigate futher but disk seems fine now… I have no clue what’s causing these reboots.

Edited September 8, 2025Sep 8 by Namecaps

2 hours ago, Namecaps said:

Unfortunately there is nothing interesting in the remote log :/

That's typical when it's a hardware problem, which is the most likely scenario.

Since you have multiple RAM sticks, try using the server with just one, if the same try with the other one, that will basically rule out bad RAM.

Thanks this is basically what I'm doing now. Will see.

I still would like to do something with udma errors but don't know how can I solve it. I have this problem only in unraid so I suspect it doesn't like the controller. There are no cables or way to reseat anything and it happens on all 6 drives so definately something with contriller but on the original system (also linux) this NAS came with no issues whatsoever. Maybe something to disable in BIOS?

[ 3240.647354] ata1.00: sense data available but port frozen

[ 3240.647363] ata1.00: exception Emask 0x11 SAct 0x80 SErr 0x0 action 0x6 frozen

[ 3240.647365] ata1.00: irq_stat 0x48000008, interface fatal error

[ 3240.647367] ata1.00: failed command: READ FPDMA QUEUED

[ 3240.647368] ata1.00: cmd 60/80:38:78:6e:e9/01:00:ee:00:00/40 tag 7 ncq dma 196608 in

res 43/84:00:00:00:00/00:00:00:00:00/40 Emask 0x10 (ATA bus error)

[ 3240.647373] ata1.00: status: { DRDY SENSE ERR }

[ 3240.647374] ata1.00: error: { ICRC ABRT }

[ 3240.647377] ata1: hard resetting link

[ 3240.952430] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)

[ 3241.057220] ata1.00: configured for UDMA/133

[ 3241.057259] ata1: EH complete

[ 3241.103343] ata1.00: sense data available but port frozen

[ 3241.103348] ata1.00: exception Emask 0x11 SAct 0x100000 SErr 0x0 action 0x6 frozen

[ 3241.103350] ata1.00: irq_stat 0x48000008, interface fatal error

[ 3241.103353] ata1.00: failed command: READ FPDMA QUEUED

[ 3241.103354] ata1.00: cmd 60/00:a0:f0:ee:08/02:00:7c:00:00/40 tag 20 ncq dma 262144 in

res 43/84:00:00:00:00/00:00:00:00:00/40 Emask 0x10 (ATA bus error)

[ 3241.103358] ata1.00: status: { DRDY SENSE ERR }

[ 3241.103359] ata1.00: error: { ICRC ABRT }

[ 3241.103362] ata1: hard resetting link

[ 3241.410345] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)

[ 3241.507577] ata1.00: configured for UDMA/133

[ 3241.507595] ata1: EH complete

[ 3241.543370] ata1.00: sense data available but port frozen

[ 3241.543375] ata1.00: exception Emask 0x11 SAct 0x8 SErr 0x0 action 0x6 frozen

[ 3241.543378] ata1.00: irq_stat 0x48000008, interface fatal error

[ 3241.543379] ata1.00: failed command: READ FPDMA QUEUED

[ 3241.543380] ata1.00: cmd 60/00:18:f0:ee:08/02:00:7c:00:00/40 tag 3 ncq dma 262144 in

res 43/84:00:00:00:00/00:00:00:00:00/40 Emask 0x10 (ATA bus error)

[ 3241.543384] ata1.00: status: { DRDY SENSE ERR }

[ 3241.543386] ata1.00: error: { ICRC ABRT }

[ 3241.543389] ata1: hard resetting link

[ 3241.849924] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)

[ 3241.949575] ata1.00: configured for UDMA/133

[ 3241.949591] ata1: EH complete

[ 3242.040303] ata1.00: sense data available but port frozen

[ 3242.040310] ata1: limiting SATA link speed to 3.0 Gbps

[ 3242.040311] ata1.00: exception Emask 0x11 SAct 0x20000 SErr 0x0 action 0x6 frozen

[ 3242.040314] ata1.00: irq_stat 0x48000008, interface fatal error

[ 3242.040316] ata1.00: failed command: READ FPDMA QUEUED

[ 3242.040317] ata1.00: cmd 60/00:88:f0:f0:08/02:00:7c:00:00/40 tag 17 ncq dma 262144 in

res 43/84:00:00:00:00/00:00:00:00:00/40 Emask 0x10 (ATA bus error)

[ 3242.040321] ata1.00: status: { DRDY SENSE ERR }

[ 3242.040323] ata1.00: error: { ICRC ABRT }

[ 3242.040325] ata1: hard resetting link

[ 3242.344742] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 320)

[ 3242.449260] ata1.00: configured for UDMA/133

[ 3242.449276] ata1: EH complete

Edited September 8, 2025Sep 8 by Namecaps

5 minutes ago, Namecaps said:

I still would like to do something with udma errors but don't know how can I solve it. I have this problem only in unraid so I suspect it doesn't like the controller.

Do they happen with all disks or just some? If the former, swap one of the affected ones positions and see where they follow.

Just now, JorgeB said:

Do they happen with all disks or just some? If the former, swap one of the affected ones positions and see where they follow.

All Disks. It started as soon as I installed Unraid and created and started array (for 2 weeks I was using original os without any issues) All 6 Disks, all new. It happens on a a disk which is currently in use.

EDIT: Forgot to add I found a reddit user with exactly the same problem on same hardware with same disks (Ironwolf pro) and also only with Unraid…

Edited September 8, 2025Sep 8 by Namecaps

In that case there's likley some kernel/controller/disk compatibility issue.

3 minutes ago, JorgeB said:

In that case there's likley some kernel/controller/disk compatibility issue.

Yeah this is what I suspect but how to address it now? Same disks were working fine under TOS (linux that NAS came with). Is there anything I could look up in bios?

Edited September 8, 2025Sep 8 by Namecaps

If you used powertop it's known to cause issues with some controllers; if not, you can try disabling pcie_aspm to see if it helps:

https://forums.unraid.net/topic/118286-nvme-drives-throwing-errors-filling-logs-instantly-how-to-resolve/?do=findComment&comment=1165009

You can also try disabling any power saving options in the BIOS.

The controller being used is an Asmedia ASM1166, and those work fine with Unraid; it's one of the recommended controllers, so it's something specific to the hardware/firmware.

2 hours ago, JorgeB said:

If you used powertop it's known to cause issues with some controllers; if not, you can try disabling pcie_aspm to see if it helps:

https://forums.unraid.net/topic/118286-nvme-drives-throwing-errors-filling-logs-instantly-how-to-resolve/?do=findComment&comment=1165009

You can also try disabling any power saving options in the BIOS.

The controller being used is an Asmedia ASM1166, and those work fine with Unraid; it's one of the recommended controllers, so it's something specific to the hardware/firmware.

I did not use powertop, ASPM disabled earlier, no dice. Unfortuanetly no way to update bios as manufacturer doesn’t release anything new yet (latest is from Aug 2024).

Well, will try to play with settings further, but for now I need to get the system stable like it was for past 200 days so let’s start with the RAM.

By the way NAS is Terramaster F6-424 MAX so someone can find this thread if they’re looking.

Will update if I find something that helps.

Edited September 8, 2025Sep 8 by Namecaps

Ok so I would like to come with the update.

Unfortunately rebooting problem is not solved, but I start to believe it's software problem.

After replies I should check the RAM I indeed removed one of two 32GB chips and lo and behold - server was working fine for a week without an Issue (mind I had 50% chance chip I removed is the faulty one). I decided to get a ram replacement. When It arrived I put it in - reboot after 1 hour.

I thought - OK I was wrong got the wrong chip replaced. Got another Replacement for the first chip and same story, reboot after few hours.
I started to test RAM chips extensively in the same hardware - 0 errors in memtest after running it for 48h so chips I replaced were actually good it seems and new ones are good too.

In the meantime I replaced the whole NAS due to UDMA errors - got replacement of same NAS, Same Hardware. And the story continues. No UDMA CRC Errors anymore but it keeps rebooting with 64GB (2x32GB).

Now I'm running unraid for 3 days no issue with 1x32GB and 1x8GB chip I found somewhere.

So why the server is rebooting without anything in the logs when I work with 2x32GB? Mind that this setup worked fine for almost a year and reboots started out of nowhere.

How do I narrow the real problem since hardware is OK? People are running same hardware with 96GB no issues and I had no issue running it for almost a year with 64GB.

That suggests to me the board may have stability issues in dual channel; it doesn't make sense the software would be the issue if it's stable with just one stick of RAM, or two different ones.

Just now, JorgeB said:

That suggests to me the board may have stability issues in dual channel; it doesn't make sense the software would be the issue if it's stable with just one stick of RAM, or two different ones.

Ok but why it started after 8 months and it's happening also on new board? No bios updates were done.

It could also be the CPU, or just a bad board design; dual channel can expose the issue.

CPU is also new... As I have two of the same devices now (they let me kept the old one) I'm running it on plain linux and stressing the memory and no issues. There most be something wrong elsewhere.
I changed 2x RAM, Board, Controller, CPU, PSU - nothing else left to change and unraid itself is the only thing left.

Edited September 25, 2025Sep 25 by Namecaps

You can try to do some more testing, but for me it's 100% hardware, and the fact that it runs stable with a single stick, or two different ones not in dual channel, confirms it.

But those chips in same board run for a week now in Ubuntu with no reboots.

You can be doing different things that don't stress the hardware in the same way. There's also a very small chance that there's some compatibility with the current Unraid kernel, but that is also easy to test. Try running, for example, 6.12 or 7.0 and see if it's the same; if it still reboots, it's not the kernel.