Hello,

 

I want to thank you to post your result of your test as I have the same problem as you since one year.

I have a NUCPRO12 with the SSD Samsung 990 pro as my system disk.

 

About once a month since one year (it's very random) , my server crashes unexpectedly. The crashes are often accompanied by a flurry of I/O errors, EXT4-fs error, Buffer I/O error on dev dm-0 and out-of-memory (OOM) killer messages, indicating severe issues.

I primarily use the server for containerization (running Docker containers) and Plex for media streaming.

 

What I've Tried:

fsck: Verified all partitions with fsck, and they are reported as clean.

smartctl : I ran smartctl on the system disk, and no errors were found. The disk is relatively new (1 year).

Memory Upgrade: I upgraded the RAM from 16GB(2x8) to 32GB(2x16) even though my previous setup never exceeded 40% of its memory usage.

I’ve already cleaned the RAM, re-seated the disks, and ensured everything is properly connected, but the issue persists.

Post-crash log issue: After I reboot the server following a crash, I’ve noticed there’s a gap in the logs. It seems like no logs are recorded from the moment of the crash until the server is rebooted, making it difficult to diagnose the exact cause.

 

I juste ordered an enclosure to test if the problem go away or not. I'm not on Unraid though , I'm on Ubuntu 22.04 with kernel generic 5.15. 

I could upgrade the kernel too , but not sure it will resolve this issue. 

 

I hope to resolve this with the enclosure, I'll post it here when after my tests !

How is testing going? Any resolution ? Im having this same consistent error but only on Unraid 7.

I am also having this problem after upgrading to Unraid 7.  Was not an issue prior to Unraid 7.

Unraid7.1-beta1 with linux kernel6.12.19 have been released, anyone have same issue on that version?

I have the same issue with my Samsung 990 Pro (2TB. latest firmware) disappearing from time to time. After shutdown and fresh reboot it works again.

 

My system: 

Unraid 7.0.1 (had this problem also on version 6 of unraid)

Motherboard: GIGABYTE B760M Gaming X DDR4 (F20 latest firmware)

Intel Core i5-12500

2x32GB DDR4 RAM

 

What I tried to solve the issue: 

- add this to Syslinux Cfg, but did not help: append initrd=/bzroot nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off

- swapped the SSD to the free 2nd possible NVME slot, did also not solve the issue

 

What else can I try, and findings?

Thanks!

Best regards,

 

I had the same problem at the start of the year, I swapped my old SSD with a new Samsung 990 Pro (2TB) using a btrfs cache pool with RAID1. I think It was already happening using Unraid 6, but I was updating Unraid to 7.0 at that same time, so I couldn't remember exactly.

It happened 2-3 times at intervals of a week. I was using a cache pool with btrfs filesystem and a docker image with a btrfs image. Every time this happened, I had to hard reboot the server (complete shutdown and restart using power button, restarting directly wasn't working). I had to completely rebuild my docker image, because it wasn't able to mount again after the reboot. After the second time, I was really frustrated.

I found this thread and the added the following config in the syslinux.cfg: 
 

For an updated syslinux config, checkout the updated post (added "pcie_port_pm=off")

 

append initrd=/bzroot nvme_core.default_ps_max_latency_us=0 pcie_aspm=off

 

I also changed the docker image from btrfs to xfs. It is working fine right now, I really hope this doesn't happen again, but it's looking good for now.

 

Edited April 28, 20251 yr by Gabriel Patzleiner

On 4/7/2025 at 8:54 PM, Gabriel Patzleiner said:

I had the same problem at the start of the year, I swapped my old SSD with a new Samsung 990 Pro (2TB) using a btrfs cache pool with RAID1. I think It was already happening using Unraid 6, but I was updating Unraid to 7.0 at that same time, so I couldn't remember exactly.

It happened 2-3 times at intervals of a week. I was using a cache pool with btrfs filesystem and a docker image with a btrfs image. Every time this happened, I had to hard reboot the server (complete shutdown and restart using power button, restarting directly wasn't working). I had to completely rebuild my docker image, because it wasn't able to mount again after the reboot. After the second time, I was really frustrated.

I found this thread and the added the following config in the syslinux.cfg: 
 

append initrd=/bzroot nvme_core.default_ps_max_latency_us=0 pcie_aspm=off

 

I also changed the docker image from btrfs to xfs. It is working fine right now, I really hope this doesn't happen again, but it's looking good for now.

 

 

 

I have same settings (same wording in Syslinux und XFS filesystem) but still this morning my system had the same disk fail after 4 days of running. What can I do? Is there no fix for this. As it is linked to Samsung SDD maybe insert another brand SSD. Will this help? Any other fix known?

38 minutes ago, ventil said:

As it is linked to Samsung SDD maybe insert another brand SSD. Will this help?

Possibly.

3 Days after my previous post, I sadly ran into the same problem again...

 

I got the error messages over night, and I saw then that the same night the scheduled monthly scrub was taking place. maybe it was connected?

After rebooting, the docker image was not mountable again and corrupt. The most annoying thing is that you have to recreate the whole docker image and reinstall all the containers. And the docker volumes that are stored in the image are lost, that's a big problem for me.

 

I have added now "pcie_port_pm=off" to the syslinux container resulting in the following line:

append initrd=/bzroot nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off

 

I saw that here:

Also, I made the update to Unraid 7.0.1.

I hope that the problem will somehow be resolved or by a kernel upgrade or by the correct syslinux config.

I find it strange that there are not more people with that problem, as the Samsung 990 Pro is quite popular I think. I switched one of my 2 Crucial 1TB NVMe drives to Samsung 990 Pro 2TB because I thought that Samsung would be more popular and better supported. Now I wanted to switch my second SSD of the pool to 2TB too, to be able to use the whole 2TB pool, but I am unsure if I should buy another Samsung.

 

 

For those interested, I use the scripts in this post to get notified of those errors. Otherwise, you don't even notice that there is a problem, only if you see the star at the drive temperature or the error spam in the syslog.

 

I’ve been experiencing the same connection loss issue with my Samsung 990 Pro (2TB) under the latest Unraid 7.1.2, despite both my motherboard and SSD being on the latest firmware. I can't understand why no one has been able to find a solution for this.

 

With numerous Docker containers and two VMs running, I find it hard to believe that the SSD is simply going into sleep mode and losing connection. There should always be read and write processes preventing that—but who knows?

 

I also reached out to Samsung but haven't received a response. It seems they’re not interested in supporting such a small user group, and pinpointing the issue as an SSD fault is difficult.

 

This is incredibly frustrating.

11 hours ago, ventil said:

I’ve been experiencing the same connection loss issue with my Samsung 990 Pro

Did you try this:

On 4/28/2025 at 5:39 PM, Gabriel Patzleiner said:

append initrd=/bzroot nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off

 

1 hour ago, JorgeB said:

Did you try this:

 

Yes, but unfortunately, it didn't resolve the issue. I've also come across posts from others experiencing the same problem, and sadly, it didn’t help them either.

I'd recommend selling the Samsung and going with WD. Not one single issue since I did that. 

 

Samsung don't care about this issue, we are a tiny minority and it doesn't make sense for them to pump any resources into it. 

1 hour ago, chrisgtl said:

I'd recommend selling the Samsung and going with WD. Not one single issue since I did that. 

 

Samsung don't care about this issue, we are a tiny minority and it doesn't make sense for them to pump any resources into it. 

 

What I don’t understand is the following:

- Samsung SSD firmware can be updated directly within Unraid, while many other SSD brands require Windows for upgrades. This led me to believe that they care for Linux users.

- Since Unraid is essentially "just" another Linux-based system, does this issue not also occur on other Linux distributions? 

I have this issue too. I have 2x 990 Pro 2TB in a ZFS mirror. One drops the other doesn't. I've tried all the things.

From all the things I've read, this is possibly a kernel interaction bug on Linux.

7 hours ago, warpspeed said:

I have this issue too. I have 2x 990 Pro 2TB in a ZFS mirror. One drops the other doesn't. I've tried all the things.

From all the things I've read, this is possibly a kernel interaction bug on Linux.

Do both Samsung SSDs have the same firmware installed?

I tried to put my Samsung 990 pro in both existing M2 slots on my motherboard but both versions brought no solution.

Seems to be a Linux problem as you mentioned.

I will change my drive to Kingston fury renegade SSD and hope that this solves my issue.

Only one of my drives has the problem, pretty sure its the one not connected directly to the CPU. I dont want to mess around with it because I don't want the stable drive to start having problems too.

I wonder if this is related:
https://bugzilla.kernel.org/show_bug.cgi?id=215742

When I boot I see the platform quirk thing in dmesg output:

root@unraid:~# dmesg | grep nvme
[    0.000000] Command line: BOOT_IMAGE=/bzimage initrd=/bzroot nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off
[    0.154215] Kernel command line: BOOT_IMAGE=/bzimage initrd=/bzroot nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off
[   36.874725] nvme 0000:02:00.0: platform quirk: setting simple suspend
[   36.879291] nvme nvme0: pci function 0000:02:00.0
[   36.885589] nvme 0000:03:00.0: platform quirk: setting simple suspend
[   36.893575] nvme nvme1: pci function 0000:03:00.0
[   37.093198] nvme nvme0: Shutdown timeout set to 10 seconds
[   37.093229] nvme nvme1: Shutdown timeout set to 10 seconds
[   37.107599] nvme nvme1: 16/0/0 default/read/poll queues
[   37.112297] nvme nvme0: 16/0/0 default/read/poll queues
[   37.123589]  nvme1n1: p1
[   37.129541]  nvme0n1: p1

Looks like a lot of discussion here:

https://bugzilla.kernel.org/show_bug.cgi?id=216809

I've seen a few posts that there's a new firmware available: 6B2QJXD7

I wonder if this fixes these dropout issues

FYI I've put 6B2QJXD7 on my one problematic drive and so far I've not had any further issues with it.

I followed the instructions on how to do the firmware upgrade from here: https://forums.unraid.net/topic/132024-guide-how-to-upgrade-samsung-nvmessd-firmware/

My other drive I've never had any issues with. I will update that one at some point too, and once I've done both and they're stable, I'll try dropping out those kernel commands.

9 hours ago, warpspeed said:

FYI I've put 6B2QJXD7 on my one problematic drive and so far I've not had any further issues with it.

I followed the instructions on how to do the firmware upgrade from here: https://forums.unraid.net/topic/132024-guide-how-to-upgrade-samsung-nvmessd-firmware/

My other drive I've never had any issues with. I will update that one at some point too, and once I've done both and they're stable, I'll try dropping out those kernel commands.

It's still working fine, did it solve the issue?

So far it has been okay, but the failure happens randomly.

I also found a recent kernel change that may be able to recover the drives without a hard power cycle, I posted about it here: https://forums.unraid.net/bug-reports/prereleases/72-beta-3-request-kernel-bug-fix-for-nvme-ssd-reset-r4044/

Samsung SSD 990 PRO with Heatsink 2TB

I am having the exact same issue /dev/nvme1n1, /dev/ng1n1, S73HNJ0XB03717R is having the issue.

I have just enabled this and also updated the firmware on both drives, let's see how I go.
linux /bzimage nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off

root@Tower:~# nvme list

Node                  Generic               SN                   Model                                    Namespace  Usage                      Format           FW Rev  

--------------------- --------------------- -------------------- ---------------------------------------- ---------- -------------------------- ---------------- --------

/dev/nvme0n1          /dev/ng0n1            S73HNJ0XB03688K      Samsung SSD 990 PRO with Heatsink 2TB    0x1          1.05  TB /   2.00  TB    512   B +  0 B   4B2QJXD7

/dev/nvme1n1          /dev/ng1n1            S73HNJ0XB03717R      Samsung SSD 990 PRO with Heatsink 2TB    0x1        888.44  GB /   2.00  TB    512   B +  0 B   4B2QJXD7

/dev/nvme2n1          /dev/ng2n1            2324E6E265B9         CT4000P3SSD8                             0x1          4.00  TB /   4.00  TB    512   B +  0 B   P9CR30A 

Update:

No issues since I did the above.

Edited June 20Jun 20 by brenton