So I have been recently trying to transition to some new hardware and am having issues with the Oracle F320 that I have been using as my cache drive for some time. From my research it seems that this is an NVME/MOBO/Linux Kernel incompatibility and I was curious on things I could test for a possible resolution. I will list what I have tried and my current plans moving forward.

 

New System:

Asrock x670E PG Lightning w/ Ryzen 9 7900x and 64GB DDR5 6000Mhz. Unraid has been tested with both the latest stable and beta.

 

- Memtest was done on memory for 24 hours without issue at 6000Mhz XMP profile

- PCIE x16 slot bifurcated to x8x8 in BIOS and using this card to split to x8x8

- F320 is in one bifurcated slot while my LSI HBA is in the other

- x520 DA2 in the x4 PCIE slot for NIC

 

System runs fine and eventually NVME controller crashes and I lose access to almost everything except local prompts, unable to clean shutdown or restart. Image shows logs from the crash. I have appended the power control to my syslinux config and still experienced the crash about 6 hours later.

 

 

After a few attempts I have moved the hardware back to my old server and plan on troubleshooting this one. I have a spare F320 Flash Accelerator to test with as well as a second Unraid license I can use for troubleshooting. My current plan is to install the beta build and test the system with only the spare F320 installed and no bifurcation to ensure that is not causing an issue. Assuming it still crashes I am considering compiling a newer kernel version of Linux to test with Unraid. Are there any other steps I should be considering or would make more sense with my test environment to try first?

 

 

Some NVMe devices have issues with power states on Linux, try this, on the main GUI page click on the flash drive, scroll down to "Syslinux Configuration", make sure it's set to "menu view" (top right) and add this to your default boot option, after "append initrd=/bzroot"

nvme_core.default_ps_max_latency_us=0 pcie_aspm=off

e.g.:

append initrd=/bzroot nvme_core.default_ps_max_latency_us=0 pcie_aspm=off

Reboot and see if it makes a difference.

Sorry, should of been more clear but I have already done this and experienced a crash afterwards.

In that case I would recommend trying a different m.2 slot or a different NVMe device (different brand/model)

Well that is an option, I was wanted to see if there is a way to make these flash accelerators work in the new system. They are PCIE NVME devices and 3.2Tb. For the price it would be impossible to get a different device that is comparable. That’s why I have it in my test UNraid system and am considering upgrading the kernel myself to see if it resolves but was asking if anyone had further or additional guidance. 

It may just be a compatibility issue between that device and the board, but difficult to say without testing in a different system.

Do you know of a good troubleshooting method to get the NVME to crash like I was experiencing? I put my old system back together and the drive has been fine for over 12 hours which is longer than it would last in the new system. I have an identical drive to test with and it’s been running in my new system for a bit now but since it’s a test bench it’s not doing anything and I’m not sure how to trigger the issue. 

8 hours ago, bubbadrk said:

Do you know of a good troubleshooting method to get the NVME to crash like I was experiencing?

Not really, just use the server normally.

For anyone following this or finding this in the future. I got it to crash again but I think I’ve narrowed down the problem to the bifurcation riser. Gonna get another and do some more testing to confirm.