Hi

 

I am trying to evaluate unraid as a replacement to my current setup (ESXi+TrusNAS).

 

I created the USB drive and tried to boot to my computer, the main screen loads but as-soon-as the boot process starts, the computer restarts itself without any error message.

 

I tried the various safe modes but nothing works.

 

Here is my setup:

 

• AMD Ryzen 9 5900x
• ASUS ROG Crosshair VIII Dark Hero with Bios version 4004
• 128GB RAM 2666MHz
• LSI Broadcom SAS 9300-8i 8-port 12Gb/s SATA+SAS PCI-Express 3.0 Low Profile Host Bus Adapter
• 10Gb SFP+ PCI-E Network Card NIC, with Broadcom BCM57810S Chip
• GPU 1: Nvidia GT710 - Main card where my external screen is connected to
• GPU 2: Nvidia Quadro P400

 

Hope you can help me

 

Thanks in advance

 

Itamar

Edited March 5, 20224 yr by hysel

How are you booting?  Via legacy or UEFI?  Try the other (rename the EFI~ to EFI or vice-versa and set the BIOS accordingly)

Thanks for the quick reply

 

Switching to legacy "helped" a little

 

I can now see more of the loading script but the PC still crashes.

 

I was able to take a picture on what was the last message on the screen before the PC restarted

 

Update

 

It looks like the motherboard is busted.

 

I ordered a new one and will update soon

 

Itamar 

 

 

Update

 

I don't think it was the Motherboard. I replaced it with Asus Crosshair VIII Hero wifi and I got the same error....

 

The next thing I did was to replace the RAM, I had a 2666Mhz kit which I replaced with a 3600MHz one. 

 

The interesting part is this, when I had all 4 DIMMS installed, the problem happened again. I removed 2 DIMMS and it worked. I re-installed the 2 other 2 DIMMS and the system worked 🤨. One interesting observation is the Motherboard is showing up post code FF even when unRaid starts...

 

Ok...

 

I tried adding my other storage devices, the system didn't boot up. I upgraded to the latest BIOS version (3801-->4004), didn't work.

 

The FF code was not good, so I tried to troubleshoot, I disconnected the CPU 2nd 4 pin power connector, didn't work. I put it back, system boot up but I still see code FF.

 

I am at a lost at the moment, I don't think FF is a good sign to have on a motherboard.

 

Any suggestions?

1 hour ago, hysel said:

I had a 2666Mhz kit which I replaced with a 3600MHz one.

Doubtful you have an actual 3600MHz kit (or 2666 for that matter)  Rather you're running an overclock on the memory (XMP or DOCP profiles).  If problems persist, disable that overclock

1 hour ago, Squid said:

Doubtful you have an actual 3600MHz kit (or 2666 for that matter)  Rather you're running an overclock on the memory (XMP or DOCP profiles).  If problems persist, disable that overclock

Tried that but it didn't work..

 

I noticed that the memory kit was not on the ASUS compatability list. I ordered a  new one just in case.

 

Will update soon..

Another observation

 

I disconnected everything from MB except for a GPU and my 2XM2 drives and I still got the same situation

 

Given that I replaced the MB and RAM already, could it be that I encountered a bad CPU as well?

 

I can successfully install ESX 7, TrueNAS and TrusNAS scale. I do get some restart issues when trying to run a VM but I am starting to wounder what else I can do here?

 

Is there a way to perform a CPU text like memtestx86? (I can google but would love to hear some suggestions)

 

Thanks in advance

 

Itamar

Make sure that ryzen C-states is disabled in the BIOS.

 

Power supplies can be the cause of these types of problems.  Look and see if you have one (or can 'borrow' one) to see if that fixes the problem. 

2 minutes ago, Frank1940 said:

Make sure that ryzen C-states is disabled in the BIOS.

 

Power supplies can be the cause of these types of problems.  Look and see if you have one (or can 'borrow' one) to see if that fixes the problem. 

 

I already did, that is the first thing I disable

What about Power Supply Typical Idle? (Set to "Typical") - And also look for BIOS updates

7 minutes ago, Squid said:

What about Power Supply Typical Idle? (Set to "Typical") - And also look for BIOS updates

 

I am running the latest BIOS (4004)

 

I will look for the Power Supply settings (I am not familiar with them)

42 minutes ago, Squid said:

What about Power Supply Typical Idle? (Set to "Typical") - And also look for BIOS updates

 

Tried to put the "Power Supply Typical Idle" to both Typical and Low but the problem still persist

Update

 

On an hunch, I tired to replace my PSU with another 750W i had in the house.

 

I was able to make some progress in terms of having unRaid load with all my PCI-E cards

 

but...

 

I have 6 x Seagate 6TB HDD, 2 Samsung 512 SSD and 1 Kingstone 256 SSD. As-soon-as I connect the 4 of the HDD drivers, everything works great,  I add the other 2 HDD, same problem... 🤨. It is not a specific drive, I can choose which 4 HDD I want to connect and they will work fantastic. 

 

So.... does a 750W PSU not enough for this setup?

 

Thanks

 

Itamar 

Spoiler

 

 

Look at the current ratings for the +12V buss(es).  You need at least 2amperes per hard drive.  (Startup current requirement)  However, I would think that this should not be an issue for most 750W power supplies with only 6 drives.  You will also have to add in the +12V current draw if those two video GPU's are plugin cards. 

 

EDIT:  The power rating of 750W is an absolute limit.  IF you exceed that limit for a couple of milliseconds, things will go wonkers.  Often the sum of the maximum power rating on the individual voltage busses will exceed the 750W rating!

Edited March 6, 20224 yr by Frank1940

44 minutes ago, Frank1940 said:

Look at the current ratings for the +12V buss(es).  You need at least 2amperes per hard drive.  (Startup current requirement)  However, I would think that this should not be an issue for most 750W power supplies with only 6 drives.  You will also have to add in the +12V current draw if those two video GPU's are plugin cards. 

 

EDIT:  The power rating of 750W is an absolute limit.  IF you exceed that limit for a couple of milliseconds, things will go wonkers.  Often the sum of the maximum power rating on the individual voltage busses will exceed the 750W rating!

Thanks for the explanation.

 

I ordered a 1000W PSU which looks like having more SATA rails available in it:

 

EVGA Supernova 1000 P6, 80 Plus Platinum 

 

https://www.amazon.com/dp/B09CRLRPBD/ref=cm_sw_r_apan_glt_i_A60Y0JAG02KPA4MND298

 

Should be here first thing in the morning so I will be able to test again.

 

 

The saga continues 

 

PSU arrived today. I plugged it in, Same problem. 😓

 

Ok then, let's switch gears. I thought maybe the fact that my SAS 9300-8I controller "doesn't like" the fact that both HDD and SSD drives so I decided to disconnect only the SSD drives. It workd!

 

I tried connecting and disconnecting the SSD drives just to verify and as long as I didn't connect the SSD drives to the SAS controller, I was able to boot unRaid.

 

Next, I connected my SSD drives to the on-board controller. Still works!

 

At this point, I was happy as a clam and decided to fully install the PSU including cable management etc.

 

I turned the PC on again, fully expecting to put this to rest and...

 

It failed again 🤣🙃😭

 

I am at a lost here.

 

Some more information about the SAS controller. I am running Firmware version 16.0.12.0

 

Anyone else has any other ideas?

 

Itamar

7 minutes ago, hysel said:

Ok then, let's switch gears. I thought maybe the fact that my SAS 9300-8I controller "doesn't like" the fact that both HDD and SSD drives so I decided to disconnect only the SSD drives. It workd!

 

I tried connecting and disconnecting the SSD drives just to verify and as long as I didn't connect the SSD drives to the SAS controller, I was able to boot unRaid.

<<<< snip   >>>

Some more information about the SAS controller. I am running Firmware version 16.0.12.0

 

Is this the latest firmware version for this card?  Is it flashed to the IT-mode?  Any indication that the card is not being properly initialized during the BIOS bootup?

 

I am going to ping @JorgeB as he might have some thoughts about this issue.  (There are not a lot of Unraid users using the 9300 series of LSI SAS cards at this point in time.)

 

9300-8i works fine, have a couple myself, firmware is also recent, so it should be fine, a rebooting server is usually due to a hardware issue.

3 hours ago, Frank1940 said:

 

Is this the latest firmware version for this card?  Is it flashed to the IT-mode?  Any indication that the card is not being properly initialized during the BIOS bootup?

 

I am going to ping @JorgeB as he might have some thoughts about this issue.  (There are not a lot of Unraid users using the 9300 series of LSI SAS cards at this point in time.)

 

Thank you, I'll wait for your response 

1 hour ago, JorgeB said:

9300-8i works fine, have a couple myself, firmware is also recent, so it should be fine, a rebooting server is usually due to a hardware issue.

Yep, I think I going to rip everything out from the case and start from scratch. Just to make sure it is not a short-circuit.

 

On 3/7/2022 at 11:11 AM, JorgeB said:

9300-8i works fine, have a couple myself, firmware is also recent, so it should be fine, a rebooting server is usually due to a hardware issue.

I now think that this was the CPU all along 😔

 

I have a personal computer with 5600x in it. I switched CPUs, and everything started working!

 

Before I go and try to replace it with the same CPU, should I expect a problem running a 5900x with my setup (see above)? are there enough PCI-E lanes? will a 5950x will be a better choice here?

 

I also saw some posts about changing the EDC and curve optimizer to try to "fix" this issue 

 

 

thanks

 

Itamar 

Edited March 8, 20224 yr by hysel

It was the CPU, got a new 5900x today and everything works now

 

thanks everyone for the support and help