Hey, just wondering if anyone has experience running Unraid on the the AsRock Rack X399D8A-2T motherboard for Threadripper seen here: https://www.asrockrack.com/general/productdetail.asp?Model=X399D8A-2T#Specifications

 

Found a couple of threads by people considering running it but not sure if anyone has actually used it. I'm especially curious if the on board  Intel X550-AT2 10gb nic will work? 

 

Thanks! 

Looking for the same info.  Did you ever get it?  NewEgg has the 2950X on sale for $599 which has me about to pull the trigger.  The only issue is that I want IPMI and this seem to be the only available board that has it.  Just want to make sure there won't be any issue with Unraid.

From what I read on the Level1 Forum, it looks like @cybrnook has one. You could ask him.

Been running it for a few months now and love it, it's been super stable (disabled C states and set power supply idle control to typical in BIOS). Both Intel NIC's work out the box and register as 10Gb and the IOMMU groupings are nice as well. Details of my system are in my signature.

 

I am on the latest 6.9.0Beta25 using a recompiled Kernel to add Nvidia Drivers from @ich777 's nice docker tool - Here

 

For BIOS and BMC:

BMC Version - 1.30.00 - Download

BIOS Version - L1.37 (BETA BIOS I requested from Support) - Download

 

For the BETA BIOS, it's been rock solid. The difference with this one is not only (current release bios doesn't even have bifurcation) does it allow you to set bifurcation, but it does so on a per PCIe slot basis. BETA BIOS prior had PCIe slots setup in groups. So if you wanted to set one slot to x4x4x4x4, then you got two slots as x4x4x4x4, I didn't like that. I am posting it in my onedrive for you since I find it a requirement.

 

 

 

 

Edited July 12, 20214 yr by cybrnook

Thanks so much for the detailed response!  Especially including the links to BMC and BIOS.  What model of SuperMicro RAM is that you're using.  I'd be curious to compare it's price to what Asrock lists on their QVL list.

23 minutes ago, RockDawg said:

Thanks so much for the detailed response!  Especially including the links to BMC and BIOS.  What model of SuperMicro RAM is that you're using.  I'd be curious to compare it's price to what Asrock lists on their QVL list.

They are discontinued now, but I got 8 x sticks of this in there: https://store.supermicro.com/16gb-ddr4-2400-mem-dr416l-hv01-eu24.html

 

MANUFACTURER Hynix

MEMORY CAPACITY 16 GB

MEMORY TECHNOLOGY DDR4

MEMORY SPEED 2400 MHz

VOLTAGE 1.2 V

ECCVLP ECC UDIMM

RANK 2 rank

CAS LATENCY CL17

MANUFACTURER PART NUMBER HMA82GU8AFR8N-UH

WARRANTY 5 year warranty

 

(here is the current equivelant: https://store.supermicro.com/memory/ddr4/16gb-ddr4-mem-dr416l-cv02-eu26.html)

 

 

Also, if you need a TR for cheap, let me know. I jumped on the 2950X deal, so want to sell off my 1920X for about $220 shipped. (I'm in SE Michigan, seemingly right above you)

Edited September 6, 20205 yr by cybrnook

Thanks for the RAM info.  Also, thanks for the generous offer on the 1920X.  That's a good deal.  But I am going to get the 2950X.

@RockDawg check this out too: https://www.phoronix.com/scan.php?page=news_item&px=ECC-DRAM-Overclocking-TR

 

Good stuff! I'm up to 2800 on my Hynix (From stock 2400), and exited to see what happens with 2nd Gen TR when it comes in since it has a better IMC.

Edited September 8, 20205 yr by cybrnook

This is now a sore spot for me. I screwed around and missed the Newegg Deal! I finally sat down to place my order yesterday and, of course, out of stock! Now they're $929. I am so ticked off at myself for waiting. Not sure what I'll do now. 

 

Good info anyhow. Thanks for reporting! 

13 minutes ago, RockDawg said:

This is now a sore spot for me. I screwed around and missed the Newegg Deal! I finally sat down to place my order yesterday and, of course, out of stock! Now they're $929. I am so ticked off at myself for waiting. Not sure what I'll do now. 

 

Good info anyhow. Thanks for reporting! 

I think you should pickup the 1920X I have 🙂 . There is nothing wrong with it, as I have been running my unraid system with it for about 6 months now. You lose 4 cores and you don't get PBO, but confirmed working on what I am running at least.

Edited September 8, 20205 yr by cybrnook

I was wanting to stay at 16 cores, but I may consider it.  Still doing some research and price comparing.  Just curious... what do you run with your system?

Did you get the ipmi for fan control working?

i got this mainboard up and running with a 2950x running nicely. but got a question about the temperature measurement which i should use

 

 

it shows here 2 different temps for the cpu. It says 32.5C and the k10temp tctl = 58 C  when i check the ipmi it says CPU 58C.

I researched this very recently as I just built a system with this board.  From what I read Tcdie is the actual CPU temp. Tctl is the CPU temp with a 27C offset.  In IPMI it will show the CPU temp reading the same as Tctl.

19 minutes ago, RockDawg said:

I researched this very recently as I just built a system with this board.  From what I read Tcdie is the actual CPU temp. Tctl is the CPU temp with a 27C offset.  In IPMI it will show the CPU temp reading the same as Tctl.

ok so i should set it to the Tdie for correctly showing the cpu temp.  nice thx

Just curious, what kind of temps are you seeing and what containers and VMs are you running?  I'm having a hell of a time trying to control the temp on mine.

Does the motherboard model and info for this board appear for you guys?  Mine just says " ASRockRack Version - s/n:".  I swear it used to show the model but I am on my 3rd board (no fault of Asrock) so maybe it was one or both of the others that showed and this one isn;t reading.  Wondering if the beta BIOS broke that or if maybe there's an issue with my board.

Anyone have any feedback on my last two questions?  I'm trying to sort out if I have an issue with my MB.

On 9/6/2020 at 8:54 AM, cybrnook said:

Been running it for a few months now and love it, it's been super stable (disabled C states and set power supply idle control to typical in BIOS). Both Intel NIC's work out the box and register as 10Gb and the IOMMU groupings are nice as well. Details of my system are in my signature.

 

I am on the latest 6.9.0Beta25 using a recompiled Kernel to add Nvidia Drivers from @ich777 's nice docker tool - Here

 

For BIOS and BMC:

BMC Version - 1.30.00 - Download

BIOS Version - L1.37 (BETA BIOS I requested from Support) - Download

 

For the BETA BIOS, it's been rock solid. The difference with this one is not only (current release bios doesn't even have bifurcation) does it allow you to set bifurcation, but it does so on a per PCIe slot basis. BETA BIOS prior had PCIe slots setup in groups. So if you wanted to set one slot to x4x4x4x4, then you got two slots as x4x4x4x4, I didn't like that. I am posting it in my onedrive for you since I find it a requirement.

 

 

 

 

Hey, Cybrnook. I finally got around to buying the X399D8A-2T. But for some reason I can't get my existing unraid usb to boot and get an IP assigned. When go to the ip address I get the IPMI login screen but I can't get to the unraid main screen. A fresh unraid install boots just fine, it gets an IP and I can navigate to the unraid page. Did you have that problem with yours?

On 5/30/2021 at 3:42 AM, JohnyClassic said:

Hey, Cybrnook. I finally got around to buying the X399D8A-2T. But for some reason I can't get my existing unraid usb to boot and get an IP assigned. When go to the ip address I get the IPMI login screen but I can't get to the unraid main screen. A fresh unraid install boots just fine, it gets an IP and I can navigate to the unraid page. Did you have that problem with yours?

There are a few things you may need to consider (nothing really specific to the board itself, but overall). Not sure what your current USB drive is, but you would want to make sure it's formatted for UEFI boot vs Legacy boot (that also means renaming the -EFI file to EFI on the unraid USB). Then in the BIOS of the board, you will need to make sure your boot options are setup correctly. My server is in the middle of it's monthly parity check, so I am not going to reboot it now, but if you want, I can send you a screenshot of my boot options the next time I reboot so you can see what I am using. High level I disabled all boot options for things I didn't need, like PCIe boot, booting from network etc.

 

Now that I think about it, the AsRock rack boards come with a lot of virtual devices enabled in the BMC, you will have to disable that for your USB drive to be listed as a bootable device. If you don't, then the boot options menu in the bios are just flooded with virtual devices.

 

 

Disable in General:

 

Set all devices to 0 in VMedia Instance:

 

 

EDIT: Also, you said 

Quote

When go to the ip address I get the IPMI login screen but I can't get to the unraid main screen.

 

Make sure that you are NOT using the same IP for BMC as you are for your UnRaid server, they need to be different IP's. On top of that, the board has a dedicated NIC for BMC, I assume that's what you have plugged in. You will need to use one of (or both) of the 10Gb nic's for your UnRaid server. So that means you will need to make sure the MAC address and NIC device are set accordingly in your existing unraid install, since that's likely setup for your old system. An easy way to get around that is to rename your existing network.cfg file and replace it with a stock one from the vanilla unraid install media and then boot. Once it's up and running with DHCP, you can log in and reconfigure your network.

Edited May 31, 20215 yr by cybrnook

Thanks for all the tips cybrnook! I got it to work after deleting both network.cfg and network-rules.cfg. But all of this is great advice to get everything set up and working correctly. 

@cybrnook Do you have an updated link to the beta bios?

16 hours ago, ryanhaver said:

@cybrnook Do you have an updated link to the beta bios?

https://1drv.ms/u/s!AlHg8pmWgJJ_gr9mLid6muLTIMOddA?e=KbfHe2

Thank you @cybrnook!

 

AsRock Support just provided me with L1.32 which they state has support for bifurcation.

 

I've asked them for clarification on the differences between BIOS versions L1.32 and L1.37.

 

5 hours ago, ryanhaver said:

Thank you @cybrnook!

 

AsRock Support just provided me with L1.32 which they state has support for bifurcation.

 

I've asked them for clarification on the differences between BIOS versions L1.32 and L1.37.

 

Bifurcation was supported in BIOSes already prior to 1.37. What 1.37 gave me, and is what I asked for, is PER SLOT bifurcation options. Prior to 1.37 bifurcation worked in groupings.