Jump to content
testdasi

The Black Bear - Threadripper 2990WX build

59 posts in this topic Last Reply

Recommended Posts

Posted (edited)

Minor updates:

  • Bought the Asus Hyper M.2 (4xM.2 -> PCIe x16) to mount the PM983 away from the GPU and open up future expandability. It now idles at under 40 degrees (5 below previous best), reaching around 46 during heavy gaming sessions (10 below previous best).
  • The SM951 (AHCI) refuses to die despite my intentional "running it to the ground" effort. Switching to the Hyper M.2 requires rejigging the config anyway so I decided to switch to the SM951 as my workstation VM boot drive (again!). Along the way, I'm reminded of how compatible the AHCI PCIe M.2 standard is - it shows up to the VM as a SATA drive so it boots natively (unlike NVMe, for example which requires a few hoops).
  • I now have a very #FirstWorldProblem of having 1x 2280 and 3x 22110 slots that I want to fill with M.2. drives. :)
  • I found my old Kingston SSDNow+ 128GB collecting dust (it was back when SATA III didn't exist) so decided to use that as my temp space for things waiting for the next gdrive batch. It becomes a half-intentional protection against crypto virus because of how unionfs works. If I were infected, it would take less than 30 minutes for the drive to fill up and error out (giving me a quick clue) while protecting my actual content.
Edited by testdasi

Share this post


Link to post
Posted (edited)
4 hours ago, testdasi said:

Asus Hyper M.2

Fascinating! Do you use my DiskSpeed Docker? I'm curious how these show up in it. Four controllers with one drive or one controller with four drives?

 

I think I might have to pick one of these up. How is the cooling? Seems like it'll be a good way to add VM drives or a larger cache pool.

Edited by jbartlett

Share this post


Link to post

Hi @testdasi

 

I read your thread with interest as I'm planning a threadripper build and considering MB options. The Designaire is high on my list and I wanted to check something with you.

 

At the beginning of your thread, you intimated that you would hesitate to recommend the designaire due to a few issues and niggles. However, reading on, it seems you've found solutions for man , (all?) the challenges. Would you still recommend looking elsewhere or would you feel it's now OK for an unRaid build?

 

Second, and if you would be so kind, would you mind sharing your passthrough method and, if possible, any config files relevant? 

 

I'm just interest to see what and how you've done it with this board. I've been passing through GPUs, Controllers and discrete devices since VMs became a thing in unRaid, but not frequently enough to remember how to do it each time. The best practice has evolved over time and I struggle to find the most up to date recommendations.

 

Cheers!

 

 

Share this post


Link to post
On 7/18/2019 at 6:27 PM, jbartlett said:

Fascinating! Do you use my DiskSpeed Docker? I'm curious how these show up in it. Four controllers with one drive or one controller with four drives?

 

I think I might have to pick one of these up. How is the cooling? Seems like it'll be a good way to add VM drives or a larger cache pool.

My vfio stubbing continues to work without any intervention so can't test it with the DiskSpeed Docker. But based on how I understand PCIe bifurcation, it should be as if you have 4 more M.2 slots. Everything (as far as I can tell) behaves just like the motherboard M.2 ports.

 

The cooling is excellent. 7-9 degrees C cooler. Most importantly, it cools back down very quickly after load.

Share this post


Link to post
4 minutes ago, meep said:

At the beginning of your thread, you intimated that you would hesitate to recommend the designaire due to a few issues and niggles. However, reading on, it seems you've found solutions for man , (all?) the challenges. Would you still recommend looking elsewhere or would you feel it's now OK for an unRaid build?

 

Second, and if you would be so kind, would you mind sharing your passthrough method and, if possible, any config files relevant? 

 

I'm just interest to see what and how you've done it with this board. I've been passing through GPUs, Controllers and discrete devices since VMs became a thing in unRaid, but not frequently enough to remember how to do it each time. The best practice has evolved over time and I struggle to find the most up to date recommendations.

 

I can fully recommend the Designare EX as long as you don't need ACS Override due to the lag issue I reported.

There may very well be some advanced fix that I don't know of but the easiest fix for me is to not use ACS Override and everything runs perfectly fine.

Come to think of it, it may be an issue with the i440fx VM machine type (I now use Q35 - that was one potential fix I didn't test).

 

There is really nothing special with my passthrough method. I basically follow the SpaceInvaderOne textbook.

  • vfio stub the devices that need to be passed through
  • dump my own vbios for the GPU and use it (eventhough it's theoretically not necessary since my motherboard BIOS allows me to boot Unraid with its cheapo GPU on slot 3)

For a refresher on PCIe passthrough, just watch SpaceInvaderOne youtube videos. They are practically just short of the guy going to your place to do it for you.

 

Of course if you have any problem, feel free to ask.

Share this post


Link to post

Thanks for the quick and comprehensive response.

Share this post


Link to post

Do you happen to have an LSI SAS controller (or anyone who has this mobo have one)? I found out the the AsRock x399 Taichi has an issue with the LSI controllers where it just resets them constantly, and requires a BIOS fix to come out for it. No news from AsRock on eta or if they will fix it, but I am looking into getting a different mobo that supports SAS controllers. Does the DESIGNARE support a sas controller?

Share this post


Link to post
On 8/26/2019 at 6:38 PM, urhellishntemre said:

Do you happen to have an LSI SAS controller (or anyone who has this mobo have one)? I found out the the AsRock x399 Taichi has an issue with the LSI controllers where it just resets them constantly, and requires a BIOS fix to come out for it. No news from AsRock on eta or if they will fix it, but I am looking into getting a different mobo that supports SAS controllers. Does the DESIGNARE support a sas controller?

No I don't use SAS controller unfortunately so can't test that. I fully embrace the cloud now so don't need a big array anymore.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.