Game development build server - help with configuration.


Sygan

Recommended Posts

Hi,

I'm responsible for setting up a build server for Unity in our company and I'm considering using Unraid as a base OS for this server. 

We already have specs closed and bought - they are as follows.

Threadripper 3960X - 24 cores.
2 x 1 TB NVMe SSDs
4 x 4 TB HDDs
128Gb RAM 3200Mhz 
Nvidia 2080 Super GPU

I've never used Unraid before so it I'd like if someone could guide me how to implement the current idea for this build - or if it's even possible. 

So what we would essentially want is to have four VMs + NAS storage as an archive. 

The way that I've envisioned it was to have all of the VMs on 1TB SSD (250Gb of space each). This VM would be backed up (in a sort of Raid1 configuration) on to the second 1TB SSD, so in case one of them fails we can just boot up the VMs from the second drive. 

I'd like to allow these machines to use all available CPU power at the moment. Memory can be limited to about 32GB (minus the RAM the Unraid needs).  

One of this machines will need to have a GPU pass-through for the purpose of baking lights in Unity (and sometimes this machine will also be used for Adobe Premiere rendering) but it would be nice if other machines would have a some kind of GPU emulation as Unity doesn't like to run its builds without a GPU.

These machines will be running Windows as an OS, each of them would have Jenkins and Unity installed. The one with a GPU passthrough would serve as an Jenkins master and the other three will be slave machines. This would allow us to run up to four simultaneous builds (as we deploy the game for multiple platforms or sometimes have a need to run multiple projects). We'd need access to these VMs through some Remote Desktop connection if needed.

All of them would have access to a NAS storage that would be made from the HDDs (we'd like to have them configured in a sort of Raid5 configuration where one of the drives could fail without losing any data). This would be used as an archive for previous game builds and some unused assets or work files for old projects. We'd need access to these storage from any computer in the network. 

A nice to have feature would be to have a VPN server that we can connect to so we can have access to these data from outside of office (in a times such as these when everybody is mostly working from home).

Another nice to have feature would be to allow the unused power to go for Folding@Home as for the most of time this machine will be turned on and not doing much. 

Do you have any ideas or guides that can help me to set this up using Unraid?  





 

Link to comment
11 hours ago, Sygan said:

...

128Gb RAM 3200Mhz 
Nvidia 2080 Super GPU
...

One of this machines will need to have a GPU pass-through for the purpose of baking lights in Unity (and sometimes this machine will also be used for Adobe Premiere rendering) but it would be nice if other machines would have a some kind of GPU emulation as Unity doesn't like to run its builds without a GPU.
...
Do you have any ideas or guides that can help me to set this up using Unraid? 

Some pointers:

  • You shouldn't run your memory at 3200MHz (even if the RAM is rated for such speed). Manufacturer-rated overclock is still an overclock and overclocking is always less stable, which isn't ideal for a hypervisor host.
  • There is no graphic emulation in Unraid. If you need graphic-card-like capabilities then you need to pass through a graphic card to the VM. Threadripper should have enough lanes for 4 GPU's. Issue may come with the 2080 because a lot of the RTX out there are 2.5-slot width varieties which means you can't fit 4x cards in the typical slot arrangement. Make sure to get 2-slot width cards.
  • Get a Gigabyte motherboard because you are embarking on a rather "interesting" adventure so any quality of life improvement will be useful. You almost certainly will need to dump your vbios rom file and/or change which graphic card the BIOS boots with. Gigabyte BIOS has the Initial Display Output BIOS setting that allows you to pick any x16 slot to boot with (instead of having to physically swap the card to the first PCIe x16 slot) - check the owner manual.
  • Expect compromises. VM is never the same as bare metal.
  • Watch SpaceInvader One youtube channel for tutorials and guides.

 

 

Link to comment

Hi,

As I said, the specs are already bought and closed, so we won't be adding any additional graphic cards for the moment (as we don't really need them performance wise, that would be just a waste of money). 

I'll check the channel, thx :)

 

Link to comment

Interesting project!

 

you will set up your 4tb drives in the unraid array. You will have one as parity, the other three as data drives. You will have 12 TB of shareable storage, and parity protection for one drive failure.

 

Parity protection != backup. If the data is mission critical, you should also have a backup to a separate system, offsite etc.

 

once set up, you can go about setting up shares on your array. You have a lot of flexibility here. You could have one share per vm, and have each windows instance mount it’s own. Or you could have one or more shares by function (queue, in progress, completed, output) and have each vm mount each of these.

 

you have a lot of options.

 

you also have a fairly steep learning curve ahead to get this set up correctly and, critically for a production environment, in a stable fashion. 

 

What motherboard do you have, as a matter of interest?

Edited by meep
Link to comment
  • 2 years later...

It's been a couple of years but I haven't noticed the message @meep.

I actually stopped working at the company I've built the machine for but the server was working fine and was (and still is to my knowledge) used when I left. But if I rememeber correctly the motherboard was a Gigabyte TRX40. 
 

On 4/23/2020 at 11:25 AM, testdasi said:

There is no graphic emulation in Unraid. If you need graphic-card-like capabilities then you need to pass through a graphic card to the VM. Threadripper should have enough lanes for 4 GPU's. Issue may come with the 2080 because a lot of the RTX out there are 2.5-slot width varieties which means you can't fit 4x cards in the typical slot arrangement. Make sure to get 2-slot width cards.

 


To this point - setting up the VMs to use VNC as a graphics card was enough for Unity to work fine. 

I've actually recently built a home workstation with similar capabilities with the exception that I'm using one VM with a GPU passthrough as a main workstation and I'm running another two VMs - one with Windows, one with macOS using VNC - for the purpose of running game builds that won't freeze my PC, and also so I can build stuff for iOS. 

This setup actually doesn't need much in terms of setting it up correctly. It's as simple as running Jenkins in a Docker container, preparing Jenkins pipelines and setting up VMs so they can work as build slaves. There's some trickery when it comes to installing PlasticSCM on a Jenkins container but it's doable. 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.