Hi,

Hopefully this is the right place to post this. I am in the process of doing some initial burn-in testing on a server I just built that I had planned to use with FreeNAS. However, lately the more research I have done on unRAID the more I am convinced I should use unRAID instead. However, this system was built with FreeNAS in mind. I checked the Hardware Recommendations and it seemed like adding a dedicated USB Drive for the boot drive was the only change I needed. My understanding is that USB drives are the only type of device supported for boot devices. I cannot use the SSDs for booting I believe.

 

Here is all of the specs on my server:

 

Chassis: Norco RPC-4224
Motherboard: Supermicro X10SRi-F
CPU: Intel Xeon E5-1650v4
Cooler: Noctua NH-U9DX i4
Cooler Fans: 2 x Noctua 90mm NF-B9 PWM
PSU: Corsair RM1000x
RAM: 4x Samsung 32GB ECC DDR4-2133 (128GB Total)
Boot Drives: 2x SanDisk SSD PLUS 2.5" 120GB
Storage Drives:    24x WD Red 8TB NAS Hard Drive (starting with 8 initially)
HBA: 3x IBM ServeRAID M1015 (crossflashed to IT Mode)
Storage Fans: 3 x 120mm Noctua NF-F12 PWM (high SP)
Exhaust Fans: 2 x 80mm Noctua NF-A8 PWM

 

Here are my questions:

 

1) Does the USB device matter? Would you recommend a particular size or brand for this setup?

 

2)  I am not sure if I should even use the original SSDs I was going to use for boot drives as cache drives. I am not sure how often I will be writing to this server at the moment, but the idea of losing data on the cache does concern me. Is their a way to setup the cache devices to be redundant? 

 

3) Is their any recommendations for configuring unRAID with data integrity in mind? Would you recommend 1 or 2 party drives? One of the main reasons I had been using FreeNAS on my prior server was ZFS. However, I like the idea that I shoudn't lose an entire pool/array of data with unRAID (atleast thats the way I understand it). Should I be using the default filesystem or would you recommend something different? I believe I saw btrfs support, but it was not stable.

 

4) Is their any unRAID apps or plugins for burn-in tests? What plugins and apps do you recommend for larger builds?

 

5) Does unRAID also require my M1015's to be crossflashed to IT mode?

 

6) Will the 128GB ECC Ram that cost around $1k go to waste in unRAID? I heard FreeNAS was very memory hungry, but was not sure about unRAID. With 24 8TB drives (1-2 being parity drives) I want to make sure this is plenty of RAM or if I need more. I have 4 more slots, but was hoping to keep the price down if possible.

 

7) Am I missing anything else before making the transition to unRAID?

 

Thank you for all of the help in advanced!

41 minutes ago, Hinatanko said:

Hi,

Hopefully this is the right place to post this. I am in the process of doing some initial burn-in testing on a server I just built that I had planned to use with FreeNAS. However, lately the more research I have done on unRAID the more I am convinced I should use unRAID instead. However, this system was built with FreeNAS in mind. I checked the Hardware Recommendations and it seemed like adding a dedicated USB Drive for the boot drive was the only change I needed. My understanding is that USB drives are the only type of device supported for boot devices. I cannot use the SSDs for booting I believe.

 

Here is all of the specs on my server:

 

Chassis: Norco RPC-4224
Motherboard: Supermicro X10SRi-F
CPU: Intel Xeon E5-1650v4
Cooler: Noctua NH-U9DX i4
Cooler Fans: 2 x Noctua 90mm NF-B9 PWM
PSU: Corsair RM1000x
RAM: 4x Samsung 32GB ECC DDR4-2133 (128GB Total)
Boot Drives: 2x SanDisk SSD PLUS 2.5" 120GB
Storage Drives:    24x WD Red 8TB NAS Hard Drive (starting with 8 initially)
HBA: 3x IBM ServeRAID M1015 (crossflashed to IT Mode)
Storage Fans: 3 x 120mm Noctua NF-F12 PWM (high SP)
Exhaust Fans: 2 x 80mm Noctua NF-A8 PWM

 

Here are my questions:

 

1) Does the USB device matter? Would you recommend a particular size or brand for this setup?

A name brand like SanDisk Cruzer Fit is a good choice.  16GB or 32GB is fine.

 

2)  I am not sure if I should even use the original SSDs I was going to use for boot drives as cache drives. I am not sure how often I will be writing to this server at the moment, but the idea of losing data on the cache does concern me. Is their a way to setup the cache devices to be redundant?

The cache disk (pool) can cache writes to the array.  It is also the default application drive for Plugins, Dockers and VMs under unRAID.  You don't need a cache drive if you are Ok writing directly to the array (slower unless you use Turbo Write) and don't plan to use Dockers/VMs or Plugins with data storage requirements.  If you do decide to use a cache drive you can implement a redundant pool of cache devices if you select BTRFS as the file system for the cache drive.

 

3) Is their any recommendations for configuring unRAID with data integrity in mind? Would you recommend 1 or 2 party drives? One of the main reasons I had been using FreeNAS on my prior server was ZFS. However, I like the idea that I shoudn't lose an entire pool/array of data with unRAID (atleast thats the way I understand it). Should I be using the default filesystem or would you recommend something different? I believe I saw btrfs support, but it was not stable.

For the number of data drives you're planning on, you should use 2 parity drives.  XFS is the preferred file system for data drives.

 

4) Is their any unRAID apps or plugins for burn-in tests? What plugins and apps do you recommend for larger builds?

You definitely should install the Community Applications plugin.  That's where you're going to get all your other plugins and Dockers.  Typically you'd also install the CA Fix Common Problems plugin as well.  And the Preclear plugin is used to burn in new disks prior to introducing them to the array.

 

5) Does unRAID also require my M1015's to be crossflashed to IT mode?

Yes

 

6) Will the 128GB ECC Ram that cost around $1k go to waste in unRAID? I heard FreeNAS was very memory hungry, but was not sure about unRAID. With 24 8TB drives (1-2 being parity drives) I want to make sure this is plenty of RAM or if I need more. I have 4 more slots, but was hoping to keep the price down if possible.

Yeah, it will largely go to waste unless you run large VMs.  unRAID can run in a couple of GB.  Dockers use memory, and VMs use even more.  unRAID uses the remainder for in-memory write-caching.  If you aren't running Dockers and VMs, though, 16GB is generous for pure NAS functions and beyond that is overkill.  But overkill is fun sometimes .

 

7) Am I missing anything else before making the transition to unRAID?

The trial license is free, give it a try!

 

Thank you for all of the help in advanced!

 

Some answers above...

 

Edited February 24, 20179 yr by tdallen

40 minutes ago, Hinatanko said:

Here are my questions:

 

1) Does the USB device matter? Would you recommend a particular size or brand for this setup?

 Don't think this matters, so long as you go with a reliable main brand. 8 or 16GB is more than enough. This is only used for initial boot. Also stores plugins and stuff. Frankly, a 2GB drive would probably be fine, but as 8GB is pretty much the smallest you can get nowadays, 8 or 16 will be more than enough.

 

2)  I am not sure if I should even use the original SSDs I was going to use for boot drives as cache drives. I am not sure how often I will be writing to this server at the moment, but the idea of losing data on the cache does concern me. Is their a way to setup the cache devices to be redundant? 

SSDs are pefect for cache. You can set them up in a cache pool so content is mirrored and data loss potential is minimized

 

3) Is their any recommendations for configuring unRAID with data integrity in mind? Would you recommend 1 or 2 party drives? One of the main reasons I had been using FreeNAS on my prior server was ZFS. However, I like the idea that I shoudn't lose an entire pool/array of data with unRAID (atleast thats the way I understand it). Should I be using the default filesystem or would you recommend something different? I believe I saw btrfs support, but it was not stable.

Depends how many drives you're going to have. You can certainly start with 2 parity drives if you have the budget, but many people recommend one is fine if you're only going to have a few data drives. Looking at your specs (24 drives), definitely go with 2 and give yourself an extra little bit of padding redundancy (You can lose 2 drives simultaneously before data loss can occur).

 

btrfs is required if you are going with a cache pool, and XFS seems to be the FS of choice for the array. If you're going with a single cache drive, XFS is also fine for it

 

4) Is their any unRAID apps or plugins for burn-in tests? What plugins and apps do you recommend for larger builds?

The preclear plugin (which you should really install and use anyways), is great for burning in drives. As far as for the cpu/ram, you can easily run Prime95 from command line. There's a thread floating around the forum with details on how-to

 

5) Does unRAID also require my M1015's to be crossflashed to IT mode?

I don't know if it is absolutely required, but it is generally the recommended way to go

 

6) Will the 128GB ECC Ram that cost around $1k go to waste in unRAID? I heard FreeNAS was very memory hungry, but was not sure about unRAID. With 24 8TB drives (1-2 being parity drives) I want to make sure this is plenty of RAM or if I need more. I have 4 more slots, but was hoping to keep the price down if possible.

If you're only using UnRaid as a NAS, technically, 4GB would be enough (some would say even less!) :). The thing is, with version 6, there is so much great stuff you can do (Dockers/VMs), that more ram becomes necessary. If you really are only going to use it as a NAS, if you go 16GB, you should be set. Problem is, if you decide down the road to do more, it becomes more onerous to source matching ram, or maybe you'll need to replace the sticks you bought... It's really up to you. 128GB is A LOT. Unless you're running a bunch of VMs, really unnecessary. Why not go 32GB. Plenty of ram for running a bunch of dockers and a few VMs down the road...

 

7) Am I missing anything else before making the transition to unRAID?

This is a great start! Nothing comes to mind right now, but poke around the forums. I get tons of great ideas from just browsing the forums regularly :). It's a great community, and you're going to love it! Welcome!!

 

Thank you for all of the help in advanced!

 

Edited February 24, 20179 yr by DoeBoye
Doh! tdallen beat me to the punch! :)

3 minutes ago, DoeBoye said:

Doh! tdallen beat me to the punch!

Lol, at least we gave him the same answers :).

Just now, tdallen said:

Lol, at least we gave him the same answers :).

lol! True! And in the same colour! I felt the blue was a comfortable, yet authoritative colour

Blue was a last minute, rather whimsical colour selection for me.  It seems it is a very versatile colour!   

Thanks guys. I probably should have mentioned my use case for this server. I will probably be running Deluge or some sort of torrent client, couchpotato and similar dockers, plex server, openvpn. A lot of the standard stuff. I will probably run 5-10 VMs for testing but probably would not have them all on at the same time. I wouldn't mind trying to run a Mac VM all the time if possible as well. Do you still think 128 GB is too much?

 

If the 128GB is still too much I may either return some (2 sticks and have 64 GB if that sounds correct) or look into more serious VMs if returning RAM is no longer possible.

For FreeNAS and ZFS, 128GB would have been great, but is excessive for a normal unRAID.  But there's a ZFS plugin that will enable ZFS usage if you want.  It's unofficial, but no complaints as far as I know.

 

I'm going to start recommending the following page for all newcomers:  Guides and Videos.  Besides guides and tutorials, the videos have a lot of ideas.

9 minutes ago, Hinatanko said:

Thanks guys. I probably should have mentioned my use case for this server. I will probably be running Deluge or some sort of torrent client, couchpotato and similar dockers, plex server, openvpn. A lot of the standard stuff. I will probably run 5-10 VMs for testing but probably would not have them all on at the same time. I wouldn't mind trying to run a Mac VM all the time if possible as well. Do you still think 128 GB is too much?

 

If the 128GB is still too much I may either return some (2 sticks and have 64 GB if that sounds correct) or look into more serious VMs if returning RAM is no longer possible.

 

Hmmm... That changes things a bit... I personally like to have 8GB per running VM, and 8GB left-over for UnRaid system and Dockers. 64GB is a good number. Plenty of overhead for a bunch of concurrent VMs, *and* you can save some $ from not buying 128GBs of ram... Keep in mind, this is opinion territory at this point... Plenty of people run VMs with less and have no issues... 

 

Also, if you drop to 16GB modules (x4), you might save a bit more over 2x32GB. That still leaves you room to add another 64GB down the road if you find 64GB to not be enough....

Use the money you save on RAM for SSDs.  You may need several by the time you are done for your Dockers and VMs.  Plan on installing the Community Applications plugin and it sounds like you might find a use for the Unassigned Devices plugin as well for the VMs.

I actually already have completed this build, but I have reached out to the seller to see if a refund is possible on 2 of the sticks so I would still have 64 GBs.

 

In terms of SSDs what would be your recommendation be? I assume the 2 SSDs I have at the moment are probably not the best choice for my use case and unRAID? How many SSDs would you recommend I be running for my use case inside this build?

 

I have seen videos in the past that use dedicated graphics cards in their VMs. Is this worth it? If so what graphics cards do people recommend for VMs?

4 hours ago, RobJ said:

For FreeNAS and ZFS, 128GB would have been great, but is excessive for a normal unRAID.  But there's a ZFS plugin that will enable ZFS usage if you want.  It's unofficial, but no complaints as far as I know.

 

I'm going to start recommending the following page for all newcomers:  Guides and Videos.  Besides guides and tutorials, the videos have a lot of ideas.

 

Does this plugin still allow for all of the unRAID benefits like adding drives to the array at different sizes and being able to not lose the whole array if the parity drives fail and another drive fails? Are the write speeds the same you would get with unRAID or are they more like FreeNAS (80-100mbs?)?

I haven't used it myself.  I would read the ZFS plugin for unRAID thread, then ask your questions there.  I doubt ZFS drives can be part of an unRAID array though.

So I will be able to get a refund on 2 sticks making my system have a total of 64 GB of RAM. From what I have heard and seen it sounds like this will do more than enough.

 

What would your recommendations be for SSD setups for caching? Should I look into returning the 2 SSDs I correctly have as well? How many SSDs should I have?

 

Does a dedicated graphics card make sense for VMs? I am not sure if this nearly as important. I would not be using the VMs to play games. At most replace my Macbook Air that is several years old with a VM running Sierra and do web development on it.

 

I also had a follow up question after installing unRAID. I am using a 32GB Sandisk USB 3.0 Drive on my motherboard internally. Is this the proper way to do the boot device or should I be using something besides USB 3.0? My assumption was that USB 3.0 would be the best because of the speed. Is this correct?

12 minutes ago, Hinatanko said:

Does a dedicated graphics card make sense for VMs? I am not sure if this nearly as important. I would not be using the VMs to play games. At most replace my Macbook Air that is several years old with a VM running Sierra and do web development on it.

Passing through a video card gives you the best performance.  Without it, you're either going to be accessing the VM via RDP or VNC  Its really up to you -> depends upon what you do with the VM.  But for web development, RDP/VNC will be easily fast enough.  But then you'll be accessing the VM via another client.  With pass through, you access the VM directly on the server's attached monitor / keyboard / mouse

14 minutes ago, Hinatanko said:

I also had a follow up question after installing unRAID. I am using a 32GB Sandisk USB 3.0 Drive on my motherboard internally. Is this the proper way to do the boot device or should I be using something besides USB 3.0? My assumption was that USB 3.0 would be the best because of the speed. Is this correct?

Speed of the USB flash is meaningless on unRaid.  It's only used during the initial boot, and for the occasional saving of settings. After the boot up is finished, the flash just sits there doing nothing.  The internal connector is best, if only because there it is safe from accidental breakage.

6) I have 32GB ECC ram, most of the time memory usage is 35%.  Running a VM, Plex server, and 3 other Dockers.

 

Make your largest drive the Parity drive.  In my case to get more storage space without increasing the SATA ports, I get a bigger Parity drive and use the old Parity drive to swap out with one the data disks.

 

For SSDs, think along these lines:

Start with the idea of a single SSD.  It will be your cache drive.  You can use it for write-caching to the array, or not.  You will use it for Docker, though.  Your Docker image file will be ~20GB and then you need enough space for all the working data of your Dockers.  120GB would work, but most folks buying something new would go with 250GB.

Now add another SSD.  You'll probably add redundancy to your cache system by using BTRFS and picking a RAID-1 setup on the cache pool.  It's easiest if the 2nd SSD is the same size as the first.

Now it's time to think about VMs.  By default the storage for your vdisks will be on the cache pool.  That can use a lot of space though, and it's shared storage.  You can use larger SSDs, or add more SSDs to the cache pool.  As an alternative, though, you can host VMs on their own, dedicated SSD.  You'd mount those SSDs for VMs using the Unassigned Devices plugin, linked above.

 

 

It looks like I won't be able to get a refund on the 2 120gb MLC SSDs I originally was going to use with FreeNAS. I may still get differents SSDs though. Is their a list of recommended SSDs I can look at or do you have any suggestions for brands, models, type, etc?

 

When looking at doing a osx vm is their any particular graphics cards I should use? I believe Apple uses AMD in their hardware. Not sure if that matters or not.