Mostly storage related questions for 1st unRaid setup


v81

Recommended Posts

Hi all,

 

I have some hardware incoming, 2 x Xeon E5 2670, ASRock EP2C602-4LD16, 8 x 8GB DDR3 1600 Reg ECC.

Looking to put it to use as an unRaid VM, and need to better understand how storage works with unRaid.

 

I have been looking around for info, most recently the Wiki here but am still clueless.

I'm thinking storage is either poorly documented or maybe it's just one of those things that makes sense after you've experienced/setup first hand.

 

The wiki explains "Assigning Devices to the Array and Cache", but does not explain what *purpose* or *role* the array or cache plays later on.

When someone says cache i think of fast volatile memory on a CPU / hard disk / raid controller etc.

 

Then there is the mention of a 'parity device' and mention of an 'array'.

What are these, what are they used for, do i need one or both?

 

I'm at a loss here, but believe i should have the aptitude for this, i have 25 years experience with computers from being a server admin in a school to building / experimenting with hardware and OS's at home!

 

So I'm looking for an explanation on both the physical and logical parts of storage at play in unRaid.

 

Also looking for an example of what hardware and setup is involved for fast and high IOPS storage that i can run several OS's on.

Will be 1 or 2 OS's with dedicated PCIE video hardware.

Several other OS's with game servers etc...

 

I am also toying with the idea of making this machine my Windows 10 desktop daily driver / gaming rig.

This will all depend on how practical the idea is, and what performance i might gain or loose vs my current machine, an [email protected]/8GB DDR3/GTX 970/240GB(OS) & 960GB(Games) SSD's

The Xeon machine would be superior in number of cores, but inferior in regards to clock speed.

 

If relevant I have a NAS for bulk / redundant data, and a separate backup solution.

The NAS is capable of NFS and iSCSI if this is at all useful to unRaid.

 

What i need to know is what is an ideal setup for fast, high IOPS storage suitable for a VM server?

 

Your help will be much appreciated, will be happy to post my results once i'm up and running.

Link to comment

The recent addition of virtualization options in unRAID has created an interesting dynamic, with folks coming to the platform first and foremost looking at it for its virtualization capabilities... Interesting.

 

unRAID's history is as a home and small business NAS storage solution, so some of the concepts and terminology may not translate exactly into your background.  The array is the collection of disks managed under unRAID.  Data drives, Cache Drives, and the Parity Drive are all part of the array.  Data drives store your data - simple enough - but given the history of unRAID think movies, TV shows, photos, etc.  You data isn't striped across drives ala traditional RAID - it's written to a single device.  unRAID supports data drives of multiple sizes in the array and supports spinning drives down that are not in use.

 

The parity drive is the "RAID" part of unRAID.  It is a data loss prevention mechanism.  When data is written to the array a mathematical calculation is done and parity data is written to the parity drive.  You can Google "parity" for a background, but the basic function of parity in unRAID is that it allows unRAID to emulate and reconstruct a failed data drive.  unRAID today supports one parity drive (and can tolerate the failure of one drive in the array without data loss) while unRAID 6.2 will support more than one parity drive when it comes out.  The parity drive protects data on the data drives, but not the cache drive.

 

The cache drive has two functions in unRAID.  First, it can optionally be used to cache writes to the parity protected array.  The parity calculation process is a little bit expensive in terms of I/O and CPU, and therefore is slower than writing directly to a hard drive.  Writes to a cache-enabled user share are written to the cache drive at full speed without parity calculations.  The data is available to you as part of the array, but it won't be protected by parity until Mover runs - typically once a day.  Mover moves data from the cache drive to the parity protected data drives.  Second, the cache drive is used as an application drive under unRAID.  Dockers, docker data, and VMs are all commonly stored on the cache drive in a cache-only (i.e., don't move to the data drives) share. 

 

The parity protected storage subsystem in unRAID is commonly used for non-volatile, high volume data - movies, photos, music, etc.  The cache drive can be a hard drive but is frequently an SSD.  Or, and I think this is where it gets interesting for you, it can be a redundant pool of SSDs using the BTRFS file system.  Your description of "fast, high IOPS storage suitable for a VM server" makes me think that you should look at the BTRFS cache pooling features of unRAID 6 for your VM storage.

Link to comment

tdallen.....

 

Excellent post ! This is very clear and concise information. Explains it all perfectly for newbs....like me.

 

V81......

 

Thank you for asking this question. I was also struggling to get a grip of it all. I am so glad you asked this because as i said above, a very clear overview given by tdallen.

 

Thank you all !

Link to comment

Thanks for the info tdallen. I've also been wondering the same questions about how the storage is supposed to work. (And still have a few more) It does make more sense realizing the background that UnRAID comes from to determine why they have chosen the model they have. This setup has the advantage for the end user of being able to swap out drives and use any sized drive they want, but it comes at a price. Basically what I'm gathering is that UnRAID uses a modified version of RAID 3, which is certainly not designed with performance in mind. The parity drive will be the weak point of the system especially since it's not handled asynchronously, so if you're writing data to multiple drives at once it's only going to go as fast as the parity drive allows. It will also probably be the first drive to fail as it's going to be handling far more I/O than any other drive. (Two of the reasons why RAID 5 was invented) For consumer usage where the majority of data is written once to disk these short comings are likely not a big deal. Obviously the cache drives can definitely help alleviate problems by holding as much data as possible so it can be written to disks as time allows.

 

The cache drives themselves seem straight forward. Any writes destined for the array go to the cache drives first, they are written to disk asynchronously at a later time. You can have multiple copies of the cache with no performance hit since it's in a software RAID 1. The part that I can't seem to figure out is if there is actually any type of read cache for any of the system. Best I can tell is that it uses a decent chunk of RAM to store data it thinks it might need. I'd really like to know if there is any more info on read performance as it seems like that could be really good if the data happened to be on different disks, or so so if all of the data you wanted to access was on the same disk. (Since we're not striping there is no guaranteed performance increase that all other RAID levels have.) Maybe 6.2 will bring improvement for read caching so you can hold large chunks of data in cache so there is a greater chance of success that the data needed will be on the fast cache and not the slow HDDs.

 

The last thing I haven't figured out yet is how you go about getting the VMs to cache at all. There is no setting for the VMs themselves to set whether or not they cache, and I believe that I put their disks onto a share that was supposed to be cached, but it didn't seem to make a difference. Can you just load and keep the VMs onto SSD cache drives and then have them periodically write the data back to the disk array for redundancy? How would you go about getting the most I/O for the VMs without taxing the parity drive or the disk drives?

 

 

Link to comment

I've heard unRAID described as closer to RAID 4, without the striping.  Read acceleration is not part of unRAID today but has been talked about as a future feature, here's a thread on it: http://lime-technology.com/forum/index.php?topic=34434.0 .

 

When you create a user share you can specify it as "Use cache disk:  Only" or cache-only.  This means that user share will be located on the cache drive and not moved to the main array.  Specifying a cache-only user share as the Primary vDisk Location when creating the VM will locate it on the cache drive and keep it there.  Another commonly used strategy is to use the Unassigned Devices Plugin to mount an SSD outside the array for VM storage.  I'm guessing that folks periodically shut down their VMs and backup the image file to the array, that's what I do with Docker.  There's a whole forum for the KVM Hypervisor used in unRAID, it's worth looking through there.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.