In my current setup, I am running 2 physical servers:

- One running OpenMediaVault as my NAS and PLEX server

- One running vSphere (with a few Debian VM's running Sonarr/Couch/Sabnzbd)

 

I'd like to consolidate onto one new server that I am building.  Ideally I would combine everything into one unRAID beast.  I'd use unRaid as my NAS/OS... and run Sonarr/Plex/etc in either Docker (docker will be new to me) or as Debian VM's.

 

I will have (20 x 5TB Reds) and (2 x 1TB EVO SSDs).  Yes, this is overkill... but i love this stuff... Love to build, learn, try new things...

 

I'd like your suggestions on how best to set up my drives.

 

I am leaning:

2 parity drives

18 data drives

1 cache ssd

1 ssd as a data drive for my VM's/Dockers

 

Thank you for your thoughts!

Edge

Looks like a good setup

Good luck with the build.

I have 26 drives in my backup server and am currently running UnRaid 6.2 beta 21 with two 8TB parity drives, runs fine.

 

You should be fine with your build, how will you transfer the data currently on your OMV server?

Thanks, guys...  Very excited about the build... and about trying out unRAID.

 

My OMV server currently has about 10TB of data.  I haven't thought about an elegant way to move the data over.  Suggestions?  I was going to just use my desktop, map to both servers and copy it over... waiting days for it to complete.. Should I use an external USB3 drive? 

 

Question on the drive setup please... As I mentioned, I'd like to use one of my SSDs as my data drive for VM's/Dockers.  Within the latest beta, how do I specify that the SSD should be used for a specific share and not part of the cache?

-For the parity drives, I selected 2 of the WD 5TB's.

-For the data drives, I selected 18 of the WD 5TB's.

-For the cache, I selected 1 of the SSD's.

-How do I set up the second SSD for use only by my dockers/vm's?

 

Thanks for the help.

Edge

 

Use "unassigned devices" plugin

 

Sent from my SM-G900W8 using Tapatalk

 

 

You can add one of your SSD's using the Unassigned devices plugin and put your VM's on that, you could then use the other SSD as a cache drive.

As for getting data to the ARRAY, are all the 18x5TB drives currently in use on your OMV server? Are you able to remove any of those drives and still keep your OMV server up and running? If so, then begin building your UnRaid server you will need at least three of those 5TB drives, one for parity and two for data, before you can start copying your data over. If you can't remove any of those 5TB drives, perhaps you have an external drive, and while tedious, copy data to it, then connect it to the UnRaid server when its up and running via the unassigned devices plugin and copy it over that way.

Thanks...  I will check out the "Unassigned Devices" plugin. Can the dockers also be stored on this unassigned SSD?  How does an unassigned device differ from a "cache only share"?

 

As for the migration... The unRAID server will hold all new drives.  I am not reusing anything from the old OMV server.  So both servers (OMV and unRAID) will be up and operational at the same time.  Probably easiest to transfer via my LAN.

 

 

 

Most people would use their cache drive to house their docker image as well as their vm images and files, I just happen to have a regular 3.5' HD as my cache drive and an SSD for my vm files. My SSD is an unassigned device but operates just fine in UnRaid, its not part of the array just an extra drive. A cache drive is not part of the array but is used by the 'mover' to move files to the array so a lot of people use an SSD as their cache drive for performance reasons.

Ok... please bear with me as I familiarize myself with the unRAID setup... I am getting closer...

 

Parity drives I get.. with the new beta, unRAID can now handle 2 parity drives... which would cover a 2 drive failure of the array. Yes?

 

Data drives are combined to serve as one big disk to the user... So any user shares created can span multiple physical disks, but to the user, will seem as one contiguous directory.  So if i create a user share called movies, it can, behind the scenes, span across multiple physical disks...

 

You could disable user shares, and instead, access the array at the disk level... disk1... disk2.. etc... shares would not span physical disk... you could however have disk1/movies and disk2/movies but no file can span those directories...

 

Cache drives are used to improve performance... so writes are stored here temporarily and then written to the disk array by the mover at some specified frequency?

 

An unassigned device is a drive that does not serve as any of the uses above (parity, array, cache).  It is a device that is mounted to the host OS outside of the unRAID array and can be used for VMs/dockers/etc.

 

Before I ask any other questions, please let me know if I got that stuff right...

 

(I appreciate your patience!)

Edge

Mate, you're spot on....

 

If only every newcomer made as much effort to get the basic knowledge right it'd make life a whole lot easier....

 

Sent from my LG-H815 using Tapatalk

 

 

Thanks, CHBMB.  With 3551 posts, I appreciate your kind words.  Way back when, I spent a few years in a QA/testing role so I try to understand/troubleshoot before asking questions.  I find you get much better answers that way too!

 

Just a couple more to ensure I got it down... 

 

- Plugins are stored on the flash drive and get loaded into memory upon boot.  Any changes to the plugin config (during runtime) get written to the USB immediately.  The mover doesn't touch the USB drive.

 

- If I put my VMs on the cache drive, I am assuming that the VM files would really live on the data array... but upon boot, the VM would be moved to the cache drive to run... and the mover process would save a copy of the VM to the data array at certain frequencies?  (this is the confusing part for me at the moment)

 

 

 

Small clarification. While user share directories span all array and cache drives that contain a root folder with that share name, no file can be split across multiple drives. That means if no single drive that could be participating in the user share has enough free space for a file, then it won't fit in the user share even if there is collectively enough space. Each drive is self contained, the user share system is just a merged view of the root folders.

 

My apologies if you already understood this, but your statement that no file can span the directories on disk shares gave me the impression that you thought it could on the user share system.

The mover does not touch the USB drive, no.

 

As far as I know, there are no VM files anywhere but on the cache drive, the VM config in UnRaid contains an XML file and knows where the VM files are located, but VM files aren't moved they live wherever you decide to put them, in this case on the cache drive.

 

 

Wow.  Another hero.  I am liking this community!  Thanks for the clarification, jonathanm.  Yes, good point.  I should have worded my sentence better.  I meant to say that the \movies user share would be spread across multiple physical disks.. and to the user it would look like one big directory, even if the files in that directory are coming from multiple disks.  Yes?

 

Thanks, ashman.  That makes sense.  I am going to try out some VMs tonight.

the \movies user share would be spread across multiple physical disks.. and to the user it would look like one big directory, even if the files in that directory are coming from multiple disks.  Yes?

Perfect.

 

Also, at the moment the cache drive is set up for write acceleration only, the mover will never put files back onto the cache drive. There has been some user discussion of a setup where small files like movie metadata or data newer than X time could be configured to live on the cache drive, but I haven't seen limetech participate in that discussion. For now, the user shares can be individually configured to live on the array drives only, the cache drive only, or be set up to put new files on the cache drive to be moved later. Edits to existing files already on the array stay on the current drive, so writes to existing files don't get moved to the cache drive.

Thanks, all.  I feel much better now.  I am off to play with the latest beta. 

 

BTW.  I just installed the Community Applications plugin.  Amazing.  333 apps I need to investigate.  I am going to have a busy weekend.

Install the powerdown plugin first....

 

Essential in my book..

 

Sent from my LG-H815 using Tapatalk

 

 

I am half way through my trial and wanted to circle back with you guys on my experience (and a few more questions please)...

 

MY TEST SETUP:

• Running 6.2.0-beta21
  
• 2 parity drives (2 x 5TB WD RED)
  
• 2 data drives (2 x 5TB WD RED)
  
• 1 cache drive (1 x 1TB EVO SSD)
  
• 1 unassigned device mounted as /mnt/disks/ssd (1 x 1TB EVO SSD)
  

 

WRITE PERFORMANCE:

• Without the cache drive, I was expecting 30-40MB/s for my writes based on forum posts.
  
• With the cache drive, I was expecting closer to 100MB/s.
  
• I created 2 shares: movies (use cache disk: NO) and shows (use cache disk: YES).
  
• I then copied the same folder from my local hard drive to each share.  The folder contained 12 .mkv files of about 1GB each.
  
• In both cases, I saw about 50MB/s on the writes.  Sometimes a few little spikes/lows, but mostly hung at 50MB/s.
  
• It doesn't seem like my cache is having any impact on my writes.  Did I miss something?
  

 

APPS:

• I absolutely love the Community Apps functionality.  It rocks!  Thanks to those who coded that.
  
• I have been playing around with some plugins (Dynamix, Unassigned Devices, Powerdown, etc), as well as the new Docker apps.
  
• I have heard alot about Docker from my developer friends at work, but I did not have any direct experience.  It has been a bit of a learning curve.  The hardest part is that many of the posts/tutorials/videos that I found were made before Community Apps came along... so they have extra steps because of the repository/templates.
  
• My current stumbling block is how best to configure Docker within my unRAID setup.  As we discussed, I'd like my Dockers and VMs to reside on the unassigned SSD.
  
• I initially tried to configure Docker to use the unassigned device (/mnt/disks/ssd).  However, no matter what I tried, I was unable to start docker when I tried to point to the unassigned device.
  
• I was only able to get Docker started when I pointed Settings/Docker/Docker Storage Location to: /mnt/disk1/appdata/docker.img... and Docker appdata storage location to: /mnt/user/appdata/
  
• I am guessing either Docker only wants to reside on a share (versus a mount)... or maybe some type of permissions issue?
  
• Your suggestions please on how to best setup Docker/VM's to use the unassigned SSD.
  

 

Thank you for your help.

Edge

I'd verify it's actually using the cache drive (check the drive's contents). I assume you are transferring to a disk/share instead of the actual share, which wouldn't use the cache drive. My 1TB EVO cache drive gets 110MB/s on my gigabit network.

 

I've heard nightmare stories about SSDs as data drives on unRAID. If your SSD starts garbage collection or TRIM, it can invalidate parity which will result in sync errors and/or failed data rebuilds. Not sure if this was ever confirmed, but makes sense to me. I personally wouldn't use an SSD as a networked data drive seems like insane overkill.

I absolutely love the Community Apps functionality.  It rocks!  Thanks to those who coded that.

It keeps me out of trouble.  Somewhat 

Docker appdata storage location to: /mnt/user/appdata/

Ideally, you should NOT use /mnt/user/appdata.  It really should be /mnt/cache/appdata (or if you're not using a cache drive, then /mnt/diskx/appdata - /mnt/user do not 100% support the symlinks that many apps want to use and there may be some issues with the containers if you continue to use /mnt/user/appdata

I am guessing either Docker only wants to reside on a share (versus a mount)...

Doesn't need to be in a share.  Mine is directly on the cache drive [/mnt/cache/docker.img]

Your suggestions please on how to best setup Docker/VM's to use the unassigned SSD.

Possibly a race condition because docker will wind up starting up before UD mounts the SSD.  Never tried it as with a cache drive, I don't particularly see the point in using another drive for docker images.  But your use case may differ.

 

Thanks for CA, Squid!  Awesome.

 

Let me focus on the Docker issue first.  Here is what I tried:

- My unassigned SSD is mounted at: /mnt/disks/ssd

- Under Settings/Docker, I have the following:

  Enable Docker: YES

  Docker vdisk size: 10GB

  Docker storage location:  /mnt/disks/ssd/appdata/docker.img

  Default appdata storage location:  /mnt/disks/ssd/appdata

  Template Authoring Mode: NO

 

When I click Apply/Done, nothing happens.  It still says docker status is stopped.

When I browse to /mnt/disks/ssd/appdata/, I do not see a docker.img.  I was under impression that it would be created automatically?

 

You'd have to post your diagnostics after trying to create a docker.img  Don't use UD for this purpose, so can't really offer much without seeing them.

 

Assuming that it is possible for docker.img to exist on a share mounted with UD (and not sure here because of the possibilities of race conditions), you are going to be better off with 6.2 than with 6.1 if you want appdata to exist on a UD mounted drive.  This is because 6.2 offers mounting options within docker mounts of "slave" mode which if you don't have set then you wind up having to stop and start the docker server every time you boot up to get it to see any volumes mounted with UD. - Just a PITA, but not a show stopper on 6.1

 

 

EDIT: re-read your settings and you are already using 6.2 

 

I decided to stop swimming upstream.  I abandoned my desire to host my VMs/Dockers on the unassigned device.  I had this mental image of keeping those separated from everything else, but it was causing problems.  I don't want to be a special case.  Much harder to get answers to your questions then.

 

I now have 2 parity drives, 2 data drives, and 2 cache drives.  Nice.  Simple. 

 

I changed my docker settings to use:

- Docker storage location:    /mnt/cache/appdata/docker.img

- Appdata storage location:  /mnt/cache/appdata/

 

Yes, and you guessed it, docker is running!  Thank you.

 

So with this configuration, I will get the performance I want for my VMs and Dockers by having them on my speedy SSD drives.

 

Since my appdata share is set to: "Use Cache Disk: ONLY", that means the mover will ignore this share, correct?

Correct

 

Sent from my SM-T560NU using Tapatalk

 

 

Awesome. 

 

Let me go play with all these dockers for a bit and educate myself on these container mappings... I will then test out the cache write speeds again to see why I am not seeing any speed increase.

 

Thanks again.