hi all,

 

i've seen a few posts in various threads go unanswered about best practices for performance, so i thought i'd just start a thread. i'd venture to say a lot of us are running VMs with applications such as sabnzbd/nzbget, sickbeard/nzbdrone, couchpotato, transmission/deluge, plex, subsonic, etc that will be doing a hefty amount of reading and writing to/from the array. does anyone have any input on the best place to be running VMs from, locations to download/postprocess to, or anything of that nature? for example, would there be a benefit to a VM postprocessing downloads inside of itself vs using a share?

 

currently my ubuntu VM has my main share (uses the cache drive) and a cache only share mounted. the VM downloads to and postprocesses on the cache only share, then moves completed downloads to the main share. this seems like a lot of io for the cache drive.

 

options i'm thinking about:

1) mount a drive outside of the array, run VM(s) off this drive, download/postprocess to cache-only share, then move to the main array

2) mount a drive outside of the array, run VM(s) off this drive, download/postprocess inside the VM, then move to the main array

3) stop using the cache drive (originally put it in for plugins), use it as the vm drive in option 2

 

anyone have input?

Currently my setup operates the same way as yours does. Use the cache to host the VM and all downloading direct (via network) to cache before it post processes to unRAID.

 

Whichever way you do this (apart from direct to share) then I don't see a difference. There will be a lot of writes to a drive be it cache or outside of the array

 

 

http://lime-technology.com/forum/index.php?topic=33221

 

Good discussion here on mounting outside of the array

http://lime-technology.com/forum/index.php?topic=33221

 

Good discussion here on mounting outside of the array

 

This is what I am doing, running a 120GB SSD outside of the array, and only using it for VM's which need performance.

 

 

I also run my VM's outside of the array, copy the IMG files to the cache drive in a temp directory then remove the backup VM IMG file and place the freshly copied file into the correct cache backup directory.

I need to start doing regular backups of the VMs now that they are setup. I assume this has to be a manual only process as the VMs need to be powered down before taking a backup?

I need to start doing regular backups of the VMs now that they are setup. I assume this has to be a manual only process as the VMs need to be powered down before taking a backup?

Someone made a script around here that will pause your VM, make a backup and resume your VM.  Not sure where it was at....

 

*edit*

 

Found it....It was created by Lonix.

#!/usr/bin/bash

vmHome="/mnt/cache/VM"
backupHome="/mnt/user/VMBackup"
logHome="/mnt/user/VMBackup"

xl list | tail -n+3 | cut -d ' ' -f 1 | while read line; do
        xl pause $line
        rsync -av --progress --"log-file=$logHome/$(date +%Y%m%d)_backup.log" $vmHome/$line $backup$
        xl unpause $line

done

Note that the rsync line was truncated in the original copy-n-paste ($ at the end of line)

Note that the rsync line was truncated in the original copy-n-paste ($ at the end of line)

after the $backup$ ?

I'd like to ask about memory and cpu configurations (since this topic notes "best practices for performance").

 

Seems like you carve up RAM as you need, so you need to know how much to give something like ARCH, or another distro, as well as Windows.

• How much should you keep for unRAID at all times? (possibly dependant on if you have any plugins as well?)
  
• How much should you give something like arch/unbuntu/distro that you are putting all your programs in, instead of using plugins (sql, plex, torrent/newsgroup clients, etc) or how do I do the math to figure out it?
  
• How much ram should I give to a windows VM (probably "as much as i can spare"
  

 

For CPU, I understand cores, and HT counts into your cpu count (so four cores + HT means you have 8 "cpus" to use for VM reference)

• Can I oversubscribe my cpus? how do i factor in vcpus vs my core/ht counts?
  
• How many cores should i give to a linux distros VM?
  
• How about a windows vm?
  
• Will the domu always take back the cpu time as it needs it, or could you really let something like plex go crazy in the VM and hamper your unRAID domu performance
  

 

I think this topic was really about hdds, and running multiple VMs on a hdd, so my last question is about running the VMs from an SSD. I'm just using a cheap 128GB SSD, but how many VM images could really be running on that one SSD before I start to see an issue? Should I split the VMs up onto multiple physical drives, or will that just move the bottleneck to the controller anyway? Since this could also be very system depentant, how would I go verifing and testing this for myself ?

 

thanks!

 

For my specific reference, i'm currently running a c2 Q9650, which is a c2 generation (4 cores, not HT) and 8GB of ram, so I'm trying to figure out what I can realisticly do with this. I'm currently just doing unRAID 6b4, and the ARCH VM from IB, with 2 vcpus and 2gb ram.

I want to throw in another question... For a DownloadVM running either Sab or any torrent application, are you building the VM with, say, 100GB and download into the VM?

Or are you setting the VM up with a, say, 15GB disk and then download on either the same hdd, just outside of the VM or another hdd?

my setup:

 

have 8g ram and 8 core intel on a supermicro mb

 

I give unraid 1024mb ram and 1 core, i have 2 plugins running on unraid, apc and swapfile, the actual swapfile is located on a SSD outside the array

 

my 240g ssd is mounted ext4 outside of the array for trim support

 

i have one major vm where all my plugins run (sab, sickbeard, couch, plex etc), all downloads are pointed to my cache drive when completed

 

its running on IB's Arch VM setup - 15g - give it 6 cores and everything but 1g of ram

 

I also made a 100g data.img that is mounted as a 2nd drive - my sb uses that for its temp download spot as 15G in the main VM can fill perty fast.  I also put my plex library there.

 

I also have a owncloud VM that i play with, and a piaf astrisk vm for my phone system - using 1 core and what ram is left over

 

Myk

 

my setup:

 

have 8g ram and 8 core intel on a supermicro mb

 

I give unraid 1024mb ram and 1 core, i have 2 plugins running on unraid, apc and swapfile, the actual swapfile is located on a SSD outside the array

 

my 240g ssd is mounted ext4 outside of the array for trim support

 

i have one major vm where all my plugins run (sab, sickbeard, couch, plex etc), all downloads are pointed to my cache drive when completed

 

its running on IB's Arch VM setup - 15g - give it 6 cores and everything but 1g of ram

 

I also made a 100g data.img that is mounted as a 2nd drive - my sb uses that for its temp download spot as 15G in the main VM can fill perty fast.  I also put my plex library there.

 

I also have a owncloud VM that i play with, and a piaf astrisk vm for my phone system - using 1 core and what ram is left over

 

Myk

 

Did you edit syslinux (i think thats the one) to limit unRAID to 1 core/1024mb, or since that is all that is left over, you leave it alone?

 

I assume you gave your Arch VM 6 cores because of plex? Wonder how much cpu time you really need for the rest of the dl'ing apps?

my setup:

 

have 8g ram and 8 core intel on a supermicro mb

 

I give unraid 1024mb ram and 1 core, i have 2 plugins running on unraid, apc and swapfile, the actual swapfile is located on a SSD outside the array

 

my 240g ssd is mounted ext4 outside of the array for trim support

 

i have one major vm where all my plugins run (sab, sickbeard, couch, plex etc), all downloads are pointed to my cache drive when completed

 

its running on IB's Arch VM setup - 15g - give it 6 cores and everything but 1g of ram

 

I also made a 100g data.img that is mounted as a 2nd drive - my sb uses that for its temp download spot as 15G in the main VM can fill perty fast.  I also put my plex library there.

 

I also have a owncloud VM that i play with, and a piaf astrisk vm for my phone system - using 1 core and what ram is left over

 

Myk

I have a similar setup with 8 core, 16GB ram, 120 SSD cache, 60GB SSD mounted outside array for domains. I give unraid 2GB/2 cores. I have one major ubuntu server domain (sab, sick, couch, plex, deluge and mythtv backend) I give it 4GB and 4 cores but I haven't moved my plex library over yet. So it might need more ram and cores.

 

I like the idea of the data.img instead of a temp directory on the cache drive. That way I could keep my domain under 10GB for faster backup and not write to the cache drive as much.

my setup:

 

have 8g ram and 8 core intel on a supermicro mb

 

I give unraid 1024mb ram and 1 core, i have 2 plugins running on unraid, apc and swapfile, the actual swapfile is located on a SSD outside the array

 

my 240g ssd is mounted ext4 outside of the array for trim support

 

i have one major vm where all my plugins run (sab, sickbeard, couch, plex etc), all downloads are pointed to my cache drive when completed

 

its running on IB's Arch VM setup - 15g - give it 6 cores and everything but 1g of ram

 

I also made a 100g data.img that is mounted as a 2nd drive - my sb uses that for its temp download spot as 15G in the main VM can fill perty fast.  I also put my plex library there.

 

I also have a owncloud VM that i play with, and a piaf astrisk vm for my phone system - using 1 core and what ram is left over

 

Myk

 

Did you edit syslinux (i think thats the one) to limit unRAID to 1 core/1024mb, or since that is all that is left over, you leave it alone?

 

I assume you gave your Arch VM 6 cores because of plex? Wonder how much cpu time you really need for the rest of the dl'ing apps?

 

Ya I gave as much as I could for plex and its transcode - as I do plan on sharing my library with family once I get on the new ISP that gives me 10mb uplink instead of the 768kb I have now....

 

Myk

 

my setup:

 

have 8g ram and 8 core intel on a supermicro mb

 

I give unraid 1024mb ram and 1 core, i have 2 plugins running on unraid, apc and swapfile, the actual swapfile is located on a SSD outside the array

 

my 240g ssd is mounted ext4 outside of the array for trim support

 

i have one major vm where all my plugins run (sab, sickbeard, couch, plex etc), all downloads are pointed to my cache drive when completed

 

its running on IB's Arch VM setup - 15g - give it 6 cores and everything but 1g of ram

 

I also made a 100g data.img that is mounted as a 2nd drive - my sb uses that for its temp download spot as 15G in the main VM can fill perty fast.  I also put my plex library there.

 

I also have a owncloud VM that i play with, and a piaf astrisk vm for my phone system - using 1 core and what ram is left over

 

Myk

I have a similar setup with 8 core, 16GB ram, 120 SSD cache, 60GB SSD mounted outside array for domains. I give unraid 2GB/2 cores. I have one major ubuntu server domain (sab, sick, couch, plex, deluge and mythtv backend) I give it 4GB and 4 cores but I haven't moved my plex library over yet. So it might need more ram and cores.

 

I like the idea of the data.img instead of a temp directory on the cache drive. That way I could keep my domain under 10GB for faster backup and not write to the cache drive as much.

 

Ya, I just moved Library and everything below it, to the / of the data.img mounted.  This way I can used that space for any of the programs I have running on that VM.  also did it because my sb puts the stuff in a wanted dir - and then I split it between home and work sabs because of my monthly limit at home, but at work because of business - no limit.  then i just move from machine at work to a 1tb portable hd to a watched folder in sickbeard and then it processes and moves them into the array etc.

 

Myk