[XEN VM IMG] ArchVM <--- deprecated 01/07/2014


688 posts in this topic Last Reply

Recommended Posts

Good points, SBD.  I think my concern is not stability, its the risk of the monthly updates doing something drastic that causes difficulty for casual users.  From what I've observed Arch has a history of dropping support for things quickly after adopting a new standard (this is arguably what makes it awesome too; it doesn't get hung up supporting old ways of doing things and can stay nimble). 

 

Agreed that Arch has some of the best documentation. 

 

You are right though, Centos probably isn't a great option for applications you describe. 

 

Sent from my Nexus 4 using Tapatalk

 

 

Link to post
  • Replies 687
  • Created
  • Last Reply

Top Posters In This Topic

Question any clue as to why this wont work? I mounted my unraid cache drive (where plex used to be installed) to /mnt/cache to match unraid. Then i changed /etc/conf.d/plexmediaserver  to

 

# Change this to be what you like

PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR="/mnt/cache/.custom/Plex/home/Application Support"

 

Which is actually a cifs mounted share to the cache drive. However, this still shows up as a new installation of Plex and does not use my old libary.

 

Reason: Alot more space on my cache drive to store 40+ gigs of my plex library than to try and implement it into arch.img

 

ref: https://plexapp.zendesk.com/hc/en-us/articles/200273978-Linux-User-and-Storage-configuration

 

and symlinks wont work due to cifs

 

[root@ArchApplianceVM_v3 Application Support]# ln -sf /opt/plexmediaserver/Library/Application\ Support/ /mnt/cache/.custom/Plex/home/Application\ Support/

ln: failed to create symbolic link â/mnt/cache/.custom/Plex/home/Application Support/Application Supportâ: Operation not supported

 

Link to post

From what I've observed Arch has a history of dropping support for things quickly after adopting a new standard (this is arguably what makes it awesome too; it doesn't get hung up supporting old ways of doing things and can stay nimble).

 

All the big things that Arch needed to switch too they have done... netctl instead of netcfg, systemd instead of sysintv, etc.

 

As of now... Arch doesn't have any major changes in the core apps / Linux Kernel aside from doing things with Pacman (Package Manager) but even those are more around package building and the common user wouldn't know / notice anyway.

 

If Ironic really wants to maintain a VM Appliance for unRAID long term...

 

I would move all the Packages from Arch to his repo, remove all the Arch repos from the VM and point them only to his. I would then lag behind Arch 3 - 6 months. That way he can manage and plan well in advance for any changes / test / let Arch deal with bugs. For example, the libnfs and grub issues would be prevented.

 

For the users who wanted to stay more current, they could enable a "testing" repo he sets up that are the packages that get rolled out in the 3+ month release.

 

That would be how I would do it if I was going to roll out a VM Appliance.

 

Even though this might offend some, if I was him I would also charge for my VM Appliance too once all the bugs get worked out. Slacklet provides a vanilla Linux OS and charges $10 where as Ironic does 1,000 more things and has a cost associated with hosting it.

Link to post

I will save you all a bunch of time / trouble.

 

1. You can use VHD, QCOW2, thin-provisioning, CoW, etc.

 

2. You need qemu installed to create qcow2.

 

3. There are other tools for VHDs.

 

4. You need to boot them with pvgrub.

 

5. You are going to need multipath and other packages if you ever want to mount / access the partitions within the VM.

 

6. Yes, you can convert images from VHD, QCOW2, VMDK to RAW format but it isn't the greatest tool. There are other opensource ones that a much better. Also, you will be running your VMs in HVM and not PV. Which means unRAID (Host) has to work a lot harder and your VMs are slower.

 

7. It's best that you create a fresh VM from scratch with the Xen PV drivers installed (like ironics). It's very easy to install a Linux VM with Xen Drivers in a VM... Do not bother creating images in VMWare and then use QEMU tools that are designed for KVM to convert them over.

 

8. Leave the VM Appliance images small.

 

Let the users create their own "data drive" image for the size they want and need. You can mount more than one "drive" in your VM using the VM cfg file. Mount the "data" drive (image file you create) as "xvdb" and leave the VM Appliance as "xvba". Your "data" drive is where you have all your stuff download / store before your Apps or a chron job move it to unRAID for you. Instead of people arguing or only being able to have VM Appliance X size, let you users decide for themselves.

 

9. Jumping through hoops for QCOW2, VHD, etc. isn't worth the trouble and using step 8 above you avoid the necessity of having to create thin provisioned images that are designed for KVM. Xen DOES NOT recommend you use them in a production environment.

 

10. Why go spend $100+ on an even bigger cache drives to store your downloads that you are going to copy over to your unRAID? Why not just download them to your unRAID instead? Do you value this data?

 

  a) Having your drive(s) on your unRAID server spin down does not increase how long they will last.

 

  b) The amount of money you spend on a new cache drive is probably 4X the cost of what you would spend for electricity for a year.

 

  c) When your files are done downloading, renamed, copied where they should be... The unRAID drive(s) will spin down.

 

11. Instead of using file based protocols like NFS or Samba...  It's time to look at block-based protocols too.

 

If I may be so bold as to make a request: This type of info is perfect for the Virtualization FAQ.  Is there any chance you could repost it there?  http://lime-technology.com/forum/index.php?topic=31716.0

Link to post

If I may be so bold as to make a request: This type of info is perfect for the Virtualization FAQ.  Is there any chance you could repost it there?  http://lime-technology.com/forum/index.php?topic=31716.0

 

Thanks... I didn't think of that. I know Ironic saw the post and I don't want to step on his toes at this point.

 

I'm sure he will want to double check / test some of things I mentioned above. If need be, move them over to that thread.

Link to post

 

 

Even though this might offend some, if I was him I would also charge for my VM Appliance too once all the bugs get worked out. Slacklet provides a vanilla Linux OS and charges $10 where as Ironic does 1,000 more things and has a cost associated with hosting it.

 

If this were to offend someone it is their own fault.  The guides are here, if they don't want to pay,  they can just as easily make the same thing themselves.  Not exactly the same but close.

 

As grumpy once said,  you will treat down and rebuild a number of times in all of this,  it is true.  I speak from experience and can say you can install and have a functional arch install pretty quickly if you just note all the commands outside the wiki.  Then just install your apps and configure them the same you you need to here,  pointing then to your shares,  etc.

 

I am not offended... I know hosting is not free.  Hell,  even donations are surely welcome.

 

Link to post

NFS bug fix until v4 is out.

 

Quote limetech:

Found the NFS problem, fixed in -beta4.

 

Here is a workaround.  First, in the webGui select the disk/share you are exporting via NFS.  Next change the security mode to "Private".  Then in the Rule field enter this:

 

*(sec=sys,rw,insecure,anongid=100,anonuid=99,all_squash)

 

Click Apply.  Your NFS client mounts should work now.

 

 

Sent from my Nexus 5 using Tapatalk

 

Link to post

with the install of sabnzbd, have you included the multicore version of par2?

 

I've found great par/unpack performance on my existing ubuntu sab box with 3 cores and multicore par2 (i believe the standard unrar is already multicore aware).  Granted, I'm using a FIO card for my esx datastore, but it might be worth investigating.

 

I'm also using an AMD CPU so any opportunity to do something multi-threaded is a huge performance jump, intel might not apply here.

Link to post

So I've looked over the youtube videos in preparation to try this out over the weekend, thanks for those. But I did notice when you go into editing syslinux that unraid (dom0) is allocated 2 gigs of memory. Is there a reason why you allocated so much memory to unraid? Right now all I see that really utilizes memory is Crashplan and Plex. Is there a guide posted somewhere on the suggested memory size?

Link to post

First, Ironic - Awesome job on the image and explanations (especially the videos). It made things very easy to follow.

 

I do have a couple of questions though:

 

So, I have SAB & SB installed, and am looking to configure. I also want to get Plex going, however on my cache drive when I look at my \Plex\Application Support folder I see the following:

 

Size: 14.1GB

Size on disk: 189GB

Contains: 196,969 Files, 122,385 Folders.

 

I am guessing that is way too large to mount into the VM (assuming I am looking at the 189GB, not the 14.1GB entry.

 

Since I had SAB, SB, Plex installed on UnRAID 5.0 I already have a share on my cache drive called /appdata that has all the relevant subdirectories for each app.

 

Is there a way to point SAB, SB & Plex to the /appdata share for it's config and default folders? I am assuming I can configure data directories as /net/Tower/mnt/user/appdata/sabnzbd (or Plex). I would guess this should limit the space required in the VM, but I am wondering if I can reuse the config as well (Since I don't remember exactly how I configured SAB/SB).

 

Any thoughts, or gotchas I need to be aware of? One that comes to mind immediately is that it takes a few seconds for ArchVM to grab an IP address. Will the apps be okay on bootup if they can't access the nfs share immediately? Or do I need to restart each of the apps that rely on a /net/Tower/share once the network is accessible?

 

Also, Ironic, you posted:

 

Quote limetech:

Found the NFS problem, fixed in -beta4.

 

Here is a workaround.  First, in the webGui select the disk/share you are exporting via NFS.  Next change the security mode to "Private".  Then in the Rule field enter this:

 

*(sec=sys,rw,insecure,anongid=100,anonuid=99,all_squash)

 

Click Apply.  Your NFS client mounts should work now.

 

Where do you edit the rule field? I don't see it on the Share webpage.

 

Link to post

The rule is on the share page after you set nfs to private

 

 

I'm also worried about the network startup it takes awhile. I'm more worried about all the disk activity that having the VM running and sick/sab/couch hammering on in addition to xbmc wanting a piece.

 

I think that I'm going to have issues with xbmc buffering and hitching.

 

I need to figure out how to mount a drive outside of unraid array to do this all on.

Link to post

The rule is on the share page after you set nfs to private

 

 

I'm also worried about the network startup it takes awhile. I'm more worried about all the disk activity that having the VM running and sick/sab/couch hammering on in addition to xbmc wanting a piece.

 

I think that I'm going to have issues with xbmc buffering and hitching.

 

I need to figure out how to mount a drive outside of unraid array to do this all on.

 

Ahh, thanks. I didn't actually apply the Private setting, so missed that.

 

I agree on XBMC. I ended up building a separate Xen server to load the virtual XBMCbuntu clients on so that I am not sharing resources with UnRAID. My bigger issue though was I want to pass-through RADEON cards to each VM and wouldn't have enough PCI-E slots along with my UnRAID SAS cards.

 

It's great having all these options now, but it's challenging to get them as consolidated as possible. :)

Link to post

Couple of questions i have to far.

 

My server = 4gigs of ram

                    Tri-Core CPU

 

 

 

Question 1

unraid takes 2GB and if i allot another 2GB (2048MB) in the xen config i get memory errors, bumping its down to 1536MB works however. Should i not be able to allot all system memory?

 

Question 2

Even if i put 2 vcpu's in the config when i use xl list  it still shows unraid as having 3 and my arch linux having 1

Link to post

Couple of questions i have to far.

 

My server = 4gigs of ram

                    Tri-Core CPU

 

 

 

Question 1

unraid takes 2GB and if i allot another 2GB (2048MB) in the xen config i get memory errors, bumping its down to 1536MB works however. Should i not be able to allot all system memory?

 

Question 2

Even if i put 2 vcpu's in the config when i use xl list  it still shows unraid as having 3 and my arch linux having 1

 

 

Some machines use a small portion of ram for the onboard video. I think I have my test box setup at 1800. There is a command to see how much memory you can give a vm but I can't remember of the top of my head

Link to post

HI I tried today to start my first vm, but I'm getting errors

 

    Press enter to boot the selected OS, 'e' to edit the

    commands before booting, 'a' to modify the kernel arguments

    before booting, or 'c' for a command line.

 

 

 

 

 

 

 

libxl: notice: libxl_numa.c:491:libxl__get_numa_candidate: NUMA placement failed                                                  , performance might be affected

                              xc: error: panic: xc_dom_boot.c:160: xc_dom_boot_                                                  mem_init: can't allocate low memory for domain: Out of memory

                                                            libxl: error: libxl                                                  _dom.c:392:libxl__build_pv: xc_dom_boot_mem_init failed: Device or resource busy

                                                                              l                                                  ibxl: error: libxl_create.c:900:domcreate_rebuild_done: cannot (re-)build domain                                                  : -3

 

root@Tower:/mnt/cache/cache_only/ArchVM# cat arch.cfg

name = "archVM"

bootloader = "pygrub"

memory = 2048

#vcpus = 2

disk = [

        'phy:/mnt/cache/cache_only/ArchVM/arch.img,xvda,w',

#      'phy:/mnt/user/series,xvdb,w'

        'phy:/mnt/user/movies,xvdc,w'

        'phy:/mnt/user/music,xvdd,w'

        ]

vif = [ 'mac=00:1B:21:C2:4F:33,bridge=br0' ]

bootloader = "pygrub"

 

so figured a few things out, but when I do a ping of Tower it cannot find it, but I can ping the host IP address.

 

%

[root@ArchApplianceVM_v3 ~]# showmount Tower -e

clnt_create: RPC: Unknown host

 

 

I have a superhub from Virgin, is that maybe the problem?

Link to post

How much memory do you have total?

 

change the mac back to 00:16:3e:xx:xx:xx  you shouldn't need to touch that at all..

 

 

 

I allocated only 1024 and it booted up

 

now the network port will not come online. I used the mac address from the Unraid network page

 

 

[root@ArchApplianceVM_v3 ~]# ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

      valid_lft forever preferred_lft forever

    inet6 ::1/128 scope host

      valid_lft forever preferred_lft forever

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000

    link/ether 00:1b:21:c2:4f:33 brd ff:ff:ff:ff:ff:ff

    inet6 fe80::21b:21ff:fec2:4f33/64 scope link

      valid_lft forever preferred_lft forever

Network Settings

MAC address:

00:1B:21:C2:4F:33

 

was I not meant to do that?

Link to post

@mrlondon

You can't mount unraid shares like you have tried directly I'm afraid.

 

You must mount them via samba or nfs from within the VM.

 

You're also out of memory. That's in your error message.

 

Sent from my Nexus 5 using Tapatalk

 

Link to post

Couple of questions i have to far.

 

My server = 4gigs of ram

                    Tri-Core CPU

 

 

 

Question 1

unraid takes 2GB and if i allot another 2GB (2048MB) in the xen config i get memory errors, bumping its down to 1536MB works however. Should i not be able to allot all system memory?

 

Question 2

Even if i put 2 vcpu's in the config when i use xl list  it still shows unraid as having 3 and my arch linux having 1

 

In the VM cfg try putting single 'quotes' around the vcpu number.

 

Eg vcpus = '4'

 

Sent from my Nexus 5 using Tapatalk

 

 

Link to post

How does one install the plexpass verion of PMS?

 

https://aur.archlinux.org/packages/plexmediaserver-plexpass/

 

Oh, cheers. I thought there was one in ironic's repo.

 

Does anyone else have issues using nano when in 'xl console'? I am unable to save a file, the commands at the bottom of the screen are all mangled and don't appear to work. It's fine when connecting directly to the VM over SSH.

Link to post
Guest
This topic is now closed to further replies.