Jump to content
ironicbadger

[XEN VM IMG] ArchVM <--- deprecated 01/07/2014

688 posts in this topic Last Reply

Recommended Posts

As part of my series of pre-made Xen compatible VM images I present to you my Arch Linux appliance.

 

The basic premise is that you boot unRAID into 'Xen mode' as I show in my video series below, then download my VM image and run it! Once you've run the image you're free to customise the installation to your hearts content with any application you can find for Arch (that's almost all of them!).

 

What is Arch Linux and why do I need it in a VM?

 

Archlinux-official-fullcolour.svg

 

Arch Linux (or Arch /??rt?/)[3] is a Linux-based operating system for i686 and x86-64 computers.[4] It is composed predominantly of free and open source software' date='[5'] and supports community involvement.[6]

The design approach of the development team focuses on elegance, code correctness, minimalism, and simplicity, and expects the user to be willing to make some effort to understand the system's operation.[7] A package manager written specifically for Arch Linux, pacman, is used to install, remove and update software packages.

Arch Linux uses a rolling release model, such that a regular system update is all that is needed to obtain the latest Arch software; the installation images released by the Arch team are simply up-to-date snapshots of the main system components.

 

I chose Arch Linux to base this particular VM on because of the way in handles software / packages in addition to it's tiny footprint on disk and in RAM. Arch Linux, just like every other Linux distro, makes use of things called 'repositories' that store packages ready for you to download. The main Arch repo's have a huge array of packages already, but they notably don't include 'those kind of apps' that we use heavily on a media server - this is where the Arch User Repository (AUR) comes in.

 

Members of the Arch community are free to create packages for their favourite apps and Arch provides an easy way to download and compile them on a full install. To make your life easier what I've done is gone ahead and compiled the majority of media apps ahead of time and put them into my own 'unraid6repo' which is preconfigured into your VM, making installing non standard apps a breeze but more on that later.

 

2014-02-02--1391345808_794x433_scrot.png

 

Ok I get it, Arch is awesome. How do I get going?!

 

See the bottom of this post for download links, and post #2 for a series of YouTube videos that take you through the process from start to finish if you prefer that method over a text guide.

 

Youtube Playlist for the setup guide - http://www.youtube.com/playlist?list=PLgUdkYSObfxotSl4c4KJlQKNyQbPAduLl

 

----------------

 

How to download and start this appliance

 

1. Login via SSH or Telnet to your unRAID box using Putty on Windows or Terminal on a Mac.

 

2. Download the .zip file to the location you want to run the VM from and unzip it.

 

cd /mnt/user/cache_only     <------ ***
wget http:// <insert download link from post #1>
unzip ArchVM.zip    <------ this will take approx 5 mins

 

*** this is up to you, you can place the .zip wherever you like. If you change this location, you will need to edit the .cfg file accordingly. If you want to do this use mcedit after you've unzipped the download.

 

mcedit ArchVM/arch.cfg

 

3. Run your VM.

 

xl create ArchVM/arch.cfg -c   <--- ***

 

*** the -c is optional, this option logs you into the VM console directly so instead of having to connect to it via SSH you can just do this at boot - a massive time saver.

 

4. Login to your VM via either SSH or Xen Console and find the IP address of your Arch Linux VM.

 

User / Passwords

root / unraid6

user / unraid6

 

xl console ArchVM
OR
ssh root@archvm

 

Once you're logged in, to find the IP address of the VM issue

 

ip addr

 

5. Once you're in you should be presented with the screenshot from above with the pretty Ascii art in the shape of the Arch logo. I compiled this image probably several days, if not weeks before you downloaded it so first we need to update the repositories as they will be stale. If there are any package upgrades, do them.

 

pacman -Syu  <-- this should be done as often as possible (weekly minimum ideally)

 

6. We are now ready to start installing packages! w00t! Arch uses something called 'pacman' as it's package manager and it couldn't be simpler. Here are the basics. All commands are CASE SENSITIVE!!!

 

pacman -Sy <insert app name>  <--- eg pacman -S sickbeard-git
pacman -Ss <insert app name>  <--- searches the repositories the VM can access for a package
pacman -R <insert app name>   <--- removes a package

 

7. Access unRAID data from the VM

 

First of all it is worth noting that Xen only supports accessing other VMs via network based protocols such as NFS and CIFS / Samba, block-based protocols are not an option! In the real world, NFS will easily be fast enough to saturate unRAID's write speeds (of around 40-50mb/s) and read speeds will be as fast as your array can give capped at around 100/110 mb/s.

 

Second, there is about a 30-60 second delay when the VM boots up whilst the unRAID Xen host gives it an IP. There is nothing I can do about this, it is not specific to this VM. Sorry! It annoys me too!

 

NFS

 

The Arch VM has something enable called autofs which automatically mounts network shares when needed. The shares will be presented under /net/<insert netBIOSname>/mnt/user/unraidShareName, for example

 

/net/unraid/mnt/user/Media

 

* Some users have reported an issue where this doesn't quite work perfectly for them so in this case use the SMB info below.

 

Samba

 

Add this entry (and change the IP of course) to /etc/fstab. Edit fstab with ' nano /etc/fstab '.

 

nano /etc/fstab
--------------
//10.0.0.2/Media    /mnt/unraid     cifs         auto,x-systemd.automount,guest,noperm,noserverino,uid=nobody,gid=users 0 0

 

Thanks to justinchase for the x-system.d.automount section, this makes the SMB share mount when a service requests it (such as Plex for example). It works a treat, just try it out.  8)

 

Manually mount shares over Samba

mount -t cifs //Tower/data /mnt/tmp -o guest,rw

 

---------------------

 

How to create a data .img to act as a datastore for Plex and others...

 

The base .img for this install is only 15G, this is mainly to keep my hosting costs down and your downloads small! Here are the steps to add more storage to your VM as some of you have ridiculous Plex libraries!! It is also useful as scratch space for Sabnzbd and/or Tranmission incomplete downloads too...

 

1. Log in to unRAID (not your VM) via SSH, telnet or the console

 

2. Power off the ArchVM if it isn't already

 

xl shutdown ArchVM
xl top <--- wait for ArchVM to disapear

 

3. cd to the directory of your VM for example:

 

cd /mnt/user/cache_only/ArchVM

 

4. Create an .img file of the required size

 

truncate -s 250G data.img    <--- adjust the size as needed, as for as I know there's no limit.

 

5. Edit your arch.cfg with mcedit

 

mcedit arch.cfg
-----------
disk = [
        'file:/mnt/user/cache_only/ArchVM/arch.img,xvda,w',
        'file:/mnt/user/cache_only/ArchVM/data.img,xvdb,w'  <--- add this line
        ]

 

6. Boot up your ArchVM

 

xl create arch.cfg -c

 

7. Check the data.img file is seen with

 

fdisk -l
-------------
Disk /dev/xvdb: 250 GiB

 

8. Create a partition structure and filesystem on the .img

 

fdisk /dev/xvdb
command: o
command: n
partition type: default p
partition num: default 1
First sector: default 
Last sector: default
command: w

mkfs.ext4 /dev/xvdb1

 

9. Mount the new partition in /etc/fstab

 

mkdir /mnt/data

nano /etc/fstab
--------------
/dev/xvdb1    /mnt/data ext4 defaults 0 1

 

Ctrl + X to save a quit the fstab file.

 

Reboot.

 

---------------------

 

How to install sabnzbd

 

* coming soon, or see my videos! *

 

---------------------

 

Known issues

 

NFS in unRAID v6 beta3 is broken and needs fixing by you. NFS shares will mount as read only meaning your usenet apps will not be able to write any data to unRAID (pretty pointless). The fix is here, supplied by Limetech and will be fixed in beta4 onwards as standard.

 

Found the NFS problem, fixed in -beta4.

 

Here is a workaround.  First, in the webGui select the disk/share you are exporting via NFS.  Next change the security mode to "Private".  Then in the Rule field enter this:

 

*(sec=sys,rw,insecure,anongid=100,anonuid=99,all_squash)

 

Click Apply.  Your NFS client mounts should work now.

 

 

The IronicBadger repository !

 

The reason for using Arch is two fold. First, it has an incredibly tiny footprint for a full distro and uses almost no RAM. Second, it's what I use and I host a repository full of my favourite apps which are not in the default repositories. The VM already has it added so needs no user interaction.

 

I take requests on adding apps to my repository, but you must first do just a little homework. You must find out first whether the package already exists in the mainline repositories and if it doesn't you must check to see if it exists on the Arch User Repository. I will only add packages from the AUR to my repo!

 

Arch Linux Main Repos - https://www.archlinux.org/packages/

AUR - https://aur.archlinux.org/packages/

 

 

For more information visit http://unraidrepo.ktz.me/.

 

Download Links

Google Drive - *discontinued for now*

Dropbox - *discontinued for now*

Personal hosting - http://unraidrepo.ktz.me/archVM/ArchVM_v5.zip

 

If you find this VM useful please consider making a donation.

 

btn_donate_SM.gif

 

Share this post


Link to post

Part 1 - How to make your USB stick and setup ready for Xen (the basics)

 

Part 2 - How to download and install the Appliance Virtual Machine including network bridge setup

 

Part 3 - How to use the VM once it's installed.

 

Part 4 - How to connect to unRAID from the VM

 

Part 5 - Edit syslinux to default boot into Xen

 

 

Share this post


Link to post

Link is up. File size is a bit larger than I would like but we can work on that.  8)

 

This version didn't include NFS mounting support. I've added it to my next image (up in about 45 mins), but until then do this to enable it.

 

1. Download some software

pacman -Sy nfs-utils autofs

 

2. Edit autofs config (this is what looks across the network to find unraid and mount it automatically when requested)

rm /etc/autofs/auto.master
nano /etc/autofs/auto.master

/etc/autofs/auto.master
--------------------------------------
/net -hosts --timeout=60

 

3. Enable services to auto start at boot (don't you just love systemd)

systemctl enable rpc-statd rpc-mountd rpc-idmapd autofs

 

4. Reboot or manually start services

systemctl start rpc-statd rpc-mountd rpc-idmapd autofs

 

5.Test it works

Try and navigate to your unraid box with the cd command, mine is:

cd /net/unraid/mnt/user/Media
ls <--- lists directory, just a quick test. if you see output, it worked.

 

Share this post


Link to post

What would the best way to DL from sabnzb ? A separate image or over LAN to unraid cache drive? Over GB network might be OK? Looking forward to try this! Any light wight desktop included? Xfce maybe ?

 

 

Sent from my iPhone using Tapatalk 2

Share this post


Link to post

What would the best way to DL from sabnzb ? A separate image or over LAN to unraid cache drive? Over GB network might be OK? Looking forward to try this! Any light wight desktop included? Xfce maybe ?

 

 

Sent from my iPhone using Tapatalk 2

 

No desktop included, it's not designed to be used in this way as it's a PV domU. No VNC enabled etc. It's supposed to be a set it and forget type VM, read up on HVM if you want that!

 

https://wiki.archlinux.org/index.php/xen#Configuring_a_hardware_virtualized_.28HVM.29_Arch_domU

 

As for SAB, what I do is mount an LVM partition of a disk outside the array directly in the VM cfg of around 150GB or so giving my downloaders plenty of room to move stuff about. You can try it via NFS to the cache drive though, can't hurt giving it a go :)

 

Here is a quote of the disk section of my config, you can probably guess what it all means...

 

disk = [
        'phy:/mnt/cache/ArchVM/arch.img,xvda,w',
#       'phy:/mnt/user/nameofshare,xvdb,w'
        ]

Share this post


Link to post

Thanks, I noticed that too.

 

I've added a new pair of mirrors on google drive and my own hosting.  8)

Share this post


Link to post

Very nice job ironicbadger, will also include samba and NFS tools in my image

Share this post


Link to post

Very nice job ironicbadger, will also include samba and NFS tools in my image

 

Just finished a quick video showing how NFS auto mounting is handled with my appliance. Enjoy  :)

 

Share this post


Link to post

Wow autofs looks really nice, def going to have a play with it :)

Share this post


Link to post

Brilliant!  You  made it a breeze to get all the essentials going.  Small foot print, low CPU usage (have not tried Plex transcoding yet), and it just works!

Thanks so much ironicbadger.  You've saved those of us that want something up and running quickly (read impatient) a heap of time.

 

All I have left is crashplan and I will be fully "converted" to 64 bit.

 

Share this post


Link to post

I'll add crashplan to the repo tomorrow. Just off to bed now!

 

Couple more videos inbound tomorrow (encoding now) and they'll be real nice icing on the cake I hope.

 

Why do you want a VB IMG? I'm sure that it's possible to convert this raw IMG, I've just never looked.

 

Sent from my Nexus 5 using Tapatalk

 

Share this post


Link to post

What performance are you seeing in Plex transcoding versus running Plex natively?

Share this post


Link to post

What performance are you seeing in Plex transcoding versus running Plex natively?

 

You won't be able to tell the difference so long as you can devote enough resources to your VM. Cpu is the big one for transcoding.

 

Here's a quick video of gaming using windows as a VM with Xen showing the performance levels there. I'd say its about a 98% performance level compared to bare metal.

 

 

Sent from my Nexus 5 using Tapatalk

 

 

Share this post


Link to post

Any chance of posting a link to your xen config  file so I can take a quick squiz?

Share this post


Link to post

You are ofc welcome to make your own Xen IMG of arch but it would seem a little wasteful (and confusing for newbies) to have two floating around to maintain. I'd stick with your KVM image and make it as good as can be, I'll do the same with my Xen one.

 

http://unraidrepo.ktz.me/archVM/arch.cfg

 

Sent from my Nexus 5 using Tapatalk

 

Share this post


Link to post

You are ofc welcome to make your own Xen IMG of arch but it would seem a little wasteful (and confusing for newbies) to have two floating around to maintain. I'd stick with your KVM image and make it as good as can be, I'll do the same with my Xen one.

 

http://unraidrepo.ktz.me/archVM/arch.cfg

 

Sent from my Nexus 5 using Tapatalk

 

yeah i wont be creating a xen image now youve done the hard work, but my inner geek wants to see the process from the start, so going to have a little tinker with building an image from scratch, just for myself :-), most probably i will bin it in the end but i just gotta know how to do this  ;D

Share this post


Link to post

Arch wiki is your friend. New video is up covering the basics of setting up v6 ready to be xen'd. The important bit, part 2 will come tomorrow. Night!

 

Sent from my Nexus 5 using Tapatalk

 

 

Share this post


Link to post

Would you kindly do a MythTV even thou its going to require x11 forwarding to setup?

 

Sent from my LG-VS980 using Tapatalk

 

 

Share this post


Link to post

really awesome badger ill be using this , just need to add in murmur and glances =)

Share this post


Link to post

Anyone else that have issue to DL from the google link ? I have no access to the file from my google acount.

 

//Peter

Share this post


Link to post
Guest
This topic is now closed to further replies.