[XEN VM IMG] ArchVM <--- deprecated 01/07/2014


Recommended Posts

  • Replies 687
  • Created
  • Last Reply

Top Posters In This Topic

Curious why this is so big. I dont use arch personally but my Debian base install + most of the tools you are hosting added and a few more is less than 1GB.

 

Is it just a safety net? What is the actual on disk size of the base install

Link to comment

The base size of the install comes to 950M or so, I haven't pruned anything yet just wanted to get it out. I reckon I could get it down by a couple hundred Mb or so quite easily. I'm also on the lookout for a better compression method / container.

 

The 15GB worth of space is just a safety net as I found that plex's cache can sometimes take up 8-10gb in extreme case it may be more, but 15gb seems to strike a good balance between being downloadable and having to teach newbies an unnecessary amount of configuration.

Link to comment

Yup OK assumed as much.

 

I think if we are pushing for Arch and this to be the defacto standard for unRAID VMs we need to consider that people might want to run and backup a few of them. 15Gb a shot quickly becomes a burden.

 

I would suggest a couple of things. /home etc be mounted as a second virtual disk. This means you can shrink the root partition safely and it becomes much easier and safer to expand /home if needed (It could even be completely replaced with a different vdisk if a user wants without too much of a headache).

 

I dont know if we can cross mount /home on multiple VMS but if we could this means you could have a single home partition across any number of VMs.

 

Saying that another thought is that as far as humanly possible we should consider using unRAID disks rather than virtual disks for data storage. I think the only hurdle is deciding on the best way to do this but using your plex example it seems sensible not to hope that a 15GB partition is enough when underneath that unRAID has 2000GB free.

 

 

I have not given this enough thought yet but you should get my meaning.

 

Link to comment

Yes, cracking idea for a separate /home partition. I'll do that later!

 

An easy solution would be the usage of symlinks to store certain apps data on the array via NFS or CIFS. You can't mount the array directly in Xen though, would have to be done via network which as there's a few second delay at boot for network connection makes this more tricky.

Link to comment

I am sure we could get around any delay.

 

I really think we should be seriously considering levering the power of nfs for /home and maybe even /root. There will be some hurdles to get over but if we concentrate our efforts as long as your VM is Linux you should be able to share /home. Not only does that reduce the size of VMs considerably it means you dont have artificial VFS limits on data storage within the VM.

 

This might not be viable but we should at least take this perfect opportunity to look at it.

Link to comment

I am sure we could get around any delay.

 

I really think we should be seriously considering levering the power of nfs for /home and maybe even /root. There will be some hurdles to get over but if we concentrate our efforts as long as your VM is Linux you should be able to share /home. Not only does that reduce the size of VMs considerably it means you dont have artificial VFS limits on data storage within the VM.

 

This might not be viable but we should at least take this perfect opportunity to look at it.

 

Would that mean if you stored your settings, local configs, DBs etc for SB SABNZBD etc on the common /home, then you could flip between various VMs with those apps and retain your settings without needing to copy/rebuild ?

Link to comment

I am sure we could get around any delay.

 

I really think we should be seriously considering levering the power of nfs for /home and maybe even /root. There will be some hurdles to get over but if we concentrate our efforts as long as your VM is Linux you should be able to share /home. Not only does that reduce the size of VMs considerably it means you dont have artificial VFS limits on data storage within the VM.

 

This might not be viable but we should at least take this perfect opportunity to look at it.

 

Would that mean if you stored your settings, local configs, DBs etc for SB SABNZBD etc on the common /home, then you could flip between various VMs with those apps and retain your settings without needing to copy/rebuild ?

Not for Arch as they install to /opt but it's trivial to copy the .ini files to your array and replace them if you did rebuild.

 

Link to comment

Excellent videos, thank you.

 

A few of questions, if I may...

1) Would it be worth waiting for the external /home tweak or can this be changed on the vm later?

2) As the VM will be living on a cache drive, how would you back it up in case of disk failure? (especially if it is in use)

3) In the scenarios above, could the home directory be on the raid and a script on a blank VM reinstall the apps and link to the settings in /home rather than backing up the vm?

 

Many thanks for your work on this!

Tony

Link to comment

Ironicbadger

 

a few questions

 

1. does arch have native packages for sab/couch/plex ?

2. as i see it now we depend on you to provide these packages  in the unraid6 repo

3. so basically we depend on you all the time to upgrade them?

which doesn't make it much different from the system we have now that a few developers create plugins till they burn out....

as the packages you make in the repo need to be compiled? plex does need to be compiled i guess?

sab and couch maybe not a biggie as they run on python but that's why question 1 comes in play if there is no native arch package then we are again depending on somebody

 

that's why i think Ubuntu is easier for the masses as they have almost all packages natively

with init scripts and all ....

 

Please note this is meant as concern ... i admire the time and energy you put in all this but i am concerned about the long run

I understand the concern about small footprint but most of us use a 250gb  to 1tb hdd for cache drive from which wee use currently like 5 % ... on a crazy download day maybe 20%

so i wouldn't mind my vm image being 50 gb or more ....

 

Link to comment

Last login: Mon Feb  3 05:48:38 from tablet

               +                OS: Arch Linux x86_64
               #                Hostname: ArchApplianceVM
              ###               Kernel Release: 3.12.9-2-ARCH
             #####              Uptime: 0:00
             ######             WM: None
            ; #####;            DE: None
           +##.#####            Packages: 159
          +##########           RAM: 120 MB / 1998 MB
         #############;         Processor Type: Six-Core AMD Opteron(tm) Processor 2419 EE
        ###############+        $EDITOR: nano
       #######   #######        Root: 1.2G / 14G (8%) (ext4)
     .######;     ;###;`".
    .#######;     ;#####.
    #########.   .########`
   ######'           '######
  ;####                 ####;
  ##'                     '##
#'                         `#

[root@ArchApplianceVM ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:16:3e:01:45:78 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.111/24 brd 192.168.2.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe01:4578/64 scope link
       valid_lft forever preferred_lft forever
[root@ArchApplianceVM ~]# showmount Tower -e
clnt_create: RPC: Unknown host
[root@ArchApplianceVM ~]# showmount 192.168.2.4 -e
clnt_create: RPC: Program not registered
[root@ArchApplianceVM ~]# cd /net/
[root@ArchApplianceVM net]# ls
[root@ArchApplianceVM net]# showmount TOWER -e
clnt_create: RPC: Unknown host
[root@ArchApplianceVM net]#

 

Its not finding the network server...

Link to comment

...

1. does arch have native packages for sab/couch/plex ?

...

 

I have found that none of the distros keep up to date with these packages however many of them can run direct from svn or git checkout.

 

I would suggest that you are spot on that we should either rely on a real repo package with the vast manpower that implys hidden behind it or some sort of auto buold script thats local.

 

Not tot take away form anyones work but the policy should be, whatever OS we choose (and arch would not be my choice - OT) that we stick with OS proper and official stuff. That way users can get support from the OS channels and not end up in some hodge podge, no one wants to really support it... rely on 1 dude solution

Link to comment

Now that we are getting to the stage that this is becoming a reality, the most familiar form for new users is a desktop linux distro. I'm really just thinking out loud here...

 

When I look at most apps nowadays the download options are Ubuntu, Fedora, centos and github by source.

 

Sooo... Those that have never seen linux would need something popular, ie Ubuntu desktop/mint etc

 

The problem with this really is that running things off a headless server (I presume is most people's circumstance) is that the software being installed is web based and doesn't need a desktop. This is for those that can install things via the command line. Again something popular that has the most support for apps, but without the overhead of a desktop GUI. Something like Ubuntu server....or even Ubuntu desktop minimal install without the desktop.

 

Then there are of course those that want to customise for whatever reason, but if you fall into this category I'm pretty sure you know enough not to rely on others and these guides.

 

Or.... Make several guides. One can then choose what distro to follow. There is not one guide to suit everyone, ironcbadger has done a great job creating the first guide, but it's not the do all guide. Just like linux distros over time many more guides will pop up to solve a particular requirement.

 

What we do need is the basic framework master guide to get people going in the right direction before it gets to the stage where one must choose their OS. Things like making the VM start on boot, utilising unraid storage array, setting up the folders on a cache drive, creating the network bridge, ensuring unraid boots into xen/unraid mode, etc

 

I thought being a new month, the forums would've been sorted out by now and we could really start to organise things?

 

This coming weekend I will dedicate time to start creating some of this stuff

 

Link to comment

I agree with the majority of your points, but disagree with you on desktop being simple.

 

This isn't always true. As you'll see in the videos, if you watch them start to finish, they take a new user from 0 to full VM running. This is why I did a video, to actually show all of the steps in real time. People can pause it etc, make notes if they want but where a video wins over a text guide is it shows every single step (even including connecting via SSH, which when i was new was a real hurdle in itself). I really beg to differ that ubuntu server is more simple than Arch, strongly.

 

How do you autostart services on ubuntu? Usually, it's an init script or some other nightmare in my experience. With Arch it's 'systemctl enable service'. Newbie friendly for sure. Correct me if I'm wrong.

 

I know your point was probably not to have a go at me, but I feel like you missed the point of the videos. Plus, I like defending myself!

 

The VM autostart / autostop business, well I'm not convinced this is good practice unless you guarantee a clean shutdown everytime of the VM (which in my experience you cannot). Again, why I showed in the vids how to do it manually. Especially important with the VMs residing on cache drives (and thus not available unless array is started).

 

Using a desktop distro will require installation of a lot of bloat that's really not required. A desktop, graphics drivers and also use HVM (full on) virtualization as opposed to the PV guest that my image is.

 

---------

 

As for the 1 man solution bit, I use this daily myself. Whenever I want to upgrade packages I'll just add them to the repo, plus it can easily be done by other people if they ask I can give FTP info etc. The last thing I actually want is to be the sole gatekeeper.

 

Link to comment

Last login: Mon Feb  3 05:48:38 from tablet

               +                OS: Arch Linux x86_64
               #                Hostname: ArchApplianceVM
              ###               Kernel Release: 3.12.9-2-ARCH
             #####              Uptime: 0:00
             ######             WM: None
            ; #####;            DE: None
           +##.#####            Packages: 159
          +##########           RAM: 120 MB / 1998 MB
         #############;         Processor Type: Six-Core AMD Opteron(tm) Processor 2419 EE
        ###############+        $EDITOR: nano
       #######   #######        Root: 1.2G / 14G (8%) (ext4)
     .######;     ;###;`".
    .#######;     ;#####.
    #########.   .########`
   ######'           '######
  ;####                 ####;
  ##'                     '##
#'                         `#

[root@ArchApplianceVM ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:16:3e:01:45:78 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.111/24 brd 192.168.2.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe01:4578/64 scope link
       valid_lft forever preferred_lft forever
[root@ArchApplianceVM ~]# showmount Tower -e
clnt_create: RPC: Unknown host
[root@ArchApplianceVM ~]# showmount 192.168.2.4 -e
clnt_create: RPC: Program not registered
[root@ArchApplianceVM ~]# cd /net/
[root@ArchApplianceVM net]# ls
[root@ArchApplianceVM net]# showmount TOWER -e
clnt_create: RPC: Unknown host
[root@ArchApplianceVM net]#

 

Its not finding the network server...

 

It takes about 10-30 seconds sometimes to pick up the network.

Link to comment

Just a comment on the Arch vs. other options.  I'm new to linux and settled with Ubuntu Server for my VM (running an ESXi box with Unraid and other guests) and it was definately not easier than what I have seen with Arch and other options with systemd, etc.  I found plenty of guides to get me where I wanted to go, but it was definately not clean getting there and for someone new to linux or the CLI, editing the init scripts with vi or nano is not going to be a cakewalk. 

 

On a side note I think it's awesome with all the work going on in the forums right now.  I'll eventually move from ESXi over to Unraid/Xen but for now ESXi is second nature to me since I use it at my job.

Link to comment

I think running 1 VM with lots of apps would be as bad as the current plugin state. And without the drain on resources that the desktop GUI would use, it would be easier to have several VMs running for different applications.

 

As a non Linux user I struggled with Ubuntu / Ubuntu server, whereas arch I found reasonably easy ( with a lot of googling)

 

Plus these VMs should be setup and left with only updates required from using CLI, and the apps accessed from http

 

 

Sent from my iPhone using Tapatalk

Link to comment

Yep, familiarity would've been a better word, I've amended it.

 

Plus your points are actually my points  :). No one guide works for everything. Your requirements for simplicity and streamline is exactly the same reason why I use arch linux on my unraid. But for others the bloat is tolerable for the simple fact it's got a desktop GUI. I'm trying to create a mass solution with choices. Everyone has their own personal experiences on what works for them.

 

My main point was to make a "getting started" guide, which is agnostic to the point when a user chooses their operating system. Some will want windows, some linux and some osx.

 

At this stage the guide would then point to your guide + others depending on what one chooses.

 

Would almost be nice to utilise the unraid wiki, but I would think most people read forums before they read a wiki.

Link to comment

The base size of the install comes to 950M or so, I haven't pruned anything yet just wanted to get it out. I reckon I could get it down by a couple hundred Mb or so quite easily. I'm also on the lookout for a better compression method / container.

 

The 15GB worth of space is just a safety net as I found that plex's cache can sometimes take up 8-10gb in extreme case it may be more, but 15gb seems to strike a good balance between being downloadable and having to teach newbies an unnecessary amount of configuration.

 

This raises an issue I've been working during about.  So we want the array to spin down and we know plex log writing will keep a drive spun up.  But what about symlinking the non-log parts of the plex database into the array?  That would eliminate the worry about the db exceeding the vm drive space, add parity protection, and prevent the array from continual spin up due to plex log writes

 

Sent from my XT1060 using Tapatalk

 

 

Link to comment
Guest
This topic is now closed to further replies.