apgood Posted January 31, 2014 Share Posted January 31, 2014 Thanks Tom great progress and updates. Sent from my Nexus 4 using Tapatalk Quote Link to comment
SchoolBusDriver Posted January 31, 2014 Share Posted January 31, 2014 I'm hoping to be able to release tomorrow to give you guys something to do during the Superbowl. But if an unforeseen problem crops up and I don't get the release out tomorrow, please don't go on a war path I just want to keep everyone informed on exactly where development stands. Couple of questions I had and wanted to make you aware of a few gotchas 1. Did you create a symbolic link to /etc/xen/auto (or change the xendomains deamon) so we can have our Xen VMs autostart? 2. Speaking of daemons, which ones did you enable? 3. If you do "lsmod | grep xen" which modules are showing up? 4. Did you add xen-pciback into the kernel or is it a module? 5. In the xendomains daemon... empty the save location of where it would save a VM on powerdown. Otherwise it will attempt to write GBs of data to /var/lib/xen/save by default. Quote Link to comment
limetech Posted January 31, 2014 Author Share Posted January 31, 2014 1. Did you create a symbolic link to /etc/xen/auto (or change the xendomains deamon) so we can have our Xen VMs autostart? Where should the symbolic link exist? There is a /etc/xen/auto directory that's empty. I've only done a cursory study of VM auto-start. 2. Speaking of daemons, which ones did you enable? There are two startup scripts invoked from rc.local: rc.xencommons which starts: xenstored xenconsoled rc.xendomains which starts: <not sure what gets started by this> As I understand it, the 'xm' command is deprecated, which means xend is also not needed, so I've used only 'xl' command and there is no xend. (But xm and xend are included in the build.) 3. If you do "lsmod | grep xen" which modules are showing up? root@Test1:~# lsmod |grep xen xen_acpi_processor 3597 0 xen_pciback 33336 0 xen_netback 20777 1 xen_blkback 15336 0 xen_gntalloc 3808 0 xen_gntdev 6056 4 4. Did you add xen-pciback into the kernel or is it a module? module 5. In the xendomains daemon... empty the save location of where it would save a VM on powerdown. Otherwise it will attempt to write GBs of data to /var/lib/xen/save by default. ok Here's the output of 'ps -ef|grep xen root@Test1:~# ps -ef |grep xen root 30 2 0 15:30 ? 00:00:00 [xenwatch] root 31 2 0 15:30 ? 00:00:00 [xenbus] root 343 2 0 15:30 ? 00:00:00 [xenbus_frontend] root 1311 2 0 15:30 ? 00:00:00 [xen_pciback_wor] root 1328 1 0 15:30 ? 00:00:00 /usr/sbin/xenstored --pid-file /var/run/xenstored.pid root 1334 1 0 15:30 ? 00:00:00 /usr/sbin/xenconsoled --pid-file=/var/run/xenconsoled.pid root 1338 1 0 15:30 ? 00:00:00 /usr/lib/xen/bin/qemu-system-i386 -xen-domid 0 -xen-attach -name dom0 -nographic -M xenpv -daemonize -monitor /dev/null -serial /dev/null -parallel /dev/null -pidfile /var/run/qemu-dom0.pid root 1663 1 0 15:41 ? 00:00:04 /usr/lib/xen/bin/qemu-system-i386 -xen-domid 2 -chardev socket,id=libxl-cmd,path=/var/run/xen/qmp-libxl-2,server,nowait -mon chardev=libxl-cmd,mode=control -xen-attach -name ubuntu.13-04.xfce.x86-64.20130424 -nographic -M xenpv -m 513 root 1707 1652 0 15:41 pts/0 00:00:00 /usr/lib64/xen/bin/xenconsole 2 --num 0 --type pv root 2275 2021 0 19:42 pts/2 00:00:00 grep xen As for the name of the bridge, probably will change to "xenbr0". Quote Link to comment
SchoolBusDriver Posted January 31, 2014 Share Posted January 31, 2014 Where should the symbolic link exist? There is a /etc/xen/auto directory that's empty. I've only done a cursory study of VM auto-start. If you drop a Xen cfg file in that folder. When the xendomains daemon starts up, it will look there and start whatever VM cfg that are there. You have a method to your madness so I would either change the setting in xendomain or symbolic link that folder to somewhere you feel is appropriate on the flash drive. 2. Speaking of daemons, which ones did you enable? There are two startup scripts invoked from rc.local: rc.xencommons which starts: xenstored xenconsoled rc.xendomains which starts: <not sure what gets started by this> As I understand it, the 'xm' command is deprecated, which means xend is also not needed, so I've used only 'xl' command and there is no xend. (But xm and xend are included in the build.) This looks good. We need xenstored, we will want xenconsole (so we can console into VMs and see them through the boot process to the login screen) and want xendomains for autostart. Speaking of console, we can add this later but if a lot of people continue to run unRAID in a VM on Xen or XenServer... We need to update inittab and securetty with the right hvc0 settings. I would hold off for now or else people are going to get respawn messages if not in a VM. I have never bothered to figure out how to do it without the respawn messages in Slackware but maybe you know. 3. If you do "lsmod | grep xen" which modules are showing up? root@Test1:~# lsmod |grep xen xen_acpi_processor 3597 0 xen_acpi_processor can cause a lot problems. Most Linux Distros, it is disabled by default due to this. For some it will work and work well but many it will not and can even cause a kernel panic. I would let them load it manually for now. In Xen 4.4 a lot of work went into addressing this. I don't have it in front of me but Xen usually drops a file in either /etc/modprobe.d or /etc/modules where it loads all the modules at boot. I would just comment out xen_acpi_processor and xen-pciback (see below). 4. Did you add xen-pciback into the kernel or is it a module? module PCI Passthrough is going to be REALLY, REALLY difficult and flaky if this isn't in the kernel. If we do not passthrough the PCI Devices in the Linux Boot command line... when Linux boots, it is going to grab / load the various Kernel modules for SATA Cards, Video Cards, Network Cards (which people will passthrough for pfSense), etc. and getting them back from the Kernel isn't a walk in the park and flaky at best. I will write a guide for PCI Passthrough for those people who need it for Video Cards, Network Cards, SATA Controllers (for unRAID within an unRAID to transfer data or back up) but it's a VERY complicated process and messy if xen-pciback isn't in the kernel and we cannot grab the PCI Devices through syslinux (boot). PS - It's the exact same for the PCISTUB driver in KVM. As a module, hell. Kernel, heaven. As for the name of the bridge, probably will change to "xenbr0". If it's not too much trouble in the WebGUI, this is a good idea as to keep confusion to a minimum. What people will see / find on the internet will all be xenbr0. Having them remember to use 00:16:3e:xx:xx:xx when they set up their MAC Addresses is going to be hard enough. Quote Link to comment
BLKMGK Posted January 31, 2014 Share Posted January 31, 2014 Awesome to get a status update Tom, thank you!! Fwiw my ESXi with passthru experience wasn't near the nightmare others have apparently had. I had no issues flashing my SAS cards to it mode. I had no issue identifying them under ESXi as I installed the ones I wanted to passthru first and then the last. I setup the bios on advice of another and used a known good revision. Biggest issue was passing the wrong USB hub and losing access to save ESXi configurations! I've run 5.0, 5.1, and now 5.5 with pretty smooth upgrades. I know/knew about the version 10 format issues and have upgraded none of my VMs and in fact disabled the trial license ASAP so I wouldn't trip over an advanced feature. It took me a few evenings to set it up and some advice but it came out solid. I use just one tool to manage my server - vSphere Client. I do see some weirdness with regards to responsiveness of Windows shares occasionally but I think that's just drive spinup. Installed XenServer tonight, looked eerily like ESX! I like the console status better actually. The Xen management tool seems very nice but it seems harder to manipulate datastores and I've found no easy way to create a VM from an ISO and a templated create has failed a few times for Ubuntu. No matter, I mostly wanted to gauge the management tool and get a sense for metics that could be provided. Overall I think with some adjustment I'd like this as much and possibly more than ESXi. The offer to join an HA pool right off the bat without signing over my firstborn was refreshing! The future looks very bright :-) Sent from my iPad using Tapatalk Quote Link to comment
SchoolBusDriver Posted January 31, 2014 Share Posted January 31, 2014 The Xen management tool seems very nice but it seems harder to manipulate datastores Learning curve. Once you learn how XenServer handles / uses datastores... You will overcome this obstacle. I've found no easy way to create a VM from an ISO and a templated create has failed a few times for Ubuntu. http://lime-technology.com/forum/index.php?topic=30719.msg275913#msg275913 If you haven't already, patch your XenServer with the latest updates / bug fixes. Quote Link to comment
limetech Posted January 31, 2014 Author Share Posted January 31, 2014 Where should the symbolic link exist? There is a /etc/xen/auto directory that's empty. I've only done a cursory study of VM auto-start. If you drop a Xen cfg file in that folder. When the xendomains daemon starts up, it will look there and start whatever VM cfg that are there. You have a method to your madness so I would either change the setting in xendomain or symbolic link that folder to somewhere you feel is appropriate on the flash drive. Ok, understood. Related question: if you have a "large number" of VM's autostarting like this, is the whole startup process going to get slowed down as each of the VM's hit potentially the same disk reading their root volumes at the same time? 3. If you do "lsmod | grep xen" which modules are showing up? root@Test1:~# lsmod |grep xen xen_acpi_processor 3597 0 xen_acpi_processor can cause a lot problems. Most Linux Distros, it is disabled by default due to this. For some it will work and work well but many it will not and can even cause a kernel panic. I would let them load it manually for now. In Xen 4.4 a lot of work went into addressing this. I don't have it in front of me but Xen usually drops a file in either /etc/modprobe.d or /etc/modules where it loads all the modules at boot. I would just comment out xen_acpi_processor and xen-pciback (see below). Ok. 4. Did you add xen-pciback into the kernel or is it a module? module PCI Passthrough is going to be REALLY, REALLY difficult and flaky if this isn't in the kernel. If we do not passthrough the PCI Devices in the Linux Boot command line... when Linux boots, it is going to grab / load the various Kernel modules for SATA Cards, Video Cards, Network Cards (which people will passthrough for pfSense), etc. and getting them back from the Kernel isn't a walk in the park and flaky at best. I will write a guide for PCI Passthrough for those people who need it for Video Cards, Network Cards, SATA Controllers (for unRAID within an unRAID to transfer data or back up) but it's a VERY complicate process and messy if xen-pciback isn't in the kernel and we cannot grab the PCI Devices through syslinux (boot). Ok this makes sense, I'll change it to a "built-in". So when someone wants to pass-through devices this will need to be done on the kernel boot command line? (This would require user to edit their syslinux.cfg file). Quote Link to comment
status Posted January 31, 2014 Share Posted January 31, 2014 I have nothing useful to add to this thread, I just wanted to add I'm really excited for this change and I fully agree with it. Quote Link to comment
SchoolBusDriver Posted January 31, 2014 Share Posted January 31, 2014 Related question: if you have a "large number" of VM's autostarting like this, is the whole startup process going to get slowed down as each of the VM's hit potentially the same disk reading their root volumes at the same time? There is an auto-delay already programmed in but you can adjust it further if needed. I have never had an issue but if you want to adjust it I believe its in /etc/xen/ in one of the config files or in /etc/default/xendomains file (don't forget to set XENDOMAINS_SAVE="" and XENDOMAINS_RESTORE=false and change XENDOMAINS_AUTO=/etc/xen/auto or symbolic link it on the flashdrive). Ok this makes sense, I'll change it to a "built-in". So when someone wants to pass-through devices this will need to be done on the kernel boot command line? (This would require user to edit their syslinux.cfg file). Correct. There are a lot of commands that users will probably add / tweak to syslinux.cfg for the amount of memory / cpus dom0 takes, PCI passthrough for AMD and Intel CPUs, etc. Example: Xen Kernel = xsave=1 cpufreq=xen:performance dom0_mem=1024M,max:2048M dom0_max_vcpus=4 dom0_vcpus_pin iommu=1 unRAID Kernel = xen-pciback.hide=(00:11.0)(00:12.2) I will write up a detailed guide on PCI Passthrough and how to configure it in SYSLINUX and the other stuff I have above for Xen. Quote Link to comment
BLKMGK Posted January 31, 2014 Share Posted January 31, 2014 Patching was a learning curve! The Citrix XenServer Patcher on github was very nice since Citrix disabled their patching tool for free use. What sort of patch process for Xen can we anticipate with unRAID? Sent from my iPad using Tapatalk Quote Link to comment
SchoolBusDriver Posted January 31, 2014 Share Posted January 31, 2014 What sort of patch process for Xen can we anticipate with unRAID? There isn't one. My guess... There would be a new unRAID 6.X release when either Xen 4.3.2 (in Release Candidate 1) or Xen 4.4 (in Release Candidate 2) is released / tested / approved by Tom. Xen 4.4 has been in feature freeze since November and they have been working on testing / bugs since then. I'd expect it to be released in a few weeks / end of February based on what I read / see in the Xen Developers Mailing list. If you are nerd, have a spare machine and want to help Xen and unRAID out... Sign up for Xen 4.4 RC3 Testing Day on Feb 3 and join us in the IRC freenode channel (#xentest). Quote Link to comment
BLKMGK Posted January 31, 2014 Share Posted January 31, 2014 Okay, fair enough. After seeing the dozen or so patches XenServer wanted I was curious is all. After multiple head butting events I've got a Mint VM running just by using the "other" template and ignoring the scary verbiage. Easy as ESXi really and I just had to link a pool of my ISO - I used my FreeNAS share that has ISO for ESX. Worked like a charm :-) Sent from my iPad using Tapatalk Quote Link to comment
MrLondon Posted January 31, 2014 Share Posted January 31, 2014 Will a WebInterface like Archipel be part of unRaid to work the Xen Hypervisor? Alternatively it would be fantastic if a VM with a compiled Archipel could be made available... Thanks MrLondon Quote Link to comment
NAS Posted January 31, 2014 Share Posted January 31, 2014 I have to say it is particularly nice to see the back and forth dev chat along with progress reports. Excited Quote Link to comment
mattkhan Posted January 31, 2014 Share Posted January 31, 2014 If you drop a Xen cfg file in that folder. When the xendomains daemon starts up, it will look there and start whatever VM cfg that are there. The naming here confuses me. AIUI xend/xm is deprecated (along with any support for managed domains) and replaced by xl and some other management layer. I took this to mean that xendomains is part of that managed domain support but is it or is it just a standalone script? Quote Link to comment
SchoolBusDriver Posted January 31, 2014 Share Posted January 31, 2014 The naming here confuses me. AIUI xend/xm is deprecated (along with any support for managed domains) and replaced by xl and some other management layer. I took this to mean that xendomains is part of that managed domain support but is it or is it just a standalone script? xendomains is a script that handles the autostart, start, stop, save, restore, etc. of the domu (VMs) when the dom0 (Host) shutsdown, starts, reboots, etc. Quote Link to comment
ironicbadger Posted January 31, 2014 Share Posted January 31, 2014 And is for the most part a huge pain in the butt. Sent from my Nexus 5 using Tapatalk Quote Link to comment
mattkhan Posted January 31, 2014 Share Posted January 31, 2014 The naming here confuses me. AIUI xend/xm is deprecated (along with any support for managed domains) and replaced by xl and some other management layer. I took this to mean that xendomains is part of that managed domain support but is it or is it just a standalone script? xendomains is a script that handles the autostart, start, stop, save, restore, etc. of the domu (VMs) when the dom0 (Host) shutsdown, starts, reboots, etc. my q was whether it is part of the deprecated xend or not (answer appears to be no) Quote Link to comment
ironicbadger Posted January 31, 2014 Share Posted January 31, 2014 It is still used very heavily by 4.3 Quote Link to comment
Kode Posted February 6, 2014 Share Posted February 6, 2014 2. The perception that each addon will come with a new vm to try and work with. My honest opinion is that naturally we will settle on one vm OS and though some process most users will be able to get most things by magically sourcing this one vm. Maybe it would be possible to integrate both Xen and docker at some point? Unraid sits all on its own, user has a button to install a VM of their choice, and use docker to install apps to that VM of choice, that way anybody could install whatever VM distro they were comfortable with, there could be a simple 1 click plugin system for users, and anything that didn't have a docker package could be installed manually on the VM. Quote Link to comment
ironicbadger Posted February 6, 2014 Share Posted February 6, 2014 Noone is stopping you dude! Quote Link to comment
bmfrosty Posted February 6, 2014 Share Posted February 6, 2014 I've been having all sorts of fun discovering all that I've ignored about virtualization in the last few years. I've been vaguely aware of things like chroot (use it for a DNS server) and vmware, and have used AWS as a hosting platform knowing that my data was basically secure. Recently I started digging into OpenVZ and Xen and KVM due to things like trying to find a good internal virtualization solution in my cage (the current winning solution is Proxmox with *many* OpenVZ containers), but know I've started looking into Docker, which seems to be a fancy front end for LXC which in turn is another implementation of OpenVZ ideas but with kernel maintainer blessings. That's not even starting to get into a proper frontend for dealing with failover of running images and such. I think I have my work cut out for me. Quote Link to comment
Kode Posted February 6, 2014 Share Posted February 6, 2014 Noone is stopping you dude! That in reply to me? If so, I wasn't making any demands, just a possible suggestion Unfortunately I have my hands full with a full time job as a PHP web developer, a 7 month old, some freelance to pay the bills, and http://fanart.tv and I use OpenVZ (via proxmox) for the 3 servers (2 x e3-1230v3 and 1 x e3-1240v2) for that so Xen is a whole new ball game. I just saw some people saying VMs were the way forward, and others saying we should use something like docker, I was just suggesting maybe combining them would be the perfect solution - personally, if I install v6 I'll probably create a Debian Wheezy VM and never use any plugins anyway Quote Link to comment
NAS Posted February 6, 2014 Share Posted February 6, 2014 Docker will have its moment I am sure just after Xen is nailed down and probably not in this Xen thread either One thing at a time as they say Quote Link to comment
CrashnBrn Posted February 15, 2014 Share Posted February 15, 2014 Just wanted to comment and say I'm super excited for this. It's going to give me a reason to upgrade all of my hardware! Can't wait for 6! Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.