Xen/unRAID-6 Discussion

[liunam]

I moved over to beta 3 as my ESXi box died (for no reason I could fathom) and for my needs it just works. I downloaded a ubuntu stacklet following the instructions at the top of this thread and now have 3 virtual machines talking via samba to the appropriate unraid user shares.

 

I was exactly in the same boat.  I had ESXi 4.1 running for over a year with Unraid, ClearOS, and WHS2011 as primary VMs.  Then it died on me two wks ago.  Spent a week trying to upgrade to ESXi 5.1/5.5, gave up, decided to give Unraid/Xen a try. 

 

I have so far loaded WHS2011 to it could not get ClearOS to load.  I am currently looking for an easy replacement.

 

[devanchya]

I have gotten my first XEN machine running but ran into these 2 issues from pure "No clue what I am doing" syndrome.  Now this is early days and I know it will be in the UI eventually but...

 

- How do I get a XEN based OS to automatically start on OS restart

- How do we get XEN to not hold the Shares open when trying to shut down...

 

Those are my 2 issues at the moment.

 

[limetech]

I have gotten my first XEN machine running but ran into these 2 issues from pure "No clue what I am doing" syndrome.  Now this is early days and I know it will be in the UI eventually but...

 

- How do I get a XEN based OS to automatically start on OS restart

- How do we get XEN to not hold the Shares open when trying to shut down...

 

Those are my 2 issues at the moment.

Those are both addressed and fixed in -beta4.

[MSattler]

I have gotten my first XEN machine running but ran into these 2 issues from pure "No clue what I am doing" syndrome.  Now this is early days and I know it will be in the UI eventually but...

 

- How do I get a XEN based OS to automatically start on OS restart

- How do we get XEN to not hold the Shares open when trying to shut down...

 

Those are my 2 issues at the moment.

Those are both addressed and fixed in -beta4.

 

Does that mean currently the shutdown's are not clean?  I was planning on setting up a VM this weekend, and switching unRAID to load as DOM0.

 

Thanks!

[limetech]

I have gotten my first XEN machine running but ran into these 2 issues from pure "No clue what I am doing" syndrome.  Now this is early days and I know it will be in the UI eventually but...

 

- How do I get a XEN based OS to automatically start on OS restart

- How do we get XEN to not hold the Shares open when trying to shut down...

 

Those are my 2 issues at the moment.

Those are both addressed and fixed in -beta4.

 

Does that mean currently the shutdown's are not clean?  I was planning on setting up a VM this weekend, and switching unRAID to load as DOM0.

 

Thanks!

 

Currently you have to manually shut things down yourself, or create and install an event script that does it.  In -beta4 the "unmounting_disks" event invokes:

 

    /etc/rc.d/rc.xendomains stop

 

This seems to cleanly shut down all the test VM's I tried but this will be a continuing work-in-process I suspect.

[apgood]

I have gotten my first XEN machine running but ran into these 2 issues from pure "No clue what I am doing" syndrome.  Now this is early days and I know it will be in the UI eventually but...

 

- How do I get a XEN based OS to automatically start on OS restart

- How do we get XEN to not hold the Shares open when trying to shut down...

 

Those are my 2 issues at the moment.

Those are both addressed and fixed in -beta4.

 

Does that mean currently the shutdown's are not clean?  I was planning on setting up a VM this weekend, and switching unRAID to load as DOM0.

 

Thanks!

 

Currently you have to manually shut things down yourself, or create and install an event script that does it.  In -beta4 the "unmounting_disks" event invokes:

 

    /etc/rc.d/rc.xendomains stop

 

This seems to cleanly shut down all the test VM's I tried but this will be a continuing work-in-process I suspect.

 

That sounds great Tom!  Sounds like you are putting a lot of effort into making it an appliance like experience.  Can't wait for when it's ready.  :-)

 

Sent from my Nexus 4 using Tapatalk

 

 

[EdgarWallace]

I made some test with the new powerdown script that dlandon came up with (great tool btw. as it it possible to start and shutdown VMs). During the test I ran some parity checks and I discovered a very low speed at only about 40 MB/sec with unRAID V6beta3.

 

Same HW with unRAID 5 was running at an average of 90-100 MB/sec.

 

My unRAID test server (with much older, slower HW, same unRAID V6beat3 with XEN) is doing the party check at about 90 MB/sec.

 

Any thoughts?

 

Yes , this time the parity was starting after an unclean shutdown but I had the same speed after clean shutdowns.

[itimpi]

You did not by any chance have the Dynamix GUI installed and open?  That is known to severely impact parity check performance if you have the real time page updates option (which is default) set.

[trurl]

I made some test with the new powerdown script that dlandon came up with (great tool btw. as it it possible to start and shutdown VMs). During the test I ran some parity checks and I discovered a very low speed at only about 40 MB/sec with unRAID V6beta3.

 

Same HW with unRAID 5 was running at an average of 90-100 MB/sec.

 

My unRAID test server (with much older, slower HW, same unRAID V6beat3 with XEN) is doing the party check at about 90 MB/sec.

 

Any thoughts?

 

Yes , this time the parity was starting after an unclean shutdown but I had the same speed after clean shutdowns.

syslog shows you're running mover every hour. I assume this is intentional. Since there is often nothing to move doesn't seem like this would matter. Are your other servers running mover this often?

[EdgarWallace]

syslog shows you're running mover every hour. I assume this is intentional. Since there is often nothing to move doesn't seem like this would matter. Are your other servers running mover this often?

Indeed trurl, running mover that often is intentional - on both servers every hour.

 

@itimpi, just a vanilla unRAID installation, w/o any addon, except the powerdown script 2.06.

 

 

What's coming to my mind: I have the sas2lp-mv8 x.x.1808 firmware installed, maybe I should upgrade to the new firmware first.

 

 

Sent from my iPad using Tapatalk

[EdgarWallace]

Upgraded the SAS2LP-MV8 to the latest firmware x.x.1812, still slow parity check at around 40-50MB/sec. 

[bkastner]

Upgraded the SAS2LP-MV8 to the latest firmware x.x.1812, still slow parity check at around 40-50MB/sec. 

 

Have you tried running the tunables script to optimize parity checks? It may be worth trying out. I know I noticed a pretty big bump in parity checks after correctly setting the tunables options (I don't remember exactly how slow I was running, but think it wasn't too far off yours, and with tunables I am back around 100MB/sec).

 

[EdgarWallace]

Good catch but unfortunately I checked it. Forecast was the speed that you see here

[Rick Sanchez]

Is there a recommended build list for v6?  As I'm currently writing a new/upgraded build list.

 

http://lime-technology.com/forum/index.php?topic=32818.0

[eroz]

Is there a recommended build list for v6?  As I'm currently writing a new/upgraded build list.

 

http://lime-technology.com/forum/index.php?topic=32818.0

Not really, but this is where people have been putting working builds.

[CUclimber]

I want to echo the comments about the ease of use of Xen.  I think that it is absolutely critical that this gets streamlined as much as possible.  Anyone who has ever used the standalone VirtualBox application should have an understanding of how simple that process is-- choose a .iso file, allocate some RAM and a CPU, and you're off and running.  No command lines, no config files copied out of a forum post somewhere, just a simple & easy process.

 

I think that advanced users forget that even though it is easy to open up a CLI and start typing away, that doesn't mean that the user understands what they're doing, why they're doing it, or what to do if something doesn't work as expected.  Troubleshooting via a CLI is totally out of the question for a lot of people, whereas a simple error message on a GUI is easy to use and understand.  Until I read this thread I never even knew that auto-starting VMs and putting in shut-down procedures was something I'd have to do manually through a config file or CLI, and I think that's really going to hurt the adoption of this whole setup.

 

I am thrilled that VMs are going to be a thing in UnRaid-- I'll be able to consolidate my HTPC onto my server's hardware, and I can install all of my applications in a nice friendly GUI where they're easy to configure, update, and remove and I don't need to worry about a plugin becoming abandonware that stops working after a Python update.  But, this all needs to move stongly in the 'Appliance' direction rather than a system that requires constant Googling and piecing together bits of advice from random forum posts.

[jonp]

We are working on significantly improving the xen VM management GUI.  We are aware that what is in the current beta is insufficient for proper management.  Stay tuned...

 

Sent from my Nexus 5 using Tapatalk

 

 

[JustinChase]

Thanks for the update.  Any chance these improvements can be pushed back to beta7, and a beta6 could be released with just the updated kernel and XEN 4.4?

 

I'm still having some issues with GPU passthru, and from what I've read XEN 4.4 may resolve these issues.  Plus, it would allow others to work on trying to get iGPU working with unRAID.  I now you haven't been successful with it yet, but perhaps others being able to test might turn up some potential solutions.

 

I understand (and very much appreciate) the desire to improve the VM system in unRAID, but I'm afraid the work on this is delaying other, more simple 'upgrades' that we might otherwise get sooner.

 

Either way, thanks again for the much improved communications of late, I know I'm not the only one that appreciates it.

[jonp]

Thanks for the update.  Any chance these improvements can be pushed back to beta7, and a beta6 could be released with just the updated kernel and XEN 4.4?

 

I'm still having some issues with GPU passthru, and from what I've read XEN 4.4 may resolve these issues.  Plus, it would allow others to work on trying to get iGPU working with unRAID.  I now you haven't been successful with it yet, but perhaps others being able to test might turn up some potential solutions.

 

I understand (and very much appreciate) the desire to improve the VM system in unRAID, but I'm afraid the work on this is delaying other, more simple 'upgrades' that we might otherwise get sooner.

 

Either way, thanks again for the much improved communications of late, I know I'm not the only one that appreciates it.

 

We are targeting a release of an updated beta and kernel for next week.  This weekend was our office move, we have had the Carlsbad fires this past month, and a lot of business development occurring in tandem with product development.

 

We also ran into a critical bug issue with the 3.14.4 Linux kernel and Xen network driver.  Crashes the VIF and there is no workaround at this time.  The updated Linux kernel + Xen 4.4 release will be next week, but instead of 3.14.4, it will be 3.12.20.

 

Sent from my Nexus 5 using Tapatalk

 

 

[JustinChase]

Wow, sounds very busy.

 

I appreciate the update, and look forward to testing the new release!

[jbartlett]

Happened a couple times when restarting a Win8 VM with PCI binding. The computer will become totally unresponsive requiring a shutdown via holding the power button down - the reset button does not do anything.

 

But when it happens, my entire LAN goes offline too. None of the computers attached to the LAN can access any other local computer or remote web site until I shut off the NAS. The only exception was my work laptop which was connected via VPN to my work network.

[RFehr]

Folks,

 

Here in Calgary, we have a short summer - or at least this year we have.  The days of +25C with sunshine seem to be numbered - hence I'm dumping a log file here without really even trying to look at it myself.  Like... not at all.

 

So... I do feel guilty - I really hate dumping problems over the wall unless I've at least put in an honest effort.  But, I'm confident that this feeling will pass with the assistance of a cold pint or two.

 

In short, b4/Xen with a single Arch VM running all plugins has been crashing every day or two - BRILLIANTLY stable prior to about a week ago.  I've changed nothing over this period.  All pluggins except Plex Media Server are running in the Arch VM (PMS performs better with direct disk access for transcoding).  Other than that, it's "off the rack" b4/Xen.

 

See attached.

UnRAID_log.zip

[RobJ]

As far as I can tell (but I could be wrong!), the issues are not related to the beta or to XEN, but rather to either the SAS card or its driver.  At 23:49:11, the SAS card appears to crash, see the following alarming message:

Jul 19 23:49:11 UnRaid kernel: mpt2sas0: _base_fault_reset_work : SAS host is non-operational !!!!

 

It attempts recovery, but isn't completely successful, so tries a DIAG reset, and it fails!

Jul 19 23:49:16 UnRaid kernel: mpt2sas0: sending diag reset !!

Jul 19 23:49:16 UnRaid kernel: mpt2sas0: diag reset: FAILED

 

From there on, all hell breaks loose, with read and write failures on numerous drives.  Since the trouble is widespread, we can assume that it is not the fault of the drives, but the card or its driver.  All or most of the Reiser file systems on these drives were changed to Read Only, which helped to limit the damage.  I would run reiserfsck on ALL of the array data drives, not the parity drive of course, see Check Disk File systems.

 

The system continued limping along, wrecking a TimeMachine and mover session, until mercifully cut short by a general protection fault at 2:32am.

 

While the logs are full of errors of all types, the primary cause was the crash of the SAS card, or its driver.  I don't have any SAS cards, and little experience with them, so perhaps others may have ideas to help you.

[RFehr]

@RobJ - thx a bunch.  Definitely a hardware problem - likely do to our 'balmy' new ambient temps.  Lifted the lid and performed a standard server BJ, re-seated SAS card and cable - issue gone.  However, no reiserfs partition detected on sdd - but the physical disk looks fine.  Partition table moved/bent - looks like I need to dust off testdisk (or the "screw it" option and rebuild from parity).

 

Thanks again.

[JonathanM]

@RobJ - thx a bunch.  Definitely a hardware problem - likely do to our 'balmy' new ambient temps.  Lifted the lid and performed a standard server BJ, re-seated SAS card and cable - issue gone.  However, no reiserfs partition detected on sdd - but the physical disk looks fine.  Partition table moved/bent - looks like I need to dust off testdisk (or the "screw it" option and rebuild from parity).

 

Thanks again.

You can see what would be recovered from a parity rebuild by removing the disk and starting the array with the disk missing. Whatever is on the simulated disk will be written to the new disk when it is rebuilt. That way you can see if a parity rebuild will gain you anything or not.