Apology and Plan Moving Forward

[limetech]

With all the turmoil surrounding the last couple of -rc's, changing kernels, workarounds, memory limits, etc., it must appear to be "Amateur Hour" here at Lime Technology - I want to apologize for that to our Customers and everyone in the unRaid Community.

 

I have managed to get into a "catch-22" situation where kernel "A" solves one nasty problem (slow writes with some motherboards) but causes another (possible system crash when Stopping array); and kernel "B" doesn't solve the first problem but does solve the second problem.  My handling of this has not been the best since I got into a kind of "scramble" mode.

 

Of the two issues, the second one (crash when Stopping array) is the more likely one to get fixed, letting us move on to the later kernel.  I'm trying to talk with a ReiserFS maintainer at SUSE to see if this problem can be looked at.  I will keep everyone informed when/if progress is made.

 

In the meantime, I will be releasing another -rc with a couple of changes, one of which is going back to the 'default' choice of a PAE kernel with no "mem=" parameter (those following that discussion know what I'm talking about).  There will be another boot option to limit RAM for those experiencing the "slow write" issue.  I should not have changed this because, being in scramble mode, I didn't fully investigate what this parameter does and I didn't test it on a server with less than 4GB of RAM - again I apologize if this has caused some grief.

 

The "real" solution moving forward is to move to a 64-bit kernel.  I've already done most of the work to do this.  So here's the overall game plan:

 

1. Release one more -rc to correct the boot menu and hopefully get a fix in there for some issues with netatalk (AFP).

2. Release 5.0 final with all the documentation completed.

3. Very soon thereafter, release 6.0 which will be same feature set as 5.0 but with 64-bit kernel/platform.

 

From that point on I will maintain two code bases:

5.x - the 32-bit, and

6.x - the 64-bit.

 

At some point it's possible the 5.x series will switch to a non-PAE kernel once the 64-bit version is rock solid.

 

UPDATE: yesterday (13 June) Jeff Mahoney of SUSE Labs, who is the current maintainer of reiserfs, sent me a patch that fixes the "continuous writes" caused by 'sync' command, and also fixes the "crash when Stopping array" bug as well.  This changes the landscape quite a bit and I'd like to release -rc15 with patched 3.9.6 kernel.

[Dephcon]

I like plans!  Sounds good Tom, keep up the good work.

 

Is it going to be a huge PITA to maintain two releases?  Will 32bit go EOL soon?

[StevenD]

Thanks for the update!

[coupas]

Thanks for the information. Looking forward to the new releases.

[Carpet3]

Is it really worth maintaining a 32bit version? How many people couldn't actually run it? I bet it's not many

[G Speed]

Out of curiosity.. what is the benefit of going 64bit, aside from PAE

[whiteatom]

Out of curiosity.. what is the benefit of going 64bit, aside from PAE

 

Using hardware that was released since 2008...

 

Thanks Tom! we really appreciate the info from the inner circle.

[JonathanM]

Out of curiosity.. what is the benefit of going 64bit, aside from PAE

Just being able to remove PAE and still access memory above the 3.5GB line is reward enough for some. To be honest, considering just the basic functionality of a NAS with no addons, there really isn't any benefit I can think of to running huge amounts of RAM. However, most of the active forum members run addons, many run LOTS of addons, and some even run virtual machines inside of unraid. I personally have 3 virtual machines running all the time, with a couple more running sporadically. If I couldn't use more than 3.5GB of RAM, I'd have to use physical machines for those functions. My lowest steady RAM usage is currently over 4GB.

[garycase]

Good plan -- I suspect there are very few folks who can't run an x64 kernel these days (but there undoubtedly are a few).

 

I definitely agree with removing the 4GB default option -- I noted that in the release thread, but there was a lot of kickback at that suggestion 

 

Looking forward to v6 (and of course the intervening v5 "final").

 

[dirtysanchez]

Sounds like a great plan for moving forward. Thanks for the update and all the hard work Tom!

[garycase]

Out of curiosity.. what is the benefit of going 64bit, aside from PAE

 

As already noted, it allows a huge extension of the address space (depends on the specific CPU, but most 64-bit CPUs support a 48-bit address register, so that's 65536 times current 4GB limit = 256TB !!  Probably more RAM than you'll be able to install anytime soon 

 

One potential disadvantage of this huge address space ... folks are likely to be more tempted to install a stack of add-ons instead of virtualizing UnRAID and using another VM to put all the other "stuff" in.    The virtualization approach allows a much more "stock" UnRAID, which tends to mean a more reliable NAS device for your storage needs.

 

Of course if you get a solid implementation of Virtual Box running "under" UnRAID, you could then run the other "stuff" in a VirtualBox VM and get essentially the same degree of isolation ... and with plenty of RAM that approach could work very well.  [sounds like jonathanm already does that]

 

By the way, just in case 256TB isn't enough, both Intel and AMD still support the 4-bit PAE registers even in 64-bit mode      [somehow I don't think anyone's going to need x64 PAE anytime soon !]

 

[moose]

Thanks Tom, no worries.  Appreciate all you do....I'm in here for the long haul!

[pfp]

Thank you for the update,  it's much appreciated.  That being said I have to add ...

 

3. Very soon thereafter, release 6.0 which will be same feature set as 5.0 but with 64-bit kernel/platform.

 

 

No worries it's all in fun.

[Ice_Black]

Thanks for update.

 

It is understandable that different version of kernel can fix some issues but its break other things, this has always been the case.

 

How did Freenas, OpenMediaVault, Openfiler and FlexRAID deal with this issues?

 

Are you saying 64-bit will fix most problems easily? I want to see your 6.x todo list

[alikuenkano]

Thank you, Tom for your hard work !!!

 

[G Speed]

Out of curiosity.. what is the benefit of going 64bit, aside from PAE

 

As already noted, it allows a huge extension of the address space (depends on the specific CPU, but most 64-bit CPUs support a 48-bit address register, so that's 65536 times current 4GB limit = 256TB !!  Probably more RAM than you'll be able to install anytime soon 

 

One potential disadvantage of this huge address space ... folks are likely to be more tempted to install a stack of add-ons instead of virtualizing UnRAID and using another VM to put all the other "stuff" in.    The virtualization approach allows a much more "stock" UnRAID, which tends to mean a more reliable NAS device for your storage needs.

 

Of course if you get a solid implementation of Virtual Box running "under" UnRAID, you could then run the other "stuff" in a VirtualBox VM and get essentially the same degree of isolation ... and with plenty of RAM that approach could work very well.  [sounds like jonathanm already does that]

 

By the way, just in case 256TB isn't enough, both Intel and AMD still support the 4-bit PAE registers even in 64-bit mode      [somehow I don't think anyone's going to need x64 PAE anytime soon !]

 

So for the average 99% of users.. nothing lol

 

Not saying that I won't use it just because lol

[rbreen]

Thanks for the clear communication - we know you are working hard on things, but its great to hear the plan!

[binhex]

Thanks for the update Tom, I for one appreciate you keeping me informed with future roadmap

[garycase]

So for the average 99% of users.. nothing lol

 

Not saying that I won't use it just because lol

 

You mean you're not planning to buy a couple 1TB memory modules ?? !!

 

You're correct that for those who run stock UnRAID without a bunch of plugins there's probably no tangible benefit.    Certainly if you're happy running an UnRAID server with 1 or 2 GB of RAM it won't make any difference.

 

But I suspect a lot of folks will install more RAM "just because" 

[nars]

Thank you for the update, glad to see things moving and to know that 64-bit version is on the way   And besides sync issue I would not hurry to get to very latest kernel, 3.4 is lts and seems very stable/mature thing... that is the most important after all...

[jaybee]

With all the turmoil surrounding the last couple of -rc's, changing kernels, workarounds, memory limits, etc., it must appear to be "Amateur Hour" here at Lime Technology - I want to apologize for that to our Customers and everyone in the unRaid Community.

 

I have managed to get into a "catch-22" situation where kernel "A" solves one nasty problem (slow writes with some motherboards) but causes another (possible system crash when Stopping array); and kernel "B" doesn't solve the first problem but does solve the second problem.  My handling of this has not been the best since I got into a kind of "scramble" mode.

 

Of the two issues, the second one (crash when Stopping array) is the more likely one to get fixed, letting us move on to the later kernel.  I'm trying to talk with a ReiserFS maintainer at SUSE to see if this problem can be looked at.  I will keep everyone informed when/if progress is made.

 

In the meantime, I will be releasing another -rc with a couple of changes, one of which is going back to the 'default' choice of a PAE kernel with no "mem=" parameter (those following that discussion know what I'm talking about).  There will be another boot option to limit RAM for those experiencing the "slow write" issue.  I should not have changed this because, being in scramble mode, I didn't fully investigate what this parameter does and I didn't test it on a server with less than 4GB of RAM - again I apologize if this has caused some grief.

 

The "real" solution moving forward is to move to a 64-bit kernel.  I've already done most of the work to do this.  So here's the overall game plan:

 

1. Release one more -rc to correct the boot menu and hopefully get a fix in there for some issues with netatalk (AFP).

2. Release 5.0 final with all the documentation completed.

3. Very soon thereafter, release 6.0 which will be same feature set as 5.0 but with 64-bit kernel/platform.

 

From that point on I will maintain two code bases:

5.x - the 32-bit, and

6.x - the 64-bit.

 

At some point it's possible the 5.x series will switch to a non-PAE kernel once the 64-bit version is rock solid.

 

 

1: So you plan to use the later Kernel for getting 5 to final, and fixing the crashing when stopping the array? Can you state why you feel this a better option than sticking with the current earlier kernel used in RC14 other than it just being newer? RC14 seems to have NO reported issues whatsoever - I am not counting 4gb issues.

 

2: Will licences bought for 32bit version be transferable to 64bit version?

 

3: Will the upgrade from 32bit to 64bit just be "another simple upgrade" for us users in that we can update our USB drive and then boot back up and expect the array to behave as before, or will there be more to it?

 

4: What approximate ETA would you set yourself for having 5 final right now?

 

5: What approximate ETA would you set yourself for having 6 released into the wild for testing?

 

 

Tom I'm happy to see you getting close with 5. One final push and it's there surely. I keep getting frustrated with you and unraids problems, but there is nothing else that fits my needs at all. NOTHING. There are competing technologies and setups that I have personally looked into myself and yet I keep coming back to unraid armed with my pro licence ready to install and use 5 final because nothing else compares. Every single other option I look at has something I don't like about it. Unraid is perfect if it could go final with no issues and the icing on the cake would be to sort NFS issues.

Keep up the good work.

[jaybee]

Oh also I wanted to say, anyone that runs hardware not capable of running a 64bit kernel...good riddance. No offense, but the world can't wait around catering for such old tech. There are times when we have to simply say, that's not supported anymore because it's too old. Sorry. I would like to see a poll as to who can afford to run unraid and keep purchasing disks and media to store, but cannot update their system to hardware capable of running 64bit. I would suspect the percentage would start with a zero.

[Dephcon]

Oh also I wanted to say, anyone that runs hardware not capable of running a 64bit kernel...good riddance. No offense, but the world can't wait around catering for such old tech. There are times when we have to simply say, that's not supported anymore because it's too old. Sorry. I would like to see a poll as to who can afford to run unraid and keep purchasing disks and media to store, but cannot update their system to hardware capable of running 64bit. I would suspect the percentage would start with a zero.

 

Well that's the thing, 5.x will probably be the last version for 32bit and once stable won't get any new features, like 4.7.

[kizer]

Thanks Tom for the info.

[JM2005]

Thanks Tom for keeping us informed!  Keep up the Good Work!