Your Chance to Chime In

[Joe L.]

cache_dir is a mem hog is it not ?

No,  on its own, it is not.  I run it on my original server, and it only has 512 meg of ram.

 

It is Linux that will cache in its disk buffer cache as much as it can.  If you have 4Gig of RAM and play a 4Gig ISO movie, all the ram will be in the disk buffer cache.

If you do a "windows" search on the shares on your server, it will do the same as cashe_dirs... basically a "find" on the directory hierarchy. 

 

Joe L.

[Pauven]

Tom, I'm just catching up and would like to cast my vote if it isn't too late:

 

1)  Freeze any feature development and resolve any open 'CRITICAL' bugs with the 5.0 RC's - a task which it sounds like it may already be complete (performance issues on select hardware configurations are not critical bugs).  As part of this, you might want to post a "Last Call for 5.0 RC10 'CRITICAL' Bug Reports" on the forum.

 

2)  Release 5.0.0-x32-Final.  Include in the release notes that this 32-bit version does not support memory configurations greater than 4GB, which have been noted to sometimes result in performance issues.  Also list any hardware platforms that may have performance issues which may or may not be related to the amount of installed RAM.  Users are certainly entitled to run unRAID on any hardware configuration they prefer, but should be informed that choosing to run certain problematic hardware or excessive memory limits their support.  Of course, any currently 'Known Issues' should be documented in the release notes.

 

3) Provide immediate and ongoing support for any new 5.0.0-x32-Final issues that are rated 'HIGH' or 'CRITICAL', releasing 5.0.n-x32 versions as necessary.

 

4) Once support calms down to only 'MEDIUM' or lower issues, give unRAID a 4-week 64-bit evaluation window.  With the clock ticking, create and release 5.1.0-x64-Alpha1 with no changes from 5.0.n-x32-Final other than those necessary to recompile under x64.  This release is purely to test the waters and determine if 64-bit is a currently viable path forward for unRAID.  Release only to a selected alpha test group, and if it proves successful, proceed to a 5.1.0-x64-Beta1 general population test release.  During the 4-week window you may code bug fixes, as long as that effort does not extend beyond the 4-week evaluation window. At the end of the 4-week evaluation window, if x64 unRAID is not ready for a Release Candidate, revert to x32 development.

 

5) Regardless of outcome of the x64 experiment, please share the results with the community.

 

6) Resume feature development on the chosen platform, either x32 or x64, but not both.

 

Other Thoughts:

You may want to take a poll to see how many users are running legacy 32-bit processors that are not 64-bit capable.  I would hate to see a large 32-bit hardware only user base alienated (if it even exists), even though I feel it is time for unRAID to make the move to 64-bit.  If the 32-bit only user base is small, then it may be worth sun-setting their support.  I also believe unRAID's development resources (Tom and, errr... just you Tom) are spread too thin to support both x32 builds and x64 builds going forward, so I see going 64-bit as abandoning 32-bit for future versions. 

 

I have concerns that future betas and release candidates may hop around between x32 and x64 builds in much the same way as I have seen test 5.0 versions hop around between various kernel builds looking for a stable base, so I would rather see x32 development take precedence over searching for a stable x64 build.  That means that if x64 alpha or beta tests come back with disappointing results, let's parking-lot x64 development in favor of x32 feature development that has long been promised.

 

As always, thank you Tom for developing and supporting this unique and very capable product. </brown-nosing>

 

Paul

 

Paul's Bug Rating Scale:

CRITICAL - On a 'Stock Installation': System crashes, data corruption, common major performance issues that limit usability or broken core functionality, all with no workarounds.

HIGH - On a 'Stock Installation':  System crashes, data corruptions, common major performance issues or broken core functionality, all with viable documented workarounds.  Also common minor performance issues.

MEDIUM - On a 'Stock Installation': Broken non-core functionality and rare performance issues (affecting less than 5% of users).  On a 'Non-Stock Installation with 3rd Party Plugins':  System crashes, data corruption, and broken core functionality.

LOW - Feature requests, module upgrade requests, specific driver requests.

[seanant]

My vote is yes for a few reasons.

 

1. Start on the next version to include feature enhancements and maitenance updates.

2. Plan on a release schedule that the community can rely on.  It may not be written in stone but gives us all a point in time to plan for upgrades and maintenance. 

3.  Plan plan plan.  At least have a plan with a process to follow or at least something we all can believe is happening.  This gives the appearance of organization and stability.

 

Just my opinion but if there is a way to set some milestones this will allow for better feature velocity and more stable code.  If we have to wait a set period of time for a feature enhancement at least we all know its coming in X amount of time.  Brake fixes should be fixed asap. 

 

Betas are great but if they last too long users lose interest. 

Thank you for a awesome product!!!

[seanant]

One last comment. Keep the code on one version.  Always have the sane code release the sane version with same features. No branches or specials. Trust me this will cause more work then anyone will have time to do. 

 

You may need two or more binary images but they should be the same version. Don't want to end up like Cisco with 3 differt operating systems that all have multiple versions for special features. What a nightmare that product is.

 

There's always the googles of the world that spends lots of cash that demand now. So keep it simple so when it's not there's room to fall back to the baseline. 

 

Just me chiming in.

[NAS]

...

 

the people using crazy amounts of ram are greatly exceeding the requirements for UNRAID. the ONLY reason for this much ram is for add-ons and plug-ins. IMO development should not cater to those users, but to the CORE functionality of unRAID and helping users diagnose their systems from the webGUI.

...

so you DO run plugins then.

 

dont get me wrong, i run plug ins myself, but i only have 3gb ram installed.

 

at some point more is just so people can say"mines bigger than yours".

 

what needs to be determined is this: What is the amount needed for unRAID and unraid alone without ANY ADD ONS.

 

cache_dir is a mem hog is it not ?

 

not trying to be an asshole here, but now we are considering delaying a fully stable release because users who are making unraid customer require more ram ?

 

Lots of good comments following my post.

 

Just to be clear I am not suggesting a delay in 5. I am merely voicing an opinion that after 5 it is time to start a development cycle for 64bit.

 

We will have to have two versions but maintaining 64 bit and 32bit builds is typical of most OS's.

 

64 bit is the way the world is going and has been for a very long time. PAE was a kludge 15 years ago and whilst it removes some memory limitations it introduces some of its very own.

 

From a usability point of view the more RAM i give unRAID the slicker it is for me. For instance i often see me resuming videos the next day when I know the disk is down. Often there is enough in RAM to make the disk spin up transparent.

 

There will be more improvements behind the scenes as well since there is a reason Linux kernel devs put tens of thousands of hours into creating a 64 bit kernel.

 

But the end of the day I want to eek every bit of performance out of my hard earned hardware and lack of 64bit is a fundamental barrier to even trying that.

 

But, V5 actually stable first (if that is 5.20 so be it) then fork and test 64 bit. This dev cycle may not be as big a deal as people think. What we are doing is not particualrly new and we are stading on the shoulders of giants that have had years of dev and test behind this already.

[tyrindor]

the people using crazy amounts of ram are greatly exceeding the requirements for UNRAID. the ONLY reason for this much ram is for add-ons and plug-ins. IMO development should not cater to those users, but to the CORE functionality of unRAID and helping users diagnose their systems from the webGUI.

 

I greatly disagree with this (no offense). Many users are exceeding 4GB and the number is growing more everyday. RAM is cheap, people are buying larger kits and are not even aware of the issues. Many people use addons like cache_dirs and want more usable RAM for it. unRAID should cater to all users that bought the software, singling out people on newer hardware is a good way to get people to go to other storage solutions. I'm sure tom knows this though, and I really hope he doesn't wait until 6.0 to get a 64bit kernel out.

 

I personally consider cache_dir a must have, as do many unRAID users. I'm pretty sure the majority of unraid user have it installed, and i'd go as far to say I wouldn't use unRAID if it didn't exist. With around 40 drives between my servers, having all my drives spin up everytime I want to access my main share is not healthy on the hardware.. and that's what happens for me without cache_dirs.

 

After 5.0 final, the next major thing after bug fixes should be 64bit in my opinion.

[itimpi]

Just for interest I have successfully run 32-bit unRAID installed on top of a 64-bit kernel (both Slackware 13.37 and 14.0) with the 32-bit compatibility layer added.  As far as I can see it all works OK so maybe the migration to 64-bit kernels might be quite easy.  However I must admit I have not tested all recovery scenarios - just normal operation. 

 

Having said that it would be much nicer if unRAID ran natively in 64-bit so that the 32-bit compatibility layer could be omitted.  Also when building the 64-bit kernel various compiler warning messages are issued about the unRAID md.c module (I assume because variables have changed size) although it does not appear to affect functionality.

[Carpet3]

I'm itching to see this go final

[NAS]

...

 

Here are some possibilities for fixing this:

...

d) provide a 64-bit kernel

...

 

The best solution IMHO is choice d).  However, this might take a while and introduce other issues.

 

Worth pointing out the OP of Limetech.

[peter_sm]

How long time shall we discuss this before we see some final release and a new 64-bit beta from limetech? 

 

//Peter

[Helmonder]

Tom has stated he wants to try and solve thr slow copy speed issue before (see other thread), unless that will prove to be to hard or undoable I think the answer is there.

 

I would seriously advice anyone holding off on using rc10 because the word final is not in the title to install...

[S80_UK]

I would seriously advice anyone holding off on using rc10 because the word final is not in the title to install...

Edit: With apologies to Helmonder since I misunderstood his meaning...

 

Disagree strongly. I agree.

 

More people using it gives more information on success or failure.  More information spread across more hardware platforms and with different usage patterns will generate more input for Tom which may help in resolving outstanding issues.  Equally, more successes will give greater confidence in the stability of the software.

 

There will be some who see a risk in using a software not named as final.  But that will always be true and that is their choice to make.  On the other hand, most of those that have used version 5 release candidates know just how stable the software is - Very.

[JackBauer]

 

Disagree strongly.

 

 

+1

 

I will probably install it about a week after it goes final.  Hopefully now that so many people have "chimed in", Tom will post back his current thinking and plans.

 

I'm very optimistic though we'll see something posted as final within 1-2 weeks now.

[jaybee]

 

I personally consider cache_dir a must have, as do many unRAID users. I'm pretty sure the majority of unraid user have it installed, and i'd go as far to say I wouldn't use unRAID if it didn't exist. With around 40 drives between my servers, having all my drives spin up everytime I want to access my main share is not healthy on the hardware.. and that's what happens for me without cache_dirs.

 

 

I do not know what this cache_dirs thing is? Can someone explain this to me? Are you saying without this plugin installed, the core functionality of unraid to spin down and up drives does not work? That is how it reads and I find that hard to believe.

 

 

[TheDragon]

Personally I find it an extremely useful plugin. It will cache the locations of files/folders in RAM. So that browsing the folder structure of your unRAID share will not result in a drive being spun up, until you access a file.

 

Drives can still spinup and down as in stock unRAID, except with cache_dirs they only will only spin up when a file is accessed.

 

Hope that makes sense.

 

 

 

Sent from my Samsung Galaxy S2 using Tapatalk 2

 

[RoninTech]

 

I personally consider cache_dir a must have, as do many unRAID users. I'm pretty sure the majority of unraid user have it installed, and i'd go as far to say I wouldn't use unRAID if it didn't exist. With around 40 drives between my servers, having all my drives spin up everytime I want to access my main share is not healthy on the hardware.. and that's what happens for me without cache_dirs.

 

 

I do not know what this cache_dirs thing is? Can someone explain this to me? Are you saying without this plugin installed, the core functionality of unraid to spin down and up drives does not work? That is how it reads and I find that hard to believe.

 

Here you go.

[RoninTech]

I would seriously advice anyone holding off on using rc10 because the word final is not in the title to install...

Disagree strongly.

 

More people using it gives more information on success or failure.  More information spread across more hardware platforms and with different usage patterns will generate more input for Tom which may help in resolving outstanding issues.  Equally, more successes will give greater confidence in the stability of the software.

 

There will be some who see a risk in using a software not named as final.  But that will always be true and that is their choice to make.  On the other hand, most of those that have used version 5 release candidates know just how stable the software is - Very.

 

Is it just me or are you guys disagreeing that you believe the same thing? 

 

Helmonder says install if you haven't just because it doesn't say final and I *think* that's what you say also S80_UK?

[Thornwood]

I think the slow write problem seems to be the processor slowing down and never speeding back up. In rc8a people were posting about this problem and I was seeing slow speeds so I set the processor to full speed and every thing is working great. Based on this I think that it has something to do with the processor throttling to save power.

 

Thornwood

[Helmonder]

I think the slow write problem seems to be the processor slowing down and never speeding back up. In rc8a people were posting about this problem and I was seeing slow speeds so I set the processor to full speed and every thing is working great. Based on this I think that it has something to do with the processor throttling to save power.

 

Thornwood

 

Only if there is some relation between the cpu slowing down and the amount of memory... The problem is solved when you change the amound of allocated memory and/or the way the system deals with it..

[dikkiedirk]

It sounds like a work around has been found for the slow writes issue. Therefore I would also vote to release as final, assuming bug #3 Joe mentions above has been fixed. If it hasn't, I think this one should be resolved prior to final release.

 

Sent from my Nexus 7 using Tapatalk HD

 

+1

Bug #3 is important, squash it please!

[kizer]

myself, i'd prefer to seee 5.0 go final as is and future dev as this:

 

 

- 5.0.x for bug fixes that *may* pop up as more users jump in the 5.0 waters.

 

- 5.1, 5.2, 5.3, etc slated for feature additions for ease of use (i.e. built in disk diag tools, rieserfs --check commands, etc.)

 

- 6.0 for 64bit.

 

the people using crazy amounts of ram are greatly exceeding the requirements for UNRAID. the ONLY reason for this much ram is for add-ons and plug-ins. IMO development should not cater to those users, but to the CORE functionality of unRAID and helping users diagnose their systems from the webGUI.

 

 

my .02.

 

I'm in agreement with this as well. Even thou I have 4gig of ram simply because it was cheap at the time and just like my Mac and my Windows7 machine more ram more better.

[S80_UK]

I would seriously advice anyone holding off on using rc10 because the word final is not in the title to install...

Disagree strongly.

 

More people using it gives more information on success or failure.  More information spread across more hardware platforms and with different usage patterns will generate more input for Tom which may help in resolving outstanding issues.  Equally, more successes will give greater confidence in the stability of the software.

 

There will be some who see a risk in using a software not named as final.  But that will always be true and that is their choice to make.  On the other hand, most of those that have used version 5 release candidates know just how stable the software is - Very.

 

Is it just me or are you guys disagreeing that you believe the same thing? 

 

Helmonder says install if you haven't just because it doesn't say final and I *think* that's what you say also S80_UK?

 

@RoninTech - I think you are right.  I read that he was advising folks not to install.  On a second reading I think he's saying the fact that it doesn't say final is not a reason to wait, which is also what I was saying.  I was tripped up by the word order.  I have edited my previous post to reflect this.

[unevent]

I think the slow write problem seems to be the processor slowing down and never speeding back up. In rc8a people were posting about this problem and I was seeing slow speeds so I set the processor to full speed and every thing is working great. Based on this I think that it has something to do with the processor throttling to save power.

 

Thornwood

 

Only if there is some relation between the cpu slowing down and the amount of memory... The problem is solved when you change the amount of allocated memory and/or the way the system deals with it..

 

Agree, there is certain hardware involved that is causing the memory allocation issues with how 32bit Linux addresses above 4GB ram (actually less) - those with flawed PAE implementation I would guess.  That said, it is a rather limited set of hardware that has that bug and it is in the BIOS, chipsets and/or processor that it affects.  32bit linux (or any OS for that matter) cannot directly address more than 4GB ram, so if the kernel has PAE support along with the motherboard & CPU, it uses the processor to map/unmap as needed to the higher memory, as I understand it.  It would be interesting to see that those with the problem disable SpeedStep or whatever is in use so their CPU runs at full speed in stock unRaid and test without the "4095M".  I ran NFS and SMB tests with an 8GB system both reads and writes and found nothing other than SMB is slower than NFS, but still 30+ for SMB and 60+ for NFS with the test setup I used.  The disk subsystem and the vm tune plays the major role with how well files are transfered over SMB or NFS with stock unRaid.  If Tom were to disable plugins by default on the next test RC and then wait and see what effect it has I am sure we will be hearing something new, perhaps better.

 

I would love to see the option for cache_dirs that allows one to change the cache_pressure easily instead of editing the file.  The default may be too aggressive to use across the board when it is unknown how much crap people are running on a low memory and underpowered systems (as an example for addons).  I find it hard for Tom to deliver a reliable 32bit product when the customization (plugin) window is wide open.  Kind of like designing a new car and give it to someone to test and they go and add in a bunch of crap to suit their tastes then go back and complain the car isn't working.  unRaid with addons should work, it has worked, but the bug in the Kernel, those with limited hardware (sure an Atom with 2GB can run 5 python/java plugins while transferring files at full speed over Ethernet, no problem), non-adjustable cache_pressure with cache_dirs, preclear scripts (and their resource requirements) and just about any addon made by just about anyone with a text editor with whatever libraries and packages they can scrap off the net to make it work with no defined rules or restrictions on how they are implemented and trying to release a piece of software as final?  Fixing addons is for a different post, but a place to start is a set list of rules (or outline framework) for addon creation.

 

 

[unevent]

myself, i'd prefer to seee 5.0 go final as is and future dev as this:

 

 

- 5.0.x for bug fixes that *may* pop up as more users jump in the 5.0 waters.

 

- 5.1, 5.2, 5.3, etc slated for feature additions for ease of use (i.e. built in disk diag tools, rieserfs --check commands, etc.)

 

- 6.0 for 64bit.

 

the people using crazy amounts of ram are greatly exceeding the requirements for UNRAID. the ONLY reason for this much ram is for add-ons and plug-ins. IMO development should not cater to those users, but to the CORE functionality of unRAID and helping users diagnose their systems from the webGUI.

 

 

my .02.

 

It is not an unRaid thing, it is a 32bit Linux thing.  unRaid is just a driver and a set of tools written for an operating system.  The fact it does not address above 4GB directly is not an unRaid thing it is a 32bit Linux thing.  Linux uses all free memory for buffers which helps unRaid performance greatly when handling multiple streams at one.  The Linux Kernel used has a bug.  Either wait for a fix or roll back to a previous one.

 

[Helmonder]

Unevent: There are a lot of mentions of the word "crap" in your post associated with plugins.. I am sure the rest of us would not use both in the same sentence.

 

If you cut all the CRAP out of your post then you are stating:

 

"make sure the amount of plugins you run is alligned with the hardware you run. The fact that a lot of plugins are there does not mean you need to run them all."