unRAID Server Release 5.0-rc14 Available

[WeeboTech]

I tried out rc14 on my unRAID1  setup. I replaced the three files but it got stuck partway through the boot. Something about losing a connection which I guess is my external enclosures. It never got to the point where it recognized the 17 drives in my external, port multiplier, enclosures. I tried a second time with the same results. So I  copied the three files from rc12a back onto the flash drive and it booted normally. I have no idea what the issue was. I'll need to try it on my second unRAID setup tomorrow, which only has eight drives in external enclosures, to see if it has a similar issue.

 

 

How much ram do you have and which boot option did you select?

Even on my ESX system, there's a really long pause as it's attaching drives until a failure shows up.

Without the mem= parameter it comes up normally. My system only has 4 drives though.

[mejutty]

If that wad the case shouldnt I see the same results either boot given there is the same amount of ram each time ?? I would have expected to see this with the other type of kernel that would be limited to 3gb

quote author=Barzija link=topic=27874.msg246906#msg246906 date=1371079651]

It's like it forces it to 3gb instead of 4gb.

No, that's still 4GB in use, with a 3/1 split.  That's how 32-bit OSes normally work.

[jumperalex]

I only have 4GB, but I'm still running the "mem=4096" option.  Not sure if that makes a difference in either direction, BUT I am seeing much better write speeds (~75-80MB/s) to the cache drive.  On rc13 i was getting very erratic speeds with long pauses with an aggregate write speed o at best 30MB/s

 

So, rc14 is a plus in my book.

[WeeboTech]

When I boot with mem=4095M and 1GB the kernel crashes.

When I boot with mem=4095M and 1.2GB the kernel crashes.

When I boot with mem=4095M and 1.3GB the kernel crashes.

I would actually be surprised if that didn't crash!

Unless of course you use the memmap= boot parameter alongside, but then things get really hairy.

 

 

If we default to mem=4095M, anyone with less then 4GB is going to have problems.

[garycase]

It's like it forces it to 3gb instead of 4gb.

No, that's still 4GB in use, with a 3/1 split.  That's how 32-bit OSes normally work.

 

Not true ... there's not 4GB in use.  When you restrict the address space to 4GB, roughly 1GB of that address space is used to address various memory mapped devices; video aperture; bus addressing; etc.    The address space used for those purposes is not available to address RAM; so only the remaining addresses can be used for RAM.    In a Windows environment, that's typically in the range of 3.2-3.5GB, although it can be much lower, depending on the video cards in use.  I'm not sure what the range is for Linux, but from the discussion earlier I'd guess it's about 3GB.

 

[garycase]

It's like it forces it to 3gb instead of 4gb.

No, that's still 4GB in use, with a 3/1 split.  That's how 32-bit OSes normally work.

 

Not true ... there's not 4GB in use.  When you restrict the address space to 4GB, roughly 1GB of that address space is used to address various memory mapped devices; video aperture; bus addressing; etc.

 

So, it is used.

 

The ADDRESS SPACE is used ... NOT the memory.

 

[garycase]

It's like you have a fixed number of lots in a neighborhood (address spaces) ... say 100 lots.  You also have 100 modular homes available to place on those lots (memory).  BUT you need a lot for a school, one for the post office, perhaps a couple for community buildings, one for a convenience store, perhaps one for a community pool, etc.      All of those ancilliary functions are placed on lots where they have an address -- but none of those can be used for the modular homes.    So you can only use as many of the homes as you have lots (addresses) left.

 

[garycase]

Your "little tool" uses PAE - which you're trying to avoid here 

 

There are several utilities in Windows that will allow the use of RAM above the 4GB limit to provide RAMDISK functions.    Even in systems with only 4GB of RAM, any unassigned RAM is re-mapped to addresses over 4K and then managed as a RAMDisk using PAE.    Note that same general technique is how 32-bit Windows Server versions can support > 4GB of RAM.

 

[tr0910]

She won't format?

 

Starting up my 2nd RC14 system (other one is fine) with a newly precleared disk in it resulting in the normal messaging.  But saying that "I want to do this" and pressing the "format" button results in the system starting the format process, but pressing refresh shows that it hasn't started formatting at all.  You can restart formatting again, but it doesn't seem to do anything if you hit refresh.  Main screen shows 6 disks and parity disk all being green balled and valid but there is a message that unformatted disks are present (disk7)

 

She's likely still loading unMenu and some addons like "powerdown now" and "screen".  otherwise she's stock....

[mejutty]

Now I get it this has nothing to do with pae as we are not turning this off we are just limiting to 4gb if I changed that value to 4.5gb i would see another 500mb of my physical ram.  When you limit to 4gb thats 4gb in total ram see by the os.  So the amount of ram assigned to your graphics card, the memory in your raid controllers all count to this 4gb ram total which is why you don't see all of the physical ram. 

[Joe L.]

She won't format?

 

Starting up my 2nd RC14 system (other one is fine) with a newly precleared disk in it resulting in the normal messaging.  But saying that "I want to do this" and pressing the "format" button results in the system starting the format process, but pressing refresh shows that it hasn't started formatting at all.  You can restart formatting again, but it doesn't seem to do anything if you hit refresh.  Main screen shows 6 disks and parity disk all being green balled and valid but there is a message that unformatted disks are present (disk7)

 

She's likely still loading unMenu and some addons like "powerdown now" and "screen".  otherwise she's stock....

Attach a syslog to your next post, otherwise, we have no clues either.

[PeterB]

Just upgraded (from rc10) to rc14.  OpenELEC v3.05 is working fine.

 

One slight oddity - my first attempt to boot reported an error that the menu.c32 file was not COM32 format.  Now, this is the same menu.c32 which I've been using for a long time (the modification date on the file is May 16 2010), with many past versions of unRAID.

 

I opened the system to get the flash drive out to copy menu.c32 from the rc14 distribution.  This allows the system to boot again.

 

I guess that my menu.c32 was not being used until the new syslinux.cfg was put in place.  I note that the first line of my old syslinux.cfg is

default unRAID OS

the new one is

default menu.c32

.

 

Anyway, all is well now and this may serve as a prompt for anyone else who encounters the same issue.

 

I will now go and test my nfs stale file handle problem when performing mkvmerge.

[madburg]

Here you go Tom:

 

I have an ISO extracting on the cache drive at the moment as well.

4 Port LSI Controller (2 ports going to an Intel (LSI based) expander) for a total of 24 drive ability.

 

Booted in without the "mem=4095M" option

 

Large Read/Write to disks as always because AFP is enabled, after several thousand per disk, it stops. No difference here.

AFP clients connected upon first try.

Other lingering AFP issues all the same, understandingly as no changes have been made in that area.

 

------

GO:

# Starting with smartmontools 5.40 the drive database file drivedb.h can be updated separately with the following command:

/usr/sbin/update-smart-drivedb

 

# Start the Management Utility

ulimit -n 20000;/usr/local/sbin/emhttp &

 

# Make sure emhttp doesnt get killed

pgrep -f "/usr/local/sbin/emhttp" | while read PID; do echo -1000 > /proc/$PID/oom_score_adj; done

------

 

Packages:

VMTools RC14 - Courtesy of Zeron

tar-1.26-i486-1.tgz

openssl-solibs-1.0.1c-i486-3.txz

openssh-5.8p1-i486-1.txz.install

utempter-1.1.5-i486-1.txz

screen-4.0.3-i486-1.txz

(Wish we had a email plugin from you/unraid)

 

-------

 

Disk Settings:

md_num_stripes = 2560

md_write_limit = 1024

md_sync_window = 1024

 

-------

 

How do we go about finding duplicate files?

[jumperalex]

I only have 4GB, but I'm still running the "mem=4096" option.  Not sure if that makes a difference in either direction, BUT I am seeing much better write speeds (~75-80MB/s)

Your better speeds are because you've started the kernel in non-PAE mode.  My observations exactly! Even with 4GB physicall RAM installed, you are still much better off in non-PAE mode.  Thank you!

 

Unlikely since I've been running with the mem parm with 13 in an attempt to solve the problem.

 

Sent from my Nexus 7 using Tapatalk HD

[nars]

Getting back to the pills from the other topic...

 

- Red Pill: I guess Tom surely haves some kind of script to automate release, packing, etc... so... I don't really see a big big problem to provide 2 kernels or even two independent download packages (maybe even better, easier for users, smaller download... just explain on site the difference of both...), just add a couple of lines to release packing script to compile kernel twice (PAE and non-PAE), automatically, pack two versions instead of one, that probably will mean just 10-20 extra minutes (processing time) to finish a release of the two flavors, am I wrong?

 

(if you followed the sync thread a few days ago you may have read that I did compiled and repacked unraid with near 10 kernel versions in one day for testing sync issue, I did it with minimal effort with a small poorly written script that did all the job for each kernel version and I ended with a lot of bzimage_x.x.x/bzroot_x.x.x files to test... 20 min to fully prepare each version... minimal manual work... that's why I don't see why not provide 2 kernels on each release?)

 

IMO this would be probably the best option possible, a real solution, that should ensure stability/compatibility for who requires non-PAE and PAE for who wants/needs it, if and only if I'm correct that Tom can fully automate it (I don't really see why not) and that it will not mean more work for him...

 

- Blue Pill: This is the easier and faster option, release just PAE enabled and the affected users will add mem option, rename sysconfig.4gb to sysconfig.cfg file as said above, or whatever to WORKAROUND it with the mem option.

 

Only Tom can select what one to "take", what one is viable for him... and I guess it's no sense to start an "your chance to chime in" topic here again for two months about this...

 

That's just my two cents.

[madburg]

5% Into Parity Check:

 

Low memory looks good (re-executed free -l in each screenshot as well as ESXi stats on utilization)

Tomorrow I will test spin ups/downs, array shutdowns/restarts

and all other clients and work loads against it.

 

 

 

 

[mr-hexen]

anyone else having trouble downloading the rc14 release? i keep getting cutoff @ 18MB.

[WeeboTech]

anyone else having trouble downloading the rc14 release? i keep getting cutoff @ 18MB.

 

 

I was able to download the whole thing earlier this eve.  I did have some difficulty with the forum. Perhaps there's something going on with the host or ISP servicing limetech's website.

[madburg]

Parity check at 9% (passed that old 250GB drive, now we're moving):

Low mem just fine

 

[needo]

Two kermels? Really? He can't bring one kernel to final in over four years, and you're now suggesting he does two kernels? What on earth are you thinking?

 

And, it's all good with the pills, but you've set it up with a bias. One might rewrite it this way:

 

- Red Pill: This is the easier and faster option, release just one non-PAE package.

 

- Blue Pill: Release a PAE enabled package, and supply different sysconfigs which force different mem= options for people with different amounts of RAM, and all the workarounds. Then continue to investigate all strange reports about unrelated crashes and problems, until sometime in 2018 you take the Red Pill and bring v.5 to final.

 

Or do what I do... every release I run I decompress compile my own kernel based off of unRAID's configuration with an option I need (Advanced Routing) and I package it back up again. It is not that difficult. However, I expect 0 support from Lime Technology running my own custom kernel.

[tr0910]

She won't format?

 

Starting up my 2nd RC14 system (other one is fine) with a newly precleared disk in it resulting in the normal messaging.  But saying that "I want to do this" and pressing the "format" button results in the system starting the format process, but pressing refresh shows that it hasn't started formatting at all.  You can restart formatting again, but it doesn't seem to do anything if you hit refresh.  Main screen shows 6 disks and parity disk all being green balled and valid but there is a message that unformatted disks are present (disk7)

 

She's likely still loading unMenu and some addons like "powerdown now" and "screen".  otherwise she's stock....

Attach a syslog to your next post, otherwise, we have no clues either.

 

Of Course, here you go...

syslog-20130612-193453_NoFormat.zip

[madburg]

Parity check at 9% (passed that old 250GB drive, now we're moving):

Low mem just fine

 

Madburg, may I ask you for a favor please? I see from your screenshots that you've allocated more than 4GB memory to that machine. Would you mind, when you have a chance, start that machine with less than 4GB, and see if you'll have any performance differences? And I don't mean just parity sync, I mean actual copying of large amounts of files (and big ones) to the server. Thanks.

 

I could, once I am done with all my tests. Parity won't be complete until tomorrow and I wont have a chance until after work, plus all the prep work for fathers day coming up

 

If you provide some details on what you consider large file copies and from a client to where? cache or array, create a scenario and I will do it when I have a moment.

 

How much memory do you want me to allocate, and under which boot option do I boot from with your choosen memory.

 

What I can tell you is once I fire up all my clients and workload aginst the unRaid server and kick off the mover my mem utilization hits above 5gb. So there is no way in hell I would settle/live with a non pae build on a permanent bases. Since I don't run apps from the unRaid server I don't experience crash, disk unmount issue etc... The issues in the past with kernel crash and oops errors seem to have been ironed out slow througth the kernel updates and Tom's tweaking the for a lack of better words unRAID script/logic etc... As well as virtualizing unRAID on my SuperMicro board. So my only issue is AFP backend running on unRaid. And if Tom has NFS licked acceptable with this release, the hopes are AFP being next, before final. May need to do a phone with remote access session with him to nip it in the bud quickly.

 

I also want to meantion (again) my experience under virtualization was poor with allocating 8gb ram, 4gb was to low when things where cooking, so I decide one day to try 6GB and it's the perfect sweet spot. I cant explain why and it's a perfect marriage.

[calypsocowboy]

I don't know what it is about that 3.4 kernel but my mobo doesn't like it. It never seems to find my USB drive.  Mine doesn't even come up with rc14. I went back to 13 and it was fine. I've rebooted a few times with 13 and no issues with stopping the array.

 

I feel for you Tom, trying to find something that works for everyone.

syslog14.txt

[nars]

Two kermels? Really? He can't bring one kernel to final in over four years, and you're now suggesting he does two kernels? What on earth are you thinking?

As I said I don't see that needing extra work, just automate it to compile kernel twice and get two files ready, simple and easy, that's why I suggested it else I would not do it. But surely it's down to him.

 

As for the four years... I can't comment, despite I did read a LOT on forum, I'm on the "boat" only since September of last year... my 1st version was rc8 (and I did purchased it with such status and knowing how much time it was rc, etc) I did see a few rc's since then, sorting multiple problems, new problems coming, some getting hard to get a solution and eventually waiting kernel or other opensource developers to get them fixed, realtek issues, etc... etc... but now... other than the slowdown issue, that can be sorted easily, we seem to have a pretty stable version, right?  Tom did recently invested on new site, Tom#2, new servers for sale, etc... then... the time to get a stable 5.0 final PROPERLY released must be now. Anyway I don't want to get on this kind of discussion as I don't think it will help on anything... I did just posted my humble opinion on what I think would be the best option nothing more

[garycase]

Agree that it's irrelevant how long it took to get to where we are today => we are where we are.

 

Clearly Tom's very focused on getting v5 final released as soon as he can;  but he's also (rightly) focused on ensuring it doesn't have any major issues when he releases it.

 

As for four years => not at all true.  The final v4 release (4.7) was released 2 yrs and 5 months ago ... so it's been 29 months since the last "official" release.    Still a long time ... but not 4 years.  [Yes, work was ongoing earlier than that -- but so is work on a 64-bit version;  work on various other "features" to be added at some point in the future; etc.    But the actual time between released has "only" been 29 months]

 

As for the one vs. two kernel discussion => clearly that's up to Tom.  Personally IF the 4GB restriction resolves the same set of issues, I'd think it's preferable to have a single PAE kernel with that optional parameter.  But Tom knows more than any of us which makes more sense to his development cycle and release management functions.