unRAID Server Release 5.0-beta11 Available

[Rembro]

Just giving some feedback on my rig with Beta 11

 

copy-ing and moving large amounts of data from disk share to disk-share crashes this version.

keeping the selected data below 20 to 25 GB everything goes fine.

 

The shares disappear and even the gui isnt available anymore

 

only a hard reset wil bring it back alive. then it hangs a very long time on ''starting.....''

 

have reproduced this same error a few times now

 

no plug-ins ,no add-ons , no simple features just plain basic Beta 11

[Auggie]

actually beta11 crashes on shutdown for many of us.

 

Good to know, though I actually keep the server running 24/7 and only shutdown if hardware maintenance is required.

[madburg]

actually beta11 crashes on shutdown for many of us.

 

Good to know, though I actually keep the server running 24/7 and only shutdown if hardware maintenance is required.

 

What are your plans if you have a power failure?

[speeding_ant]

Really? I'm not sure why you're not running 4.7 if you want stability...  What other solution are you talking about, and what could possibly be so bad that you have to move platform completely?

 

I'm working with someone else to embed SSMTP with full SSL/TLS support. However, Tom has made this clear from the beginning. For one person developing this product, he doesn't have time to invest in plugins. That's where our great community comes in. If you want to make something better, collaborate with us to code in these features. It's not hard!

 

 

 

So my crystal ball was quite clear, that this would turn into a large issue as time moves on. Not happy about it or sticking to anyone...

 

I ran out of space and had to move to a different solution than unRAID for my storage needs temporarily. As I just cant trust it at this stage. Beta 11 took longer to come out than I had hoped and I was ok with an older stable kernal version as per Tom to gain the stability. But seems it has not cured it. I will upgrade just to be able to submit the feedback to help Tom.

 

What I would like to request is nothing more than 1) stability/speed of unRAID. 2) Embedded smtp client with SSL/TLS support and apcupsd client by TOM, so its offically supported. Just about every offerring out alerts you via email and can gracefully shutdown itself in an event of a power failure.

 

The frustration is quite simple, dont run anything Tom did not official make and your supported. But the basics are not there for stability and trust. Yes I can load the simplefeatures add-on for a great GUI, complie and add the ssmtp add-on, apcupsd plugin, shutdown script, cleanup scripts, mover scripts, this one shows temps, this one does not show temps, this one does this, that one those that. Edit this one to do this, do that for that. I want and need the basics FIRST and FOREMOST.

 

Forgive me but Squeezebox does not move me any closer to getting me on unRAID and the long awaited plugin manager will not get us anything we run today official supported because it was installed by this plugin manager. Not that I dont want them just that the carriage came before the horse at this point.

 

I have much invested and not much to show for it, so I am speaking out loud, how I feel. I would love to know where this is headed for the next beta.

[Interstellar]

Seeing that this has been in beta for awhile now, I haven't really been checking on it...

 

But is it now (b11), as you say, "stable" enough to install on a "production" server.  I use "production" loosely as it's just my personal media server.  I'm not too worried about bugs so long as data corruption is pretty much non-existent (yes, I know stuff can happen when you least expect it), but if that aspect is pretty close to reliability as any "final" version, then that's good enough for me!

 

I hope the final version is on the horizon. Want to migrate to unraid but only on a final 5.0 version.

 

Whats the difference between 5b11 and 5 final?

 

What if a small change to 5 final causes a new bug, but you've upgraded because it's final.

 

Titles mean nothing unless it's alpha software.

 

I have critical data on my UnRAID rig too however given the amount of people who have run the server perfectly fine (including me) without critical data loss bugs I think you should just upgrade anyway.

 

 

actually beta11 crashes on shutdown for many of us.

 

Yes, but I was referring to data loss. Which has been a very small number of cases and probably the same with 4.x?

 

Either way my point is that beta or "final" is just a tag, and data loss can happen with any version

[madburg]

Really? I'm not sure why you're not running 4.7 if you want stability...   What other solution are you talking about, and what could possibly be so bad that you have to move platform completely?

 

I'm working with someone else to embed SSMTP with full SSL/TLS support. However, Tom has made this clear from the beginning. For one person developing this product, he doesn't have time to invest in plugins. That's where our great community comes in. If you want to make something better, collaborate with us to code in these features. It's not hard!

 

 

 

So my crystal ball was quite clear, that this would turn into a large issue as time moves on. Not happy about it or sticking to anyone...

 

I ran out of space and had to move to a different solution than unRAID for my storage needs temporarily. As I just cant trust it at this stage. Beta 11 took longer to come out than I had hoped and I was ok with an older stable kernal version as per Tom to gain the stability. But seems it has not cured it. I will upgrade just to be able to submit the feedback to help Tom.

 

What I would like to request is nothing more than 1) stability/speed of unRAID. 2) Embedded smtp client with SSL/TLS support and apcupsd client by TOM, so its offically supported. Just about every offerring out alerts you via email and can gracefully shutdown itself in an event of a power failure.

 

The frustration is quite simple, dont run anything Tom did not official make and your supported. But the basics are not there for stability and trust. Yes I can load the simplefeatures add-on for a great GUI, complie and add the ssmtp add-on, apcupsd plugin, shutdown script, cleanup scripts, mover scripts, this one shows temps, this one does not show temps, this one does this, that one those that. Edit this one to do this, do that for that. I want and need the basics FIRST and FOREMOST.

 

Forgive me but Squeezebox does not move me any closer to getting me on unRAID and the long awaited plugin manager will not get us anything we run today official supported because it was installed by this plugin manager. Not that I dont want them just that the carriage came before the horse at this point.

 

I have much invested and not much to show for it, so I am speaking out loud, how I feel. I would love to know where this is headed for the next beta.

 

Very much so, 4.7 does not support my controllers, and 3TB drives. I ended up buying more HP SAS drives for my HP MSA70 and added another LUN. So yeah REALLY. I have lost data before and know what its like, if you lost baby pictures, wedding, home movies, etc.. its gone.

 

I am aware Tom is one person, but I am not asking for a bunch of plugins, or squeeze box, The two plugins i stated should be core in unRAID. So if you got hit by a bus i could deal with not having a nice GUI, but cant afford not having email alerts and ups support. So he found the time for squeeze box, but not these? come on.

 

P.S.  I  am the one working with you on the ssmtp plugin in SF  

But it would be better for all if it was apart of unRAID. Then no one would be questioned are you running 'stock' unRAID. The answer would be Yes, i am!

[speeding_ant]

LOL That is quite funny. Slow day for me!   

 

I agree that there should be a mailer built into unRAID, makes complete sense.

 

UPS support, maybe not... Support for UPS can get very messy, especially as there are so many different ways manufacturers interface with software. I've compiled a few drivers for different UPS products, and not one was the same.

[limetech]

Update: This week should see -beta12 which uses the linux 3.0.3 kernel.

 

So here's the explanation.  First, it is very desirable to be keeping up with linux kernel releases.  This is because the latest drivers and bug fixes almost always go into the "current" kernel, and it's up to various maintainers to port fixes into previous releases as they see fit.

 

The Realtek r8169 driver is a good case in point.  Looking through the change logs, since 2.6.39 there have been quite a few fixes, all the way up through and including kernel 3.0.  For some reason Realtek NIC's and their drivers have been problematic from time-to-time, on many platforms besides linux; and, it's mainly updates to this driver which I watch to determine when to upgrade the kernel 

 

But something changed starting in 2.6.39 which "broke" parity-sync, or rather, caused it to slow waaaaay down, i.e., running at 25% of normal speed.  I'd say I've spent probably a solid 40-50 hours trying to figure out what this problem was.  I didn't know if this was a kernel problem, or an unraid driver problem being brought out now, or some other systemic problem.  Turns out it was caused by a significant change in the kernel which resulted in about a 6-line change in the unraid driver to fix.

 

So.. I am completing testing on -beta12 and should be able to release very soon.  I don't know if this will solve the 'shutdown' crash since I can't reproduce that for some reason.

[hackztor]

I had issue with my 1.5tb being seen as 2tb and this was my drive with red ball. Ended up having to replace it with a 2tb because it kept saying my spare 1.5 was too small.

 

I'm doing some research into what I suspect might be a bug.

 

With 5.0-beta10:

 

A new 1TB drive recently inserted into array (passed pre-clear) failed with 4000 errors. However, it appears to be a driver issue with my motherboard chipset, as the drive was just reported missing (much research gone into this, many others having this same issue). I've had this happen before. SMART status shows no problems. I restarted the server, and the array auto started with a drive un-assigned. I stopped the array, and the gui was showing the "failed" 1Tb drive as a 1Tb drive, with 1.5Tb drive available sector size, and wanted to rebuild and expand the file system. I restarted again, and the drive showed up as 1Tb.

 

Few issues with this:

 

• Why did the array auto start when the array was degraded

• Why did the 1Tb drive show up as 1.5Tb, especially when it reported as a 1Tb.

• Why was the drive automatically assigned to the replacement slot.

 

I'll be posting syslogs and screenshots as soon as I've finished rebuilding the drive.

 

[mejutty]

Does beta 11 or 12 for that matter have the issue I am seeing with 10 to do with the time.  I set the time through the gui to be UTC+8 for Perth then set ntp time.  This works I get the correct time files written to the disk are correct.  On shutdown UTC time is written to the cmos not my local UTC+8 on the mobo obvioulsy 8 hours behind my real time.  On reboot obivoulsy the time is still 8 hours behind until the network comes up and it does an ntp then suddenly time jumps forward 8 hours and instantly my disk stop spinning.  I have tried a few things but the only one that I have working now is to manually set the time with the timzone set to UTC this ensures that the cmos time is correct and the unraid time is sort of correct if you ignore the timezone but means that windows reports files as being 8 hours old.

 

[PeterB]

Update: This week should see -beta12 which uses the linux 3.0.3 kernel.

 

Does this include any changes to the mptsas/mpt2sas drivers?  (See this post to understand my reason for asking)

[SSD]

Either way my point is that beta or "final" is just a tag, and data loss can happen with any version

 

+1  I like the way you put that!

[limetech]

Does beta 11 or 12 for that matter have the issue I am seeing with 10 to do with the time.  I set the time through the gui to be UTC+8 for Perth then set ntp time.  This works I get the correct time files written to the disk are correct.  On shutdown UTC time is written to the cmos not my local UTC+8 on the mobo obvioulsy 8 hours behind my real time.  On reboot obivoulsy the time is still 8 hours behind until the network comes up and it does an ntp then suddenly time jumps forward 8 hours and instantly my disk stop spinning.  I have tried a few things but the only one that I have working now is to manually set the time with the timzone set to UTC this ensures that the cmos time is correct and the unraid time is sort of correct if you ignore the timezone but means that windows reports files as being 8 hours old.

 

 

Time is stored in the cmos as UTC - this is how almost every O.S. does it (except Windows of course).  Upon boot, your timezone is read from the 'config/ident.cfg' file and shouldn't require ntp to display proper local time.  This sequence should work:

 

(assume 'Use NTP' is already set to No)

1. Select your 'Time zone' and click Apply.

2. Edit the 'New date & time' to reflect your actual local date and time and click Apply.

3. Change 'Use NTP' to Yes; define 'NTP Server 1' (e.g., pool.ntp.org) and click Apply.

 

After step 3 there might be a delay but eventually you should see a time correction message show up in the system log.

[limetech]

Update: This week should see -beta12 which uses the linux 3.0.3 kernel.

 

Does this include any changes to the mptsas/mpt2sas drivers?  (See this post to understand my reason for asking)

 

If I counted correctly, since 2.6.37.6 there have been 19 linux kernel releases (not counting -rc releases), so the answer is "probably".  You can look here:

http://www.kernel.org/pub/linux/kernel/v2.6/

and here:

http://www.kernel.org/pub/linux/kernel/v3.0/

and examine the text files named "ChangeLog-x.x.x".

 

Ok, just picking ChangeLog-3.0, "mpt2sas" is mentioned several times.  Here one such instance:

 

commit 5fd5cc83a886d691ab8cd6d4404ca35e09ce88e3

Author: Kashyap, Desai <[email protected]>

Date:  Wed May 4 16:37:04 2011 +0530

 

    [sCSI] mpt2sas: Driver version upgrade 08.100.00.02

   

    Signed-off-by: Kashyap Desai <[email protected]>

    Signed-off-by: James Bottomley <[email protected]>

 

 

Is this going to fix the issue you are seeing?  Hard to say, but probably there is a h/w problem causing this issue and it's possible the driver is not recovering correctly.  I suggest you bypass the cages and hook both power and data cables directly to the drives to see if problem persists (and don't use any jenky power splitters).

[speeding_ant]

I can't upload any attachments due to upload folder being full.

[limetech]

Forgive me but Squeezebox does not move me any closer to getting me on unRAID and the long awaited plugin manager will not get us anything we run today official supported because it was installed by this plugin manager. Not that I dont want them just that the carriage came before the horse at this point.

 

I coded the slimserver plugin mainly as "proof of concept" (as well as serve a practical need I had at the time) because it's fairly complex in requirements.  I was hoping to have a refined plugin manager and more plugins this week, but seriously, as mentioned in prior post, I got "stuck" at kernel 2.6.37.6 because all later versions result in very serious parity sync performance that I couldn't explain at the time - so all effort has been to understand and fix this issue.  The 'shutdown' bug appears to be another kernel issue, but I can't post any questions (and syslog trace) to the lkml unless I'm using the latest kernel.

[limetech]

I can't upload any attachments due to upload folder being full.

Should be fixed now.

[speeding_ant]

Here's a screenshot and syslog of this weird hard drive replacement issue.

 

The replacement hard drive is the original hard drive, which was 1Tb. Restarted, appeared as a 1Tb drive again. Also check previous post for the "auto start degraded array" issue.

Syslog.zip

[mejutty]

Does beta 11 or 12 for that matter have the issue I am seeing with 10 to do with the time.  I set the time through the gui to be UTC+8 for Perth then set ntp time.  This works I get the correct time files written to the disk are correct.  On shutdown UTC time is written to the cmos not my local UTC+8 on the mobo obvioulsy 8 hours behind my real time.  On reboot obivoulsy the time is still 8 hours behind until the network comes up and it does an ntp then suddenly time jumps forward 8 hours and instantly my disk stop spinning.  I have tried a few things but the only one that I have working now is to manually set the time with the timzone set to UTC this ensures that the cmos time is correct and the unraid time is sort of correct if you ignore the timezone but means that windows reports files as being 8 hours old.

 

 

Time is stored in the cmos as UTC - this is how almost every O.S. does it (except Windows of course).  Upon boot, your timezone is read from the 'config/ident.cfg' file and shouldn't require ntp to display proper local time.  This sequence should work:

 

(assume 'Use NTP' is already set to No)

1. Select your 'Time zone' and click Apply.

2. Edit the 'New date & time' to reflect your actual local date and time and click Apply.

3. Change 'Use NTP' to Yes; define 'NTP Server 1' (e.g., pool.ntp.org) and click Apply.

 

After step 3 there might be a delay but eventually you should see a time correction message show up in the system log.

 

That would be what I am seeing but while unraid is booting the syslog is utc time up until it either loads the timezone or does NTP so at the point when the syslog goes from the UTC time to my local time the disks spin down as they see it as the server being idle for 1 hour here is the syslog from my posting in the general support

 

Aug 21 22:05:52 Tower emhttp: shcmd (23): killall -HUP smbd

Aug 21 22:05:52 Tower emhttp: shcmd (24): /usr/local/sbin/emhttp_event svcs_restarted

Aug 21 22:05:52 Tower emhttp_event: svcs_restarted

Aug 21 22:05:53 Tower kernel: tg3 0000:02:00.0: eth0: Link is up at 1000 Mbps, full duplex

Aug 21 22:05:53 Tower kernel: tg3 0000:02:00.0: eth0: Flow control is on for TX and on for RX

Aug 21 14:06:12 Tower kernel: mdcmd (16): spindown 1

Aug 21 14:06:12 Tower kernel: mdcmd (17): spindown 2

Aug 21 14:07:01 Tower crond[1087]: time disparity of -479 minutes detected

[PeterB]

Tom,

 

Many thanks for your reply!

 

Update: This week should see -beta12 which uses the linux 3.0.3 kernel.

 

Does this include any changes to the mptsas/mpt2sas drivers?  (See this post to understand my reason for asking)

 

If I counted correctly, since 2.6.37.6 there have been 19 linux kernel releases (not counting -rc releases), so the answer is "probably".  You can look here:

http://www.kernel.org/pub/linux/kernel/v2.6/

and here:

http://www.kernel.org/pub/linux/kernel/v3.0/

and examine the text files named "ChangeLog-x.x.x".

 

Ok, just picking ChangeLog-3.0, "mpt2sas" is mentioned several times.  Here one such instance:

 

commit 5fd5cc83a886d691ab8cd6d4404ca35e09ce88e3

Author: Kashyap, Desai <[email protected]>

Date:   Wed May 4 16:37:04 2011 +0530

 

   [sCSI] mpt2sas: Driver version upgrade 08.100.00.02

   

   Signed-off-by: Kashyap Desai <[email protected]>

   Signed-off-by: James Bottomley <[email protected]>

 

 

Great - I had been googling for change logs, but hadn't found that.

 

Is this going to fix the issue you are seeing?  Hard to say, but probably there is a h/w problem causing this issue and it's possible the driver is not recovering correctly.  I suggest you bypass the cages and hook both power and data cables directly to the drives to see if problem persists (and don't use any jenky power splitters).

 

Yes, I was thinking that it was a hardware problem.  I had been running the same drives, controller and cables, without problems, for a couple of months.  I put the cages in and immediately experience 'attempting task abort'.  However, the problem occurs with two cages, both connected to the Supermicro controller, but not the third cage, connected to the mobo controller.  It was this which had me looking at it as a driver problem.

 

I have had no problems during 'normal' array usage (playing flacs and mkvs/isos). I can rebuild a 2TB drive without problems, but a non-correcting verify always causes the failure.  Preclear produces the error message immediately the first read test starts.

 

Oh, and I avoid power splitters like the plague - one of the factors in choosing my psu was the availability of 8 Molex and/or 8 SATA connections.

[Interstellar]

Update: This week should see -beta12 which uses the linux 3.0.3 kernel.

 

So here's the explanation.  First, it is very desirable to be keeping up with linux kernel releases.  This is because the latest drivers and bug fixes almost always go into the "current" kernel, and it's up to various maintainers to port fixes into previous releases as they see fit.

 

The Realtek r8169 driver is a good case in point.  Looking through the change logs, since 2.6.39 there have been quite a few fixes, all the way up through and including kernel 3.0.  For some reason Realtek NIC's and their drivers have been problematic from time-to-time, on many platforms besides linux; and, it's mainly updates to this driver which I watch to determine when to upgrade the kernel 

 

But something changed starting in 2.6.39 which "broke" parity-sync, or rather, caused it to slow waaaaay down, i.e., running at 25% of normal speed.  I'd say I've spent probably a solid 40-50 hours trying to figure out what this problem was.  I didn't know if this was a kernel problem, or an unraid driver problem being brought out now, or some other systemic problem.  Turns out it was caused by a significant change in the kernel which resulted in about a 6-line change in the unraid driver to fix.

 

So.. I am completing testing on -beta12 and should be able to release very soon.  I don't know if this will solve the 'shutdown' crash since I can't reproduce that for some reason.

 

Good stuff.

 

Look forward to it

 

Do you think the issues some of us have with the r8169 driver will be fixed with the 3.0.3 kernel?

 

Thanks!

 

 

PS: The shutdown crash has stopped for me now, and seems to be quite random.

[madburg]

Forgive me but Squeezebox does not move me any closer to getting me on unRAID and the long awaited plugin manager will not get us anything we run today official supported because it was installed by this plugin manager. Not that I dont want them just that the carriage came before the horse at this point.

 

I coded the slimserver plugin mainly as "proof of concept" (as well as serve a practical need I had at the time) because it's fairly complex in requirements.  I was hoping to have a refined plugin manager and more plugins this week, but seriously, as mentioned in prior post, I got "stuck" at kernel 2.6.37.6 because all later versions result in very serious parity sync performance that I couldn't explain at the time - so all effort has been to understand and fix this issue.  The 'shutdown' bug appears to be another kernel issue, but I can't post any questions (and syslog trace) to the lkml unless I'm using the latest kernel.

 

I know, I know Tom. You always try to work things out for the greater good/masses. The realtek driver is an example. You tried both the Linux and the vendors drivers to see what the feedback would be which was a great thing to do in a beta and that’s what they are for. But as you stated and I know from experience myself some hw/drivers are shady, realtek is one of them. So a decision will need to be made which to keep to serve the greater good.

 

That is the same for an smtp/ups client that I am requesting come baked in from you. If they work for 90% of the user base it meets the requirement. The ssmtp suite is very lite once compiled and will serve the masses. The apcusd (daemon) client is also lite and works with APC and other third party UPS’s, not all but will serve the masses.

 

I know your product is very flexible, but I see no issue if a line is drawn and something unfortunately at this time is not supported. Ex. SAS controllers are not supported in 4.7, 3TB are not supported in 4.7

 

Many products/appliances don’t support things. Buy a ReadyNAS, it comes with baked in email/ups clients you can add a few plugins.

VMware has a strict hardware model, because they want it stable (they won’t support realtek). FreeNAS has its baked in email/ups clients, tons of others.

 

I do understand you getting started with the plugin manager and sample plug-in, I am merely stating it came a bit before what is needed to get 5.0 stable and perform the core functions we need, I would have preferred a first pass at testing the email/ups client provided by you versus the plugin manager.

 

I can send you all what I have learned from the ssmtp suite that serves a multitude of mail configurations. Anonymous relays/ MSExchange / Google / SSL / TLS / with certs, etc… if that would help. If you dont want to take the route baking these two packages in, then it just needs to be stated, and we will just move on, knowing we need to do our own thing.

 

To add for some reason there is a community powerdown script that overwrites yours to be able to successfully shutdown unRAID, not sure why, but may need to be looked into as well to be incorporated….

[prostuff1]

To add for some reason there is a community powerdown script that overwrites yours to be able to successfully shutdown unRAID, not sure why, but may need to be looked into as well to be incorporated….

No, Tom specifically set up his version of powerdown to not kill processes keeping disks busy.  There are a number of reasons for this and it IS the correct approach.  The community powerdown package takes a much more forceful approach, which for people running a UPS MIGHT be needed, but is not necessarily.

 

Tom's approach in this case is the correct one.

[madburg]

To add for some reason there is a community powerdown script that overwrites yours to be able to successfully shutdown unRAID, not sure why, but may need to be looked into as well to be incorporated….

No, Tom specifically set up his version of powerdown to not kill processes keeping disks busy.  There are a number of reasons for this and it IS the correct approach.  The community powerdown package takes a much more forceful approach, which for people running a UPS MIGHT be needed, but is not necessarily.

 

Tom's approach in this case is the correct one.

 

Thanks for the info!

 

I have setup two Poll threads that maybe of use once everyone votes:

 

Question: Do you require email support with unRAID 

http://lime-technology.com/forum/index.php?topic=14862.0

 

Question: What brand of UPS do you have connected to your unRAID server

http://lime-technology.com/forum/index.php?topic=14866.0

 

 

[Auggie]

What are your plans if you have a power failure?

 

I have an APC Home Theater battery backup and conditioner, and only the unRAID and PCH are connected to the battery outlets.